Conversion Optimization (CRO)

Conversion optimization is a discipline within the field of online marketing. This discipline has continued to gain popularity since its introduction. In this article I provide a comprehensive picture of what the meaning of conversion optimization is and what a CRO specialist can do for you in such a process.

What is conversion optimization (CRO)?

What is conversion optimization? Wikipedia gives the following definition of conversion optimization: "In online marketing, conversion optimization, which is also called Conversion Rate Optimization (abbreviation: CRO), is a system for increasing the percentage of visitors to a website that are converted into customers, or more broadly, who take a desired action."

The term Conversion Rate Optimization was introduced in 2007 by the British company Conversion Rate Experts (CRE). From that moment on, however, there has been doubt as to the extent to which the name covers the scope of the activities it covers. When introducing the term, CRE itself indicated that they had a broader scope in mind, namely optimizing the entire company (and not just the website). CRE: "Well-done CRO optimization involves assessing the entire process from the first ad to lead generation through the post-sale follow-up (i.e. the entire customer journey). The real purpose of CRO is to determine which parts of the sales funnel yield the biggest profits with the least work.". In this way, optimizing the conversion takes on a broader meaning.

Customer Experience Optimization (CXO)

In addition to the term conversion optimization, other terms are also used to describe this discipline. The most popular alternative term used is Customer Experience Optimization (CXO). CXO is often described as a variant of CRO optimization in which the entire customer journey is viewed (instead of just the website), just as this was intended when the term was introduced.

In practice, however, you won't always notice a huge difference between a company that does 'CRO' or one that does 'CXO'. Different companies will or will not carry out different activities within this discipline. For example, sometimes user research or strategy will fall under CXO but not under CRO, while another company will have a division between CRO and UX, and yet another company does indeed optimize the entire customer journey, but calls it this discipline for recognisability. still just CRO. In other words, every company gives it its own meaning.

Why conversion optimization

Organizations work hard to better achieve their goals. In many cases this will involve realizing more turnover, but also a larger number of donations or increased awareness for a certain issue are common goals. Simply put, as a company you have two aspects that you can work on to better achieve your online goals. These are:

1. Increasing the number of visitors to your website or webshop.
2. Convert more of the website visitors.

Why is conversion optimization important?

Previously, companies mainly focused on increasing the number of visitors by, for example, working on search engine optimization (SEO) or online advertising (SEA, display). However, more and more organizations are starting to realize that just investing in this one aspect is not the most effective use of the online marketing budget. This can be compared to constantly refilling a leaking bucket. However, if you were to spend a little less money on refilling the bucket every time and a little more money on closing the holes, more water would remain in the bucket in the long run. That's exactly why conversion optimization is so important.

Effect of CRO on revenue

The conversion rate of your website or webshop is the number of visitors who take a certain action, divided by the total number of visitors. If you then add the average order value (AOV) to that formula, you can calculate the effect of conversion optimization on your turnover.

It is also important to realize that if these 5% extra visitors are purchased through advertisements, the costs to realize this increase in turnover will continue to occur monthly. This is in contrast to an increase that is achieved by increasing the conversion rate. This immediately shows that website optimization must be viewed from a broader perspective than just one department in order to make the best possible use of the benefits. However, the above does not mean that optimizing conversions is a one-time action that you perform (and pay for) after which you no longer have to worry about it. For example, research by Econsultancy shows that 'high performers' much more often have a structured CRO process than 'mainstream performers' (52% vs. 28%).

Organization

Every organization is of course unique. Just as the role of HR is different in an organization with 100 or 10,000 employees, the same applies to the role of conversion optimization. However, it is important to realize that CRO needs the support of other departments to achieve good results. Good mutual cooperation and overarching strategy is therefore very important.

Freelancer, agency, or in-house

Who is responsible for CRO optimization within a company? This depends on several factors, including the size of the company, its digital maturity, and the available budget. In many cases, however, it will involve a combination of the following:

• In-house online marketer: In many cases, a conversion optimization process starts within a company with an in-house online marketer who takes on an additional task as CRO or is assigned by management. The level of optimization is often relatively low, as is the amount of time available for this. One of the advantages of this is that you can switch quickly as a one-man team.
• Freelancer or agency: After the importance of conversion optimization within a company has become clear, in many cases people decide to hire a freelance conversion optimization specialist or CRO agency. Because they are specialized, this will generally result in a significant increase in the level of optimization. Of course, the effectiveness of the freelancer or agency still depends on the support from the company.
• In-house specialist or team: In many cases, the next step is to hire an in-house specialist and/or set up an in-house team. This is usually done in collaboration with the external party with which we are currently working. The role of the external party is therefore slowly shifting from executive to advisory.
• Center of excellence: The final step in this process is to make CRO a center of excellence. In addition, the ability to validate or test aspects within the company then becomes the 'product' of this team instead of them just carrying out the optimization themselves.

Roles within a CRO team

Multiple roles must be fulfilled within a CRO team. However, especially in organizations that have been working on conversion optimization for less time, you will see that some roles only have limited fulfillment, or that multiple roles are fulfilled by one person.

• CRO manager: A CRO manager is someone who ensures that the specialists can do their work properly. He or she also often ensures contact with other teams and/or with the external customer. Finally, the CRO manager is often responsible for planning and in many cases also project management.
• Conversion specialist: Depending on how large the team is, this person can/must fulfill multiple roles at the same time. In fact, in small CRO teams this specialist sometimes fulfills all substantive roles at the same time. For example, if there is no developer available within the CRO team, then that role must also be fulfilled by this specialist. Within larger teams, a CRO specialist is often mainly concerned with A/B testing. Consider maintaining the A/B testing roadmap, drawing up hypotheses, setting up A/B tests, etc.
• Web analyst: The person who fulfills this role is concerned with matters such as tagging, implementation tools, click tracking, setting up dashboards, etc. In some cases, however, the web analyst will (also) act as a data scientist and be more concerned with predictive models for recommendations, for example.
• Psychologist: He or she will provide insights from psychology. Consider recent studies that are relevant to a certain hypothesis or insights into the behavior of certain target groups.
• Developer: When things need to be built, this person is called in. For example, he or she will take on the task of building A/B tests, as well as developing complex personalization.

Collaboration with other teams

In most cases, teams from multiple disciplines work simultaneously to improve a website. By ensuring good cooperation between these teams, there is as little friction as possible and the best possible results can be achieved for the company.

• Marketing: Many companies work with a marketing calendar. The planned campaigns have been set out well in advance that you can take into account when optimizing the website. Of course, there will always be unplanned campaigns or promotions, so make sure you are kept informed of these in a permanent place.
• UX: Is there a separate UX department in addition to the people who are busy optimizing conversions? Then make sure that you carefully coordinate with them which user research is planned.
• Design: Some companies have strict rules regarding corporate identity. Consider the use of color, fonts, or even distances between elements. By taking these rules into account early on, you avoid having to make major adjustments to A/B tests later to bring them in line with the corporate identity.
• Development: Is a specific working method used, such as Agile, Scrum, or sprints? Is it necessary to obtain data from the backend for certain tests, or is server-side testing perhaps used? Where can the release calendar be found? For such questions, good cooperation with the developers is very important.
• Management: Ensure that management is as informed as possible about the results achieved. It can also help to discuss the possibilities of CRO optimization that are currently untapped. For example, consider validating new features before they are developed by the developers.

Managing expectations well

In addition to good cooperation, managing expectations is also an important part. For example, have targets been set for the number of tests that need to be run? Or expectations about a certain increase in conversion rate or turnover? Goals that need to be achieved by CRO within a certain period? In many organizations, 'CRO' is on relatively thin ice (especially in the early days). Therefore, make sure that you are well informed about the expectations and adjust them where possible.

Percentage A/B test winners

The highly experienced expert in online experiments Ronny Kohavi has made the following statements about his time at Microsoft.

Ronny Kohavi - Vice President at AirBNB

"1/3 of the ideas gave a positive and statistically significant result,
1/3 of the ideas were not statistically significantly different,
1/3 of the ideas gave a negative and statistically significant result."

Please keep in mind that on further optimized platforms the percentage of A/B test winners with sufficient significance is often only around 10-20% positive. Another important point here is that a lot can of course also be learned from tests that are not significant 'winners'!

Research

In order to arrive at good hypotheses for possible optimizations of the website, thorough conversion optimization research is of great importance. With this research you ensure that not just 'nice ideas', best practices, or general tips are tested, but that these ideas arise from (possible) wishes or needs of the visitors. Another advantage of a CRO process built on research is that it becomes less likely that the HiPPO will have too much influence on what will be tested.

Quantitative research

Research called quantitative is characterized by high numbers of data points, anonymous data, and data that focuses on answering questions that start with what, where, and how (and not with who or why).

Statistics

Valid and reliable statistics about what actions visitors take on the website are indispensable. This includes data about users, page views, events, goals, transactions, etc. By storing this data in tools such as Google Analytics or Adobe Analytics, you can gain insight into how the website can be improved. It is very important that the data quality of the statistics is in order. If this is not the case, you are drawing conclusions based on incorrect data.

Due to the critical role that web statistics play in a successful conversion optimization process, it is important that at least one team member has in-depth knowledge of this. However, it is preferable that several (or even all) team members have this knowledge so that findings and issues can be discussed at a different level.

Visitor behavior

In addition to visitor behavior in the form of figures and graphs, it can also be helpful to be able to observe this behavior visually. Tools like Hotjar can help you with this. Within these types of tools, you can, for example, visually display click behavior using so-called heat maps, movement maps, and scroll maps. In essence, these are screenshots of a specific page on the website (such as the homepage, category page, product page, shopping cart, or checkout) over which a layer has been placed on which things such as clicks, mouse movements, and scrolls are made visible.

Various tools now also offer screen recordings. This allows you to see how a visitor navigates across the website, which parts of the website are viewed, which filters are used on an e-commerce webshop, etc. Because this involves moving images, you can often get a much better idea of how a certain issue arises when you are only looking at static data or heat maps.

Qualitative research

Research that is called qualitative is characterized by low numbers of data points, personal data, and data that focuses on answering questions that start with who and why (and not with what, where, or how).

User tests

This research method (also called user testing) is a popular method to gain in-depth insights into the behavior and mindset of the target group. User testing often uses panels from which people from the target group are selected. They are then asked to carry out certain assignments on a website (such as purchasing a certain product) and to answer a few questions about it. The participant's screen is recorded, as is their voice, and in some cases even a video of their face. The combination of all this input ensures that you learn more about how the target group experiences a certain department, option, filter, form, etc. Companies such as Userfeel or the Dutch UserSense can help make this type of user research possible.

Written feedback

This method has many different forms, with associated advantages and disadvantages. The written feedback can vary from a simple exit poll with one question to extensive questionnaires. Reviews left by existing customers are also an important source of written feedback. With this written feedback, keep in mind that the wording of the question is often very important. By asking a question in a slightly different way (or using a different trigger before a question is shown), the level of the answers can differ considerably.

Interviews

This is probably the method with which the most in-depth insights can be obtained. It is therefore important that this is really about one person who thinks about something in a certain way. However, this issue can of course be addressed by conducting a larger number of interviews and examining the extent to which consensus can be found in the responses. When conducting interviews, also try to take psychological pitfalls such as the value-action gap and choice-supportive bias into account.

Website audits

In addition to user research, you can also have an expert perform analyzes of your website. These are often efficient processes that can quickly reveal a large number of points for improvement for the website. Depending on the specific type of website audit that is carried out, this may involve broad analyzes or research focused on a specific aspect of the website.

Expert review

With an expert review, a conversion optimization expert takes a detailed look at a specific website. He or she will use a wide range of knowledge, tools, and experiences. Many organizations seem to expect that the web builder also has such knowledge and has therefore already taken this into account when building the website. Unfortunately, in practice it appears that this is almost never the case. Some web builders will know more about optimizing a website for conversion than others, but in general the knowledge is unfortunately very limited.

Competitive analysis

When a competitive analysis is performed, an expert looks at how competitors' websites compare to the website being analyzed. For example, are there other unique selling points, is more information provided about the services, are more photos available, etc.

This input from what competitors are doing can be included in ideas about how your own website can be further optimized. It is often the case that competitors do not necessarily know what they are doing, but this can provide very valuable information just for inspiration.

Experiments

No article about conversion optimization is complete without also talking extensively about experiments. The most commonly used term for such experiments is A/B testing, but other (related) terms such as split testing, multivariate testing, and online controlled experiments are also used.

The points below provide a detailed explanation of various matters related to online experiments, but are by no means a complete overview. If you are interested in learning more about this subject, I can highly recommend the book Trustworthy Online Controlled Experiments.

Testing tools

There is a wide range of tools available to run A/B testing on a website. Well-known examples are VWO, Optimizely, AB Tasty, and Convert. Each of these tools has its own advantages and disadvantages.

In addition to the previously mentioned tools that you can use out of the box, it is of course also possible to build your own testing tool. This can range from a simple piece of code that you load into Google Tag Manager to split visitors up to extremely complex A/B testing suites that effectively replace the production website. Keep in mind that building a complete A/B testing tool is extremely complex and should only be considered as a serious option when there is a strongly mature CRO process and significant resources to help such a project succeed.

Clientside or serverside?

Client-side testing tools are loaded using a piece of JavaScript that is placed on the website, webshop or landing page. This JavaScript is then used to divide the visitors and apply the variants. The big advantage of client-side tools is that they are easier to implement and work well regardless of a certain server setup and the programming language of the website. The major disadvantage of client-side tools, however, is that they often slow down the loading time of the website and that so-called flickering can occur. The control version of the website is then temporarily visible before changes from the variant are implemented.

However, server-side testing tools require that a large part of the operations such as splitting visitors between variants are handled by website code. The server-side testing tool is mainly used as a means to analyze tests. However, as with client-side testing tools, the exact functionality that is or is not offered varies considerably per tool. The big advantage of server-side testing tools is that there is no longer any impact on loading time (and no more flickering). With serverside testing you can also use data from the backend to make price testing much easier, for example. However, the major disadvantage of server-side testing is that implementation is many times more difficult than with client-side testing tools. My advice would be not to start this if you have serious developer capacity available.

Roadmap for experiments

To ensure that there is a structured approach to the conversion optimization process, a roadmap is an important part. This contains details about all experiments from the present, past and future. Examples of how you can use this data include creating alerts when an experiment needs to go live or segmenting the test results based on certain characteristics.

Spreadsheet tools such as Google Sheets are widely used for setting up a roadmap, but hybrid spreadsheet databases such as AirTable are also common. In addition, there are targeted CRO management tools such as Effective Experiments.

Create a good hypothesis

Solid experiments are built on good hypotheses. As mentioned earlier, these hypotheses preferably arise from previous research as much as possible. However, the structure of the hypothesis itself can also contribute to how valuable it will be. To arrive at a well-constructed hypothesis, CRO veteran Craig Sullivan, in collaboration with many other CRO experts, has drawn up two hypothesis kits.

Craig Sullivan - Owner at OptimiseOrDie

"1. Because we observed (data/feedback).
2. Do we expect that (change) will cause (impact)
3. We are going to measure this using (data point)"

The usefulness of A/A testing

With an A/A test you can investigate the validity and reliability of your testing tool. The idea of an A/A test is that you put an experiment live, but without making any changes to the variant. If the tool does its job well (and you have collected enough data), this should ensure a non-significant difference between the control and the variant. If there is a significant difference (despite the fact that no changes have been made to the variant that could cause this), this may be an indication that something went wrong with the implementation of the testing tool.

A/A tests are often used at the start of a CRO process, but it would be better to run them on a regular basis. The biggest problem with A/A testing is that they provide more 'tested visitors', which can be expensive with tools with a visitor quota.

Setting up an A/B test

There is a lot involved in properly setting up an A/B test. Consider, for example, writing the code for the variations, but also matters such as choosing the right traffic allocation, drawing up a solid hypothesis, the way in which the experiment should be triggered, additional click tracking, the KPIs on which the experiment is based. will be assessed, the use of asynchronous blocks of code, the structure of the URLs, links to the statistics tool, extensive quality assurance, dealing with statistical outliers, etc.

Unfortunately, the drag-and-drop editors of popular testing tools make it seem like running a valid and reliable A/B test is a piece of cake. Unfortunately, with modern websites (which often load parts of the website dynamically with JavaScript and work responsively on multiple screen sizes) this is a distortion of reality. If you set up an A/B test without in-depth knowledge of the above aspects, the test will almost certainly produce results that are not valid and reliable. The result of this is that you start making decisions based on 'nonsense data', which means you are even further away from simply guessing whether a change will lead to an improved conversion or not.

Analyzing experiments

After sufficient conversions have been achieved, you should stop an experiment and analyze the results. However, this immediately raises a number of questions:

• When do you stop an A/B test? After sufficient data has been collected. As a rule of thumb, you need at least 100 conversions per variant.
• How many days must an experiment run at least? The rule of thumb is that an experiment must have been run for at least 7 days to rule out any effects associated with certain weekdays. However, this is more complex for websites that have products or services with a long sales cycle, because the conversions as a result of the experiment only take place after a longer period.
• How many visitors do you need at least? The minimum number of visitors for successful experiments strongly depends on the conversion rate. If 5 out of 100 visitors convert, you need much fewer visitors in an experiment than if this is 1 out of 100.
• How many conversions per month do you need? The rule of thumb here is that you need at least 1,000 macro conversions per month. If you have significantly fewer conversions, you will have very irregular winning tests. As a result, enthusiasm and support for the testing program often quickly disappears.

Analyzing experiments is usually done in a statistics tool such as Google Analytics. By hanging the variants of the test as segments above the reports, you can easily view the differences. Which reports you view next depends greatly on the testing goals you have set up. However, in many cases, reports like 'Device Groups' and 'Ecommerce' can provide valuable insights into the experiment.

Frequentist or Bayesian

Experiments can be analyzed using various statistical methods. Almost all testing tools nowadays use (variants of) the frequentist or Bayesian approach. Long articles have been written about the differences between these approaches, such as Bayesian vs. Frequentist. Personally, however, I am in favor of the Bayesian approach because it better suits the business context in which A/B testing is performed (this is in contrast to the scientific world from which the frequentist approach emerged). Conversion optimization expert Annemarie Klaassen has written an extensive article about Bayesian A/B test analysis, which I definitely recommend if you want to read more information on this subject.

Personalization

In many cases, personalization is little more than putting a variant live for a certain segment of visitors. Examples of this are logged in visitors, visitors from a certain region of the country, visitors who have already viewed certain pages, visitors who have reached the website via a certain channel, etc. In many cases, personalization will result from the analysis of an A/ B test. This then shows that a certain change has a significant positive effect on the intended goals, after which this variant can be put live as personalization for that segment.

However, sites like Booking, Netflix and AirBNB take the concept of personalization to the next level. They can (with the large amounts of data they have about their users) create almost completely unique experiences per user. Unfortunately, this is not possible for almost all 'normal' websites.

Conclusion

Conversion optimization is a potentially very powerful tool to work on the goals of your company. However, as the above article shows, there are a large number of aspects that need to be taken into account. However, if you ensure that you gather sufficient information about each step (and involve experts where necessary), I certainly expect positive results to emerge from the CRO process.