Matik development director Artyom Ovechkin has written a note about the reason why a great many of A/B-tests fail and why it’s necessary to pay more attention to preparation rather than testing itself.
According to marketingcharts data, А/В-testing is considered to be the most efficient method of managing conversion. 60% out of more than 1000 polled marketing specialists in Europe consider this type of page testing to be more effective. Another quite popular method is the analysis of visitors’ paths throughout a website. It got 52% of positive reviews. A method of comparing to competitors had the lowest rates — 26%.
According to another investigation of the same source, only one out of seven А/В-tests is recognized as statistically presentable and successful. I can tell it from my own experience that А/В-tests haven’t entered the pool of obligatory marketing tools. It’s the destiny of either the biggest or the “most advanced” companies. As practice shows, testing is a long period which can take more than 1 month.
The majority of companies prefer to work faster rather than focus on quality. But taking into account market reduction and lack of money in business, I can say that the stimulus “to spend money faster” will calm down significantly. There’s an evident tendency to “spend the budget wisely with a prior preparation work”. A/B-testing is a thorough preparatory work that will help you to increase website conversion many times.
But as we have already mentioned, the majority of the test show insignificant improvement. Why does it happen? There are several reasons.
Solid data ground
The main reason for failures is the absence of preparatory work. Yes, the preparation stage itself needs some preparation! Don’t be confused: the more thorough is the preparation, the better is the effect. We need to get used to it.
A/B-test is the final stage of the process. Before carrying out the test it’s necessary to collect the following data:
- Usability-analysis of the website made by a specialist. The analysis will indicate key mistakes if the page, show you the most important things, and will clear what and for that should you test. Usability-analysis is a map with roads from A to B. Periods of a usability-audit depends on the capacity, age of the website, and depth of the analysis. So it maybe 1 day to 2 weeks.
- Data on website attendance for quite a long period (from half a year). They will show how users interact with the website, help to find out weaknesses and what you need to improve. It costs you nothing to introduce an analytics service (for example, Google Analytics).
- And finally, use heatmaps.
The idea preparation steps are to collect maximum data about the current state of the website. In order to compare A and B, you need to have not only an enhanced B variant but also a current A variant. Otherwise, you won’t be able to understand what exactly gets better.
60-70% of time should be spent on the pre-preparatory stage (before A/B-testing). 30-40% is for creating pages and launching tests.
Now we have solid data of what’s wrong with a website and what “places and zones” with what aim should you test. It’s the first difference between the right tests from the wrong ones. “Losers” rely on intuition and common sense (“Buy” red button should work better). Winners hold statistics in their hands and auditors’ detailed analysis. That’s why it’s so important to collect statistics from the first days of your website.
According to marketing charts, only 13% of companies have solid data. Others prefer intuition and common ideas of “what’s good and what’s bad” in website usability.
Solid understanding of your client
Another part of the preparation work is understanding why a client chooses your company and your website. Forget about “search results”. You need to answer the question of what’s is so special about your website that people make purchases?
Surely, you need to answer a negative question as well: what’s there on your website that people don’t make purchases?
As you understand a well-known “Buy” red button may increase conversion, but if a problem lies deeper, then slight page optimization (cosmetic changes) won’t give you significant results worth your efforts.
You should formulate hypotheses on the basis of your answers to the questions: what can I do to turn “no” into “yes”.
Eric Reyes, a popular marketing specialist, suggests finding answers with the help of a series of “Why?” questions. Put your intuition aside and go to people. Ask your clients and visitors why they have or haven’t chosen your company. Ask them “Why?” several times:
- Why did you think about purchasing?
- Why did you click our website link?
- Why did you choose our goods?
- Why did you fill out the application form?
- Why will you make another purchase with us?
As you have already understood each question may have a negative form.
I personally don’t recommend to use online-polls as people answer them thoughtlessly. It would be better to ask 10 real people individually by phone rather than cover 100 people with the help of a soulless program. Contact with a living person will give you an insight.
Solid design foundation
Experts recommend using frameworks as a solid base for the creating of design for tested pages. Models are sketches of pages used as hypotheses. It would be better to see how they look.
They are created faster than ready design-project pages. You work on enhancing hypotheses and a scenario of how a visitor reacts to this or that variant on this stage. In some cases, several real users may be involved in the process. But as for me, I don’t think that it will have a significant effect, and evaluation of hypotheses and design is a complex topic, and not all the clients have a sufficient level of abstract thinking so they could evaluate models.
Once you have solid analytics, report on client polls and several approved models, you’re ready for a real A/B-testing. Order the creation of ready pages on the basis of your models and launch them. And then you’ll be one of the 7 companies with significant A/B-test results.