When wanting to improve a page’s conversions (for instance, sign-ups
or sales), we often use multivariate testing to see which combination of
changes will effect the page in the best possible way. By testing
different elements of your page like titles, buttons, layout changes and
color, you can find out which combination will maximize your page’s
conversion rate.
There are, of course, different ways to analyze the results of your test and choosing the right one is important as it will determine how quickly you can choose to make a combination a winner or a loser. There are three main testing methods which are used – Two are established while the third is a newer concept that aims to be the best option out of the three.
Google Website Optimizer
uses this system so you can easily set up a full factorial multivariate
test at absolutely no cost. The main problem with this approach is the
traffic needed to quickly determine a winner. Since the number of
combinations can increase so easily, you will need a lot of traffic to
determine a winner. This means that quite often, you will have to limit
your test to just a few changes to avoid waiting weeks, if not months,
for a winner.
When you do find a winner, however, you will know that the winning combination is definitely the best – The full factorial method is the most reliable method to use if you want to be absolutely sure.
The main advantage of this Fractional Factorial method is the shorter time needed to find a winner. This means it is possible to evaluate a larger number of page elements and variations than the Full Factorial method. Unfortunately, since only a sample is tested, it means that not all possible combinations are tested and as a result, you will have less accurate estimations of how well each variation performed.
Like
the Full Factorial method, each test will only be complete once
statistical confidence is reached, which means you still need quite a
bit of traffic to be able to draw some conclusions. Keep in mind,
however, that the results will never be as accurate as the Full
Factorial method.
HiConversion.net,
the creators of this method, claim that this system will initially show
a few random page combinations to live visitors and process their
reactions in real time, creating statistical data that is used by the
adaptive algorithm to generate a new page combination that will be shown
to live visitors.
Although the method is new, it does sound very promising. The main advantage of this method is the fact that you need just 1% of the traffic you’d need for the Full Factorial in order to determine a winning combination. Although I’m not sure what the maths is being the algorithm, they claim to put your conversion rate on autopilot by monitoring page performance and adapting to new market dynamics on-the-fly.
I’m pretty curious about how well this last method works – If any of you have any experience using it, please do share below.
There are, of course, different ways to analyze the results of your test and choosing the right one is important as it will determine how quickly you can choose to make a combination a winner or a loser. There are three main testing methods which are used – Two are established while the third is a newer concept that aims to be the best option out of the three.
Full Factorial Method
This method involves testing every single possible combination until a winner is found. If you are testing two different titles and two different buttons, your test will have 4 different combinations:- Title 1 – Button 1
- Title 2 – Button 1
- Title 1 – Button 2
- Title 2 – Button 2
Image from http://www.websiteoptimization.com/services/marketing/multivariate-testing/
When you do find a winner, however, you will know that the winning combination is definitely the best – The full factorial method is the most reliable method to use if you want to be absolutely sure.
Fractional Factorial Method
This method is more sophisticated than the full factorial and usually costs quite a bit to set up. Rather than testing every variation, a smaller sample is tested and mathematical modeling is used to predict a winner. The most common method used in fractional factorial testing is the Taguchi Method.The main advantage of this Fractional Factorial method is the shorter time needed to find a winner. This means it is possible to evaluate a larger number of page elements and variations than the Full Factorial method. Unfortunately, since only a sample is tested, it means that not all possible combinations are tested and as a result, you will have less accurate estimations of how well each variation performed.
Image from http://www.websiteoptimization.com/services/marketing/multivariate-testing/
Adaptive Multivariate Testing Method
The newer method of the three, the Adaptive Multivariate Testing Method is a new approach to multivariate testing which aims to adapt to visitor behavior in real time while simultaneously getting closer to the winning combination. In other words, it’s a test that continually learns and adjusts, as it will constantly configure itself to be the best performing page.
Image from http://www.websiteoptimization.com/services/marketing/multivariate-testing/
Although the method is new, it does sound very promising. The main advantage of this method is the fact that you need just 1% of the traffic you’d need for the Full Factorial in order to determine a winning combination. Although I’m not sure what the maths is being the algorithm, they claim to put your conversion rate on autopilot by monitoring page performance and adapting to new market dynamics on-the-fly.
I’m pretty curious about how well this last method works – If any of you have any experience using it, please do share below.
0 comments:
Post a Comment