In October 2015 VWO is launching its new SmartStats which runs on Bayesian statistics. This will provide better understanding on how long to run a test for conversion rate optimization. They do not get into the algorithm but discuss the benefits of being able to view the data in three perspectives (below).
In the end I do not think this will affect testing to much but will allow those who do not understand statistics and the phases of an A-B test to read the data clearer and make the right decisions. Too often I see decisions made too quickly in a test and this will help in preventing inaccurate conclusions.
Here are the three modes VWO is providing
For finding quick trends where tests don’t affect your revenue directly
You can choose this mode when testing non-revenue goals such as bounce rate and time spent on a page, or for quick headline tests. With this mode you can reduce your testing time for non-critical tests where there isn’t a risk of hurting your revenue directly by deploying a false winner.
Ideal for most tests.
This is the default mode and can be used for almost all tests. As the name suggests it is the best balance between testing time and minimizing potential loss.
Best for revenue-critical tests when you want to absolutely minimize potential loss. Usually takes the longest to conclude a test.
Suppose you have an e-Commerce website and you want to test changes to your checkout flow. You want to be as certain as possible to minimize potential loss from deploying a false winner even if it takes a lot of time. This is the best mode for such critical tests which affect your revenue directly.
Latest posts by Greg Ahern (see all)
- Which Test Won? Request vs Schedule vs Get a Demo - November 15, 2017
- Which Test Won? Paragraph vs Bullets - October 31, 2017
- Which Test Won? Submit Registration vs Get Started vs Watch Videos - October 19, 2017