{"id":766,"date":"2017-04-20T12:46:51","date_gmt":"2017-04-20T18:46:51","guid":{"rendered":"https:\/\/www.ometrics.com\/blog\/?p=766"},"modified":"2025-01-17T16:14:57","modified_gmt":"2025-01-17T23:14:57","slug":"2017-ab-testing-tools-review","status":"publish","type":"post","link":"https:\/\/www.ometrics.com\/blog\/ab-testing-tools-review\/","title":{"rendered":"2025 A\/B Testing Tools Review for Conversion Rate Optimization Experts"},"content":{"rendered":"\n[et_pb_section fb_built=”1″ _builder_version=”4.17.4″ _module_preset=”default” global_colors_info=”{}”][et_pb_row column_structure=”3_5,2_5″ _builder_version=”4.17.4″ _module_preset=”default” global_colors_info=”{}”][et_pb_column type=”3_5″ _builder_version=”4.17.4″ _module_preset=”default” global_colors_info=”{}”][et_pb_text _builder_version=”4.17.4″ _module_preset=”default” global_colors_info=”{}”]

There are many A\/B testing tools on the market right now. If you are planning to run an A\/B test on your website or landing page, it\u2019s often hard to tell which tool will work best for your business.<\/span><\/p>\n

As experts in conversion rate optimization (CRO), we have used many of the A\/B testing tools listed in this review. However, we have also included newer tools that we have not tested. This list is merely a starting point for your own investigations.\u00a0<\/span><\/p>[\/et_pb_text][\/et_pb_column][et_pb_column type=”2_5″ _builder_version=”4.17.4″ _module_preset=”default” global_colors_info=”{}”][et_pb_image src=”https:\/\/www.ometrics.com\/blog\/wp-content\/uploads\/2023\/02\/AB-Testing.jpg” alt=”A\/B Testing” title_text=”AB Testing” _builder_version=”4.17.4″ _module_preset=”default” global_colors_info=”{}”][\/et_pb_image][\/et_pb_column][\/et_pb_row][et_pb_row _builder_version=”4.17.4″ _module_preset=”default” global_colors_info=”{}”][et_pb_column type=”4_4″ _builder_version=”4.17.4″ _module_preset=”default” global_colors_info=”{}”][et_pb_text _builder_version=”4.17.4″ _module_preset=”default” hover_enabled=”0″ global_colors_info=”{}” sticky_enabled=”0″]

1.<\/strong> Optimizely<\/span><\/a> is an enterprise-level experimentation platform. It is the most well-known platform in this space due to a lot of VC funding and advertising. At $1,440 per month (paid annually) it is one of the most expensive of A\/B testing tools. They also offer a pay-as-you-go pricing model that costs $49 per 1000 unique monthly visitors. The platform supports testing on the web, mobile web, and mobile apps.<\/span>
<\/span>\u00a0<\/span><\/p>\n

2.<\/strong> VWO<\/span><\/a> (Visual Website Optimizer) [Updated 2-14-23]<\/strong> With the sunsetting of Google Optimize VWO has released a free plan to attract the Optimize users. They have also reversed their earlier decision to <\/span>show pricing on the site<\/span><\/a>. Since 2019 VWO has improved the features across the platform. The code is significantly less buggy than it was when it first launched.\u00a0 <\/span><\/p>\n

[Updated 1-31-19]<\/strong> I just finished a meeting with VWO where they provided a sneak peek of VWO\u2019<\/a>s new interface. It has some new features that make it very easy to see with visual graphs if a test is stable. They also removed some of the confusion when comparing variations and how they are doing against control. The data is presented so a novice can understand it and an expert can dig into the statistics. The data and graphs will also be exportable to a PDF to make reporting easier for clients. The new version should be out in the second quarter of this year. I will update this post when they have a firm date. Regarding my next comment\u2026 they are reinvesting in the tool to justify the cost and this will be going beyond just the presentation layer of the system.<\/p>\n

[Updated 1-17-19]<\/strong> VWO has completely changed its pricing model to follow Optimizely. There are now only annual subscriptions and the price has doubled. VWO offers other CRO tools like heatmaps and videos. VWO works across mobile, tablet, and desktop websites.<\/span><\/p>\n

3.<\/strong> Freshmarketer<\/span><\/a> has acquired Zarget. [Updated 1-17-19] The pricing is very good and the support is excellent. 50k users under test for $159 a month and no annual commitment. They also have a number of other conversion tools similar to VWO. The testing system presents nice reports and does conversion rate over time which I feel is a key visual indicator of test stability when comparing variations. Setting up some tests is a bit different but it is clear they are building better functionality. The acquisition included all the developers so they did not lose any talent<\/span>
<\/span><\/p>\n

4.<\/strong> Convert<\/span><\/a> has received high praise from its users, who tend to be SMB customers. It was the first solid A\/B testing tool on the market before Optimizely. Their focus is on being the best in the market for website A\/B testing.\u00a0 [Updated 1-17-19] Price starts at $699\/month or $599 a month (paid annually).\u00a0 It has an intuitive interface and excellent customer support<\/span>
<\/span><\/p>\n

5.<\/strong> Crazyegg<\/span><\/a> known for its excellent heatmap tools has added A\/B testing. Pricing starts at $29 a month for 30k visitors. Note this is not users under test, so it is harder to compare to other testing pricing plans and will have to throttle the traffic on the pages for larger sites. That said I would love some feedback on their A\/B tool.<\/span><\/p>\n

6.<\/strong> Google Optimize<\/span><\/a> [updated 2-14-23]<\/strong> Google has decided to <\/span>sunset its Optimize product<\/span><\/a> in September of 2023. You can learn more about it <\/span>here<\/b><\/a>. <\/span>Firebase<\/span><\/a> testing for apps will still use the code from Optimize and is not sunsetting at this time.\u00a0 Optimize was an A\/B testing tool that connects to Google Analytics. Since you already have all your data in your Google Analytics account. You can\u2019t beat the cost and convenience of this tool since it is free. It allows you to split test up to 10 full versions of a single page, each delivered from a distinct URL. However, goal setting is very cumbersome relying on Google Analytic goals and segments, and in some cases would take Tag Manager to set up goals. Hopefully, in the future, they will build features that take seconds in other tools instead of 10 to 30 minutes.\u00a0<\/span><\/p>\n

[Update 5-7-17] <\/strong>I asked Google<\/span><\/a> about the goal issue and they said\u00a0 that within a year you will be able to set goals on elements and that next month they will be adding Adwords to Optimize.]<\/span><\/p>\n

[Update 1-17-19]<\/strong>\u00a0 Google launched new features today. I have not played with them yet but if they can set goals quickly without google analytics then they are moving in the right direction.\u00a0<\/span><\/p>\n

[1-31-19]<\/strong> The new goals called Custom Objectives get around the biggest issue of only using the Google Analytics goals which were limited. They still don\u2019t have one-click goal setting but it is getting there. So the time it takes to set all of these up compared to clicking a button and calling it a goal can be significant. What is nice is it is easy to set purchase and revenue goals if you have enhanced eCommerce already set up. This can be difficult in other systems<\/span>
<\/span><\/p>\n

7.<\/strong> Omniconvert<\/span><\/a> provides an A\/B testing tool for tablets, mobile, and desktops. Some of the unique features are ready-to-use overlays, multiple overlay formats, OnPage variables, frequency capping, and IP address targeting. Pricing starts at $38\/month (paid annually).<\/span><\/p>\n

8.<\/strong> AB Tasty<\/span><\/a> [Update 2-14-23] Google had listed AB Tasty as one of the tools it recommends once Optimizley sunsets. Pricing is not shown on the site anymore.\u00a0 AB Tasty is a tool that tests user re-engagement and it is designed for marketing teams. You can modify pages using a visual editor, so you don\u2019t need a technical background to use it. This is a good tool to start A\/B testing if you\u2019ve never done it before and you don\u2019t want to invest a lot of money. You must schedule a consultation with one of their experts as pricing is not available on their site<\/span>
<\/span><\/p>\n

9.<\/strong> Intellimize<\/span><\/a> helps marketers drive conversions and revenue growth by using artificial intelligence to personalize websites to deliver the right experience to each visitor on any device. \u00a0 Since this is a newer approach, I reached out to them for an interview so that they can clarify how they do what they do.<\/span><\/span><\/p>\n

 <\/p>\n

\n

Pricing?<\/b> \u2013 We are a SaaS-based pricing model with an annual term and monthly pricing.\u00a0 Cost varies considerably based on the needs of the client and we don\u2019t publish pricing as our work for clients.\u00a0 We work mostly with middle-market companies. As you noted, it\u2019s common for companies to run their entire site through our system as we can optimize flows for them.<\/span>
<\/span>
<\/span>Does this affect SEO?<\/b> \u2013 Great question. Google has established guidelines for companies like us to follow.<\/span>
<\/span>1. Google asks personalization companies like ours to serve a random experience to them. We do not \u201clisten\u201d for Google.<\/span>
<\/span>2. For keywords that are important to you, we ensure that variations of your page contain those keywords and metadata to ensure that your page will keep ranking.<\/span>
<\/span>3. It\u2019s important to remember we are changing elements of the page (headlines, CTAs, images, etc.) rather than setting up new pages so your existing work will remain intact.<\/span><\/p>\n


<\/span>How do you measure\/define the audiences for each variation? Is it from data points in the site or data found outside of the site or both?<\/b><\/p>\n


<\/b>Our system does not require any data integration to get started and can accept data from 1st and 3rd party sources.\u00a0 The most commonly used data by our system would be contextual data because it\u2019s readily available for each visitor.\u00a0 This includes features like: IP address, device type, browser, time of day, day of week, and geographic location (city level). We also commonly use firmographic and demographic data as well as behavioral data about how the user interacts on the site.<\/span><\/p>\n


<\/span>Finally, our approach to A\/B testing is rooted in the principle that A is probably best for some people and B is probably best for others.\u00a0 Instead of finding a global winner (the best average), our AI-based system learns which version of your site is best for each visitor and automatically allocates more traffic to your best-performing variations.<\/span><\/p>\n<\/blockquote>[\/et_pb_text][\/et_pb_column][\/et_pb_row][et_pb_row column_structure=”1_2,1_2″ _builder_version=”4.17.4″ _module_preset=”default” link_option_url=”https:\/\/www.ometrics.com\/ochatbot-for-enterprise\/” border_radii=”on|3px|3px|3px|3px” border_width_all=”3px” border_color_all=”#68AB37″ global_module=”6437″ saved_tabs=”all” global_colors_info=”{}”][et_pb_column type=”1_2″ _builder_version=”4.16″ _module_preset=”default” global_colors_info=”{}”][et_pb_image src=”https:\/\/www.ometrics.com\/blog\/wp-content\/uploads\/2022\/08\/Ochatbot_ecomm-cart-420×420-1-300×300.png” alt=”shopify plugins” title_text=”Ochatbot_ecomm-cart 420×420″ align=”center” _builder_version=”4.17.4″ _module_preset=”default” global_colors_info=”{}”][\/et_pb_image][\/et_pb_column][et_pb_column type=”1_2″ _builder_version=”4.16″ _module_preset=”default” global_colors_info=”{}”][et_pb_text _builder_version=”4.17.4″ _module_preset=”default” custom_margin=”|3px||3px|false|false” border_width_all=”0px” global_colors_info=”{}”]

True AI to engage customers for eCommerce, business leads, and customer support.<\/b><\/h4>\n
<\/div>\n