A/B testing - Which option is better?

A/B testing - Which option is better?

In my last post I talked about A/B testing ecommerce video performance, and why we’ve opted to automate this process for clients using our platform. The truth is that the only way anyone can truly expect to improve their site’s performance is via a meticulously implemented optimization program, based on consistent execution of A/B testing, for all the variables one can tweak on one’s site.

A/B testing is extra work

The problem with A/B testing is that it requires you to make an extra effort – you need to have two versions (at least) for the site element (text, design, image, button,video, etc.) you want to optimize, and you need to have an effective framework for planning, implementing, and analyzing your test results.

Sadly this additional effort is enough to make the majority of ecommerce site operators decide to give up on the entire exercise.

Is it worth it? You tell us!

Since site owners & designers tend to be protective of their current efforts, and are often less than receptive to the idea they might be able to improve on them, we decided to setup an experiment to enable people to test their intuitions and compare them against verifiable data.
If your up for the challenge take a look at the following nearly identical Biotone body crème videos and try to estimate what effect, if at all, the voice-over had on their sales conversion rates:


Ecommerce Product Video by Treepodia - A/B Testing the effect of Voice Over on Conversion Rates - Version 1 - No Voice Over


Ecommerce Product Video by Treepodia - A/B Testing the effect of Voice Over on Conversion Rates - Version 2 - With Voice Over

So are you ready to make a guess?

If you guessed the voice-over increased conversion rates by 50% (from 3% to 4.5%) you’re spot on.

The problem is that when we conducted a survey, to see whether people’s intuitions were able to predict this result, we discovered the vast majority of those surveyed came nowhere close to guessing the right answer. This isn’t really very surprising, considering that answering the question with any type of credence would require one to be an ecommerce expert with a fair deal of experience in video marketing and the effects of voice overs on conversion rates – not exactly common experience…

Lets Try Again Shall We?

The following two videos are both for the same vendor – an online retailer that specializes in providing golfing goods online. The first video is a production grade video ordered from a professional studio. The second video was created automatically via Treepodia’s automated video solution. All you have to do is guess which video provided a higher conversion rate:


Ecommerce Product Video by Treepodia - A/B Testing - Production vs. Automated Video - Production Version


Ecommerce Product Video by Treepodia - A/B Testing - Production vs. Automated Video - Automated Version

Ready to make your guess?

If you guessed the automated video out-preformed the man-made one by nearly 300% you’re correct. You’re also unique. Not one of our survey respondents guessed the answer to this question correctly. This is precisely what makes A/B testing such a powerful tool:

A/B Testing Provides Answers You Can’t Get Any Other Way

Without A/B testing it’s simply impossible to isolate which elements of a page provide better performance and by how much. Occasionally the results of this type of testing will prove insights that are the exact opposite of what your own gut instinct tells you, but that still doesn’t change the fact that the result you got is based on cold hard data, and is PRECISE & ACTIONABLE.

Are You Doing the Best You Can?

If you’re not implementing A/B testing on your site now you’ll never figure out what exact elements of your site are providing you with the highest value in terms of CVR. You may be doing well for yourself but you have no idea what you might be missing.

We’re Doing the Best We Can For You

In order to guarantee that your Treepodia Smart Video platform is giving you the best bang for your buck we’ve integrated and automated A/B testing platform directly into your system. So, although you might not be aware of it, you can rest assured that your Treepodia Product Videos are constantly being tested for performance with top performers replacing less optimal videos constantly. This is one of the reasons our clients consistently report CVR increases of +35% and above since implementing our system on their ecommerce sites.

Posted by mike On July - 19 - 2010 Promoted Uncategorized

Subscribe here