A/B testing - Which option is better?

A/B testing - Which option is better?

In my last post I talked about A/B testing ecommerce video performance, and why we’ve opted to automate this process for clients using our platform. The truth is that the only way anyone can truly expect to improve their site’s performance is via a meticulously implemented optimization program, based on consistent execution of A/B testing, for all the variables one can tweak on one’s site.

A/B testing is extra work

The problem with A/B testing is that it requires you to make an extra effort – you need to have two versions (at least) for the site element (text, design, image, button,video, etc.) you want to optimize, and you need to have an effective framework for planning, implementing, and analyzing your test results.

Sadly this additional effort is enough to make the majority of ecommerce site operators decide to give up on the entire exercise.

Is it worth it? You tell us!

Since site owners & designers tend to be protective of their current efforts, and are often less than receptive to the idea they might be able to improve on them, we decided to setup an experiment to enable people to test their intuitions and compare them against verifiable data.
If your up for the challenge take a look at the following nearly identical Biotone body crème videos and try to estimate what effect, if at all, the voice-over had on their sales conversion rates:


Ecommerce Product Video by Treepodia - A/B Testing the effect of Voice Over on Conversion Rates - Version 1 - No Voice Over


Ecommerce Product Video by Treepodia - A/B Testing the effect of Voice Over on Conversion Rates - Version 2 - With Voice Over

So are you ready to make a guess?

If you guessed the voice-over increased conversion rates by 50% (from 3% to 4.5%) you’re spot on.

The problem is that when we conducted a survey, to see whether people’s intuitions were able to predict this result, we discovered the vast majority of those surveyed came nowhere close to guessing the right answer. This isn’t really very surprising, considering that answering the question with any type of credence would require one to be an ecommerce expert with a fair deal of experience in video marketing and the effects of voice overs on conversion rates – not exactly common experience…

Lets Try Again Shall We?

The following two videos are both for the same vendor – an online retailer that specializes in providing golfing goods online. The first video is a production grade video ordered from a professional studio. The second video was created automatically via Treepodia’s automated video solution. All you have to do is guess which video provided a higher conversion rate:


Ecommerce Product Video by Treepodia - A/B Testing - Production vs. Automated Video - Production Version


Ecommerce Product Video by Treepodia - A/B Testing - Production vs. Automated Video - Automated Version

Ready to make your guess?

If you guessed the automated video out-preformed the man-made one by nearly 300% you’re correct. You’re also unique. Not one of our survey respondents guessed the answer to this question correctly. This is precisely what makes A/B testing such a powerful tool:

A/B Testing Provides Answers You Can’t Get Any Other Way

Without A/B testing it’s simply impossible to isolate which elements of a page provide better performance and by how much. Occasionally the results of this type of testing will prove insights that are the exact opposite of what your own gut instinct tells you, but that still doesn’t change the fact that the result you got is based on cold hard data, and is PRECISE & ACTIONABLE.

Are You Doing the Best You Can?

If you’re not implementing A/B testing on your site now you’ll never figure out what exact elements of your site are providing you with the highest value in terms of CVR. You may be doing well for yourself but you have no idea what you might be missing.

We’re Doing the Best We Can For You

In order to guarantee that your Treepodia Smart Video platform is giving you the best bang for your buck we’ve integrated and automated A/B testing platform directly into your system. So, although you might not be aware of it, you can rest assured that your Treepodia Product Videos are constantly being tested for performance with top performers replacing less optimal videos constantly. This is one of the reasons our clients consistently report CVR increases of +35% and above since implementing our system on their ecommerce sites.

Share
Posted by mike On July - 19 - 2010 Promoted Uncategorized

Most Etailers Have Poor Decision Making Processes

It never ceases to amaze me how despite the extensive online availability of massively powerful tools for improving conversions and sales, the vast majority of marketers & e-tailers still rely, almost exclusively, on “gut-instinct” (their own, or that of others) as the only system for making the decisions that manage their business. In my humble opinion this may well be the main cause such a vast percentage of online businesses fail, despite the relatively low overhead cost that going online offers when compared to a brick-and-mortar business.

GOOD Technology Works ...without forcing YOU to understand it

Great Technology is Friendly - it works without forcing YOU to understand it

From my experiences only few online retailers bother to study the statistics of their website traffic closely, and even fewer do so effectively – i.e. know what indicators are the most important for their business and how their day-to-day operations influence them.

This is a particularly sad state of affairs because the ineptitude can’t be traced back to a root cause that makes sense, for example a desire to avoid expenditure on software or resources. Some of the most powerful tools available for gathering statistics and acting on them effectively are available online for FREE.

Are You Optimizing Yet?

One example of an extremely powerful tool, that far too few people are using, is Google’s “Website Optimizer” service, which allows etailers to test how various combinations of changes to their website content will affect conversions, in order to choose the most effective one. Website Optimizer lets etailers choose what parts of a page they’d like to test – headlines, images, design styles, etc., and then runs an online experiment directing a portion of the site’s traffic to each variation in order to determine which content users respond to best.

Why Not?

Over the years I’ve studied this failing rather extensively as it represents what to me is a very intriguing paradox:
On the one hand the very fact a person succeeds at opening a business would seem to indicate a certain level of competence and diligence, however on the other hand that very same person is often unwilling or unable to carry that effort through by providing it with all the opportunities it requires for success. This despite the fact that this person often acknowledges freely their awareness that their ineffectiveness means they’re “Leaving money on the table” and losing their business as a result. The most obvious question to ask would be WHY???

Why Not Indeed…

After spending a fair deal of time pondering this I’ve come to the conclusion that the fault is NOT exclusively that of the entrepreneurs, a degree of blame must be laid at the feet of the developers and engineers that provide these incredible marketing tools.

Most Powerful Online Marketing Tools Are Too Complicated To Use

A well known paradigm of technology development is that all too often engineers develop products without consulting enough, or even at all, with their target audience. At the end of the process they end up with a product that is intuitive and easy enough for THEM to use, but NOT so for anybody else…

I think this may well be the case with many of the marketing tools available online today. Although they’re powerful and effective, and at times even free, they’re engineer-friendly design makes them simply unusable by the vast majority of the population. The fact someone is a great marketer, a driven sales person, and an industrious entrepreneur, is no guarantee that one has the skill sets required to operate a bit of software designed with an engineering audience in mind. Some might say that the exact opposite might be true (think why so few engineers become effective business owners…).

What could be done otherwise

One of the reasons so many people love Apple’s products, despite their technological limitations, poorer features and complete reliance on Apple’s closed marketplace, is because these products don’t require you to learn how to make them work. They simply work for you.

Apple’s products are designed around the core understanding that:

People don’t care about technology. They care about what technology can DO FOR THEM.

Why We’ve Opted For Automation

This truth can be extended just as easily to the world of ecommerce – At the end of the day when presented with the question:

“Do YOU want to increase your revenues by becoming an expert at operating our software, or do you want US to increase your revenues for you?”

9 business owners out of 10 will prefer to simply have their revenues increased (& the 10th is likely to be a “Power User” type with a strong affinity for technology…). This is precisely why we at Treepodia opted to automate as much of our service as possible, and are constantly pursuing new methods for increasing this automation.

We believe wholeheartedly that we serve you better by not burdening you with technobabble you don’t really care about ,and focusing you on what you do care about instead:

Your business results and bottom line revenue.

Now you tell me:
Assuming we GUARANTEE your ROI,
don’t you actually prefer it that way?…

Image credit: http://www.flickr.com/photos/firepile/438134633

Share
Posted by mike On July - 13 - 2010 Methodology Promoted
French Shopping

French Shopping (Image Credit: http://www.flickr.com/photos/shibuya246)

Although France may have been trounced in the world cup not all the news coming from the land of baguettes and liberty is bad. It turns out that in terms of e-commerce revenues Q1 2010 wasn’t only great for our clients, as we discussed in our last post, it was excellent for online retailers in general, and nowhere more so than in France.

As a recent article in eMarketer quotes, according to FEVAD (Fédération du E-commerce et de la Vente à Distance), Q1 2010 saw a 30% increase in ecommerce sales over Q1 2009, valued at €7.2 billion (or $10 billion).

B2C E-commerce sales growth in France

What is powering this growth?

Much of this overall increase is attributed to the addition of some 16,000 online retail sites. But the growth is more than a matter of more stores, it also applies to overall basket value which was up to a healthy to €93 (or $129).

In addition, another factor is the fact that buying online is becoming more common in France. While the average consumer made 11 online purchases in 2009, already in Q1 2010 the average consumer has made four purchases online (at that rate online purchases would average 16 per year throughout 2010, representing an almost 50% increase over the previous year).

In other words, the French market is experiencing a positive overall trend with the combined factors of more online stores, more purchases made online at a higher value.

French growth set to exceed FEVAD expectations

While growth in the French market was anticipated this year (FEVAD predicted in the 24% range), if the current trend continues the numbers could far surpass what was expected.

While the largest online retailers (eBay, PriceMinister and Le Redoute) still garner the lion’s share of the pie, the overall growth trend is positive news for French online retailers across the board.

This positive trend also means there’s lots of room for ecommerce sites to expand and resume investing in the future of their businesses by updating and modernizing their sites to meet the developing demands of consumers.

Share
Posted by mike On July - 1 - 2010 Promoted Trends

Subscribe here