A/B Testing is how sites like Amazon, Yahoo, Google, Facebook and others became the best in the world. They used data to drive decisions about content, layout, wording and other aspects of their sites.
Smashing magazine covers the details of A/B testing in this post.
At its core, A/B testing is exactly what it sounds like: you have two versions of an element (A and B) and a metric that defines success. To determine which version is better, you subject both versions to experimentation simultaneously. In the end, you measure which version was more successful and select that version for real-world use.
37signals tested the headline on its pricing page. It found that “30-Day Free Trial on All Accounts” generated 30% more sign-ups than the original “Start a Highrise Account.”
Dustin found that “You should follow me on Twitter here” worked 173% better than his control text, “I’m on Twitter.”
Along with its other A/B tests, CareLogger increased its conversion rate by 34% simply by changing the color of the sign-up button from green to red!
The article covers the results from others of testing, as well as details on how to get started for your own site.
What I find interesting is that the tools that used to require significant investment in time and development are increasingly available as easy to use, free solutions.