Ask HN: How do you use A/B testing?
1 comments
I did ad hoc A/B testing for a while, switch to Google Website Optimizer, and am going to get around to releasing a homegrown Rails A/B framework one of these months...
You can A/B test the heck out of everything, but the best ROI on time for me has been taking key interactions on the funnel and testing changes to them. For me, the big ones are viewer of webpage to download of software, and view purchasing page to completing sale.
Some of the changes are obviously going to have significant results (redoing your shopping cart), some might be subtle (tweaking a button's color).
In general, I don't bother A/B testing things outside of the funnels. I'm sure that given infinite time I could rewrite my copy on tertiary web pages to decrease my bounce rate by a few points, but that does not move the needle for my business. Rewriting my shopping cart did.
You can A/B test the heck out of everything, but the best ROI on time for me has been taking key interactions on the funnel and testing changes to them. For me, the big ones are viewer of webpage to download of software, and view purchasing page to completing sale.
Some of the changes are obviously going to have significant results (redoing your shopping cart), some might be subtle (tweaking a button's color).
In general, I don't bother A/B testing things outside of the funnels. I'm sure that given infinite time I could rewrite my copy on tertiary web pages to decrease my bounce rate by a few points, but that does not move the needle for my business. Rewriting my shopping cart did.
thanks, that's helpful... yes we're thinking of exactly those two points...registration funnel and purchase funnel to begin with.
I'm curious what other startups are already using A/B testing for and where it helps most? (and not)