Does A/B testing lead to crappy products?(andrewchenblog.com)

32 pointsby wmorein17 tahun yang lalu4 comments

Does A/B testing lead to crappy products?

http://andrewchenblog.com/2009/03/02/does-ab-testing-lead-to-crappy-products/

3 comments

joe_the_user17 tahun yang lalu

Are crappy products unprofitable?

He says Hollywood keeps making sequels even though they turn out to be bad. But does he show that Hollywood loses money making bad sequels?

IF bad inevitably translated to unprofitable, Hollywood would have a strong incentive to make good movies. If movie goers flock to movies that are just a mediocre mishmash of the themes they've seen before, then Hollywood has every reason to keep putting them out.

[deleted]17 tahun yang lalu

aaronblohowiak17 tahun yang lalu

From TFA: "Thus, my answer to the title question is that no, A/B testing does NOT lead to crappy products, but poor decision-making around data can absolutely lead to it"

A more accurate title would have been "Metrics don't replace strategy."

madh17 tahun yang lalu

I like the point. With the gobs of data available in web product development, I think there is sometimes too much focus on metrics. Metrics are absolutely critical but can be wrong. They are really just an input into a sound strategy.

suhail17 tahun yang lalu

Metrics aren't wrong, it's how you perceive the results of what you're testing. If you make too many assumptions, it's your fault.

There's too many variables involved for metrics to be the final solution towards success, part of building a startup is taking risks and your intelligence to come out on top.