Stop guessing. Use A/B testing to determine ideal price for your product(visualwebsiteoptimizer.com)
visualwebsiteoptimizer.com
Stop guessing. Use A/B testing to determine ideal price for your product
http://visualwebsiteoptimizer.com/split-testing-blog/ab-testing-price-testing/
14 comments
In 2000, Amazon got caught doing price testing and the news was covered on all major blogs. Here is a link: http://news.cnet.com/2100-1017-245631.html
What's your source on a/b testing price on identical products being illegal?
I've never heard that and none of the links provided in the comments so far justify your claim. It is a pretty extreme assertion and you provided no links justifying it either.
I can see this rapidly turning into an urban myth, whispered wild eyed at 3am over ramen noodles as founders try and figure out a price for their product.
I've never heard that and none of the links provided in the comments so far justify your claim. It is a pretty extreme assertion and you provided no links justifying it either.
I can see this rapidly turning into an urban myth, whispered wild eyed at 3am over ramen noodles as founders try and figure out a price for their product.
The concept is called "price discrimination". There is a great discussion going in this thread and I may be wrong at this. Even if it may not be illegal, it would leave a bad taste in your potential customers' mouth especially since there are better alternatives.
> The concept is called "price discrimination".
Not quite - price discrimination is finding a way to take people with more money, and charge them more money. The challenge is, that prices people with less money out of the market - so price discrimination is about finding ways to justify cutting prices for some people while charging more to people who can afford more.
Examples: Student discounts, senior citizen discounts, kid's meals, and matinee prices at movie theaters are all price discrimation. A/B testing prices is... just A/B testing prices.
Though, the wise A/B price-tester does so by offering two different sales - one discounted product, and one more discounted product. This seems to minimize backlash - finding someone else got a better discount doesn't bother people as much as finding out they paid a higher price.
Not quite - price discrimination is finding a way to take people with more money, and charge them more money. The challenge is, that prices people with less money out of the market - so price discrimination is about finding ways to justify cutting prices for some people while charging more to people who can afford more.
Examples: Student discounts, senior citizen discounts, kid's meals, and matinee prices at movie theaters are all price discrimation. A/B testing prices is... just A/B testing prices.
Though, the wise A/B price-tester does so by offering two different sales - one discounted product, and one more discounted product. This seems to minimize backlash - finding someone else got a better discount doesn't bother people as much as finding out they paid a higher price.
Price discrimination doesn't have to be on just cutting prices, although it is often used this way to increase your potential pool of customers. I believe Amazon was using their knowledge of you purchase history, to decide if you were more likely to buy a product. If you were, the price would be higher.
I don't believe it's legal to charge different prices to different people for exactly the same product. Changing the price (for everyone) is fine (you could even change the price 5 times a day), or adding a back scratcher and bumping the price by $500 is fine, but doing an AB test where you flip the price back and forth on the same thing isn't legal as far as I know. IANAL.
Don't car dealerships do this all the time? Or small shops in Chinatown that negotiate for tourist souvenirs? Safeway or CVS cards? Google AdWords? Priceline.com? Employee/corporate discounts? Heck, on my recent trip to the Grand Canyon, my sister and I both checked out the exact same rental car through our respective employers, and it was about $30 cheaper through mine, presumably because my employer gives a better corporate discount than her employer. Really, anything where you negotiate the price will end up with different prices for the same item.
You're thinking that just because people don't do it, it must be illegal. The idea of "one price for identical items" is a pretty recent invention, dating back to the rise of department stores (Woolworth's, Macy's, Sears) at the turn of the 20th century. It's for economic reasons, not legal ones. They found that it was cheaper to forego the extra revenue than it was to hire all the salespeople needed to negotiate each individual purchase. Charge less and make it up in volume. It also has the advantage of fairness and simplicity, which customers tend to like.
You're thinking that just because people don't do it, it must be illegal. The idea of "one price for identical items" is a pretty recent invention, dating back to the rise of department stores (Woolworth's, Macy's, Sears) at the turn of the 20th century. It's for economic reasons, not legal ones. They found that it was cheaper to forego the extra revenue than it was to hire all the salespeople needed to negotiate each individual purchase. Charge less and make it up in volume. It also has the advantage of fairness and simplicity, which customers tend to like.
How would that be so? How do you 'believe' a factual thing like this? I can charge one guy 10$ for a pie in my bakery shop, and the next guy 100$ because I don't like his face. There is a narrow set of classes on which it is not allowed to discriminate but all else is fair game.
I'm not claiming to know this of all jurisdictions, there may be specific circumstances in which this does not hold, but these are the exceptions.
I'm not claiming to know this of all jurisdictions, there may be specific circumstances in which this does not hold, but these are the exceptions.
I don't know if the other comment was edited, but he doesn't say anything about 'illegal'. People certainly found it distasteful though.
I think he wasn't referring to my comment but to my post where I write: "It’s illegal and can lead to huge potential lawsuit."
Oops, indeed. Hadn't seen who submitted the article.
I think he's right though that there needs to be some linkage to "what you can and can't do, legally".
I think he's right though that there needs to be some linkage to "what you can and can't do, legally".
Depends what country your in really. Always check your local laws and regulations before taking advice on the internet; the laws for someone may not be the same as yours.
[deleted]
"Pricing for physical goods is simple. For example, if you are manufacturing staplers, all you need to do is to calculate cost of production and distribution, slam 20% margin on it and there you have the price you can sell your shiny stapler machines for."
Not true. In fact, a lot of Maker businesses get hung up here. The pricing of physical goods depends on the market every bit as much as the pricing of software or web services. In fact, large companies rely on that all the time--case in point is P&Gs Swiffer Sweeper, that they charge much more than the manufacturing cost for the sweeper and for the refills.
Many small business owners I've run across actually feel guilty if they charge "too much" (say 100% markup) for their physical goods they are selling, even though their competitors sell at that price. The price elasticity of demand applies just as much for physical goods as it does for non-physical.
Not true. In fact, a lot of Maker businesses get hung up here. The pricing of physical goods depends on the market every bit as much as the pricing of software or web services. In fact, large companies rely on that all the time--case in point is P&Gs Swiffer Sweeper, that they charge much more than the manufacturing cost for the sweeper and for the refills.
Many small business owners I've run across actually feel guilty if they charge "too much" (say 100% markup) for their physical goods they are selling, even though their competitors sell at that price. The price elasticity of demand applies just as much for physical goods as it does for non-physical.
To an extent I agree with the sentiment of your post, but the practicalities are different imo, and "The pricing of physical goods depends on the market every bit as much as the pricing of software or web services." I don't agree with at all. Many non-software products and services (staples, pies, chimney sweeping, plumbing, ...) are much more commodities than software. Efficient markets, lemon problem etc. drive down the markup to a low (single to low-double digit) percentage. Since cost is also much less variable (costs in terms of resources, labor, capital investments etc. for a pie from one baker is going to be very similar to those of another down the street) the end-price is going to be not very flexible; and any eventual elasticity will just drive down consumption rather than price. The degree of availability of substitution options is a major influence. That's why even major supermarket chains operate with low single-digit profit margins, even on a product (food) where there is obviously a constant and high demand.
Compare that to software where no two offerings are direct substitutions (the degree varies, of course) and where one company can upset a market with new technology very easily without his competitors necessarily being able to duplicate (for skills reasons, legacy investments, ...). An incumbent can enter the market with fixed costs that are 75 or 50 or 25 percent of the established players. Such an incumbent can do two things: go deep on price to get market share, or get fat off 500% profit margins. I don't know of many such situations in physical goods businesses.
(of course the same could happen with physical goods, it's just much more common in technology)
Compare that to software where no two offerings are direct substitutions (the degree varies, of course) and where one company can upset a market with new technology very easily without his competitors necessarily being able to duplicate (for skills reasons, legacy investments, ...). An incumbent can enter the market with fixed costs that are 75 or 50 or 25 percent of the established players. Such an incumbent can do two things: go deep on price to get market share, or get fat off 500% profit margins. I don't know of many such situations in physical goods businesses.
(of course the same could happen with physical goods, it's just much more common in technology)
This article was overall a very shallow economic analysis, they should have gone into marginal costs and price differentiation for this problem. Maximising income from customers by charging more to those willing to pay more is a very well studied problem, and solutions to it are observable in every shop on the high street.
Has anybody tried using hill climbing to arrive at an optimal price?
eg:
1. Pick a pivot price
2. Make N buckets with prices at fixed intervals from the pivot
3. Find the bucket with the most profit, and use its price as the new pivot
4. Repeat
eg:
1. Pick a pivot price
2. Make N buckets with prices at fixed intervals from the pivot
3. Find the bucket with the most profit, and use its price as the new pivot
4. Repeat
Fascinating concept. Makes me want to start selling something in large volumes, just so I could apply other optimization techniques like simulated annealing or beam search to price points and other scalar variables.
A major complication that you do not address is how to do confidence testing on the results of your pricing difference. There is an important difference between "A did better than B but who knows why" and "We're confident that there is a real difference." Figuring that out is not easy.
Yes, that's right I should have touched that point. I think the revenue for each variation can be approximated as normal curve and then a hypothesis test can be done to see if difference in mean is statistically significant.
That approach absolutely works. But fairly few people have enough of a math background to figure out how to do it.
There are also complications around noise from large outliers. Though this matters less with a fixed price service.
There are also complications around noise from large outliers. Though this matters less with a fixed price service.
Keep in mind that this is secondary to finding the market that puts the greatest value on your product.
Interesting. Can you clarify a bit more? An example would really help. Thanks.
Stormpulse has recently shifted from B2C to B2B. Pricing is so completely different there's no A/B test you could use to determine that.
Say you were selling tickets to hear Lang Lang play the piano. You could try selling those on the corner of 3rd and Main and get $20 for them because no one you talk to (unless you get lucky) has heard of him (and you could waste a lot of time convincing them that he's great). Or you could move your booth to the more fashionable part of town and sell them in 5 minutes for $250 each.
Say you were selling tickets to hear Lang Lang play the piano. You could try selling those on the corner of 3rd and Main and get $20 for them because no one you talk to (unless you get lucky) has heard of him (and you could waste a lot of time convincing them that he's great). Or you could move your booth to the more fashionable part of town and sell them in 5 minutes for $250 each.
This is why I think the golden goose of web optimization tools is going to be the one that automagically segments a user base into population/profiles based on their repeated behaviours and their origin IP (consumer/biz) (provide a klout.com type of analysis on everyone that visits/registers). It will allow startups to see their potential market segments and follow the most promising ones.
Oooh there's this company that starts with an F and ends with a k. Ah but the valuation's too high. Fuck!
Pro tip: Have at least three pricing levels with directly comparable features. It will increase the number of sales for the intermediary level.
I've heard that one before and I've seen sites that do it but I have yet to find a place where there is actual evidence that it works, do you have a reference for that?
We're debating trying this and I'd like a bit more evidence that it really works and why it does.
We're debating trying this and I'd like a bit more evidence that it really works and why it does.
Here are a few citations, not all directly relevant, from Dan Ariely's _Predictably Irrational_ (itself a good read for any startup founder):
___________________________________
1. Amos Tversky, "Features of Similarity," Psychological Review, Vol. 84 (1977).
2. Amos Tversky and Daniel Kahneman, "The Framing of Decisions and the Psychology of Choice," Science (1981).
3. Joel Huber, John Payne, and Chris Puto, "Adding Asymetrically Dominated Alternatives: Violations of Regularity and the Similarity Hypothesis," Journal of Consumer Research (1982).
4. Itamar Simonson, "Choice Based on Reasons: The Case of Attraction and Compromise Effects," Journal of Consumer Research (1993).
5. Amos Tversky and Itamar Simonson, "Context-Dependent Preferences," Management Science (1993).
6. Dan Ariely and Tom Wallsten, "Seeking Subjective Dominance in Multidimensional Space: An Explanation of the Asymmetric Dominance Effect," Organizational Behavior and Human Decision Processes (1995).
7. Constantine Sedikides, Dan Ariely, and Nils Olsen, "Contextual and Procedural Determinants of Partner Selection: On Asymmetric Dominance and Prominence," Social Cognition (1999).
___________________________________
1. Amos Tversky, "Features of Similarity," Psychological Review, Vol. 84 (1977).
2. Amos Tversky and Daniel Kahneman, "The Framing of Decisions and the Psychology of Choice," Science (1981).
3. Joel Huber, John Payne, and Chris Puto, "Adding Asymetrically Dominated Alternatives: Violations of Regularity and the Similarity Hypothesis," Journal of Consumer Research (1982).
4. Itamar Simonson, "Choice Based on Reasons: The Case of Attraction and Compromise Effects," Journal of Consumer Research (1993).
5. Amos Tversky and Itamar Simonson, "Context-Dependent Preferences," Management Science (1993).
6. Dan Ariely and Tom Wallsten, "Seeking Subjective Dominance in Multidimensional Space: An Explanation of the Asymmetric Dominance Effect," Organizational Behavior and Human Decision Processes (1995).
7. Constantine Sedikides, Dan Ariely, and Nils Olsen, "Contextual and Procedural Determinants of Partner Selection: On Asymmetric Dominance and Prominence," Social Cognition (1999).
Robert Cialdini's work also has some examples - http://www.influenceatwork.com/
Super! Thank you very much!
The problem with doing this is that if your customers find out they're likely to be pissed off. If this doesn't matter to you then feel free.
I don't like the first paragraph.
"For example, if you are manufacturing staplers, all you need to do is to calculate cost of production and distribution, slam 20% margin on it and there you have the price you can sell your shiny stapler machines for."
That's called Cost Based Pricing and is one of many options. That's a decision that was made. You can argue that's the wrong way to price. If you make something in the US and charge 20% on top of costs and have a competitor making Staplers in China, you're out of business.
Boom. Roasted.
"For example, if you are manufacturing staplers, all you need to do is to calculate cost of production and distribution, slam 20% margin on it and there you have the price you can sell your shiny stapler machines for."
That's called Cost Based Pricing and is one of many options. That's a decision that was made. You can argue that's the wrong way to price. If you make something in the US and charge 20% on top of costs and have a competitor making Staplers in China, you're out of business.
Boom. Roasted.
If 99% of your value is in the base product and 1% is in the advanced features, most people will chooses the cheapest plan available.
My customers don't care about ssl or extra storage so they'll always choose the cheaper plan. But if the basic plan was +$10 what percentage would still buy it?
My customers don't care about ssl or extra storage so they'll always choose the cheaper plan. But if the basic plan was +$10 what percentage would still buy it?
He is not saying you should offer a second basic plan next to the primary basic plan. He is saying that you should AB-test two different basic plans, only one of them is visible for any given customer.
Why is it illegal to offer different prices for the same service (as this article says)?
http://definitions.uslegal.com/p/price-discrimination/
this doesn't address this case directly, but it seems to say it's only illegal if done to harm your competitors.
this doesn't address this case directly, but it seems to say it's only illegal if done to harm your competitors.
It is not illegal to offer different prices. Case in point - a lowly 12 oz of Budweiser. If I buy it in a keg it costs me around 50 cents (USD)each. In the bottle it would be closer to $1. At a local bar that has good beer prices and good food it is $3.00 for draft normally, except on Thursdays it is $2 (Bisonwitches, Tucson AZ). If I go to a fancy restaurant it will likely be $4 or even as high as $6.
In a free market you are free to charge what you want. Consumers are free not to purchase your item. Rules do vary around the world, so check with your local authorities. I am looking at this from a North American perspective.
In a free market you are free to charge what you want. Consumers are free not to purchase your item. Rules do vary around the world, so check with your local authorities. I am looking at this from a North American perspective.
I think you're wrong. 12 oz of bud in a can is a different product than 12 oz in a bottle. 12 oz by the can is different than by the case. But if you line it all up and make everything else equal, and have a line of customers in front of your till, it's illegal to charge different prices to those customers in line.
Ah so it's OK to charge people you don't like (wrong skin color say) more because it's a free market?
There's a difference between giving bulk discounts and selling the same product for different prices to different customers.
There's a difference between giving bulk discounts and selling the same product for different prices to different customers.
That's not price discrimination, that's discrimination, period, this article has nothing to do with that kind of discrimination so I don't understand why you're pulling it in.
And that is illegal anyway, as well it should be.
Selling the same product at different price points is done all the time (bulk discounts, account discounts and so on).
The issue here is not whether it is legal to sell the same product at different prices, the issue is whether or not it is legal to offer the same product to the same class of buyers at different prices at the same time.
And that is legal, even if not always in the best interest of the business:
http://en.wikipedia.org/wiki/Price_discrimination
I've done it, briefly, to test different price points and I made sure to refund those we tested at a higher level after the experiment was over.
And that is illegal anyway, as well it should be.
Selling the same product at different price points is done all the time (bulk discounts, account discounts and so on).
The issue here is not whether it is legal to sell the same product at different prices, the issue is whether or not it is legal to offer the same product to the same class of buyers at different prices at the same time.
And that is legal, even if not always in the best interest of the business:
http://en.wikipedia.org/wiki/Price_discrimination
I've done it, briefly, to test different price points and I made sure to refund those we tested at a higher level after the experiment was over.
No, it is not legal to charge people a different price if their skin is purple instead of blue. In the OP example, he is talking about web delivered services where it would be difficult to determine someones skin color anyway. In my example the price of budweiser varies by location and circumstance, not by the color of the purchasers skin.
Read this: "PRICE DISCRIMINATION AND THE LAW" http://www.imakenews.com/strategicpricing/e_article000649775...
Interesting. That applies to USA. Is there any EU law on the matter?
The article you linked specifies that the statute in question only applies to goods, not services. Is that applicable here?
I am not a lawyer but I think it should apply to digital products. Perhaps not consultancy services, if you are offering a product, it should apply.
To quote the article you linked to :
"This complex, Depression-era legislation was enacted to protect small businesses by outlawing discriminatory price and promotional allowances obtained by large businesses. As is the case with the other antitrust laws, the Department of Justice, the FTC, and private parties may each bring Robinson-Patman cases, although the enforcement agencies have not focused on this area for some time. Indeed, the Justice Department has criminal powers in this area that have gone unused for many years, while the FTC today brings few significant cases in this area after being particularly active through the 1970s. Private suits on behalf of businesses (consumers have no standing to sue under the statute) account for most of the enforcement activity. Successful plaintiffs are entitled to the same remedies as those available under the antitrust laws (e.g injunctions, treble damages, attorneys’ fees and costs)."
The article lists the very specific reasons a case may be brought, and the various defenses. I do not see how any of this is relevant to your argument that "charging different prices is illegal". I do however agree with your concept that you should try out different prices strategies.
I have been amazed in my own experience when I make a package of my products priced beyond any shade of reason (from my view, as seller) and someone goes ahead and buys it. I had a manufacturing company and the 'big' package was to buy $1200 worth of goods and it was sometimes a tough sale. We tried a new 'big' package that was priced a little over $10,000 and it was an easier sale (to bigger customers). Lesson learned - don't ever think you know what the limits are until you test them. You might think that nobody would ever pay $100 per month for your widget, when some division of a Fortune 500 company would happily pay you a $1000 per month. Try it, you may be surprised.
The article lists the very specific reasons a case may be brought, and the various defenses. I do not see how any of this is relevant to your argument that "charging different prices is illegal". I do however agree with your concept that you should try out different prices strategies.
I have been amazed in my own experience when I make a package of my products priced beyond any shade of reason (from my view, as seller) and someone goes ahead and buys it. I had a manufacturing company and the 'big' package was to buy $1200 worth of goods and it was sometimes a tough sale. We tried a new 'big' package that was priced a little over $10,000 and it was an easier sale (to bigger customers). Lesson learned - don't ever think you know what the limits are until you test them. You might think that nobody would ever pay $100 per month for your widget, when some division of a Fortune 500 company would happily pay you a $1000 per month. Try it, you may be surprised.
It refers to the Robinson-Patman act, about which you can find more information at http://en.wikipedia.org/wiki/Robinson%E2%80%93Patman_Act including a link to the relevant statute at http://www.law.cornell.edu/uscode/15/13.html.
My non-lawyerly reading says that you only need to worry if you're shipping physical goods for resale. And even then you have room to be OK as long as the price differences aren't creating a monopoly, and aren't enough to economically injure the people who receive a higher price.
My non-lawyerly reading says that you only need to worry if you're shipping physical goods for resale. And even then you have room to be OK as long as the price differences aren't creating a monopoly, and aren't enough to economically injure the people who receive a higher price.
Note that selling a product below its cost is required to break the law (aka "dumping"). I would assume this means below its "marginal" cost; which, for digital goods, is nearly zero.
What if you do display different prices, but at the end of checkout you surprise them and charge everyone the lowest price in your range.
That is a really interesting idea.
The only issue I would see with this is that having the sudden "We lowered the price!" message would influence conversion rates at the final step.
You would have to measure and compare the number of "add to baskets" and "proceed to check outs" and not the revenue / conversions.
The only issue I would see with this is that having the sudden "We lowered the price!" message would influence conversion rates at the final step.
You would have to measure and compare the number of "add to baskets" and "proceed to check outs" and not the revenue / conversions.
Well, if one has to offer "completely" different products at different prices for testing purposes (say product "x", "y", "z") like the author suggests, how can you determine the ideal price for product "x" in the end?
Good post, I think more people should be testing different values (I remember reading something where someone kept doubling the price til they reached the tipping point).
Cheers
Cheers
You're thinking about http://jacquesmattheij.com/Double+your+price+%28and+no%2C+I%... which was discussed in http://news.ycombinator.com/item?id=1639712.
Yes, price testing is very important at initial stages of launch. It becomes progressively harder as your product gets established.
After your product is established, the only way to test price sensitivity is to launch pro version.
After your product is established, the only way to test price sensitivity is to launch pro version.
A pro version isn't the only way, you could launch:
* light version
* experiment with different sales and promo codes
* offer different rates for paying ahead a certain amount
* light version
* experiment with different sales and promo codes
* offer different rates for paying ahead a certain amount
Any best practices for A/B testing on the iOS App Store?
I have done this with an iphone app. The ideal case for testing:
-Your app has little publicity and is generally found through keyword searching. This means that you have few repeat visitors to the app page
-Your daily sales data has little variation. You typically see this 2 weeks after release when it is no longer in the new apps list. Avoid holidays and other sources of variation for your test.
-Schedule pricing into the future on one week increments. Try not to have pricing go generally up or down, perhaps by using some randomization or shuffling of possible prices.
-Use the revenue generated during each of the weeks coupled with the pricing to identify your ideal price.
Using poor scientific measurements (I crossed holidays, and did a declining price from a high point) I came up with $2.99 as an ideal price for my app.
Keep in mind that your sales rank influences your revenue, and if you think your app can break the top 100 apps list, then price it at $.99, because once you get into that list any losses prior due to underpricing an app are nullified.
Too bad the app store doesn't primarily sort by revenue (aka value to the consumer) and doesn't provide a continuous scroll past the top 100 which would make such devaluations unnecessary.
-Your app has little publicity and is generally found through keyword searching. This means that you have few repeat visitors to the app page
-Your daily sales data has little variation. You typically see this 2 weeks after release when it is no longer in the new apps list. Avoid holidays and other sources of variation for your test.
-Schedule pricing into the future on one week increments. Try not to have pricing go generally up or down, perhaps by using some randomization or shuffling of possible prices.
-Use the revenue generated during each of the weeks coupled with the pricing to identify your ideal price.
Using poor scientific measurements (I crossed holidays, and did a declining price from a high point) I came up with $2.99 as an ideal price for my app.
Keep in mind that your sales rank influences your revenue, and if you think your app can break the top 100 apps list, then price it at $.99, because once you get into that list any losses prior due to underpricing an app are nullified.
Too bad the app store doesn't primarily sort by revenue (aka value to the consumer) and doesn't provide a continuous scroll past the top 100 which would make such devaluations unnecessary.