Sock Puppets. Online Reviews: Completely Worthless or Only Mostly Worthless?(techcrunch.com)
techcrunch.com
Sock Puppets. Online Reviews: Completely Worthless or Only Mostly Worthless?
http://techcrunch.com/2012/09/08/sock-puppet-spectacular-are-online-reviews-completely-worthless-or-only-mostly-worthless/
17 comments
(meta question)
image credit here links to flickr, where the photo is shared under CC non-commercial license (http://creativecommons.org/licenses/by-nc-sa/2.0/). Is a techcrunch post really considered non-commercial usage? (non-rhetoric question)
A for-profit blog is definitely commercial usage.
When I'm browsing products on Amazon, I've found the reviews there to be very helpful and they've saved me from buying a lot of terrible products.
And on the flipside, I've never bought something because of good reviews that all turned out to be lies when I had the product in my hands a couple of days later.
And on the flipside, I've never bought something because of good reviews that all turned out to be lies when I had the product in my hands a couple of days later.
I find the reviews themselves helpful, the scores less so.
It seems that scores follow an interesting distribution, people mostly review things they love or hate. People rarely seem to want to take the time to point out something is just ok.
You see this bimodal distribution for Yelp restaurant reviews. Reading through the reviews I find the reasoning of a lot of reviewers quite spurious. I think it would be an useful application of machine learning to determine the validity of claims and generate a real score, an automatic Zagat-machine.
It's worse for complicated products, if I was a manufacturer I'd want to be able to rebut them. For example, I was looking to buy a coffee grinder and the reviews were all over the place. It's obvious that a lot didn't read the instructions or had higher expectations than a product at that price point would provide. People complain that a $30 item is made mainly of plastic.
One of the advantages of Amazon is the sheer quantity of reviews for most items, the outliers--whether positive or negative--get lost in the noise. It does penalize products with only a couple of reviews, it starts to make you nervous and reconsider the purchase.
This really is the tyranny of choice, you waste a lot of time looking for the "perfect" product, when traditionally you'd just walk into your local store, make a decision based on imperfect knowledge, and probably be happy with what you ended up buying.
A good example is my recent purchase of a nail brush from Amazon. It's a sub-$10 item that in the past I'd have just walked into my local drugstore and picked up the cheapest out of whatever they had in stock. The category[1] has 843 products that I wasted a good hour deciding on the "best" to purchase.
[1] http://www.amazon.com/s/?url=node=11063531&rh=n:3760911,...
It seems that scores follow an interesting distribution, people mostly review things they love or hate. People rarely seem to want to take the time to point out something is just ok.
You see this bimodal distribution for Yelp restaurant reviews. Reading through the reviews I find the reasoning of a lot of reviewers quite spurious. I think it would be an useful application of machine learning to determine the validity of claims and generate a real score, an automatic Zagat-machine.
It's worse for complicated products, if I was a manufacturer I'd want to be able to rebut them. For example, I was looking to buy a coffee grinder and the reviews were all over the place. It's obvious that a lot didn't read the instructions or had higher expectations than a product at that price point would provide. People complain that a $30 item is made mainly of plastic.
One of the advantages of Amazon is the sheer quantity of reviews for most items, the outliers--whether positive or negative--get lost in the noise. It does penalize products with only a couple of reviews, it starts to make you nervous and reconsider the purchase.
This really is the tyranny of choice, you waste a lot of time looking for the "perfect" product, when traditionally you'd just walk into your local store, make a decision based on imperfect knowledge, and probably be happy with what you ended up buying.
A good example is my recent purchase of a nail brush from Amazon. It's a sub-$10 item that in the past I'd have just walked into my local drugstore and picked up the cheapest out of whatever they had in stock. The category[1] has 843 products that I wasted a good hour deciding on the "best" to purchase.
[1] http://www.amazon.com/s/?url=node=11063531&rh=n:3760911,...
>It's worse for complicated products, if I was a manufacturer I'd want to be able to rebut them. For example, I was looking to buy a coffee grinder and the reviews were all over the place. It's obvious that a lot didn't read the instructions or had higher expectations than a product at that price point would provide. People complain that a $30 item is made mainly of plastic.
You can rebut. I mean, for a long time the first hit for 'prgmr.com' was a thread on webhostingtalk[1] where someone was complaining that I hadn't set them up and that I had problems. Fair complaints, on the other hand, at the time, five dollars was an unheard of price for what I was providing (It's no longer an unheard-of price. Moore's law abides.) I responded and eh, I think I ended up coming out looking okay.
I guess the thing is, acknowledging your mistakes, yes, is very unprofessional, but really? I think people are sick of dealing with professionals. Showing some humanity and some humility (even when it's the 'false humility' we all put on so that our arrogance is less irritating to others.) can help you a lot.
On the other hand, on that same website, I'm trying to sell burstable 10G cogent ports[2] for a price that is currently unheard of, and I've gotten no bites, so It's possible that people prefer professionals for expensive things and humans for cheap things.
Note, though, (and this is the 'false humility' I describe) you don't want to directly say "you are too dumb to use the product" - even if they are. I mean, you want to explain the proper use of the product, and you want to back out the deals (refunds, usually) with the people that are unwilling or unable to do it correctly. Saying things here like 'Oh, I should improve this documentation here' or 'oh, I should link to this other documentation over there' is also good, especially if you really can improve that documentation. Personally, I think having a reputation for being not particularly easy to use is very good. You don't want to deal with the people that want handholding, especially when your product is as difficult to use safely as a Linux server.
[1]http://www.webhostingtalk.com/archive/index.php/t-541344.htm...
[2]http://www.webhostingtalk.com/showthread.php?t=1186971
You can rebut. I mean, for a long time the first hit for 'prgmr.com' was a thread on webhostingtalk[1] where someone was complaining that I hadn't set them up and that I had problems. Fair complaints, on the other hand, at the time, five dollars was an unheard of price for what I was providing (It's no longer an unheard-of price. Moore's law abides.) I responded and eh, I think I ended up coming out looking okay.
I guess the thing is, acknowledging your mistakes, yes, is very unprofessional, but really? I think people are sick of dealing with professionals. Showing some humanity and some humility (even when it's the 'false humility' we all put on so that our arrogance is less irritating to others.) can help you a lot.
On the other hand, on that same website, I'm trying to sell burstable 10G cogent ports[2] for a price that is currently unheard of, and I've gotten no bites, so It's possible that people prefer professionals for expensive things and humans for cheap things.
Note, though, (and this is the 'false humility' I describe) you don't want to directly say "you are too dumb to use the product" - even if they are. I mean, you want to explain the proper use of the product, and you want to back out the deals (refunds, usually) with the people that are unwilling or unable to do it correctly. Saying things here like 'Oh, I should improve this documentation here' or 'oh, I should link to this other documentation over there' is also good, especially if you really can improve that documentation. Personally, I think having a reputation for being not particularly easy to use is very good. You don't want to deal with the people that want handholding, especially when your product is as difficult to use safely as a Linux server.
[1]http://www.webhostingtalk.com/archive/index.php/t-541344.htm...
[2]http://www.webhostingtalk.com/showthread.php?t=1186971
If a review is littered with phrases such as "modern masterpiece", "will touch your soul" or "a magnificent book", I'm not likely to take it seriously. I pay attention to reviews in which the reviewer can (a) convince me that they've actually read the book and (b) show me that their criteria for a good book have something in common with mine.
Exactly; maybe it's just because of age, but by the time on-line reviews became important my BS filters were well developed. Its not very difficult to detect the chaff, and it doesn't take too many heuristics---I like "specificity" a lot---to recognize the honest ones. Then doing your step b is easy.
Maybe adding the social factor could improve things? A platform that finds me a review from my third degree connection would be nifty. The biggest problem probably being not enough close connections who bothered to review. Sigh, back to good old asking around (maybe at least using e-mail or IM.)
I find this article ironic...
Techcrunch is a "publication" I've always found questionable and wondered how much sock-puppetry was hiding under the covers. Note I have no hard evidence, just a feel.
Sock puppets complaining about sock puppets... Go figure.
Techcrunch is a "publication" I've always found questionable and wondered how much sock-puppetry was hiding under the covers. Note I have no hard evidence, just a feel.
Sock puppets complaining about sock puppets... Go figure.
I assure you, I am not a sock puppet.
Of course, that's exactly what a sock puppet would say. Hmm. Is there a sock-puppet Turing test?
Of course, that's exactly what a sock puppet would say. Hmm. Is there a sock-puppet Turing test?
>>Hmm. Is there a sock-puppet Turing test?<<
You could make millions! :)
You could make millions! :)
Disappointed - I thought this was going to be an honest review about sock puppets. I was hoping to see how well our mascot fared (http://bidkat.com) ;)
Hypothesis: Yelp relies on blackmail.
Disprove it.
Disprove it.
[deleted]
Sock Puppets. Tech Crunch: Completely Worthless or Only Mostly Worthless?
Seriously, though. Every source with a grain of salt.[1,2]
______________
[1] http://paulgraham.com/submarine.html: "If anyone is dishonest, it's...reporters. The main reason PR firms exist is that reporters are lazy. Or, to put it more nicely, overworked. Really they ought to be out there digging up stories for themselves. But it's so tempting to sit in their offices and let PR firms bring the stories to them."
[2] That's not a knock on this submission. If anything, just an application what we learned inside (ie,its principles). Or, as the author un-ironically notes, "Even the honest reviews are mostly written by people who are stupid...". That sounds fair and objective, right?
Seriously, though. Every source with a grain of salt.[1,2]
______________
[1] http://paulgraham.com/submarine.html: "If anyone is dishonest, it's...reporters. The main reason PR firms exist is that reporters are lazy. Or, to put it more nicely, overworked. Really they ought to be out there digging up stories for themselves. But it's so tempting to sit in their offices and let PR firms bring the stories to them."
[2] That's not a knock on this submission. If anything, just an application what we learned inside (ie,its principles). Or, as the author un-ironically notes, "Even the honest reviews are mostly written by people who are stupid...". That sounds fair and objective, right?
only somewhat. But techcrunch cant think outside of superlative and hyperbole.
"Real Actors Reading Yelp Reviews" justifies any amount of sock puppetry.