Most common words unique to 1-star and 5-star App Store reviews(marco.org)
marco.org
Most common words unique to 1-star and 5-star App Store reviews
http://www.marco.org/1111087530
17 comments
It would be useful to separate the common words into two buckets (paid and free). Freeloaders reviewing free apps can be merciless...
I think it is the opposite. There is nothing like an user scorned for wasting their 99 cents.
I strongly think that he is likely to be empirically accurate with regards to the free apps.
Relatedly, you should always have a price tag on anything you are selling so that you do not compromise people's perception of its value. (This is not necessarily going to make you the most popular person in your techy circle of friends, but then again since they won't actually buy your stuff their opinion is not the world's most relevant.)
If you give it away for free, give it away by special arrangement. (Which will make people feel like they're in the in-crowd with backstage passes, rather than entitled louts.)
Relatedly, you should always have a price tag on anything you are selling so that you do not compromise people's perception of its value. (This is not necessarily going to make you the most popular person in your techy circle of friends, but then again since they won't actually buy your stuff their opinion is not the world's most relevant.)
If you give it away for free, give it away by special arrangement. (Which will make people feel like they're in the in-crowd with backstage passes, rather than entitled louts.)
While this does happen sometimes, the more expensive the app the higher the app rating (in general).
Free apps, people will download haphazardly and then delete, leading to many 1-star reviews.
With more expensive apps, people are much more likely to appreciate the app as they do the research before spending their money.
Source: taking a look at the App Store plus my own apps.
Free apps, people will download haphazardly and then delete, leading to many 1-star reviews.
With more expensive apps, people are much more likely to appreciate the app as they do the research before spending their money.
Source: taking a look at the App Store plus my own apps.
There might also be an effect of people wanting more badly to like the app and "deluding" themselves... if I pay $0.99 for an app, I don't care too much if it works or not, but if I pay $29.99 I'm going to want to like it for reasons of self-validation.
Alternatively, you could conclude that the difference is in the apps. 75%+ of free apps in the App Store are junk. Paid apps tend to be much higher quality offerings in my experience.
The words that leap out at me from the five-star list are "never" and "always". If ever a piece of data underscored the value of consistency, it's that.
The words that leap out at me from the one-star list are "actually" and "says", words indicating that real behavior differed from expected/advertised behavior. The app says X but actually does Y, and therefore sucks. This strongly underscores the value of consistency between what your customers think they're getting and what you actually deliver.
If only this was based on any kind of logic. The problem with the app store users is that they often just look at the picture, and do not read the description. There are lots and lots of cases where people are asking for something that is specifically mentioned not available in the app store description.
It goes back to understanding your users. If you thoroughly understand what problem they're trying to solve and what they normally do when solving it, you can design completely around that and make them happy.
I agree that this is a good design philosophy. But I also think that the problem of "the user not reading the app description" (as described by the parent) is a separate issue.
For example, I found that there were several problems to be solved that were all related to the same domain. Apps existed that do a decent job of solving problem X. So I wrote an app to solve problem Y. I have received numerous 1-star reviews because the user is disappointed that my app does not solve Problem X. In my case, I am pretty confident that I clearly described the problem solved by my app. I even explicitly say that it is not meant to solve problem X. I have also received many positive reviews from users looking for an app that solves Problem Y. So in my case, I don't think it was a problem with understanding the user.
For example, I found that there were several problems to be solved that were all related to the same domain. Apps existed that do a decent job of solving problem X. So I wrote an app to solve problem Y. I have received numerous 1-star reviews because the user is disappointed that my app does not solve Problem X. In my case, I am pretty confident that I clearly described the problem solved by my app. I even explicitly say that it is not meant to solve problem X. I have also received many positive reviews from users looking for an app that solves Problem Y. So in my case, I don't think it was a problem with understanding the user.
This might describe a supernatural level of user-understanding; but it must be possible to create an app description which gets past the barrier of what the customer expects your app to do, and actually communicates to them the true capabilities of your app.
I'm just speculating, but it's possible that the people who misunderstand the purpose of your app because they didn't read the description are people you never intended to be part of your demographic of users. For [a hypothetical] niche-puchose software, catering to idiots may be very detrimental. I could imagine a situation where intended software goals and mass market acceptance would never line up.
As I guessed, the words used by those writing 1-star reviews suggest bitterness and idiocy on the part of the reviewer. Most nice people grade on a curve.
The words used in 5-star reviews aren't bad, though, except for the word "perfect". It's often misused when describing software.
The words used in 5-star reviews aren't bad, though, except for the word "perfect". It's often misused when describing software.
> Most nice people grade on a curve.
I don't understand this, could you explain?
I don't understand this, could you explain?
What I mean by grading on a curve is that, rather than give as many 1-star reviews as 5-star reviews, nice people almost always give between 3 and 5 stars. Even if they find an app to be useless to them, they often give the benefit of the doubt. Perhaps it will improve in a future version, or be more useful to someone else.
By doing so, of course, they make the whole system roughly 1/3 less useful[1] for everyone; that isn't my idea of "nice".
[1] log 3 / log 5 ~= 0.68.
[1] log 3 / log 5 ~= 0.68.
True. It's mitigated somewhat by having half-stars in the reviews (not sure Apple does this but Yelp does), but it still makes it a pretty slim margin between crappy and excellent.
But we're talking about the best and worst here- so presumably at the ends of the curve, and deserving of superlatives.
Yes, but I think there's a different type of person that regularly leaves one-star and two-star reviews. I think there are a lot of people who don't want to trash something, even if it sucks, and don't care about statistical integrity.
In my experience the #1 cause of "crashes" is being killed by the memory watchdog. An app I publish deals with photos, and I ran into such a problem with people trying to load in massive photos that I had to enforce an artificial pixel-size cap to stop it.
The iPad has a memory fragmentation problem. The longer the unit is up, the less you can guarantee that a request for a large amount of memory won't be met with a watchdog kill. What works once when it's from a fresh boot may not work a few days or a few weeks later.
The iPad has a memory fragmentation problem. The longer the unit is up, the less you can guarantee that a request for a large amount of memory won't be met with a watchdog kill. What works once when it's from a fresh boot may not work a few days or a few weeks later.
We deal with large photos as well, and it's usually just a matter of making sure the only thing in memory is what's currently showing on the screen, and optimizing which size of photos we're using so that we can use as little memory as possible. It's a pain though. We also run into problems with orientation changes, because the animation for the orientation change gets very, very choppy if you're using too much memory.
Your experience echoes mine. There are only two situations where I use the full-resolution image, and I had to go to great lengths to make sure NOTHING else was loaded. The rest of the time it uses pre-rendered representations at various smaller sizes.
Why do apps get killed for requesting memory? Can't malloc return NULL and the app act accordingly ("your picture is too big, so i am not going to edit it").
Now, it seems most people never handle malloc returning NULL, and "you can't do this today" is a bad user experience. But crashing is what happens in the first case anyway, and being auto-killed is a bad experience too.
Now, it seems most people never handle malloc returning NULL, and "you can't do this today" is a bad user experience. But crashing is what happens in the first case anyway, and being auto-killed is a bad experience too.
Apps don't get killed for requesting memory. They get killed when they're running in the background and other apps request memory.
Can't malloc return NULL and the app act accordingly ("your picture is too big, so i am not going to edit it").
Thats exactly what happens. You're free to run:
// edit: Here's what happens when you do run that (the last line is a NSLog() to show the address of foo):
Can't malloc return NULL and the app act accordingly ("your picture is too big, so i am not going to edit it").
Thats exactly what happens. You're free to run:
void *foo = malloc(INT_MAX);
// edit: Here's what happens when you do run that (the last line is a NSLog() to show the address of foo):
Malloc(4972,0x3e088868) malloc: *** mmap(size=2147483648) failed (error code=12)
*** error: can't allocate region
*** set a breakpoint in malloc_error_break to debug
2010-09-12 19:44:59.154 Malloc[4972:307] malloc: 0x0
I don't know. But it'll usually give you the memory, send the level 2 memory warning (level 3 is kill), and then kill you shortly after. The memory warning is supposed to be a warning to release cached data, so it's sort of hard to release the data you're working on right at that point in time gracefully.
Ok I don't know a lot about iOS development or ARM development in general, but there's something I don't understand. I had the impression that the ARM architecture uses a memory paging system not unlike the one used by x86. This would imply that memory fragmentation shouldn't be an issue, unless we're talking about in-process logical memory address space fragmentation, which would be largely the developer's fault anyway. Am I missing something here?
I haven't done any real investigation on the whole thing, but the general feeling I get about the whole thing is that over time, system-related processes may hold on to more and more memory. I don't know what specifically is doing it or why it's doing it, but the end result is that trying to load a 2048x2048 pixel image will work one time and the next time your app will get killed for it.
iOS devices have NO virtual memory at the OS level. Once it's out of physical RAM, that's when it really begins to clamp down on app memory usage. There's only 256 MB of RAM on the iPad, so loading that one image needs at least 40 MB of that. If a bunch of the Apple-originated processes are resident in memory, and it seems like it hangs on to a bunch of them even after the app is closed, then it may or may not let you use that much memory.
iOS devices have NO virtual memory at the OS level. Once it's out of physical RAM, that's when it really begins to clamp down on app memory usage. There's only 256 MB of RAM on the iPad, so loading that one image needs at least 40 MB of that. If a bunch of the Apple-originated processes are resident in memory, and it seems like it hangs on to a bunch of them even after the app is closed, then it may or may not let you use that much memory.
The word "useless" has been popping up in my free apps' reviews too. I don't know how to interpret it.
I'm wondering if this is a new word among young people now. Kind-of the new "sucks" maybe?
I'm wondering if this is a new word among young people now. Kind-of the new "sucks" maybe?
"useless" is more extreme than "sucks". I think it is dismissive in the same way as describing something as "FAIL". People are being exposed to more options and are being asked to make more and more judgements. I think making a gray analysis like "It's good at this but bad at that" is hard to keep track of with the quantity of options. It is far easier to simply lump things into black and white buckets of good and useless.
But ultimately what's "useless" is the comment itself. It does nothing to better anything. It leaves the developer in a void of wonderment. What's useless? Is it lacking a feature? Is it not working at all? What void were you looking to fill?
I also get an overwhelming amount of 5 star ratings with well thought out reviews. So when one of these "useless" comments pops in, it throws me back to high school days, when everything "sucked" - which is why I wonder if the word is the new "sucks".
Thinking back, saying "sucks" so often was quite useless.
I also get an overwhelming amount of 5 star ratings with well thought out reviews. So when one of these "useless" comments pops in, it throws me back to high school days, when everything "sucked" - which is why I wonder if the word is the new "sucks".
Thinking back, saying "sucks" so often was quite useless.
I don't think the 1-star reviewer is thinking of "you" (the developers), at all. They want to bury the app so that no other user will have to waste their time in the same way they just did. It "sucks" but I don't know how you could change it.
It's interesting how the negative list seems to be heavily-populated by unigrams from a phrase something like "doesn't actually do anything". The app store must have a significant misrepresentation-of-functionality problem.
It may well be a usability problem - the app does what it says on the tin, but the user can't figure out how. In my experience the usual meaning of "It won't do x" is in fact "I know it does x, but I can't figure out how to make it". Rightly so in my opinion - I think it's part of the great cultural divide in computing, where developers see software as a toolkit but users expect appliances.
Marco also notes that the app has to contend with the user's expectation of what it should do. Which might not have any basis in reality. If an app doesn't do what a user expects, regardless of where that expectation originates, it earns the 'useless' designation.
The biggest problem with App Store 'reviews' is that iOS asks for a rating on uninstall, this leads to many '1 star' ratings, with no review at all, leaving the developer in the dark as to what the user's issues are.
This was a problem, but it was fixed with the release of iOS 4.0. Now users have to go to the app store to rate an app.
It would be fun to expose this data (perhaps with a few additional dimensions?) and hack up an extension/greasemonkey script to rip out reviews that fit a certain criteria.
Would you pay money for this?
Me personally, no, but it raises an interesting point. I could see a market in "scoring" reviews using models based on data sets similar to the one examined here.
Another great thing to measure would be the common words in the app descriptions. I would love to find out why people still download 1- star rated apps.
It would be interesting to see such list for the app descriptions (may be gauged by sales, not by the app score). It might be that some harmless-looking words trigger positive or negative reaction.
Freaconomics has interesting chapter on words used in house advertisement - some word like "wonderful" send negative message.
Someone should do this for the most upvoted posts/commented on Hacker News. I'm too lazy for that :)
Surely, we will find content with a lot less use of subjective terms like "awesome" but I'm curious by how much. That's one way to get a sense how objectively critical the posts/comments mostly are.
Surely, we will find content with a lot less use of subjective terms like "awesome" but I'm curious by how much. That's one way to get a sense how objectively critical the posts/comments mostly are.
What I am going to be doing immediately: composing mock 5- and 1-star reviews based on these words, and running through my apps' future updates to make sure they are much more more readily described by the former.
Surprisingly appropriate works to describe the best and worst people, as well!
haha, it matches my app's 5 star and 1 star reviews surprisingly well.
Most apps settle into the 3 star range for that reason.
Most apps settle into the 3 star range for that reason.
Funny to find "open" in the 1-star list. (not saying that that actually means anything)
Probably along the lines of "...as soon as I open the app it crashes..."
Most likely due to the memory manager issues mentioned above. I've seen some quality apps get massive numbers of one star reviews for this type of problem.
The star rating system really irks me for this reason. Something that could be fixed to perfect condition in less than 24hours permanently carries the baggage beyond its relevance.
Most likely due to the memory manager issues mentioned above. I've seen some quality apps get massive numbers of one star reviews for this type of problem.
The star rating system really irks me for this reason. Something that could be fixed to perfect condition in less than 24hours permanently carries the baggage beyond its relevance.
GPL apps arent allowed on iphone you know that right?
really really this analysis is so lame, if you take any product review and applied this analysis it would give similar results.
This is equivalent of all those spamfographics
This is equivalent of all those spamfographics