Android 2.2 demolishes iOS4 in JavaScript benchmarks(feeds.arstechnica.com)
feeds.arstechnica.com
Android 2.2 demolishes iOS4 in JavaScript benchmarks
http://feeds.arstechnica.com/~r/arstechnica/everything/~3/Fe2vBjQR9KU/android-22-demolishes-ios4-in-javascript-benchmarks.ars
17 comments
Apple only quote numbers when both of the following apply:
1) They beat the competition at that point on it
2) They can present it as a benefit to the UX
Current example: Resolution. They beat the competition, and they present it as a distinct experience benefit.
By and large, they neither strongly compete nor communicate numbers elsewhere. In fact they go so far as to obscure or hide numbers and instead focus exclusively on the quality and experience angles instead.
I'm not a fruit fanboy, but I do think that the way to play the numbers game is to show why they matter in ways that the layman can comprehend. On that front, if Google aren't using this advantage in the sunspiders tests to communicate on a user experience angle then they're missing the advantage of being ahead in the numbers game.
Current example: Resolution. They beat the competition, and they present it as a distinct experience benefit.
By and large, they neither strongly compete nor communicate numbers elsewhere. In fact they go so far as to obscure or hide numbers and instead focus exclusively on the quality and experience angles instead.
I'm not a fruit fanboy, but I do think that the way to play the numbers game is to show why they matter in ways that the layman can comprehend. On that front, if Google aren't using this advantage in the sunspiders tests to communicate on a user experience angle then they're missing the advantage of being ahead in the numbers game.
While that's certainly true, this is also to the credit of Lars Bak, the lead programmer on V8. There just aren't many people on earth who have his experience building VM's [1].
Disclosure: Googler, huge Apple fanboy.
1. http://en.wikipedia.org/wiki/Lars_Bak_(computer_programmer)
Disclosure: Googler, huge Apple fanboy.
1. http://en.wikipedia.org/wiki/Lars_Bak_(computer_programmer)
No kidding. This page[1] of Ars' Froyo review made me cringe, and I'm a geek who understands all of this stuff! No way I could recommend this to friends & family who aren't into tech for the sake of tech. If that's the cost of removable and upgradeable storage count me out.
http://arstechnica.com/open-source/reviews/2010/07/android-2...
This particular problem is with the handset, not the OS, but it's basically the flagship device. Ugh.
http://arstechnica.com/open-source/reviews/2010/07/android-2...
This particular problem is with the handset, not the OS, but it's basically the flagship device. Ugh.
By default Froyo manages moving apps between internal storage and the SD card automatically. Normal users never have to see those (admittedly ugly) detail screens. Your complaint is like looking at the man page for vi in Terminal.app and concluding that Mac OS X is only usable by geeks.
I didn't know that. That sounds like the way it should work and would alleviate my concerns. I'm not against knobs for knowledgable users in general.
But they'll be bitten by this. The Android team are clearly not smart enough to understand why people like the iPhone.
"One obvious downside of external application storage is that the software installed on the SD card will only be available when the card is mounted on the device. If you plug your phone into your computer and enable USB mass storage mode, for example, the software installed on the SD card will not be accessible. Due to this limitation, Google's developer documentation warns that long-running software like background services, live wallpapers, and widgets should not allow SD installation."
Can you imagine explaining this to end users? "Well, if you install to the SD card to get more storage, you can't use this app when your phone is plugged into your computer." This is just... stupid.
"One obvious downside of external application storage is that the software installed on the SD card will only be available when the card is mounted on the device. If you plug your phone into your computer and enable USB mass storage mode, for example, the software installed on the SD card will not be accessible. Due to this limitation, Google's developer documentation warns that long-running software like background services, live wallpapers, and widgets should not allow SD installation."
Can you imagine explaining this to end users? "Well, if you install to the SD card to get more storage, you can't use this app when your phone is plugged into your computer." This is just... stupid.
"Well, if you install to the SD card to get more storage, you can't use this app when your phone is plugged into your computer."
Plugged in is fine, it's only when the SD card is explicitly mounted as a USB drive that the apps are unavailable. Still mildly annoying, but not as annoying as the iPhone refusing to let you access its filesystem at all.
Plugged in is fine, it's only when the SD card is explicitly mounted as a USB drive that the apps are unavailable. Still mildly annoying, but not as annoying as the iPhone refusing to let you access its filesystem at all.
Out of curiosity, which part in particular made you cringe?
First the default limited storage for apps. WebOS had the same problem up until about 6 months ago. That's just poor design and planning.
Then the choice of where to install apps being left to developers.
Last, the choice to expose a method for shuffling apps around. Sure it's nice to have because of the default limited space for apps. However the correct solution isn't to burden users with this low-level crap, but to fix the design so this kind of thing is unnecessary.
If they insist on supporting a removable SD card then they should install apps wherever there is room, perhaps notifying the user that if they remove the SD card then they will no longer be able to run apps installed to it.
It seems like a small thing, but these kinds of hacks pile up on each other and soon your left with a Windows-esque system where users are burdened with system maintenance crap instead of the device being more like an appliance that they can just use.
If a user goes to install an app and the device says there's no room when they have a 16GB SD card, most would just throw up their hands and think that it's broken, not that they should contact the developer of the app to request that they enable installation to SD cards, or that they should change the install location or whatever.
Then the choice of where to install apps being left to developers.
Last, the choice to expose a method for shuffling apps around. Sure it's nice to have because of the default limited space for apps. However the correct solution isn't to burden users with this low-level crap, but to fix the design so this kind of thing is unnecessary.
If they insist on supporting a removable SD card then they should install apps wherever there is room, perhaps notifying the user that if they remove the SD card then they will no longer be able to run apps installed to it.
It seems like a small thing, but these kinds of hacks pile up on each other and soon your left with a Windows-esque system where users are burdened with system maintenance crap instead of the device being more like an appliance that they can just use.
If a user goes to install an app and the device says there's no room when they have a 16GB SD card, most would just throw up their hands and think that it's broken, not that they should contact the developer of the app to request that they enable installation to SD cards, or that they should change the install location or whatever.
Then the choice of where to install apps being left to developers.
Yeah, I'm not thrilled about that. They have a point that some apps types fundamentally won't work well when run from the SD card, but I'd prefer it to be an opt-out rather than opt-in process.
Yeah, I'm not thrilled about that. They have a point that some apps types fundamentally won't work well when run from the SD card, but I'd prefer it to be an opt-out rather than opt-in process.
>Apple's products have almost always been lagging in terms of spec sheet
This is incredibly untrue. The iPhone has dominated virtually every benchmark until very, very recently, owing to a more efficient CPU, and a vastly superior GPU (the Samsung Galaxy S is pretty much the only device that beats even the 3GS).
The iPhone is always talked up on the merits of its spec sheet, whether it's screen technology, case technology, GPU speed, browser speed, and on and on. This has always been its edge.
This is incredibly untrue. The iPhone has dominated virtually every benchmark until very, very recently, owing to a more efficient CPU, and a vastly superior GPU (the Samsung Galaxy S is pretty much the only device that beats even the 3GS).
The iPhone is always talked up on the merits of its spec sheet, whether it's screen technology, case technology, GPU speed, browser speed, and on and on. This has always been its edge.
Mac hardware and iPods have indeed been lagging in terms of spec sheet for quite a while, so no, it's not "incredibly untrue." Note that there's also a difference between lagging in specs and talking up specs. When Apple talks about specs, it's almost always in comparison to previous Apple products. You'll never see them comparing the latest MacBook Pros to Alienware laptops.
Also, Apple has no qualms trumpeting better specs, but they frequently don't. The GPU, for instance, has been completely absent from iDevice discussion since the iPad. (It's part of the A4, but Apple doesn't tell you that.) They also failed to mention the RAM bump for the iPhone 4.
Also, Apple has no qualms trumpeting better specs, but they frequently don't. The GPU, for instance, has been completely absent from iDevice discussion since the iPad. (It's part of the A4, but Apple doesn't tell you that.) They also failed to mention the RAM bump for the iPhone 4.
According to Wikipedia, over 200MM people had 3G phones before the iPhone even launched[1]. It wasn't for another year that Apple introduced the iPhone 3G.
Whether or not Apple wins or loses on specs (cf. the "I don't care" bear), they're winning big on overall experience, mindshare and ecosystem. Arguments about who has the best GPU misses the point when you can only play the newest, hottest games (or whatever) on the iPhone.
It's sort of like the argument about the Wii vs. the X360 and PS3 or the DS vs. the PSP. Who cares what the specs? The apps are driving the platform.
[1] http://en.wikipedia.org/wiki/3G#History
Whether or not Apple wins or loses on specs (cf. the "I don't care" bear), they're winning big on overall experience, mindshare and ecosystem. Arguments about who has the best GPU misses the point when you can only play the newest, hottest games (or whatever) on the iPhone.
It's sort of like the argument about the Wii vs. the X360 and PS3 or the DS vs. the PSP. Who cares what the specs? The apps are driving the platform.
[1] http://en.wikipedia.org/wiki/3G#History
Is there any commonly used web app which have cpu intensive javascript code?
I would assume that most of the time would be wasted on rendering and downloading, and that javascript runs a fraction, even unmeasurable time.
From SunSpider website: "This includes tests to generate a tagcloud from JSON input, a 3D raytracer, cryptography tests, code decompression, and many more "
How often are this things really done in a browser, and are they really the bottleneck?
I would assume that most of the time would be wasted on rendering and downloading, and that javascript runs a fraction, even unmeasurable time.
From SunSpider website: "This includes tests to generate a tagcloud from JSON input, a 3D raytracer, cryptography tests, code decompression, and many more "
How often are this things really done in a browser, and are they really the bottleneck?
I feel the same but in a good world (Flash dies) upcoming html5/javascript games might be important.
More and more of these tasks are getting to be commonplace in the client side scripting of a website.
However, you raise a valid point: Mobile websites, the kinds most likely to be served to you via MobileSafari, will likely not have things like Tag clouds or 3D raytracers for the foreseeable future.
Maybe there should be a mobile edition of the JS benchmarks run?
And also, as much as speed is an issue, what about a benchmark for compliance? Safari/Webkit is still (to some sites) a red headed step child. The "it works in FF and IE" mentality applies there. My bank is an example of that. It has gotten better, but there are still plenty of web apps where webkit is unsupported.
However, you raise a valid point: Mobile websites, the kinds most likely to be served to you via MobileSafari, will likely not have things like Tag clouds or 3D raytracers for the foreseeable future.
Maybe there should be a mobile edition of the JS benchmarks run?
And also, as much as speed is an issue, what about a benchmark for compliance? Safari/Webkit is still (to some sites) a red headed step child. The "it works in FF and IE" mentality applies there. My bank is an example of that. It has gotten better, but there are still plenty of web apps where webkit is unsupported.
> there are still plenty of web apps where webkit is unsupported.
I hope one of these days we will get to the point where web apps are standards complaint, rather than rendering engine complaint.
We've largely won the battle on earning recognition and support for Gecko, but now we have to do most of it over again for Webkit? What new and exciting engine will we be griping about in two years? Does this make sense?
I hope one of these days we will get to the point where web apps are standards complaint, rather than rendering engine complaint.
We've largely won the battle on earning recognition and support for Gecko, but now we have to do most of it over again for Webkit? What new and exciting engine will we be griping about in two years? Does this make sense?
Anecdotal case: I'm writing a simple vector drawing program for tabletop RPG mapping inspired by Ben Robbin's series of posts about his 'West Marches' campaign which included a game artifact called the 'table map' the conceit being that the map was carved into a tabletop at the adventurers' home tavern.
To do this im rendering my scene to one canvas buffer, and then using that as the stencil to run a 3x3 convolution filter to produced a 'carved' look on a wood texture. I want the map editing to update the carved surface as quickly as possible, so yes javascript performance is a bottle neck.
ideally i want this to run on the iPad so its useful at the table as well as post game. The current implementation isn't fast enough on iPad so the browser kills the script.
Stuff like you are talking about isn't common at the moment, but its only going to become more viable and more prevalent. If a guy like me can knock together something like the above in a couple of days, imagine the possiblities for someone working full time on a real project.
To do this im rendering my scene to one canvas buffer, and then using that as the stencil to run a 3x3 convolution filter to produced a 'carved' look on a wood texture. I want the map editing to update the carved surface as quickly as possible, so yes javascript performance is a bottle neck.
ideally i want this to run on the iPad so its useful at the table as well as post game. The current implementation isn't fast enough on iPad so the browser kills the script.
Stuff like you are talking about isn't common at the moment, but its only going to become more viable and more prevalent. If a guy like me can knock together something like the above in a couple of days, imagine the possiblities for someone working full time on a real project.
I'd love to see how fast my Android phone runs with 2.2. Too bad I can't. One thing the iOS releases have going for them is that users can actually get them when they are released. 48 Days later, I still don't have the 2.2 update.
Well, it's not really 48 days later. It's 48 days after the announcement, but the final source wasn't released until a week or two ago.
Umm I don't know for sure. I just went with the "Latest stable release" date on Wikipedia's page about it.
http://android-developers.blogspot.com/2010/06/froyo-code-dr...
"Today is one of those days that has my heart racing; we’ve just released the source code for Android 2.2."
...
"Posted by Tim Bray on 23 June 2010"
"Today is one of those days that has my heart racing; we’ve just released the source code for Android 2.2."
...
"Posted by Tim Bray on 23 June 2010"
releasing the source != pushing out over the air OS updates
Wiki is wrong, then. The official 2.2 update for the Nexus One only went out about a week ago, and Froyo code was added to AOSP about a week before that.
http://www.pcworld.com/article/200041/google_android_22_froy...
http://www.pcworld.com/article/200041/google_android_22_froy...
I installed Ultimate Droid Froyo on my Motorola Droid, and I saw an enormous speed improvement over 2.1, so I think there will be a pretty significant difference.
You can safely install it manually if you wish.
Regardless, your statement seems to contradict itself. If you can't install 2.2 on your phone, then it isn't really "released," is it?
Regardless, your statement seems to contradict itself. If you can't install 2.2 on your phone, then it isn't really "released," is it?
It seems to me that Apple really doesn't care all that much about the iPhone's browser performance. Anything that presents a good alternative to the App Store universe is not in Apple's best interest.
Then why are they pushing so hard on HTML5? If what you say were true they should be trying to stall it, no?
Because web apps are still preferable to Flash
I don't see this as an advantage per se. Javascript on iPhone is indeed slower, but not to the point that it renders web apps unusable.
It could become an advantage over iPhone if Google plans to boost web apps on the platform. Otherwise this benchmark is meaningless to the average user, in my opinion.
It could become an advantage over iPhone if Google plans to boost web apps on the platform. Otherwise this benchmark is meaningless to the average user, in my opinion.
It renders some web apps unusable. Try reorganizing your Netflix queue on the iPad. Netflix's JS doesn't scale well to a queue with hundreds of items. On a computer running a modern browser, this means a lag of a couple of seconds. But on the iPad, it can be s full minute.
(The post isn't comparing Android 2.2 to the iPad but rather to the iPhone 4 hardware and OS...)
You're being downvoted because the article is comparing iOS4 to Android 2.2. Both are operating systems. iOS4 is on both the iPad and the iPhone.
In fact, the iPad and iPhone 4 are very similar in hardware as well. The most significant difference is in RAM, where the iPhone has twice as much as the iPad.
Mentioning his or her experience on the iPad is completely valid.
In fact, the iPad and iPhone 4 are very similar in hardware as well. The most significant difference is in RAM, where the iPhone has twice as much as the iPad.
Mentioning his or her experience on the iPad is completely valid.
iOS4 isn't on the iPad and won't be for several months.
That said, of course faster javascript is a benefit. Shaving even fractions of a second off web operations has huge benefits for users and results in much higher usage and happier customers.
When the performance bumps are imperceptible, it will be time to make a distinction. But on mobile devices that's still quite a ways out.
That said, of course faster javascript is a benefit. Shaving even fractions of a second off web operations has huge benefits for users and results in much higher usage and happier customers.
When the performance bumps are imperceptible, it will be time to make a distinction. But on mobile devices that's still quite a ways out.
And that said, the CPU of the iPad is believed to be 1GHz, where the iPhone 4 CPU is believed to be a 750MHz model.
Its likely that the experience will be faster on the iPad when iOS 4 comes out for it in October.
Its likely that the experience will be faster on the iPad when iOS 4 comes out for it in October.
Not to mention that a faster, more efficient JS engine means less time pegging the CPU while loading and interacting with a webpage, which means longer battery life for your mobile device.
I would probably recommend doing it the iPhone way - downloading one of the number of Netflix apps
It renders some web apps unusable. Try reorganizing your Netflix queue on the iPad. Netflix's JS doesn't scale well to a queue with hundreds of items. On a computer running a modern browser, this means a lag of a couple of seconds. But on the iPad, it can be s full minute.
The most interesting part of this will be how Gruber eventually spins this... can't wait to see that.
My guess is he'll be completely silent about it, UNTIL there's some future iOS update the lets iOS score higher in the same benchmark. Then it'll be an example of Apple's attention to detail and planning, and how Google doesn't care about their user experience. :-)
Exactly. Just like pixels per inch, where it didn't matter that the Droid and N1 were massively superior to the iPhone for months. Then the iPhone 4 comes out with slightly higher PPI than the Droid and now it's all about the magical retina display.
Sigh. That’s just not true. Here is what Gruber said about Nexus One’s screen (January 2010):
“The high pixel density of the display is marvelous for reading text. Letterforms are, as you’d expect, very crisp. My biggest gripe about the Nexus One display is that certain colors are way over-saturated. All skin tones look very orange to me. Everyone gets that spray-on tan look. Reds, pinks, and especially oranges all go fluorescent. In short, I love the pixel density and brightness (and, so far, the battery life), but I do not like the color reproduction. I don’t know if that’s the nature of OLED, or if it’s specific to the Nexus One.“
— http://daringfireball.net/linked/2010/01/19/nexus-one-oled
“The high pixel density of the display is marvelous for reading text. Letterforms are, as you’d expect, very crisp. My biggest gripe about the Nexus One display is that certain colors are way over-saturated. All skin tones look very orange to me. Everyone gets that spray-on tan look. Reds, pinks, and especially oranges all go fluorescent. In short, I love the pixel density and brightness (and, so far, the battery life), but I do not like the color reproduction. I don’t know if that’s the nature of OLED, or if it’s specific to the Nexus One.“
— http://daringfireball.net/linked/2010/01/19/nexus-one-oled
Gruber only very seldomly comments about data points like these unless they are a larger point. He takes most things from the perspective of a user and tends to ignore nerd whistles like this one.
Right now this information is cool (and an engineering triumph, yet another data point for my love of VMs and how they are the future of all modern computing), but put this in perspective: it only affects a very small number of users with an expensive and difficult-to-obtain handset. From the perspective of the average customer, it's pretty meaningless. Gruber probably won't respond until many phones have 2.2 and suddenly it's a public issue that "iPhone is slow."
Right now this information is cool (and an engineering triumph, yet another data point for my love of VMs and how they are the future of all modern computing), but put this in perspective: it only affects a very small number of users with an expensive and difficult-to-obtain handset. From the perspective of the average customer, it's pretty meaningless. Gruber probably won't respond until many phones have 2.2 and suddenly it's a public issue that "iPhone is slow."
Personally, I'd rather see a aggregate subjective ratings of "responsiveness" and "speed", rather than raw horsepower, as it were.
If the end goal is enhancing user experience, why not benchmark against those metrics?
If the end goal is enhancing user experience, why not benchmark against those metrics?
And what would be examples of benchmarking a subjective experience? I mean, a shitty JavaScript engineer can make any UX shitty regardless of speed of JavaScript tests, whereas a good JavaScript engineer can make a UX good with a crap interpreter.
Right, so it's entirely appropriate to benchmark arbitrary computational metrics, but benchmarking UI metrics is inconceivable?
I don't understand your argument - true shitty engineers can ruin a UI, as well as they can ruin a JS interpreter. The point of testing and benchmarking is to evaluate the relative "performance" in each of these domains.
Measuring page loads on an array of popular, representative websites, or measuring the latency between user interactions and some visible response are some examples.
Also, having a group of real people give subjective ratings I think carries more weight than people give it credit for.
I don't understand your argument - true shitty engineers can ruin a UI, as well as they can ruin a JS interpreter. The point of testing and benchmarking is to evaluate the relative "performance" in each of these domains.
Measuring page loads on an array of popular, representative websites, or measuring the latency between user interactions and some visible response are some examples.
Also, having a group of real people give subjective ratings I think carries more weight than people give it credit for.
it's not that hard to measure latency. point a video camera at the screen and measure how long it takes from click to completion of new screen load.
then pick ten web pages and collect the data.
then pick ten other common UI actions (not in the web browser) and collect that data too. there's some fudge here because the phones have different UIs so you won't necessarily be able to find identical actions but you could get a good idea.
it's entirely possible that the dev teams have instrumented builds that give them this info automatically. i would be surprised if they didn't, in fact.
then pick ten web pages and collect the data.
then pick ten other common UI actions (not in the web browser) and collect that data too. there's some fudge here because the phones have different UIs so you won't necessarily be able to find identical actions but you could get a good idea.
it's entirely possible that the dev teams have instrumented builds that give them this info automatically. i would be surprised if they didn't, in fact.
That's great because that's the main thing most people look at when they're buying a phone. Forget the display and the camera, what are the JavaScript benchmarks like?
Don't get me wrong, it's good that these sort of enhancements are made and that the platforms drive each other on in this area, but it's not actually going to sell any phones because the gap will never become big enough to become a genuine selling point.
Don't get me wrong, it's good that these sort of enhancements are made and that the platforms drive each other on in this area, but it's not actually going to sell any phones because the gap will never become big enough to become a genuine selling point.
Seems like a duplicate of [http://news.ycombinator.com/item?id=1493937]. I really wish there was a way to stop the blatant duplicate submission. This is of their feeds.arstechnica.com instead of the already submitted arstechnica.com article.
Maybe this is coincidental, but I am not sure anymore...
Maybe this is coincidental, but I am not sure anymore...
That would make for a good feature request.
On the right of flag we could add "merge with" and take a news.yc url and if enough people submitted the same url a mod could merge all the comments into the old story.
On the right of flag we could add "merge with" and take a news.yc url and if enough people submitted the same url a mod could merge all the comments into the old story.
I have been thinking of this for some time. A "merge with" would be great to help the clutter and then the good conversations that happen would not be lost once one is closed.
This isn't the duplicate, the other one is. This was submitted 2 hours ago (at the moment), the other 1 hour ago. I've flagged the other one, though.
And when the time appears the same (both at 2 hours at the moment), there is always the item number, 1493907 for this submission and 1493937 for the duplicate.
Is there something fundamentally different in the approach taken in Froyo that makes for this kind of speed improvement?
Everything has been enhanced by a JIT compiler for the apps. It should result in a 250-300% speed-up in real-world usage on most devices.
[deleted]
I won't be surprised if V8 is faster than the laggy Java Dalvik JIT VM in 2.2.
I don't care. iPhone 4. I want iPhone 4.
I get the reference. Funny. (if you haven't seen the video http://www.youtube.com/watch?v=FL7yD-0pqZg&feature=playe...)
I just benchmarked my Palm Pre against my collegue HTC Android phone, and the Pre beats the HTC hands down : V8 score 52 to 9...
You don't say which HTC phone it is, but I don't think any of them have 2.2 yet...
iOS 4 demolishes Android 2.2 in not responding to touch events like molasses
They made the screen sweet and sticky on Android devices in a _software_ update? That's even more impressive than the JavaScript optimizations.
got anything quantitative to back that up?
have you even run 2.2?
http://dl.dropbox.com/u/2316004/Mobile%20Photo%20Jul%207%2C%...
Had to take it with my iPhone 4, so it's not included in the picture. Before you ask, I'm a mobile developer.
Had to take it with my iPhone 4, so it's not included in the picture. Before you ask, I'm a mobile developer.
I don't know, iOS4 has made my 3G dog slow and laggy.
[deleted]
Yeah, Android still doesn't have the Apple logo so you can give it a chance (just saying; And getting downvoted)
Good for Android. They should be good at something. At least one thing.
People trying to beat them on spec never will because it's not the game they're playing (at least not right now). They're building and selling user experience; it's the sum of the parts that makes them great, not the quality or performance of individual parts.