Android Needs A Simulator, Not An Emulator(jakewharton.com)
jakewharton.com
Android Needs A Simulator, Not An Emulator
http://jakewharton.com/android-needs-a-simulator/
10 comments
The point of a simulator is it drives deployment time to zero and increases the number of virtual devices on which you can run without adding a new AVD for each. Deploying to a phone takes maybe 40-60 seconds? Maybe less? But I'm not a very patient person. At 60 seconds per deploy, if I make a tiny tweak and then have to wait a full minute for feedback on how it feels, it has become a slow and arduous process to develop. If I deploy to device and immediately discover a null-pointer exception, then that's 120 seconds of my day gone. The amount of time is even longer for an emulator, where it can be as long as five or ten minutes.
It is much much less. I just timed it on my machine. It was 8 seconds for a decent sized(~12M) app. A build and deploy takes much longer, but the long pole isn't the time it takes to install the app on the hardware. If you are having a lot of latency, I don't think a simulator will help.
This is still far longer than it should be.
I just built a not-particularly-large app after making a single-line change and it took 16 seconds to compile, dex, package and install on a running x86 emulator. The installation time is appreciable (4 seconds for my 10MB APK), which would be cut to zero for a simulator. Similarly, removing the dexing and packaging would be a great help.
I'm jealous when I see how fast my iOS colleagues can make code tweaks and see them running on a simulator one second later.
I just built a not-particularly-large app after making a single-line change and it took 16 seconds to compile, dex, package and install on a running x86 emulator. The installation time is appreciable (4 seconds for my 10MB APK), which would be cut to zero for a simulator. Similarly, removing the dexing and packaging would be a great help.
I'm jealous when I see how fast my iOS colleagues can make code tweaks and see them running on a simulator one second later.
I wonder if an Android device as a service would work. No, I don't mean loaning devices. I mean by the hour remote control of an Android device you can take over. Your end looks like the simulator, my end is a farm of Android devices which get a factory reset between every client connecting. You reserve it for as long as you want, or just do one-off testing.
Edit: I suppose I should ask a more concrete question: would you pay for a service like this? If not, what would you use instead?
Edit: I suppose I should ask a more concrete question: would you pay for a service like this? If not, what would you use instead?
"I wonder if an Android device as a service would work"
Doubtful it would work any better than the custom VM solution that Genymotion (mentioned in the article) does (for this specific pain-point).
I would still need to adb deploy my apk to the device (which is a process that really drags development even when doing it to local devices) and now its even worse because it is remote. Also this remoteness would add all sorts of new issues like that fact that if I'm developing a server in tandem with the client app, now I have to jump through hoops to make the server remotely accessible (so your service devices can see it) whereas otherwise I might just be running it locally in a non-publicly accessible way for development purposes (solvable problem, but still a lot of friction and this request for a simulator is all about removing such friction).
Android devices as a service is still a good idea for other things like testing & QA because while Android fragmentation isn't as bad as it used to be, you still do see device-specific bugs (or at least platform-specific, like TouchWiz bugs on Samsung devices that don't impact other versions of Android) quite often when developing and having access to specific devices would help in such situations, but I don't think it addresses any of the main problems that are driving this request for iOS-style simulation (which, FWIW, I totally agree with Jake Wharton on).
Doubtful it would work any better than the custom VM solution that Genymotion (mentioned in the article) does (for this specific pain-point).
I would still need to adb deploy my apk to the device (which is a process that really drags development even when doing it to local devices) and now its even worse because it is remote. Also this remoteness would add all sorts of new issues like that fact that if I'm developing a server in tandem with the client app, now I have to jump through hoops to make the server remotely accessible (so your service devices can see it) whereas otherwise I might just be running it locally in a non-publicly accessible way for development purposes (solvable problem, but still a lot of friction and this request for a simulator is all about removing such friction).
Android devices as a service is still a good idea for other things like testing & QA because while Android fragmentation isn't as bad as it used to be, you still do see device-specific bugs (or at least platform-specific, like TouchWiz bugs on Samsung devices that don't impact other versions of Android) quite often when developing and having access to specific devices would help in such situations, but I don't think it addresses any of the main problems that are driving this request for iOS-style simulation (which, FWIW, I totally agree with Jake Wharton on).
Genymotion has one problem for me, which is that things that rely on the device having an ARM processor cannot be tested, so I do like the idea and the suggestions for existing services by other people in this thread. Shall check some of them out.
The Samsung Remote Test Lab[1] provides this for free (with limits)— they have a range of real devices (S5—S2, Notes, Tabs, even Gears) that you remote into and can install APKs, screenshot, and otherwise have full control over.
Somewhat unfortunately, I've found the connection fairly unstable, and they don't factory reset between uses (so unless you remove your APK, it'll stay on the device and configurations persist).
[1]: http://developer.samsung.com/remotetestlab/rtlDeviceList.act...
Somewhat unfortunately, I've found the connection fairly unstable, and they don't factory reset between uses (so unless you remove your APK, it'll stay on the device and configurations persist).
[1]: http://developer.samsung.com/remotetestlab/rtlDeviceList.act...
There have been several attempts at device testing as a service. I don't know of any that have really thrived. To be interesting, some have tried offering testing in diverse locations. In theory, they could also move their testing rigs, heat and cool them, pressurize the room, and/or rotate them in order to test sensors.
There are some kinds of sensors with variable quality that some developers claim leads to a need for testing every device. I would write code that characterizes performance instead, and possibly blacklist some known bad devices.
There are some kinds of sensors with variable quality that some developers claim leads to a need for testing every device. I would write code that characterizes performance instead, and possibly blacklist some known bad devices.
This exists: http://www.keynote.com/solutions/testing/mobile-testing
I've used it fairly extensively, it can be a bit buggy and slow, but it was instrumental in tracking down a bug just after launching an Android app.
People were leaving 1-star reviews, saying the app simply hard crashed at launch. We couldn't replicate the issue in any way on any of our ~10 physical devices we used for testing, or in any VM, but, with the service, we were able to isolate it exclusively to a handful of Android 2.3.3 devices with less than 512mb of memory. After that it was much easier to track down the source of the bug, fix it, and stem the flow of bad reviews.
Android is so fragmented among versions and devices that in the future I'd always want to use a service like this to at the very least make sure the app opens and runs across a slew of devices before launching.
I've used it fairly extensively, it can be a bit buggy and slow, but it was instrumental in tracking down a bug just after launching an Android app.
People were leaving 1-star reviews, saying the app simply hard crashed at launch. We couldn't replicate the issue in any way on any of our ~10 physical devices we used for testing, or in any VM, but, with the service, we were able to isolate it exclusively to a handful of Android 2.3.3 devices with less than 512mb of memory. After that it was much easier to track down the source of the bug, fix it, and stem the flow of bad reviews.
Android is so fragmented among versions and devices that in the future I'd always want to use a service like this to at the very least make sure the app opens and runs across a slew of devices before launching.
There are a few of these out there including http://developer.samsung.com/remotetestlab/rtlDeviceList.act...
I like this idea as well, it seems fruitful enough that at least one organisation is providing it [0]. I would have thought overheads of getting new devices might cut into revenue though. I'd be interested to see what the difference in experience is for something on an emulated device (with HAXM) and a remote real device though, and if it's substantial enough to be worth paying for.
[0] : http://www.keynote.com/solutions/testing/mobile-testing
[0] : http://www.keynote.com/solutions/testing/mobile-testing
How would you avoid/handle abuses like using it as a proxy to access extremely illegal things?
How does AWS do that with their remote desktop in the cloud service? If you use this service, I have your name and credit card number.
You can do this with almost any service now where you can execute arbitrary code, and a lot easier than running it through an android board.
You might like Manymo -- our company -- it's Android emulators as a service. Every recent OS and screen size combo; launches in a couple of second. Accessible from a browser, or in ADB.
https://www.manymo.com/
https://www.manymo.com/
First party support for Robolectric would be wonderful. It is a nightmare any time you need something Robolectric doesn't implement, particularly in Context classes because there are so many different types of Context: Application, Context, ContextWrapper, ContextImpl, Activity, etc. and I've never gotten Robolectric's @Config and @Implementation annotations to successfully add to them without blowing up about weird super class not matching errors.
The only thing I'd like is maybe a gradle build for ADT. I like my workflow I just hate placing required android libs on the path by hand, so time consuming. I only test on devices so the emulator thing to me is really a non issue.
That said, I would love a "simulator" since it would probably mean that Google has wrapped the JDK with an Android interface which frees me from having to every use Swing/SWT/any other java gui toolkit here ever again.
That said, I would love a "simulator" since it would probably mean that Google has wrapped the JDK with an Android interface which frees me from having to every use Swing/SWT/any other java gui toolkit here ever again.
Are you doing your integration and instrumentation tests on actual devices as well?
I only do manual testing since Im a hobbyist android dev.
When I started a new job doing iOS development, I was a bit surprised that Apple provided a simulator and not an emulator for iPhone/iPad development. An emulator seemed like a better choice, because it would be more realistic. Realism turned out not to be a big issue, at least for the kind of app we were developing, and I guess iOS and Android devices are just too fast to emulate on a desktop with reasonable performance.
I'm not sure how necessary true emulation really is. The fact that you can easily plug a phone in and run there means that the emulator isn't as valuable so would be if it took 20 minutes of hoops to get your code on a device.
True, and I think Apple's simulator turned out to be the right move.
What platform(s) would the simulator support? Just GNU/Linux? Even that is quite different from actual Android in some ways, e.g. glibc instead of bionic. It's easy by comparison for Apple to provide a simulator, because the simulator only has to run on Mac OS X, which basically shares everything below the UI toolkit with iOS.
You're not forced to use glibc on GNU/Linux. You could link against bionic if you want, assuming it runs in a stock kernel.
[deleted]
When I develop in Android, I have this feeling that Google was thinking something along the lines "Its good enough and now we can just switch it into maintenance mode"
> "The most well known simulator to any Android developer is probably (and ironically) the one that iOS developers use from Apple."
I get it. Ironically Google didn't copy Apple for once, and their solution turned out bad. So now they need to ironically copy it, please, Google, pretty please. But the real irony is that Android developers both get to boast about Android being open source, yet when they need something, they go beg Google implement it for them.
And Google really can't implement an efficient iOS simulator for them, because there's no Android desktop operating system.
You see, while it's getting a bit old to hear Tim Cook brag at every keynote about how "only Apple can make everything work seamlessly together", truth is, they indeed share a kernel and over half their APIs between their desktop and mobile operating systems. This is why the iOS simulator is that good. Tim Cook knows it.
While a supposed "thin layer" Android simulator has to interface all its APIs with Windows and then do it all over again for OSX, and then again for Linux (common POSIX APIs notwithstanding), every version with its unique quirks and bugs, not matching the platform it's simulating.
So instead of that ball of pain, Google chose the easier task of virtualizing the actual hardware and then running the whole Android stack on top of that.
In their particular circumstances, this was, and remains the only feasible way of doing it, and no amount of blog whining will change that.
Turns out you really can't copy some things so easily.
I get it. Ironically Google didn't copy Apple for once, and their solution turned out bad. So now they need to ironically copy it, please, Google, pretty please. But the real irony is that Android developers both get to boast about Android being open source, yet when they need something, they go beg Google implement it for them.
And Google really can't implement an efficient iOS simulator for them, because there's no Android desktop operating system.
You see, while it's getting a bit old to hear Tim Cook brag at every keynote about how "only Apple can make everything work seamlessly together", truth is, they indeed share a kernel and over half their APIs between their desktop and mobile operating systems. This is why the iOS simulator is that good. Tim Cook knows it.
While a supposed "thin layer" Android simulator has to interface all its APIs with Windows and then do it all over again for OSX, and then again for Linux (common POSIX APIs notwithstanding), every version with its unique quirks and bugs, not matching the platform it's simulating.
So instead of that ball of pain, Google chose the easier task of virtualizing the actual hardware and then running the whole Android stack on top of that.
In their particular circumstances, this was, and remains the only feasible way of doing it, and no amount of blog whining will change that.
Turns out you really can't copy some things so easily.
> Google really can't implement an efficient iOS simulator for them, because there's no Android desktop operating system.
The x86 Android build provided by GenyMotion (cited in the article) performs at the same speed as the x86 iPhone simulator.
It has two major issues:
- The UI is quirky (but usable)
- It's expensive
- GenyMotion devices have no Google Play by default. Want to test how your website runs in Chrome? Spend ages hacking Google Play onto the Android device.
Google could fix all of these things easily.
The x86 Android build provided by GenyMotion (cited in the article) performs at the same speed as the x86 iPhone simulator.
It has two major issues:
- The UI is quirky (but usable)
- It's expensive
- GenyMotion devices have no Google Play by default. Want to test how your website runs in Chrome? Spend ages hacking Google Play onto the Android device.
Google could fix all of these things easily.
The x86 images provided by Google run just as fast as the Genymotion images, and have Google Play. Just FYI.
Thanks for the suggestion. I spent an hour setting these up today - despite supposedly using x64, HAXE etc they're unusably slow. I have no idea what's wrong. The UI tools run at unacceptable latency (they're Java based) so I can't debug easily.
This is on a 2013 Haswell MBA with 8GB RAM and a 512GB SSD.
Google may as well throw the official emulator away.
This is on a 2013 Haswell MBA with 8GB RAM and a 512GB SSD.
Google may as well throw the official emulator away.
When you run the emulator, do you get this message (on stdout): "HAX is working and emulator runs in fast virt mode"?
Did you enable the "Use host GPU" option?
Did you enable the "Use host GPU" option?
Have you guys heard of GennyMotion?
It's way faster than the actual Android Emulator.
My Professor for my Android Class in college recommended it to use for debugging for our assignments if we didn't have an Android Device.
http://www.genymotion.com/
http://www.genymotion.com/
Genymotion is mentioned in the article, along with the reasons why it's not ideal (aside from the fact that it's not a first-party solution, and that you can't get a version with the Google SDKs running).
> You can't get a version with the Google SDKs running
Technically you can, it's just hard to do and should be there by default (I very much agree in spirit).
Technically you can, it's just hard to do and should be there by default (I very much agree in spirit).
Wow, that page has a 2.5mb png as a background image on one of the elements.
web 4.0
It's mentioned in the article, but the downsides are almost all due to the licensing and pricing model. Doesn't really make it a bad solution from a technical standpoint (and for others to do something similar).
To me Genymotion is what Android "should" have, and it's already here.
To me Genymotion is what Android "should" have, and it's already here.
Genymotion is the way to go. Google's emulator plain sucks.
I guess I don't see the point of a simulator either. If the performance is that bad, why not just use hardware?