Developing a mobile cross-platform library in C++(skyscanner.net)
skyscanner.net
Developing a mobile cross-platform library in C++
http://www.skyscanner.net/blogs/developing-mobile-cross-platform-library-part-2-c
15 comments
Yeah I think Python would have been a good choice. Something that looks like kivy perhaps.
http://kivy.org/#home
Java is just too popular and too average. Has enough people knowing it, works well with an IDE, compiles (and thus hides the source better).
The apps I wrote for example, was more about the API and editing XML files. I've used Java before but in this case it felt different, I would say easier. Not a lot of FactoryFactoryManager type mess many stereotypically associate with Enterprise Java back-ends. So it wasn't too bad.
Another interesting point. At one of the PyCons someone asked Guido publicly (he was still at Google back then), why doesn't Android use Python more? And Guido replied it was a "political decision". Moreover the tone he replied it was betraying that he was a bit upset or annoyed at it. So I imagine there was some internal discussion, perhaps more than one and more than once, and one camp one and one lost.
http://kivy.org/#home
Java is just too popular and too average. Has enough people knowing it, works well with an IDE, compiles (and thus hides the source better).
The apps I wrote for example, was more about the API and editing XML files. I've used Java before but in this case it felt different, I would say easier. Not a lot of FactoryFactoryManager type mess many stereotypically associate with Enterprise Java back-ends. So it wasn't too bad.
Another interesting point. At one of the PyCons someone asked Guido publicly (he was still at Google back then), why doesn't Android use Python more? And Guido replied it was a "political decision". Moreover the tone he replied it was betraying that he was a bit upset or annoyed at it. So I imagine there was some internal discussion, perhaps more than one and more than once, and one camp one and one lost.
I would really love a Python API on Android. Unfortunately, Python as it exists now is way too inefficient to use on a mobile phone. Have you tried running a Kivy app for more than a few minutes and watched your battery run down?
Don't get me wrong -- that's not Kivy's fault really. It's the Python interpreter that's eating up the battery, not Kivy per se. And I'd like to say that there's hope in Pypy, but for the fact that PyPy uses up to twice the memory of regular Python.
Now, there's no reason why Python couldn't efficiently run on mobile, but it'd have to be with an entirely different interpreter, or at least a significantly modified CPython. One that uses a register-based VM instead of a stack-based one, and makes a number of other bytecode and VM changes toward greater efficiency.
Don't get me wrong -- that's not Kivy's fault really. It's the Python interpreter that's eating up the battery, not Kivy per se. And I'd like to say that there's hope in Pypy, but for the fact that PyPy uses up to twice the memory of regular Python.
Now, there's no reason why Python couldn't efficiently run on mobile, but it'd have to be with an entirely different interpreter, or at least a significantly modified CPython. One that uses a register-based VM instead of a stack-based one, and makes a number of other bytecode and VM changes toward greater efficiency.
I would love it if Tcl were a first class option. W/o getting into Tcl vs. Lua vs. Python vs. Ruby vs. Perl vs. Chicken Scheme vs... if the underlying platform really were C, all these things would be viable options. C would have let all these flourish.
Worse, for the Java fans like myself, they really forked Java.
So now Java 8 is around the corner and Android is still playing catchup with Java 7, which is only fully supported on KitKat.
So I end up going with C++, since what I care about is covered by libraries like SDL and openframeworks.
They had a talk about ART planned for FOSDEM and it was:
- cancelled at last minute
- Google was also not letting anyone film the talks on the Java room.
http://neugens.wordpress.com/2014/02/09/fosdem-2014/#comment...
So now Java 8 is around the corner and Android is still playing catchup with Java 7, which is only fully supported on KitKat.
So I end up going with C++, since what I care about is covered by libraries like SDL and openframeworks.
They had a talk about ART planned for FOSDEM and it was:
- cancelled at last minute
- Google was also not letting anyone film the talks on the Java room.
http://neugens.wordpress.com/2014/02/09/fosdem-2014/#comment...
We agree on with native and opening up the platform at Apportable. It's why we built Apportable. It provides everything you expect from iOS on top of Android and makes it just as open for mixing code. We try to fill in the gaps that people with the native experience on Android.
Apportable's promise is huge. The prospect of being able to compile the same project for both iOS and Android makes me want to jump. But the platform is not quite there yet, as far as I know.
Without divulging anything, I can safely say it's responsible for at least 32 apps that made it in the Google Play Top 100 games last year, so it's doing pretty good.
For anyone else that wants to focus on iOS where they will make the most money but not compromise on Android and provide it with the same experience without having to rewrite everything, it's working great. I will say that if you are building a game it works out of the box today better but we have a lot of people using in ways we could never imagined.
The newly rewritten UIKit support is coming soon which will make it a viable solution for anyone that wants a single push button solution for all kinds of apps. For a few users, they choose to use the native UI on Android and bind that to their Objective-C/C++ backends with Apportable. There are a lot of options with Apportable.
For anyone else that wants to focus on iOS where they will make the most money but not compromise on Android and provide it with the same experience without having to rewrite everything, it's working great. I will say that if you are building a game it works out of the box today better but we have a lot of people using in ways we could never imagined.
The newly rewritten UIKit support is coming soon which will make it a viable solution for anyone that wants a single push button solution for all kinds of apps. For a few users, they choose to use the native UI on Android and bind that to their Objective-C/C++ backends with Apportable. There are a lot of options with Apportable.
Apportable doesn't let you build libraries. You can only translate whole iOS apps to Android.
It actually does let you build libraries, but those libraries also depend on Apportable's improved libc and libc++ environment and other libs you link. Admittedly the documentation is lacking how to do that exactly. That will improve soon.
I have written a good chuck of common code in C++ for my diagramming app "Lekh Diagram"(http://www.avabodh.com/lekh). The shape recognition, shape management etc all done in C++ code. The app is available for Android and iOS.
Integrating the C++ code with Objective C is much smoother experience than the Java in android.
As the Objective C (more precisely objective C++) is superset of C++, I can directly call C++ code from objective C and for other way communication (calling objective C code from C++), I had defined an C++ interface. The C++ code calls on interface only and the interface is implemented in objective C.
For Android, little more work was required. I had write wrappers code. I also had to hit many stupid issues like local reference table overflow etc.
Integrating the C++ code with Objective C is much smoother experience than the Java in android.
As the Objective C (more precisely objective C++) is superset of C++, I can directly call C++ code from objective C and for other way communication (calling objective C code from C++), I had defined an C++ interface. The C++ code calls on interface only and the interface is implemented in objective C.
For Android, little more work was required. I had write wrappers code. I also had to hit many stupid issues like local reference table overflow etc.
The shape recognition from drawings looks great! I've been looking to use something like this in a project of mine, any pointers you would like to provide, if possible?
Objective-C also runs on Android with Apportable. Less of a headache. :-)
It might, but unfortunately, you can't build a library with it. It only transforms whole apps. Apportable was investigated in the first part here http://www.skyscanner.net/blogs/developing-mobile-cross-plat...
Not once did the article talk about the C++ ABI which is one of the most important things a library developer needs to be aware of (as opposed to an application developer).
The C++ ABI is unstable. You can't effectively deploy 3rd party libraries with that either.
Android ships with multiple versions of the compiler, each one of those are incompatible with each other's standard library. On top of that, Android ships multiple C++ standard libraries for each compiler, all of which are not-interoperable either. Then Android updates the compiler with each NDK drop introducing more potential incompatibilities.
The end result is you can't deploy prebuilt C++ libraries (a prickly issue for commercial/proprietary code) unless you can force users to use your exact toolchain. But library writers usually can't fairly dictate those terms, especially if the users want intermix libraries from other sources which may use a different toolchain.
The C++ ABI is unstable. You can't effectively deploy 3rd party libraries with that either.
Android ships with multiple versions of the compiler, each one of those are incompatible with each other's standard library. On top of that, Android ships multiple C++ standard libraries for each compiler, all of which are not-interoperable either. Then Android updates the compiler with each NDK drop introducing more potential incompatibilities.
The end result is you can't deploy prebuilt C++ libraries (a prickly issue for commercial/proprietary code) unless you can force users to use your exact toolchain. But library writers usually can't fairly dictate those terms, especially if the users want intermix libraries from other sources which may use a different toolchain.
It does actually let you build libraries, just not entirely documented.
You are right though. Those libraries will link Apportable's extended/improved libc (called libv) and libc++ and would require distributing more parts of the platform (it can get a little hairy there). It is also true that the out of the box experience is one that is meant for just apps but it's built on a layer cake and at a lower level you can use it to compile libraries. This is how it builds dependencies in your Xcode project today.
You are right though. Those libraries will link Apportable's extended/improved libc (called libv) and libc++ and would require distributing more parts of the platform (it can get a little hairy there). It is also true that the out of the box experience is one that is meant for just apps but it's built on a layer cake and at a lower level you can use it to compile libraries. This is how it builds dependencies in your Xcode project today.
Or you know you can use Apportable and write Objective-C which is entirely cross-platform. :-) Don't have to sell your soul to C++ if you don't want too and you don't have to deal with the NDK.
If really want to write C++, Apportable also is the only place you will get full C++11 support (using libc++) and the same clang as iOS to make your life easier with just a little bit of constancy.
It will seriously make your life easier (the big thing is not having to maintain Makefiles for Android).
If really want to write C++, Apportable also is the only place you will get full C++11 support (using libc++) and the same clang as iOS to make your life easier with just a little bit of constancy.
It will seriously make your life easier (the big thing is not having to maintain Makefiles for Android).
If you can use libc++, what's stopping me from doing the same? Why should I use Apportable instead of just heading over to clang's website and downloading/building/linking it myself?
It won't work out of the box. There is actually a project in the AOSP world that attempted to get it working but it's labeled experimental and it fails half the tests. There are lot of missing features in Bionic (Android's libc) that libc++ needs that we provide in Apportable and doing so we are able to pass all the tests.
Apportable is more than just C++ though. It's about building a better NDK or rather a platform that makes things consistent across the board with other native environments like iOS. Native in Android is an afterthought which we aim to change. We fix things like libc, C++, and provide you with a lot of low level libraries you expect would be included.
Apportable also is the only place you are going to find a native profiler today (baring awful hacks) or will get the latest version of clang that run on Android either.
The idea is to save you time with inconstancies and make it easier to deal with building native code for both (you can even use the same Xcode project and target both from there).
We also support Objective-C and everything that you get from iOS so you can even bring that code over.
Apportable is more than just C++ though. It's about building a better NDK or rather a platform that makes things consistent across the board with other native environments like iOS. Native in Android is an afterthought which we aim to change. We fix things like libc, C++, and provide you with a lot of low level libraries you expect would be included.
Apportable also is the only place you are going to find a native profiler today (baring awful hacks) or will get the latest version of clang that run on Android either.
The idea is to save you time with inconstancies and make it easier to deal with building native code for both (you can even use the same Xcode project and target both from there).
We also support Objective-C and everything that you get from iOS so you can even bring that code over.
Thanks for the detailed response. I'll definitely give you guys a try.
I made a painless way to integrate native libraries into Java (Android NDK support too) with some automatic JNI generation [1]. It uses GWT's javascript comment syntax (edit: see [2] for examples).
[1] https://bitbucket.org/wetherbeei/acceljava
[2] https://bitbucket.org/wetherbeei/acceljava/src/4140d721c11b5...
[1] https://bitbucket.org/wetherbeei/acceljava
[2] https://bitbucket.org/wetherbeei/acceljava/src/4140d721c11b5...
Most game engines are built as C/C++ and wrapped with Objective-C, Java, etc wrappers. It is really easy with games because you just need an OpenGL context and some basic views, everything else can happen in the render. When you have to add native UI and integrate with libs like StoreKit, GameCenter, audio etc those are also wrapped.
Still the best underlying tech is C/C++ for games. Even top engines like Unity export to this format. It would be entirely limiting to build an entire engine in Objective-C or Java for mobile multiplatform. C/C++ can be used everywhere with a little bit of wrapping.
Still the best underlying tech is C/C++ for games. Even top engines like Unity export to this format. It would be entirely limiting to build an entire engine in Objective-C or Java for mobile multiplatform. C/C++ can be used everywhere with a little bit of wrapping.
I'm not quite sure I understand his objections to Xamarin. He says, "It can only output to intermediate Xamarin libraries, not native ones."
On iOS Xamarin does compile down to native code and on Android is compiled to dex format. They are not interpreted on those platforms.
Xamarin can utilize PCL assemblies, so non-ui code could be written once and used multiple places without recompilation. This is actually very handy since those libraries can be built on Windows, where the toolchain is better (IMO VS is better than XS) and the system is more powerful.
On iOS Xamarin does compile down to native code and on Android is compiled to dex format. They are not interpreted on those platforms.
Xamarin can utilize PCL assemblies, so non-ui code could be written once and used multiple places without recompilation. This is actually very handy since those libraries can be built on Windows, where the toolchain is better (IMO VS is better than XS) and the system is more powerful.
I'm sorry if this wasn't clear. The purpose of building this library was to have something which could be distributable to any mobile developer, not just the ones using Xamarin. If you did it through Xamarin, you could only give it to Xamarin developers, which is very limiting. We need something that could be importable by most developers, especially native SDK ones.
We are developing a game engine in C++ that compiles and runs on iOS, Android, Windows and OSX.
We use the Marmalade SDK and though there are some issues with it we are actually running the builds on all these environments without issues which is pretty cool.
And if Marmalade goes under we can still port the lower layers to OpenGL.
Just because support for multiple platforms is the goal doesn't mean that it has to share the same code base.
Actually, for the author's simple use case (REST API), writing per-platform would probably have taken as much code as the wrappers themselves. Not to mention improved maintainability and a slew of other factors.
Diving down to C++ for cross-platform libraries starts to make sense when there are 4-5+ platforms (certainly not 2), or if there are complex algorithms (certainly not just a REST API), or if there's a very wide range of functionality (i.e. 100+ API calls).
Actually, for the author's simple use case (REST API), writing per-platform would probably have taken as much code as the wrappers themselves. Not to mention improved maintainability and a slew of other factors.
Diving down to C++ for cross-platform libraries starts to make sense when there are 4-5+ platforms (certainly not 2), or if there are complex algorithms (certainly not just a REST API), or if there's a very wide range of functionality (i.e. 100+ API calls).
C++ is currently my language to go on mobile development, even if I nowadays prefer GC enabled languages, because of:
- experience with C++ since the ARM was published
- only language common to all mobile OS SDKs and I not want to rely on third party toolchains
- C++11 makes it feel refreshing (except for compile times :\ )
- I am only doing portable graphics related stuff as an hobby
- RoboVM would be nice, but it is still kind of work in progress
In the OP case, you are right, but it could also be compensated with third party libraries like Qt, instead of the DYI approach that was taken.
However with bigger teams with disparate skillsets, maybe something like Mono would be a better approach.
- experience with C++ since the ARM was published
- only language common to all mobile OS SDKs and I not want to rely on third party toolchains
- C++11 makes it feel refreshing (except for compile times :\ )
- I am only doing portable graphics related stuff as an hobby
- RoboVM would be nice, but it is still kind of work in progress
In the OP case, you are right, but it could also be compensated with third party libraries like Qt, instead of the DYI approach that was taken.
However with bigger teams with disparate skillsets, maybe something like Mono would be a better approach.
I hope you also support non ARM Android devices like Samsung Galaxy Tab 3 (Anrdoid 4x on Intel Atom x86) and there are also devices with MIPS CPU.
So far only hobby stuff, so I only target my own devices, but it is quite easy to support multiple devices.
I am a bit older than C++, so I do know how to write portable code before we had VMs everywhere. :)
I am a bit older than C++, so I do know how to write portable code before we had VMs everywhere. :)
I think if you want to write xplatform c++, Qt is the way to go. Qt also has QML that solves all the pains of flexible dynamic UI framework.
I have a decent code base written in C/C++. Now as I am thinking about mobile apps, I wonder if it is worth all the hassle on Andriod to go through all of this or if it is just better to do a rewrite in Java. Does anyone have any experience about this?
That depends on what your "decent code base" is, what it is meant to accomplish, how much coverage of your library is going to exist in your mobile app, whether (and how often) you will actually re-use the core libraries, etc.
Realistically, what the OP is doing is pushing the maintenance and platform-specific code into a different part of the stack. In either case, code in different languages, with different native-interface specifications and idioms, is required to be written and maintained. Unless you have some certainty that the maintenance of the multiple different platform wrapper code bases will be less laborious than simply writing platform-specific code, it's probably not worth it.
In the worst case, for example, a new function in C or C++ that you write may have to have one (or more!) new functions written in the platform-specific wrapper code in order to utilize it.
Realistically, what the OP is doing is pushing the maintenance and platform-specific code into a different part of the stack. In either case, code in different languages, with different native-interface specifications and idioms, is required to be written and maintained. Unless you have some certainty that the maintenance of the multiple different platform wrapper code bases will be less laborious than simply writing platform-specific code, it's probably not worth it.
In the worst case, for example, a new function in C or C++ that you write may have to have one (or more!) new functions written in the platform-specific wrapper code in order to utilize it.
Have you tried to use the latest Qt to integrate your code in mobile apps ?
Why use explicit memory management in the iOS wrapper (delete in dealloc) when ARC in conjunction with std::shared_ptr would take care of it automatically?
In iOS, things are simple. The deallocating code is very concise and safe. Why having the performance overhead of smart pointers?
On the other hand, it could be useful to have them in Java and avoid manual destruction. However, this wasn't investigated :(
link to Google cache...
http://webcache.googleusercontent.com/search?q=cache:Y2AE0GA...
http://webcache.googleusercontent.com/search?q=cache:Y2AE0GA...
it's back up
Great! Let's do the same thing for Browser Plugins now :)
Interesting.
this is not open sourced?
No, unfortunately, it needs polishing before open-sourcing it. If there's enough demand, I might consider it...
I wish they had considered just making the top layer in Luajit, as well as exposing a C API. Luajit would have been just as efficient battery-wise as Dalvik, and perhaps even better-performing. Best of all, people then could have much more easily built libraries for Python, Ruby, and any other language with good C interop.
Instead, we're left with Java. It's not a bad language, but it is very bad at integrating with other languages and platforms.
I prefer Android to iOS, but the iOS architecture is definitely more open to third-party platforms. Unfortunately, Apple is not so welcoming and puts up roadblocks.