Swift-X tool to cross compile Swift and Objective-C(++) code to Android(swift-x.org)
swift-x.org
Swift-X tool to cross compile Swift and Objective-C(++) code to Android
http://swift-x.org/?p=17
7 comments
(This is a reply to this whole sub-thread, not just this comment.)
Most well-known approaches to cross-platform mobile app development have left me dissatisfied. The root problem is the required bridging overhead between the cross-platform code and the platform's native runtime (the ObjC runtime on iOS; Dalvik/ART on Android). Using C/C++ for the cross-platform code is the lowest overhead approach on iOS, but it's suboptimal on Android.
So why does this matter in practice? A few reasons:
Multiple coexisting memory management systems: In general, my understanding is that a garbage collector behaves best when all memory, or very near all of it, is under the collector's control. My fuzzy intuition is that the least optimal situation is two coexisting garbage collectors, e.g. Xamarin.Android or a JavaScript engine other than Rhino on Android. But mixing a garbage collector with reference counting or manual memory management, such as with native code on Android or RoboVM or Xamarin.iOS, also seems to be suboptimal. If a significant amount of memory is allocated outside the scope of the GC, then the GC doesn't have as much information with which to intelligently decide when and how much to collect. Also, in hybrid systems like the ones being discussed here, objects under the GC's control sometimes have to be pinned so they can be accessed by the outside code; this interferes with copying or compacting collectors.
More information:
http://developer.android.com/training/articles/perf-tips.htm... (see the section "Use Native Methods Carefully")
http://developer.xamarin.com/guides/android/advanced_topics/...
http://developer.xamarin.com/guides/cross-platform/applicati...
http://www.virtualdub.org/blog/pivot/entry.php?id=360
Basically, having two coexisting memory management systems is messy. I'd rather avoid it.
Bridging adds overhead: Consider the cost of copying strings between the cross-platform component and the native string implementation, or mapping an array of cross-platform objects to a native array of wrappers so the objects can be used from UI code. There's always some overhead for marshaling. Maybe it's not much in practice, and I'm just being anal. But just knowing that it's there bothers me. The best way to keep software fast is simply to minimize the amount of pointless crap it has to do. So it seems best to minimize indirection and overhead by having just one implementation of strings, collections, etc.
Heavy use of external functions obstructs whole-program optimization: The Android performance tips document referenced above mentions this in the context of JNI methods and JIT compilation. I imagine the same applies to ART's AOT compilation, since the native methods are still opaque to the compiler, unlike the Dex bytecode. And for combinations such as Xamarin.Android or a JS engine other than Rhino on Android, there are two JIT compilers working suboptimally, because neither of them can see into the other's world.
Hybrid setups thwart obfuscation on Android: It's common for Android developers to use ProGuard (or its commercial big brother DexGuard) to shrink, obfuscate, and optimize their apps. Having a significant non-Java component in the app seriously limits the effectiveness of this practice.
Hybrid setups limit the usefulness of debuggers: If you're mixing Java and native code on Android, or using Xamarin, RoboVM, or a JavaScript engine, can you get a single useful stack trace across the platform-specific and cross-platform components when something goes wrong, as it inevitably will?
So are we doomed to rewrite the same common logic in Objective-C for iOS, Java for Android, JavaScript for client-side web apps, and (C#|JavaScript|C++) for Windows Phone, if we want that code to integrate well with the host platform? Luckily, no.
RubyMotion has the right idea, I think, and there's apparently an Android version on the way, but nothing for Windows Phone/WinRT or client-side web apps.
Another option is to write common code in Java, then use J2ObjC for iOS, IKVM.NET for Windows and Windows Phone, and GWT for the Web. Some teams within Google are doing this (minus the part about IKVM.net on Windows, AFAIK), including the team that developed J2ObjC. This approach certainly doesn't have the polish of a commercial product like RubyMotion or the Xamarin tools. But on the other hand, all the tools are free, so this should appeal to cash-strapped indie developers. One thing I don't like about J2ObjC is that the JRE emulation library has a larger footprint than I'd like. On a more aesthetic level, I don't like the fact that this approach takes Java, which is designed to be a platform unto itself, and tries to make it work on other host platforms.
And then there's RemObjects Elements (http://www.remobjects.com/elements/), a compiler toolchain with two language front-ends and three platform back-ends. On the language side, we have a choice of C# or Oxygene (a language based on Object Pascal). Either of these languages can be compiled to .NET bytecode, JVM bytecode (and from there to Dalvik bytecode for Android), or native code for the Objective-C runtime. (Note that the C# language front-end only provides the language; it doesn't attempt to bring the .NET class library to the other platforms.) Each platform back-end uses the target platform's preferred memory management system. That means GC for .NET and Java, and automatic reference counting for the Objective-C runtime. There's a cross-platform standard library called Sugar for things like collections, date/time manipulation, making HTTP requests, and XML, but emphatically not UI. Sugar has a very small footprint; in fact, many Sugar classes and methods are mapped to their platform-provided counterparts at compile time. Like the Xamarin suite, RemObjects C# and Oxygene are commercial products, but the pricing is very reasonable. I'm looking forward to trying this toolchain on a real project, probably using the C# front-end.
I know this has been a long comment; it covers a topic I've been thinking about a lot for more than a year. I know a lot of developers will just accept the added complexity and inefficiency I've described, for the sake of getting things done, i.e. accepting complexity and runtime inefficiency in exchange for developer efficiency. But then, added complexity translates to developer inefficiency at some point, especially when something goes wrong. And inflicting runtime inefficiency feels wrong when the machines in question aren't ours. That is, the trade-off between programmer time and machine time is different for client-side apps than for server-side apps. Luckily, it appears we can have the best of all worlds, especially with the RemObjects products, at least if we're willing and able to pay for excellent tools and accept that some of those tools will be closed-source.
(I guess readers will have to trust that this whole comment isn't actually a marketing piece for RemObjects. I just discovered the RemObjects products last night, and they don't seem to be well known, so I'm excited about them.)
Most well-known approaches to cross-platform mobile app development have left me dissatisfied. The root problem is the required bridging overhead between the cross-platform code and the platform's native runtime (the ObjC runtime on iOS; Dalvik/ART on Android). Using C/C++ for the cross-platform code is the lowest overhead approach on iOS, but it's suboptimal on Android.
So why does this matter in practice? A few reasons:
Multiple coexisting memory management systems: In general, my understanding is that a garbage collector behaves best when all memory, or very near all of it, is under the collector's control. My fuzzy intuition is that the least optimal situation is two coexisting garbage collectors, e.g. Xamarin.Android or a JavaScript engine other than Rhino on Android. But mixing a garbage collector with reference counting or manual memory management, such as with native code on Android or RoboVM or Xamarin.iOS, also seems to be suboptimal. If a significant amount of memory is allocated outside the scope of the GC, then the GC doesn't have as much information with which to intelligently decide when and how much to collect. Also, in hybrid systems like the ones being discussed here, objects under the GC's control sometimes have to be pinned so they can be accessed by the outside code; this interferes with copying or compacting collectors.
More information:
http://developer.android.com/training/articles/perf-tips.htm... (see the section "Use Native Methods Carefully")
http://developer.xamarin.com/guides/android/advanced_topics/...
http://developer.xamarin.com/guides/cross-platform/applicati...
http://www.virtualdub.org/blog/pivot/entry.php?id=360
Basically, having two coexisting memory management systems is messy. I'd rather avoid it.
Bridging adds overhead: Consider the cost of copying strings between the cross-platform component and the native string implementation, or mapping an array of cross-platform objects to a native array of wrappers so the objects can be used from UI code. There's always some overhead for marshaling. Maybe it's not much in practice, and I'm just being anal. But just knowing that it's there bothers me. The best way to keep software fast is simply to minimize the amount of pointless crap it has to do. So it seems best to minimize indirection and overhead by having just one implementation of strings, collections, etc.
Heavy use of external functions obstructs whole-program optimization: The Android performance tips document referenced above mentions this in the context of JNI methods and JIT compilation. I imagine the same applies to ART's AOT compilation, since the native methods are still opaque to the compiler, unlike the Dex bytecode. And for combinations such as Xamarin.Android or a JS engine other than Rhino on Android, there are two JIT compilers working suboptimally, because neither of them can see into the other's world.
Hybrid setups thwart obfuscation on Android: It's common for Android developers to use ProGuard (or its commercial big brother DexGuard) to shrink, obfuscate, and optimize their apps. Having a significant non-Java component in the app seriously limits the effectiveness of this practice.
Hybrid setups limit the usefulness of debuggers: If you're mixing Java and native code on Android, or using Xamarin, RoboVM, or a JavaScript engine, can you get a single useful stack trace across the platform-specific and cross-platform components when something goes wrong, as it inevitably will?
So are we doomed to rewrite the same common logic in Objective-C for iOS, Java for Android, JavaScript for client-side web apps, and (C#|JavaScript|C++) for Windows Phone, if we want that code to integrate well with the host platform? Luckily, no.
RubyMotion has the right idea, I think, and there's apparently an Android version on the way, but nothing for Windows Phone/WinRT or client-side web apps.
Another option is to write common code in Java, then use J2ObjC for iOS, IKVM.NET for Windows and Windows Phone, and GWT for the Web. Some teams within Google are doing this (minus the part about IKVM.net on Windows, AFAIK), including the team that developed J2ObjC. This approach certainly doesn't have the polish of a commercial product like RubyMotion or the Xamarin tools. But on the other hand, all the tools are free, so this should appeal to cash-strapped indie developers. One thing I don't like about J2ObjC is that the JRE emulation library has a larger footprint than I'd like. On a more aesthetic level, I don't like the fact that this approach takes Java, which is designed to be a platform unto itself, and tries to make it work on other host platforms.
And then there's RemObjects Elements (http://www.remobjects.com/elements/), a compiler toolchain with two language front-ends and three platform back-ends. On the language side, we have a choice of C# or Oxygene (a language based on Object Pascal). Either of these languages can be compiled to .NET bytecode, JVM bytecode (and from there to Dalvik bytecode for Android), or native code for the Objective-C runtime. (Note that the C# language front-end only provides the language; it doesn't attempt to bring the .NET class library to the other platforms.) Each platform back-end uses the target platform's preferred memory management system. That means GC for .NET and Java, and automatic reference counting for the Objective-C runtime. There's a cross-platform standard library called Sugar for things like collections, date/time manipulation, making HTTP requests, and XML, but emphatically not UI. Sugar has a very small footprint; in fact, many Sugar classes and methods are mapped to their platform-provided counterparts at compile time. Like the Xamarin suite, RemObjects C# and Oxygene are commercial products, but the pricing is very reasonable. I'm looking forward to trying this toolchain on a real project, probably using the C# front-end.
I know this has been a long comment; it covers a topic I've been thinking about a lot for more than a year. I know a lot of developers will just accept the added complexity and inefficiency I've described, for the sake of getting things done, i.e. accepting complexity and runtime inefficiency in exchange for developer efficiency. But then, added complexity translates to developer inefficiency at some point, especially when something goes wrong. And inflicting runtime inefficiency feels wrong when the machines in question aren't ours. That is, the trade-off between programmer time and machine time is different for client-side apps than for server-side apps. Luckily, it appears we can have the best of all worlds, especially with the RemObjects products, at least if we're willing and able to pay for excellent tools and accept that some of those tools will be closed-source.
(I guess readers will have to trust that this whole comment isn't actually a marketing piece for RemObjects. I just discovered the RemObjects products last night, and they don't seem to be well known, so I'm excited about them.)
Having worked on mobile code sharing a lot at Dropbox, I would say that the bridging overhead may not be a huge issue depending on how you structure your code. The approach we took was to keep all the UI completely native in the platform's native language, and put only the networking/database/model logic in the shared code. This means that typically you only have to cross the bridging layer when you hit disk or network, which is already going to be heavily IO bound anyway.
I've heard of companies being successful using C# as the shared language, but in our case we wanted to distribute SDKs, so we picked C++ since it wouldn't require us to include a runtime with the SDK. It's definitely not the easiest way to do things though if you're not already familiar with C++.
I'm not as familiar with Android compared to iOS, but ease of debugging is definitely a real problem with C++ on Android. I haven't tried very many of the other cross-platforms options, but would consider ease of debugging as one of my highest priorities since it's one of the hardest things to engineer around.
I've heard of companies being successful using C# as the shared language, but in our case we wanted to distribute SDKs, so we picked C++ since it wouldn't require us to include a runtime with the SDK. It's definitely not the easiest way to do things though if you're not already familiar with C++.
I'm not as familiar with Android compared to iOS, but ease of debugging is definitely a real problem with C++ on Android. I haven't tried very many of the other cross-platforms options, but would consider ease of debugging as one of my highest priorities since it's one of the hardest things to engineer around.
Another viable — and possibly more enjoyable — approach would be to use C# for the full stack, irrespective of platform. See http://xamarin.com
No offense against C# (I like the language) but Xamarin is a commercial product https://store.xamarin.com/ which makes you dependent on Xamarin. Xamarin is definitely more advanced and I guess you can also do cross compiling UI apps. But before going the Xamarin road I would look more into QT.
> But before going the Xamarin road I would look more into QT.
Which is also commercial, unless you plan to release an open source app.
Which is also commercial, unless you plan to release an open source app.
Qt has been available under the LGPL for several years now.
Static linking for iOS generally means an Qt enterprise license for proprietary / closed applications.
Obviously every situation is unique.
Obviously every situation is unique.
Not all mobile platforms allow for dynamic linking of third party code.
Digia also recently changed the license to LGPL3, which has some implications in embedded platforms, like mobiles.
Personally, I am the opinion when one gets money from others work, it should pay back in some form.
Digia also recently changed the license to LGPL3, which has some implications in embedded platforms, like mobiles.
Personally, I am the opinion when one gets money from others work, it should pay back in some form.
iOS 8 finally allows dynamically linked frameworks.
There's also j2objc, another interesting tool for creating a common layer for iOS and Android apps.
https://github.com/google/j2objc
I'm playing with it for a personal project, and as long as you're not too adventurous with your use of the JRE, it produces pretty usable Objective-C code.
https://github.com/google/j2objc
I'm playing with it for a personal project, and as long as you're not too adventurous with your use of the JRE, it produces pretty usable Objective-C code.
Another option that I'm currently using for a prototype at work is Java (or any JVM language) via RoboVM http://www.robovm.org - it works similar to Xamarin so it compiles Java bytecode to LLVM to a native binary. The advantage over Xamarin is, that it uses the Apache 2 license so doesn't require hefty license fees or a subscription and you are independent. RoboVM will be released in the next couple of month in version 1.0 and it works very very well.
My current prototype uses Java for all the backend / business logic (data base communication, file access, network access, threading etc.) and it communicates via a JavaScript bridge to a WebView UI layer for which I have targets for iOS (RoboVM), desktop (JavaFX), HTTP server (Jetty) and later possibly Android. All from a single Java code base which shares large amounts of code between the platforms.
My current prototype uses Java for all the backend / business logic (data base communication, file access, network access, threading etc.) and it communicates via a JavaScript bridge to a WebView UI layer for which I have targets for iOS (RoboVM), desktop (JavaFX), HTTP server (Jetty) and later possibly Android. All from a single Java code base which shares large amounts of code between the platforms.
Lisp is another option: https://wukix.com/mocl for iOS, Android, and OS X
From the project's GitHub page, some more details on what it actually does:
swift-x is a python script that simplifies the compiling of Swift and Objective-C(++) code into object files.
... [It] is a two stage wrapper for clang and the swift compiler that uses the Android NDK tools to produce android compatible .o files.
The first stage uses clang or swift to compile source code and emit LLVM-IR code which is passed to the Android second stage tools which compile the IR into .o files.
So you get working object code, but you still need to figure out how to get Android versions of the Cocoa Touch libraries if you use any of those APIs.
swift-x is a python script that simplifies the compiling of Swift and Objective-C(++) code into object files.
... [It] is a two stage wrapper for clang and the swift compiler that uses the Android NDK tools to produce android compatible .o files.
The first stage uses clang or swift to compile source code and emit LLVM-IR code which is passed to the Android second stage tools which compile the IR into .o files.
So you get working object code, but you still need to figure out how to get Android versions of the Cocoa Touch libraries if you use any of those APIs.
You can't just take a Swift app for iOS and compile it for Android, sure.
But this could be useful for people who enjoy Swift, and want to reuse blocks of logic from their iOS app to an Android port.
I'm wondering though, I thought Swift compiled down to code that uses things like the Foundation framework (NSString etc), which would also be missing on Android. It would be hard to do anything without these classes, wouldn't it?
But this could be useful for people who enjoy Swift, and want to reuse blocks of logic from their iOS app to an Android port.
I'm wondering though, I thought Swift compiled down to code that uses things like the Foundation framework (NSString etc), which would also be missing on Android. It would be hard to do anything without these classes, wouldn't it?
There are several independent open-source implementations of Foundation like GNUStep and Cocotron.
Once you have the Obj-C runtime and cross-compiler working, porting either of those to Android should be fairly simple... But I suspect that Swift may have dependencies on new or recent Foundation features which could be missing from the open-source versions (for understandable reasons, they lag behind Apple by several years).
Once you have the Obj-C runtime and cross-compiler working, porting either of those to Android should be fairly simple... But I suspect that Swift may have dependencies on new or recent Foundation features which could be missing from the open-source versions (for understandable reasons, they lag behind Apple by several years).
These guys already did port those frameworks to Android:
http://www.apportable.com
In what concerns GNUStep, has it improved the situation of a partially supported NeXTStep and Cocoa around 10.4?
I'm working building a UI library in swift that uses OpenGL for all rendering, etc. A page about this project is here:
http://photovivaapp.com/SwiftUI/
This runs on iOS for now, but it would be nice to get it running on android. I'm wondering how to call C code from Swift and how to call Swift from C for this project to work.
http://photovivaapp.com/SwiftUI/
This runs on iOS for now, but it would be nice to get it running on android. I'm wondering how to call C code from Swift and how to call Swift from C for this project to work.
FWIW I took a recent look at porting the Apple Objective-C runtime to Linux (with the vague idea that if Apple open source Swift, it might be easier to get it working with their own runtime rather than the GNUstep one).
One nice thing about clang is that there's no problem outputting Objective-C code that uses the Apple runtime on other platforms: just use -fobjc-runtime=macosx. The catch is that because class, protocol, etc definitions are just data, you need a way to get notified by the runtime linker when an object is mapped into the process address space. (You could scan the current address space at startup, but that makes it difficult to support dynamic loading, although probably not impossible now that I think about it. I thought I was nearly there with ld.so audit interface, but audit modules are loaded into their own namespace, so you can't easily communicate with the global libobjc.)
One nice thing about clang is that there's no problem outputting Objective-C code that uses the Apple runtime on other platforms: just use -fobjc-runtime=macosx. The catch is that because class, protocol, etc definitions are just data, you need a way to get notified by the runtime linker when an object is mapped into the process address space. (You could scan the current address space at startup, but that makes it difficult to support dynamic loading, although probably not impossible now that I think about it. I thought I was nearly there with ld.so audit interface, but audit modules are loaded into their own namespace, so you can't easily communicate with the global libobjc.)
I got some minimal Objective-C code running on Android a while ago, see http://blog.metaobject.com/2012/12/android-ndk-now-includes-...
This tool appears to fix the problem with the NDK make tool not recognizing .m files. I worked around this once by naming them with the .c extension and using compiler flags to set the language to Objective-C. Another time, I think I hacked the makefiles, but that was extremely painful.
Of course, there's apportable (http://www.apportable.com/open_source), their low-level stuff (runtime, Foundation) is open-source.
This tool appears to fix the problem with the NDK make tool not recognizing .m files. I worked around this once by naming them with the .c extension and using compiler flags to set the language to Objective-C. Another time, I think I hacked the makefiles, but that was extremely painful.
Of course, there's apportable (http://www.apportable.com/open_source), their low-level stuff (runtime, Foundation) is open-source.
Its great to see work being put into building community tools like this for Swift, thanks for the contribution.
BTW if you are in SF, we are running a Swift hackathon @ GitHub HQ in a few weeks: http://www.swifthack.splashthat.com
BTW if you are in SF, we are running a Swift hackathon @ GitHub HQ in a few weeks: http://www.swifthack.splashthat.com
Anything that makes it easier for entrepreneurs to make their apps available on both iOS and Android is productive and welcome.
Super nice work. Thanks for posting.
Also interesting http://kivy.org allows you to use Python for iOS, Android, OS X and Windows.
I would not bet on Swift and cross compiling it today... too bleeding edge. But interesting project... hope it matures. Good luck!