Glad I could get my first open source effort up after all these years. There are few if any modern OpenGL ES 3.0 / 3.1 demos for Java with Android and this is important tech. I spent a good chunk of the last week pulling out code from my commercial middleware (TyphonRT) to make a lightweight and concise GL framework that makes working with modern GL much easier. I also provide a few demos with more forthcoming though this is a spare time effort. The nice thing is that the demo code is separated from the framework, so the framework can be used separately. Eventually I'll be porting the NVidia Gameworks demos to Java / Android. Lots more to say, but I'd be glad to answer any questions about modern OpenGL on Android. I'll be expanding the wiki on the repos linked here in the next week.
If you think the code above is useful or cool it's the basis for where I'm headed with building a next-gen video engine for Android. I have ~9 hours left on a crowd funding campaign that will help me get it out sooner than later having bootstrapped to 95% of the way there spending 2.5k hours over the last year building it. I'll keep it short as there is a video available to check out of it in action. Also bumped into Robert Scoble a week or so ago and he gave me a couple of amazing endorsements:
KS here: http://kck.st/1xA3R61 At this point I'd really just like to connect with interested early users as the funding goal is well... yeah... Thanks folks!
Hi, my name is Michael and I'm the founder of TyphonRT Media, Inc. It's just me and I have been bootstrapping various game
and performance middleware for Java for over 10 years. My story is one of a consultancy attempting to turn into a product
oriented company. I however chose a difficult category, new media tech (music-tech, initially, creative engine tech now), to
innovate in and one that requires hardcore engineering. Last year I was in the right place at the right time with the right
tools to start to build a next-gen video engine from scratch on Android. Android 4.3, last August, introduced the ability for
the MediaCodec API (low level hardware accelerated encoding / decoding of audio & video) to be combined with OpenGL ES.
Android being Android there were many challenges (re: bugs) to overcome with MediaCodec and such over the past year, but I
got things stable on the majority hardware configurations. In that time I have spent $80k out of pocket (all professional
earnings in last ~1.5 years) on the video engine and roughly 2500 hours on the engineering side building the engine over the
last year.
I am running a Kickstarter to fund final launch efforts / release engineering; I'm ~95% of the way to launching and can
do it by the end of Q1 '15! A video engine of this complexity takes a lot of testing and I'll be staggering the launch initially on well tested hardware
configurations (Snapdragon & Tegra K1 SoCs). I am attempting to raise $20k (28% funded presently). The crowdfunding attempt
first and foremost is to raise awareness of the tech and to reach out to early enthusiasts for new photo / video tech on
Android. If I don't raise that amount it simply means the launch will be delayed as I go out and do another 3rd party contract
to keep the lights on in the meantime. Ultimately I have funded ~80%+ of this effort out of pocket.
Now "next-gen" is a bit buzz-wordy, but indeed I built the graphics capabilities of the video engine on OpenGL ES 3.0 and
will also be supporting 3.1 which is the latest standards (compute shaders!). This allows many internal engine improvements
over GLES 2.0. The engine itself supports ~150 image operations and the first app in the suite is a creative video capture
app that offers deep effects composition capabilities while still providing an easy to use and expressive GUI on phones. It's
possible to stack up to 8 different image operations and modify them in real time while recording video. A high-concept product comparison is: "Adobe After Effects in your pocket." It so happens that
many of the previous limitations (FBO limits) on mobile have been lifted over the last year with new mobile GPU tech allowing
significantly more realtime post-processing capabilities. The Snapdragon 800 / Adreno 330 GPU (Nexus 5) being the first phone
form factor SoC capable of truly deep real time effects manipulation. Of course this is continued with the Snapdragon
805 / Adreno 420 and others like the NVidia K1 all of which are currently stable running my video engine.
I suppose the really cool thing is that I essentially made the missing video / audio middleware for Android that can service
many apps and am immediately interested in releasing some of my own, but also am considering licensing the engine to 3rd parties.
I am aiming to launch a suite of modern creative video apps for Android which is a category sorely lacking on Android. Mainly
this is due to the fact that Google doesn't release a high quality video engine to app developers to create media apps with. In
comparison Apple does release a fully featured video engine and quality APIs for media development and thus there are a lot of
video / media apps for iOS and compartively very few for Android.
I do want to give an acknowledgement to Brad Larson and the GPUImage open source effort. While my video engine efforts don't
share any software architecture with GPUImage I found the catalog of image operations (mainly fragment shaders) very helpful
when building my tech and as a starting point this allowed me to really focus on overcoming issues with MediaCodec versus
recreating a stable of fundamental image operations in GLSL. I fixed several issues with the shaders in regard to how they
work in combination and upgraded everything to OpenGL ES 3.0 as the baseline GLSL version.
I'd be glad to answer any questions on the tech or even bootstrapping while doing "hardcore engineering". It's been a long road
to launching product and I'm almost there. Join the announcement email list at www.typhonvideo.com
For a direct link to the sizzle reel video and better quality check it out on YouTube: https://www.youtube.com/watch?v=AriO_JktgjI
Thanks in advance for any support from the HN community!
There are always trade offs in life.. On the dating front at least. If I spent more time in general hanging out and out and about on weekends goofing around like many folks I'm sure things would work out differently. One has to consider that for me often I'm left to just generally low stakes / OkC derived meet ups.
Statistically speaking at this point I can say that coming across as passionate about ones life's work and mentioning that entails a day job at times and often night / weekend effort doesn't set the tone of being emotionally or otherwise available even though that may not be the case nonetheless it is best to can it on the first meetup... That is all.. It's bad enough being an introvert in an extrovert's world.. :)
Oh my... Well, meh perhaps this is longer than it needs to be..
I didn't think my post to Erik's blog would be reposted here in entirety. I felt moved to post quickly on Erik's blog FWIW and didn't mince words. His post reminds me of certain aspects of thought regarding where I was at after my degree. The will to create is a mysterious thing.
As things go jrogers65 probably has no idea what I'm trying to accomplish or how far along I actually am or the personal funding I've sunk into this path (everything) over the years.. I too had a deep fascination w/ graphics & audio tech for some time and I knew in advance 10 years ago that I was choosing the harder path to innovate particularly through large scale 2D / 3D spatial audio, upping immersion in games / movies / all media in the home and venues, such that the path was fraught with a long term time span.
In pursuit of this path I assure you I probably have the best hacker den around while also supporting the R&D goals at hand. It wasn't cheap to build or now maintain while bootstrapping and is in a prime SF spot; can support upwards of ~6 initial employees; etc. Indeed this leads to a challenging month to month reality in regard to long term debt / build out costs and trying to free up time (IE rent $$$ in advance) to work in waves unadulterated at the tech at hand.
Besides relentless innovation often comes hand in hand with perfection, but sometimes there is simply just time to get things right. After the general economic dip around '08 it's clear I'm at earliest presently 5 years out to perhaps 10 years w/ my audio aspirations regarding product / market fit just from an economic angle; I can be more specific, but will save it. This general mismatch and the rise of Android being the fortuitous tech occurrence has given me the time to focus on creating middleware tools supporting the long term effort while "doing it right" and taking into consideration the next 10-15 years of software dev.
If one is going to commercialize middleware tools related efforts the bar is high when money is to change hands.. One is free to go the open / social path and release early and evolve an effort in the public eye and likely never gain revenue. This is a valid direction... I however plan to make a long term living from my efforts and the sweat spent now _should_ pay off later. It reminds me of a supposed African proverb, “If you want to go fast, go alone. If you want to go far, go with others.”
One can wish to get funded for something innovative / artistic / complex / before its time; one can wish to be independently wealthy and be free to pursue ones goals. Outside of those two general hypotheticals what is often left from there is just pure dogged persistence and yeah it's not going to be the most balanced life, but them the breaks. I at least love what I do and am fully committed to seeing my long term vision become reality not just for me (it already is here for me!), but for "you" / others; commercially too...
I hope Erik finds his path without losing the will to create something great; individually (at first at least) w/ commercial aims or more publicly via open source during the conception phase, etc.
In short... You do not know what you speak of jrogers65.. ;P
Nice.. I too was working with embedded C++ dev w/ a Blackfin & Sharc combo and yes the standard build cycle was kind of lame. I attended a Blackfin course that just happened to be located at Analog Devices HQ. In the 4 days that I was there I got an OSC (Open Sound Control) library w/ message parsing and Lua support up and running hooked up to desktop GUI audio control apps built with a framework I've created in Java. I continued to try and get things going with the OSC message parsing to Sharc communication for DSP control (Ambisonics mostly), but then Android steam rolled me..
I already had a real time app dev runtime / framework in Java for J2SE that immediately started porting to also run on Android. A modern SoC and Android w/ my framework on the desktop and on the device for 3rd party app dev is now driving my future audio hardware dev efforts. First though I get to do a general release of the software dev framework as a product which never really would have happened without Android showing up. Things changed a lot recently!
Anyway, good idea with the scripting / Blackfin deal though as that can pay off big.
> OK, just going to answer a few points, to try to wrap this up. :)
And for the wrap up wrap up ;P
> I write a lot of the core "engine" in C++ for speed and lack of garbage creation.
Most definitely. That was all I was getting at is that the core engine should be as fast as possible.
What game on Android Market BTW? Would be glad to check it out and support yah! :)
> Geting UI right is really, really hard... But if I want a list dialog, or a grid view...
Indeed.. UI is hard though I'd say list dialogs and grid view is still GL possible without too much difficulty given various tool kits. Yes a WebKit view is difficult to pull off cross-platform presently. There are options like Awesomium though I've yet to see it used on a mobile platform, however I'm sure something will come up soon.
Yes.. a keyboard entry field depending w/ a GL UI in conjunction with say popping up the default native UI keyboard for touch only devices is something that would need to be fine tuned.
I plan to tackle general GL UI matters at a future point to offer a nicely integrated set of optional components. You are right though it's generally difficult.
For aspects in TyphonRT that require local / native peers the main application container is loaded declaratively with a cross-platform interface component backed by the proper native implementation, so from the devs perspective they can just request the common interface and the platform specific aspects are active behind the scene.
>... most platforms I write for don't already have a guaranteed JVM
Yes, for a desktop release it might be pertinent to distribute a private stripped down JVM. For mobile.. Well.. It's not easily feasible or likely allowed if I recall in the TOS to have a JVM on the iPhone. Cross-compilation is the likely way to go if / when TyphonRT supports iOS.
>How much overhead does Scala add to an app on Android?
Yeah you got me there.. ;) If one uses the Scala standard library that jar file is ~8MB. Whether that can be split up and paired down I'm not sure yet. I'm close to starting on Scala integration (mind you this is optional w/ TyphonRT).
There are also platform aspects of TyphonRT that will reduce overhead for multiple apps installed, but this gets into a bit more conversation here than necessary; essentially though side-loading shared components installed by other apps thus minimizing each app requiring a large download.
So yeah, some work and experimentation ahead. I'll definitely be working on a minimum profile (if possible) for Scala integration.
>Hmm...getting very much off topic here, so I want to keep this brief ... the problems of the language being very verbose and missing a lot of useful features
Yes, let's not belabor things, but I do want to point out that Scala essentially covers all of the points you raised. It supports duck typing (type safe too!), coroutines, first-class functions, and is more syntactically elegant. Scala even supports a particular mechanism to "sort of" support primitives w/ generics with the @Specialized annotation. So yes, I'm turning to Scala to provide additional language features and it's especially cool keeping type-safe aspects and being able to do joint compilation, so this works well for desktop / J2SE & Android integration. Of course Scala on top of the finely tuned TyphonRT / Java COP API and such is nice; more concise code for sure; this in addition to having access to all of the Java and/or Android SDK and 3rd party libraries. On the Java tip too regarding generics / Object situation TyphonRT has primitive collections available as well which is the main use case.
> Lua can act object-oriented if you want it to, but you can also use it in other ways that simply make more sense to the problem at hand.
I'd argue it's not a silver bullet as a primary language to write an entire game / engine in w/ more discussion continuing along those lines; there seems to be flexibility gained and lost. As a scripting engine that's one thing, but not a catch all for everything.
The important part here though and hopefully this ties back in a little with the OP and your concerns is that there are options available to take the edge off of mobile dev Android or otherwise and things should radically change in that regard in the next couple of years.
@seclorum is certainly correct that cross-platform mobile dev with OpenGL/ES is not out reach at all and once the C/C++ or Java decision is made depending on dev flexibility one can be creating rich content games not limited by a particular mobile OS concern.
Ultimately you want to pick the tool you are most comfortable and proficient with as in regard to indie game dev there are many other fish to fry let alone the hit based nature of the current app stores.
>For what it's worth, Moai does come with things like physics; not sure how they bind things together, but I would have guessed that they have a very basic engine with game objects. Could be wrong.
From what I read Moai supports Box2D. You know who else supports Box2D; everyone... BatteryTech, libgdx, TyphonRT, the list goes on, etc. etc. ;) It does sound like there is a basic game object API in Moai, but it only is geared towards 2D presently.
I know Robert of BatteryTech is doing a whole lot of Lua integration and work with BatteryTech, so do take a 2nd look and drop him an email perhaps.
>JavaCPP looks interesting
Indeed.. It's the 1st JNI / native marshaling assistance API that I like that takes the pain away of hand rolling JNI
code. :)
>Yes, but it's the API for getting events
This is abstracted with most middleware mentioned. In fact input control is the largest category of core architecture components in TyphonRT (almost 10%).
>putting up native dialogs, etc. that will be non-native everywhere.
Why would you do this if you are creating an OpenGL/ES cross-platform game? Use a GL based GUI that will be cross-platform!
>It's just not my thing, since I'm not a Java person.
Fair enough hence why I'll stop posting on the thread! :) Though I did want mention some responses on the off the top general criticisms at the front of this reply. JVM languages will breathe some life into the Java sphere of things. This in concert with well developed middleware will make a lot of things that are hard now much easier soon enough.
>Thanks, but I guess I didn't get across how much I still hate Java. ;)
Can't solve that.. ;) ;)
>I feel it has 95% of the problems of C++, without the advantages (predictable speed, mostly).
I suppose you'd have to get a little more particular in the 95% of problems angle. I have been working with Java for quite some time and TyphonRT contains necessary workarounds or API extensions enabling predictable speed. For instance TyphonRT being a Java based component architecture is highly dependent on iteration over collections of components. The custom collections API in TyphonRT has recycled and resetable iterators overcoming the "iterator problem" of the standard collections API. There is a bit more of other things done too to provide predictable runtime performance. A lot of that involves staying as memory efficient as possible and not triggering GC. Of course when things get too hairy yeah native code is used. One has to do that for physics engines (Box2D / Bullet) especially on Android. I'm really loving JavaCPP which is a relatively new effort for managing native bindings (http://code.google.com/p/javacpp/).
>I'm sure your platform will be useful for anyone porting J2SE code to Android, but that's not me.
If you are most familiar with C/C++ certainly by all means go with that and the NDK and use BatteryTech, Proton SDK, or something like that to provide the core cross-platform architecture support.
>And it doesn't look like you're supporting iPhone?
Not yet, but just like Unity which you mention cross-compilation is not out of the question. Like libgdx once the main framework is mostly finalized I'll be looking at say an Angle backed for the desktop and / or other options for expanding device and environment support. The primary goal is to create a leading edge Java framework / platform first. The platform / PaaS aspects also are a priority in the short term among many other things like strong Scala support / integration over say iPhone support. A thing to keep in kind too is that TyphonRT is not just aimed at game dev, but is applicable for all dev efforts including enterprise.
So yes, if you need cross-platform support go native or use a canned 3D engine like Unity if it matches the type of game you are trying to make.
>I know that middleware providers will step in to fill in the gaps, but it's just annoying to be non-native on ALL platforms.
Hrm? C / C++ is available on the majority of platforms... ?
>http://www.batterypoweredgames.com/batterytech is one that I know that's low level, for instance -- it gives you events, OpenGL and sound on Windows, Mac, iPhone, and Android. It's pretty bare-bones, though, as far as I know.
Yes it's not a complete game engine nor is Moai which you mention you have your eye on presently. BatteryTech, libgdx, and TyphonRT provide the core architecture to abstract the common requirements (input, audio, etc.) for real time apps & games. TyphonRT also has higher level optional components for game dev including a fully featured component oriented entity system (libgdx / BatteryTech, etc. don't). This class of tech is meant for folks that can build their own engines and perhaps their own creative games rather than trying to force canned engines to do things they are not great at doing. As mentioned though there are optional components with my efforts to further assist game dev efforts beyond core architecture abstractions.
Regardless of what you choose though I hope you make some cool games for Android! :)
@SomeCallMeTim
>I submit it would be better for everyone if there were a single cross-platform engine that everyone used.
>What would have been better is a library that abstracts away the fact that Android is its own unique OS, period. When I write code, I don't want it to be Android code, or iOS code, or WebOS code, or Windows [Mobile] code, or Linux code.
---
There are options out there. I'm one of those 3rd parties soon releasing a comprehensive middleware solution for app dev including a lot of game dev oriented effort that provides a clean abstraction layer via a component architecture that uses a large majority of advanced Java language features under the hood without sacrificing performance.
Keep a watch out for TyphonRT; drop me an email via the contact address and I'll be glad to discuss more and make sure you get early access. I'm aiming to get it out ASAP and before I give an all day game dev workshop at AnDevCon II in November. I suppose one thing that makes TyphonRT unique is that it's a tool, framework, and future platform (PaaS). At the heart of things you can pick and choose individual blocks of functionality (tool), or utilize the runtime framework (framework), and soon there will be optional PaaS features aiding Android and cross-platform dev in the coming year after launch.
Another Java based cross-platform effort which is similar in purpose though significantly different at the architecture level is libgdx. For C/C++ check out Battery Tech and the Proton SDK for similar cross-platform efforts (both of these with even more device coverage).
TyphonRT provides that layer of sugar you seek without losing performance and creating full cross-platform execution for OpenGL apps without changing a line of code for J2SE -> Android. TyphonRT is not limited to OpenGL though and can utilize any graphics API potentially allowing a hybrid UI, but the rest of an app still being cross-platform.
Having worked intensely with Android since v1.0 and the G1 hit my hands in Oct '08 I have more than a few stories on the difficulty of low level development / games, etc. across different OS and major device hardware generations. I've experienced first hand a good deal of general engineering blunders and have found and reported a few major ones myself (you know when big G fixes a bug immediately upon being reported you found a good one; the 3.0 & 3.1 / NIO immutable endian swap issue comes to mind!).
Middleware will mature and soon provide a lot that the Android SDK lacks for certain app development areas. In many ways the Android SDK while novel in several areas is still being developed from an old engineering model, but that is seemingly unavoidable for a large organization such as Google. There are of course additional inherent problems with tying the SDK to firmware, but such is life.
I do agree with your last point though that generally the SDK in addition to the particular ecosystem quirks that have occurred over time isn't exactly indicative of satisfying developers.
I commented on the linked articles in more depth regarding performance game dev w/ Java. I see someone mentioned libgdx in a comment here which is awesome. Also awesome to see mention of Slick. I'll just mention that I have been working for many years developing a mature component oriented platform for real time app & game dev in Java that spans J2SE & Android utilizing all the language features of Java without sacrificing performance. I am barreling towards initial release. It's called TyphonRT and a little more info is available here: http://www.typhonrt.org/ I'm aiming for an end of summer release and I hope you might want to consider Java (w/ Scala) & TyphonRT for you next project.. :)
@ChuckMcM - "Disclaimer though is that I was one of the Java guys and I had completely bought into the notion of 'write once, run anywhere'. Making that true turned out to be impossible for a number of reasons, not the least of which that the underlying OS providers were hostile to the idea."
"Configuration Management is a Hard Problem (tm) on any system and people see the Windows eco-system or the MacOS eco-system and feel like they should be able to do that cross-platform."
---
I'll just pipe in here real quick as cross-platform Java apps are core to the open source / middleware product I'm releasing called TyphonRT this summer. It's something I've personally worked on for many years well before Android's emergence. Fragmentation is difficult to solve alone across the variety of OS and device differentiation in the Android ecosystem without even considering cross-platform compatibility with the J2SE.
TyphonRT is one such effort though and is built with a scalable configuration system via a component architecture. Major engineering of the client SDK / runtime will hopefully wrap up by the end of June with a closed beta for those interested w/ public launch end of summer / Q3.
To support distribution across the Android ecosystem and desktop configurations part of TyphonRT is a PaaS system such as when a TyphonRT app is installed it will download any OS or device specific modifications / alternate components to "patch" or provide a stable middleware layer in attempt to keep that write once / run anywhere promise or get as close as possible.
Of note I'll also be dumping over 10 years of solid Java real time app / game dev knowledge too through an expanding tutorial series w/ lots of code examples styled in a similar manner to Khan Academy.
Ok not to hijack this thread, but the work I am doing as an indie / startup is relevant to this discussion. This kind of solution with the current landscape surrounding Java is only going to come from the open source / indie area.
Glad to see the comments flood in for this and see it hit #1 on HN so quickly as I was beginning to wonder when it would ship. I was contracted to work on the Amazon MP3 v2.0 app rearchitecting the download architecture and adding cloud drive download support. It was "very interesting" being the only outside contractor / software architect level dev to be hired by A2Z / Amazon to work on core architecture for Amazon MP3. I finished my involvement mid-Feb. I guess I'm just posting to get an account started here on HN as I'm launching some very compelling Android platform / middleware soon called TyphonRT. I've been bootstrapping for years and this recent Amazon MP3 contract has opened up enough runway for me to launch my tech in the coming months. Hopefully I'll have some more time to post in the future too.
Yeah account sharing is watched and when detected the suspect account will get flagged for a TOS violation. I can't tell you how quickly the ban hammer may come down as things go, but 2 devices is likely not going to trigger it.
BLE stack is even worse in Lollipop.