Hidden limitations of Google App Engine(stackoverflow.com)
stackoverflow.com
Hidden limitations of Google App Engine
http://stackoverflow.com/questions/565963/hidden-limitations-of-google-app-engine
20 comments
Is your last point the reasoning behind not using Tornado's templates? And why not Jinja? Django templates have always rubbed me the wrong way in performance and flexibility measurements.
Using Django with my prototyping framework is by no means an endorsement, it just happens to be what I fell into because when I designed the generic HTML5 template I plugged in Django vars and helpers (stubify, etc.) without really thinking through a decision on which templating lib to use.
That said, I did create a generic sketch.template wrapper for Django templates, and I have managed to swap in web.py, tornado and cheetah (sketch being the name of the framework). I have never used Jinja in production, but have been meaning to go back to it, your comment will prompt me to do just that (I assume by asking about it, it is your pref).
I will eventually push the entire framework public once I clean up some parts - it is simple and has a narrow scope/purpose but it is working really well for me
That said, I did create a generic sketch.template wrapper for Django templates, and I have managed to swap in web.py, tornado and cheetah (sketch being the name of the framework). I have never used Jinja in production, but have been meaning to go back to it, your comment will prompt me to do just that (I assume by asking about it, it is your pref).
I will eventually push the entire framework public once I clean up some parts - it is simple and has a narrow scope/purpose but it is working really well for me
Do you also use it beyond just prototypes? Or do you find it too limited?
I have only used it for prototypes so far because that is all that I have been doing recently. One of the projects will eventually be in PHP - but I see no reason why I can't clean the code up a bit, check it in and publish it.
It is just a collection of libs for GAE with my own glue code for the app, controller, a templating wrapper and some JS glue to get the models from the server into the client. You define the models and it auto generates scaffolding which you can re-arrange with the JS. Super-easy to create form based apps, I will push it to github next week and post it here on HN (I haven't had this much fun coding in a while, especially with being able to whip up mini-projects from idea to prototype really quickly - for eg. I did a simple todo list in a few hours, and a simple crm in a day (both auth with google), the actual code is just the models, over-riding the controllers where needed and placing form/record elements
It is just a collection of libs for GAE with my own glue code for the app, controller, a templating wrapper and some JS glue to get the models from the server into the client. You define the models and it auto generates scaffolding which you can re-arrange with the JS. Super-easy to create form based apps, I will push it to github next week and post it here on HN (I haven't had this much fun coding in a while, especially with being able to whip up mini-projects from idea to prototype really quickly - for eg. I did a simple todo list in a few hours, and a simple crm in a day (both auth with google), the actual code is just the models, over-riding the controllers where needed and placing form/record elements
AppEngine has made some great leaps in the last year. We built http://gri.pe on it earlier this year and have been impressed by the platform.
The last two months have seen the platform mature a lot as well. The latest 1.4 release has some amazing features: async datastore queries, app keep-alive, longer timeouts for background tasks, etc.
The last two months have seen the platform mature a lot as well. The latest 1.4 release has some amazing features: async datastore queries, app keep-alive, longer timeouts for background tasks, etc.
And 1.4 has the Channel API for everyone!
The architecture limitations exist b/c App Engine essentially scales effortlessly once you abide by them.
If you insist on using SQL, then use a company like Heroku, but realize that if your app gets popular you will be paying Heroku a lot of money for the same level of performance you could get for a few bucks a month using app engine.
After I wrote my first toy app a few years ago, App Engine doesn't really feel like it has any significant limitations other than the absence of naked domains and SSL. The recent reliability problems have been a nightmare, but as of Nov 6, I feel like they have been resolved.
I should add that I love app engine and use it whenever I don't need naked domains or SSL. (SSL is only available via .appspot.com domains)
If you insist on using SQL, then use a company like Heroku, but realize that if your app gets popular you will be paying Heroku a lot of money for the same level of performance you could get for a few bucks a month using app engine.
After I wrote my first toy app a few years ago, App Engine doesn't really feel like it has any significant limitations other than the absence of naked domains and SSL. The recent reliability problems have been a nightmare, but as of Nov 6, I feel like they have been resolved.
I should add that I love app engine and use it whenever I don't need naked domains or SSL. (SSL is only available via .appspot.com domains)
On Heroku ... I chose to build by own stuff on EC2, and it is a lot cheaper. And now with the free tier account I also get some freebies.
And yeah, app engine is great for entry-level prototipes, since you can forget about sysadmin tasks.
And yeah, app engine is great for entry-level prototipes, since you can forget about sysadmin tasks.
The biggest limitation I had was while writing to the database, the database would sometimes return an error for no apparent reason. I'd then be forced to catch these errors, and resubmit. But if you had lots of traffic and resubmit too quickly, Google App Engine would throw more errors back at you. This was all over a year ago, not sure how it is now, I would never go back to using it.
From http://code.google.com/appengine/articles/life_of_write.html
There is an expected failure rate on writes as Bigtable tablets are sometimes unavailable, for example, when they are being moved or split. The presence of more indexes increases the probability of hitting an unavailable tablet as an exception will be raised if a write fails for any of the indexes. In those situations, your application will need to decide how to handle the exception. One option is to add a task to the task queue to retry the write at a later point in time. Another idea would be to respond with an error from the app and have the client retry. This tends to work with things like AJAX requests where there is client side logic which can handle an error message from the server.
From http://code.google.com/appengine/articles/life_of_write.html
There is an expected failure rate on writes as Bigtable tablets are sometimes unavailable, for example, when they are being moved or split. The presence of more indexes increases the probability of hitting an unavailable tablet as an exception will be raised if a write fails for any of the indexes. In those situations, your application will need to decide how to handle the exception. One option is to add a task to the task queue to retry the write at a later point in time. Another idea would be to respond with an error from the app and have the client retry. This tends to work with things like AJAX requests where there is client side logic which can handle an error message from the server.
This might have been reduced by the release in February:
http://googleappengine.blogspot.com/2010/02/app-engine-sdk-1...
See the section "Reduced error rate with Automatic Datastore Retries".
I haven't kept a close watch on what's changed since then though.
http://googleappengine.blogspot.com/2010/02/app-engine-sdk-1...
See the section "Reduced error rate with Automatic Datastore Retries".
I haven't kept a close watch on what's changed since then though.
That helped some, but there was a release (to the backend infrastructure, not the AppEngine SDK) about a month ago that made a huge difference.
Edit: Here's the link that shows the improvement: http://code.google.com/status/appengine/detail/datastore/201...
Edit: Here's the link that shows the improvement: http://code.google.com/status/appengine/detail/datastore/201...
Yes that day looks impressive, but if you look at the following days, the improvement does not look anywhere near as dramatic.
http://code.google.com/status/appengine/detail/datastore/201...
Weird. If the entire keyspace isn't affected, seems like you might want to use another region as the 'task queue'. That is, if you get an error where you want to insert something, add a prefix that lands it on another tablet. Occasionally scan those failover regions for values that can be moved back to their real keys, later.
checkout the following issues:
http://code.google.com/p/googleappengine/issues/detail?id=17...
http://code.google.com/p/googleappengine/issues/detail?id=14...
you need to trap DeadlineExceededError for long-running queries, CapabilityDisabledError (from google.appengine.runtime.apiproxy_errors) for when the datastore is in maintenance mode and other misc errors. If it is a deadline error, you can re-submit it (but prob advisable to only re-sub once), on CapabilityDisabledError, return a server error page or load from memcache
http://code.google.com/p/googleappengine/issues/detail?id=17...
http://code.google.com/p/googleappengine/issues/detail?id=14...
you need to trap DeadlineExceededError for long-running queries, CapabilityDisabledError (from google.appengine.runtime.apiproxy_errors) for when the datastore is in maintenance mode and other misc errors. If it is a deadline error, you can re-submit it (but prob advisable to only re-sub once), on CapabilityDisabledError, return a server error page or load from memcache
Notice the date of this question is Feb 20th, 2009. Almost 2 years out of date. I imagine at least some of the limitations mentioned are gone by now.
If you're sick of these limitations (and use Django) head over to www.djangy.com and sign up for the private beta :-)
I wonder if the simplenote guys can provide some input?
http://simplenoteapp.com/about/
http://simplenoteapp.com/about/
The biggest limitation is that they only support two languages - python and java.
Not that there's anything wrong with these particular languages, but I'd prefer more choice. It would be cool if they supported C++, PHP, Javascript (V8), Ruby, Erlang.
Not that there's anything wrong with these particular languages, but I'd prefer more choice. It would be cool if they supported C++, PHP, Javascript (V8), Ruby, Erlang.
You can run Ruby and Javascript stuff on top of the Java support. The pipeline has to be pretty light though and I wouldn't run Rails on it.
My prototyping platform is essentially: tornado.web with the tornado request, response, escape, auth and locale. OpenID from web.py. My own WSGIApplication that inherits from the built-in so that it uses my Controller class (which has a plugin arch), Django for templates, JSON, BeautifulSoup, Markdown, a generic 'asset' module implementing something akin to ActiveRecord on the standard DB with memcache support, a generic HTML5 getting started template, a CSS toolkit (can't recall the name) then jQuery and backbone.js with some of my own JS glue code that maps the server models to client models.
Using this stack, I can prototype form-driven web apps in hours.
I wrote a relevance view for Twitter (with Twitter oauth support) in half a day. I don't think I will use the RDBMS even when they release it - BigTable is excellent (it will be confusing if you schema the same way you schema with an RDBMS).
It is great to be able to have an idea for a quick app, register an AppEngine app, fire the code up, commit and deploy and have the app live for others to try out within hours.
The admin interface allows you to spot errors and choke points in production and fix them quickly. The pricing is better than AWS or any of the others and the platform includes hidden gems such as the MapReduce lib, multitenancy, xmpp and Google data libs (including auth)
The only real limitations that I have bumped into are:
* The max request time is around 20 seconds, so you can't have open Ajax requests
* No HTTPS support as yet (coming soon, apparently)
* Limits on the number and size of files. It isn't ideal for serving a large number of static files, but it does have zipimport support for libs
* For best performance, you need to define the indexes yourself rather than relying on the defaults that the lib generates
* There are a lot of sys calls blocked, so you can't run Tornado or any cpython modules