Is Google App Engine the wrong product for the market?(25hoursaday.com)
25hoursaday.com
Is Google App Engine the wrong product for the market?
http://www.25hoursaday.com/weblog/2009/03/01/IsGoogleAppEngineTheWrongProductForTheMarket.aspx
9 comments
If you're running a small site (say, 1,000,000 pageviews per month), you aren't going to have trouble scaling a MySQL/PostgreSQL database and application. You'll normally be showing one pageview per second at that rate. So, it's just as easy to go with Slicehost or Amazon.
It is significantly easier to go with App Engine, though. There's a sizable group of people who don't want to deal with all the messy details of SliceHost or EC2. When you consider that all they really want to do is host a web application and have it scale, requiring them to start with a bare metal VM is total overkill.
It is significantly easier to go with App Engine, though. There's a sizable group of people who don't want to deal with all the messy details of SliceHost or EC2. When you consider that all they really want to do is host a web application and have it scale, requiring them to start with a bare metal VM is total overkill.
I'd disagree. App Engine places huge restrictions on what you can do. With a bare VM, I can install a database, webserver and go to town. With App Engine, I have to deal with all of the small quirks of their data store.
Examples:
You are doing something location-based and what to find things within 10 miles of a location. With SQL, you can do a crude bounding box query if it doesn't have to be that accurate. You can even do better geometric queries with PostGIS. With App Engine, you can't even do a bounding box query. Inequality queries can only be on one field at a time. Beyond the location example, you can't say get me the items where comment_count > 5 and author_karma > 10.
You can't do joins and so you have to denormalize a lot of data that will perform fine as normalized data.
There is no OR operator at all.
Saying that it allows you to ignore the messy details is like saying that you can avoid having to get egg on your fingers making an omelet if you create a machine that can automatically create food. It is not easier to go with App Engine for the majority of applications which would benefit from simple things that App Engine doesn't support.
It is far easier to scale something written for App Engine. That's the payoff. Even if you have to port away from App Engine, you've still created a data storage strategy that doesn't require joins; you've still moved a lot of the data processing out of the database and into Python manually; you've still made it so that you aren't using any advanced queries. App Engine forces you to write applications in a way that scales. App Engine doesn't allow you to do any of the operations that make programming quick to do. It's a trade-off and depending on whether your application needs it, it can be worthwhile or not.
Examples:
You are doing something location-based and what to find things within 10 miles of a location. With SQL, you can do a crude bounding box query if it doesn't have to be that accurate. You can even do better geometric queries with PostGIS. With App Engine, you can't even do a bounding box query. Inequality queries can only be on one field at a time. Beyond the location example, you can't say get me the items where comment_count > 5 and author_karma > 10.
You can't do joins and so you have to denormalize a lot of data that will perform fine as normalized data.
There is no OR operator at all.
Saying that it allows you to ignore the messy details is like saying that you can avoid having to get egg on your fingers making an omelet if you create a machine that can automatically create food. It is not easier to go with App Engine for the majority of applications which would benefit from simple things that App Engine doesn't support.
It is far easier to scale something written for App Engine. That's the payoff. Even if you have to port away from App Engine, you've still created a data storage strategy that doesn't require joins; you've still moved a lot of the data processing out of the database and into Python manually; you've still made it so that you aren't using any advanced queries. App Engine forces you to write applications in a way that scales. App Engine doesn't allow you to do any of the operations that make programming quick to do. It's a trade-off and depending on whether your application needs it, it can be worthwhile or not.
You are talking past each other. neilc pointed out that it is easier to go with App Engine, which is probably true. Your criticisms are that App Engine does not have as many features, which is also true. App Engine isn't aimed at "enterprise" so much as it's aimed at "people who don't want to bother setting up a VPS."
Here's one obvious use case: social apps. Most apps on facebook, myspace, etc don't really require perfectly normalized data, most of the stuff is just for fun, or games to generate ad revenue etc. Plus, you don't have to worry much about the scalability issue other than making your queries run fast, once it's on app engine, a million myspace users hitting your app won't even make you worry.
I've got my own colocation facility with virtual machines and the like as well as an amazon ec2 account with several vms, but I still make use of google app engine any time I want to write a facebook/orkut/myspace app.
I've got my own colocation facility with virtual machines and the like as well as an amazon ec2 account with several vms, but I still make use of google app engine any time I want to write a facebook/orkut/myspace app.
It is significantly easier to go with App Engine, though.
For values of easier which include:
1) Learn to program in Python.
2) Forget everything you know about databases.
3) Lose the accumulated value of all reference work on the Internet on, e.g., "How do I send an email to someone 3 days after they sign up for my free trial?"
For values of easier which include:
1) Learn to program in Python.
2) Forget everything you know about databases.
3) Lose the accumulated value of all reference work on the Internet on, e.g., "How do I send an email to someone 3 days after they sign up for my free trial?"
> Third, App Engine doesn't have an SLA. That's a harsh sell for enterprise stuff.
How many SLAs actually pay for the incurred loss? Sure, a month of free hosting is nice, but does it really make up for the loss of a day?
> It doesn't even have a way to make a backup of the database in a dump file. That makes it an absolute no-go.
Why does the backup format matter?
How many SLAs actually pay for the incurred loss? Sure, a month of free hosting is nice, but does it really make up for the loss of a day?
> It doesn't even have a way to make a backup of the database in a dump file. That makes it an absolute no-go.
Why does the backup format matter?
The format doesn't matter. The fact that there is no backup mechanism matters. App Engine does have some built-in redundancy, but as we've seen in some high-profile incidents, backups are a necessity. App Engine provides no facility to create backups.
We are working on database export, it's on the roadmap:
http://code.google.com/appengine/docs/roadmap.html
-- @mdrcode / App Engine PM
http://code.google.com/appengine/docs/roadmap.html
-- @mdrcode / App Engine PM
As a few people have already pointed out, right now App Engine is missing functionality that a lot of apps need. Right now coding to App Engine means lock-in to a single hosting provider.
But that's today, and I think the business will grow up fast. Hosting apps and databases rather than VMs and block devices (Amazon's current model) allow more granular control of resource allocation. At the same time the app hosting model lets you export problems such as geographic replication and failover to the hosting provider. The app hosting business can be accelerated by building on top of VM hosting infrastructure from companies like Amazon.
The way I see this playing out: VM hosting will continue to proliferate. App hosting will start to get popular and the frameworks will get fleshed out with the functionality 80% app developers need. App hosting companies will build their infrastructure on top of VM hosting infrastructure. Interoperability standards will get created (probably in the form of extensions to frameworks like Rails and Django). As there are more companies offering app hosting and the hosting environments get more feature-complete companies will start to switch for new development. In another 3-5 years we will see most new services being built on app hosting rather than dedicated/shared/VM hosting.
For legacy apps and apps that don't fit in the limitations of something like App Engine the VM hosting model works well. I've written one app on App Engine and I'm in the process of moving my company's legacy code base to EC2 hosting right now. VM hosting is always going to give you more control than app hosting just like running your own hardware will always give you more control than VM hosting. That said app hosting will give developers more pre-built infrastructure. How many devs actually want to patch servers or set up MySQL site-site replication? Negotiate bandwidth contracts? Rack servers? Not only is that stuff irritating to deal with, it costs money. App hosting is a pretty big win.
So is App Engine the wrong product? I don't think so, it's just not done yet.
But that's today, and I think the business will grow up fast. Hosting apps and databases rather than VMs and block devices (Amazon's current model) allow more granular control of resource allocation. At the same time the app hosting model lets you export problems such as geographic replication and failover to the hosting provider. The app hosting business can be accelerated by building on top of VM hosting infrastructure from companies like Amazon.
The way I see this playing out: VM hosting will continue to proliferate. App hosting will start to get popular and the frameworks will get fleshed out with the functionality 80% app developers need. App hosting companies will build their infrastructure on top of VM hosting infrastructure. Interoperability standards will get created (probably in the form of extensions to frameworks like Rails and Django). As there are more companies offering app hosting and the hosting environments get more feature-complete companies will start to switch for new development. In another 3-5 years we will see most new services being built on app hosting rather than dedicated/shared/VM hosting.
For legacy apps and apps that don't fit in the limitations of something like App Engine the VM hosting model works well. I've written one app on App Engine and I'm in the process of moving my company's legacy code base to EC2 hosting right now. VM hosting is always going to give you more control than app hosting just like running your own hardware will always give you more control than VM hosting. That said app hosting will give developers more pre-built infrastructure. How many devs actually want to patch servers or set up MySQL site-site replication? Negotiate bandwidth contracts? Rack servers? Not only is that stuff irritating to deal with, it costs money. App hosting is a pretty big win.
So is App Engine the wrong product? I don't think so, it's just not done yet.
Google App Engine represent one more step towards lowering the barrier to entry to web applications and services.
Any small team of developers can deploy a successful web application with no system administrators and no VP of operations. GAE takes away all the complexity of databases, replication, redundancy, monitoring.
For my last project (http://www.btexty.com , an open mobile messaging platform), we switched to Python and GAE, moving away from a Ruby On Rails setup on dedicated hosting, that we used from a previous project (http://www.moqpon.com mobile coupons).
I'm very happy with that decision. For a duo of developers, forgetting about the hosting platform allows to focus on product development and dramatically increases productivity.
Any small team of developers can deploy a successful web application with no system administrators and no VP of operations. GAE takes away all the complexity of databases, replication, redundancy, monitoring.
For my last project (http://www.btexty.com , an open mobile messaging platform), we switched to Python and GAE, moving away from a Ruby On Rails setup on dedicated hosting, that we used from a previous project (http://www.moqpon.com mobile coupons).
I'm very happy with that decision. For a duo of developers, forgetting about the hosting platform allows to focus on product development and dramatically increases productivity.
Criticism of Google App Engine for lack of backwards compatibility with status quo web stacks misses the point. GAE's use case is interoperable small pieces that scale, not traditional monolithic web apps.
As a platform for startups, rather than the enterprise, GAE makes a lot of sense.
As a platform for startups, rather than the enterprise, GAE makes a lot of sense.
I'm not sure I'd build anything more than a startup prototype on AppEngine. The biggest reason is that I don't like being dependent on one company for "cost of scale". With no hosting competitor for AppEngine apps, if google decides to raise prices you're screwed.
It seems best suited for small, focused applications that are developed quickly and expected to live hard and die young (lots of traffic for a short amount of time).
I can also imagine applications that designed for the user to be able to install it on their own AppEngine account. Maybe something like Wordpress for GAE, or imagine a distributed/federated version of Twitter. People might find a new app very useful, but have concerns about investing time and effort in an application from some startup that could vanish on short notice. If they had it instaled on their own app engine account, they'd at least have a chance to move their data over to another application before entropy took its toll.
I can also imagine applications that designed for the user to be able to install it on their own AppEngine account. Maybe something like Wordpress for GAE, or imagine a distributed/federated version of Twitter. People might find a new app very useful, but have concerns about investing time and effort in an application from some startup that could vanish on short notice. If they had it instaled on their own app engine account, they'd at least have a chance to move their data over to another application before entropy took its toll.
I personally prefer the simplicity of App Engine, for the types of apps that fits in that space. As I heard they have plans to support other languages later on. They are constantly improving the service.
Indeed, we're working on adding support for a new runtime language. It's on the roadmap, along with lots of other coming improvements:
http://code.google.com/appengine/docs/roadmap.html
-- @mdrcode / App Engine PM
http://code.google.com/appengine/docs/roadmap.html
-- @mdrcode / App Engine PM
Can I state the obvious first competitor: Web hosting. It's dirty cheap, has SQL, supports basic scripting python, perl, and some support even C cgi. Typical accounts have 20+GB and even more GB on daily transfers.
The main drawback is it doesn't scale, but you can move things to S3 or App Engine if you really need to. Use client-side processing with AJAX, CDN, etc.
Google's lock-in architecture isn't tempting for starters unless you plan to have a massive number of visitors.
The main drawback is it doesn't scale, but you can move things to S3 or App Engine if you really need to. Use client-side processing with AJAX, CDN, etc.
Google's lock-in architecture isn't tempting for starters unless you plan to have a massive number of visitors.
App Engine isn't aimed at the "enterprise" market that Dare is talking about. It is aimed at the Python programmer who wants an easier solution. Dare says
Hobbyists either don't spend money or spend relatively little so I discounted them as a target audience of interest.
But App Engine is a free service... people who don't want to spend money are exactly the target audience.
Hobbyists either don't spend money or spend relatively little so I discounted them as a target audience of interest.
But App Engine is a free service... people who don't want to spend money are exactly the target audience.
One thing GAE does do, in my opinion, is put the final nail in the coffin for shared hosting. Who would pay even a few dollars a month for low quality, limited shared hosting when they can get their app on Google's infrastructure for free? The only group I can think of is those who only serve static content.
I disagree. Many people are using shared hosting to run apps such as WordPress or Drupal that don't run on GAE. It's more likely that $10/month VPSes will be the death of shared hosting.
With a VPS you have to manage an entire operating system. With shared hosting you usually don't have to manage even a single application (since they usually have "one-click" installers and auto-updaters). For that reason, the vast majority of people would be better off paying $10/month for shared hosting than $10/month for a VPS.
Indeed, I did just that. I moved all my wordpress installations over to a $10 account with webfaction. I considered a VPS, but dropped that idea quickly. I wasn't interested in being a sysadmin, and I didn't have to worry about shoehorning the DB server, OS and web/application server into the space afforded by a cheap VPS.
On a VPS you should be able to install some kind of control panel that would give all the advantages of shared hosting but the flexibility (if you need it) and performance isolation of a VPS.
At least for now, App Engine only supports Python. So the coffin for shared hosting still needs a lot more nails.
[deleted]
First, App Engine will really only run things that are written for App Engine. If you have a Python web application, there's a good likelihood that it won't run on App Engine because it does things that App Engine doesn't (like relational database stuff or cron jobs or whatnot). Enterprise applications are things already written.
Second, enterprise applications are usually things that don't need to support millions of users so much as they need to have features. You're not serving like Digg or Google so much as doing non-real-time data processing. So, App Engine's infiniscale thing isn't as important as a relational database's ability to have triggers.
Third, App Engine doesn't have an SLA. That's a harsh sell for enterprise stuff. It doesn't even have a way to make a backup of the database in a dump file. That makes it an absolute no-go.
Fourth, App Engine's low cost usually isn't a concern on the enterprise level (even if it should be).
App Engine is a cool service, but whether it will become important is yet to be seen. If you're running a small site (say, 1,000,000 pageviews per month), you aren't going to have trouble scaling a MySQL/PostgreSQL database and application. You'll normally be showing one pageview per second at that rate. So, it's just as easy to go with Slicehost or Amazon. And using Amazon or Slicehost or the like has advantages - you get to use any software you find useful, you can have relational databases, etc. So, for small sites, App Engine's scaling capabilities aren't so important.
For larger sites, you just can't deal without a backup strategy, SLA, and cron jobs. It's too limiting. Plus, when you're a big site, you can hire people to deal with scaling issues.
So, it misses a little on both sides. I really want to love it, but there are just some gaps in the offering that are really important. I don't think Google needs to worry about the low side since they can't practically offer things like JOINs off their architecture, but on the high side, they need to offer data dumps and cron jobs and a little more information about the data integrity in the system (how many copies, how fast is it replicated, etc.)