Ask YC: Startup using Google App Engine?
27 comments
Not sure what you mean by lack of proper database layer. The AppEngine datastore is a different beast than the usual SQL relational database, but it still has reasonable queries, sorts, etc. (Better than Amazon's simple DB IMHO.)
Main issues for a startup using AppEngine:
1) Quotas. Until they let you pay-as-you-go, AppEngine isn't really production ready. There are file # limits and quotas you may trip, depending on your target market. Google says they'll open things up by the end of this year.
2) Processing issues. AppEngine isn't a good platform for CPU-intensive processing apps. Transcoding videos, for example, is kind of a no-brainer for Amazon EC2 versus AppEngine. I think the AppEngine team is working on background processing systems but nothing is public yet.
3) HTTPS. There's none right now. You can use Google authentication, so that's one workaround but this is a deal-breaker for a number of possible apps.
These are the main ones that jump out at me. On the positive side, I think it's awesome to outsource significant infrastructure issues (servers, DB) to Google. I've got too much to code as is, so I'd much rather delegate the chunks I'm not as competent or interested in.
I'm not as concerned about lock-in, although I understand how others may take a different viewpoint. Also, tethering yourself to Google may have some repercussions if you are trying to get acquired by one of their competitors. (But I think you should just worry about your business first and not the exit strategy.)
Main issues for a startup using AppEngine:
1) Quotas. Until they let you pay-as-you-go, AppEngine isn't really production ready. There are file # limits and quotas you may trip, depending on your target market. Google says they'll open things up by the end of this year.
2) Processing issues. AppEngine isn't a good platform for CPU-intensive processing apps. Transcoding videos, for example, is kind of a no-brainer for Amazon EC2 versus AppEngine. I think the AppEngine team is working on background processing systems but nothing is public yet.
3) HTTPS. There's none right now. You can use Google authentication, so that's one workaround but this is a deal-breaker for a number of possible apps.
These are the main ones that jump out at me. On the positive side, I think it's awesome to outsource significant infrastructure issues (servers, DB) to Google. I've got too much to code as is, so I'd much rather delegate the chunks I'm not as competent or interested in.
I'm not as concerned about lock-in, although I understand how others may take a different viewpoint. Also, tethering yourself to Google may have some repercussions if you are trying to get acquired by one of their competitors. (But I think you should just worry about your business first and not the exit strategy.)
Google's AppEngine is NOT a good option for a startup today and may never be a good option for most startups.
Today, there is no service level agreement; there is no way to buy more capacity; the service has rigid standards that can be unreasonable for many applications.
I like the idea of AppEngine for both reliability and scalability, but today it offers neither. It has seen a good bit of downtime and it's limited to small sites (5 million page-views a month). Without those two things, what does it offer? Nothing that you couldn't do better on another host.
Probably more important are the rigid standards you must adhere to. File uploads are capped at 1MB. Someone tries to upload a photo larger than that and it just fails. Likewise, the datastore isn't flexible. Your advantage as a startup is that you don't have to worry about scalling to 100 million users at launch. You can use things like database joins to make your development faster.
Not only that, but the AppEngine can't handle this simple SQL:
select * from articles where author_id in (:mike_id, :adam_id) order by date desc
Why? Because you must order based on the first filter in the where clause! Now, that's an important restriction when you're dealing with a petabyte of information. It's just a handicap when you're a startup. There are restrictions all over like this - restrictions that make sense when you're storing information for hundreds of millions of users.
Basically, AppEngine forces you to scale your app from day one. That means that you loose your main advantage that you as a startup have against large firms.
And it just isn't that hard to set up a server. Install Apache and a SQL database and start coding! It's a lot easier than AppEngine's restrictions.
Today, there is no service level agreement; there is no way to buy more capacity; the service has rigid standards that can be unreasonable for many applications.
I like the idea of AppEngine for both reliability and scalability, but today it offers neither. It has seen a good bit of downtime and it's limited to small sites (5 million page-views a month). Without those two things, what does it offer? Nothing that you couldn't do better on another host.
Probably more important are the rigid standards you must adhere to. File uploads are capped at 1MB. Someone tries to upload a photo larger than that and it just fails. Likewise, the datastore isn't flexible. Your advantage as a startup is that you don't have to worry about scalling to 100 million users at launch. You can use things like database joins to make your development faster.
Not only that, but the AppEngine can't handle this simple SQL:
select * from articles where author_id in (:mike_id, :adam_id) order by date desc
Why? Because you must order based on the first filter in the where clause! Now, that's an important restriction when you're dealing with a petabyte of information. It's just a handicap when you're a startup. There are restrictions all over like this - restrictions that make sense when you're storing information for hundreds of millions of users.
Basically, AppEngine forces you to scale your app from day one. That means that you loose your main advantage that you as a startup have against large firms.
And it just isn't that hard to set up a server. Install Apache and a SQL database and start coding! It's a lot easier than AppEngine's restrictions.
> "Because you must order based on the first filter in the where clause! Now, that's an important restriction when you're dealing with a petabyte of information."
If you're dealing with queries on a petabyte of information, you'd likely have to deal with a number of issues, whether sharding, managing clusters of DB servers, etc. Sure, you have to structure your queries differently if you use Google's datastore but it's a trade-off.
Being forced to think about scale-friendly coding isn't necessarily a bad thing.
> "the datastore isn't flexible"
I think you mean the datastore doesn't support the full spectrum of queries you expect from relational DBs. You can add new properties on-the-fly, so the schemas are extremely flexible in that regard. Just throw data at your datastore.
If you're dealing with queries on a petabyte of information, you'd likely have to deal with a number of issues, whether sharding, managing clusters of DB servers, etc. Sure, you have to structure your queries differently if you use Google's datastore but it's a trade-off.
Being forced to think about scale-friendly coding isn't necessarily a bad thing.
> "the datastore isn't flexible"
I think you mean the datastore doesn't support the full spectrum of queries you expect from relational DBs. You can add new properties on-the-fly, so the schemas are extremely flexible in that regard. Just throw data at your datastore.
It depends on what you want to do with your startup. With App Engine, you're limited to Google's sandbox, that means that third party libraries can't be installed, and Python has many, many third party libraries that make a developer's life so much easier.
The initial example was PIL, or the Python imaging library, that a lot of web developers use to resize images for their sites. App Engine didn't support using PIL because it was a third party library, and when enough developers complained they opened up the API that's used for Picasa's web albums.
Another issue that you might run into is merchant accounts and libraries for credit card purchases. You can't use them, and as of a month ago when I was looking at GAE, there was no solution.
The initial example was PIL, or the Python imaging library, that a lot of web developers use to resize images for their sites. App Engine didn't support using PIL because it was a third party library, and when enough developers complained they opened up the API that's used for Picasa's web albums.
Another issue that you might run into is merchant accounts and libraries for credit card purchases. You can't use them, and as of a month ago when I was looking at GAE, there was no solution.
Google supports many third-party libraries, (e.g. Django, etc.) and many that they don't explicitly support work anyway.
What doesn't work is Python code that has C extensions. There are obviously security issues there. That's the reason the PIL was initially disabled.
What doesn't work is Python code that has C extensions. There are obviously security issues there. That's the reason the PIL was initially disabled.
I'm using GAE (for a blog and a wiki with basic functionalities) and was surprised to like to play with the BigTables.
I don't think you should care for a startup. The only thing I'm missing is the use of external apps like LaTeX to build png images from math formulas.
I don't think you should care for a startup. The only thing I'm missing is the use of external apps like LaTeX to build png images from math formulas.
You can use Javascript to convert LaTeX style notation into MathML: http://www1.chapman.edu/~jipsen/mathml/asciimath.html
I think the bottom line is AppEngine is pre-release.
There are no quota packages available and the lack of HTTPS is a real problem.
There are no quota packages available and the lack of HTTPS is a real problem.
Why limit yourself to their technology without any need? The only reason to use it would be if they let you do something you could not do otherwise.
I see one application where this could be true: giving people an application they can install for themselves, without any costs. With GAE, for the first time everybody can just install a web application, same way they can just install a plugin on their own computer. Even better, they might sometimes not be able to install something on their computer (no admin rights), but they would probably always be able to install a GAE thing.
I see one application where this could be true: giving people an application they can install for themselves, without any costs. With GAE, for the first time everybody can just install a web application, same way they can just install a plugin on their own computer. Even better, they might sometimes not be able to install something on their computer (no admin rights), but they would probably always be able to install a GAE thing.
For $100 you can get a halfway decent dedicated server, or, a high-end VPS (virtual private server, essentially a Xen or VMware instance on top of a big machine). And then you can put whatever you want on there.
I agree, go with slicehost.
Gdata store is good, but they will surely release new features. Fulltext search isn't available. But some guy who calls himself 'appengineguy' found code in the .py files of the dev server that actually supports fulltext.
Here's the link which shows how to do it: http://appengineguy.com/2008/06/how-to-full-text-search-in-g...
There's are other places like appenginefan.com and appengineguy.com which provide tips and tricks for appengine developers.
I am waiting for the pay-as-you-go model, because until that, the app isn't 'really' scalable.
I just posted a thread in the Google AppEngine groups asking about the launch of the pay-as-you-go model.
Here's the link to the thread: http://groups.google.com/group/google-appengine/browse_threa...
Here's the link which shows how to do it: http://appengineguy.com/2008/06/how-to-full-text-search-in-g...
There's are other places like appenginefan.com and appengineguy.com which provide tips and tricks for appengine developers.
I am waiting for the pay-as-you-go model, because until that, the app isn't 'really' scalable.
I just posted a thread in the Google AppEngine groups asking about the launch of the pay-as-you-go model.
Here's the link to the thread: http://groups.google.com/group/google-appengine/browse_threa...
If by "proper database layer" you mean a relational DBMS then:
First go read the responses to the article "Maybe Normalizing Isn't Normal" at http://www.codinghorror.com/blog/archives/001152.html which should convince you to normalize your database.
Secondly peruse the Google App Engine newsgroup at http://groups.google.com/group/google-appengine wherein much ado is made about how normalization is a _bad_ thing for Google App Engine databases.
Finally choose: do you want a consistent database or a very, very fast database that may be inconsistent?
I chose consistency and consequently abandoned the idea of using Google App Engine. I gain further peace of mind by _not_ abandoning decades of relational database wisdom.
Google isn't infallible.
First go read the responses to the article "Maybe Normalizing Isn't Normal" at http://www.codinghorror.com/blog/archives/001152.html which should convince you to normalize your database.
Secondly peruse the Google App Engine newsgroup at http://groups.google.com/group/google-appengine wherein much ado is made about how normalization is a _bad_ thing for Google App Engine databases.
Finally choose: do you want a consistent database or a very, very fast database that may be inconsistent?
I chose consistency and consequently abandoned the idea of using Google App Engine. I gain further peace of mind by _not_ abandoning decades of relational database wisdom.
Google isn't infallible.
I don't think the technology makes a difference.
In my opinion, a startup is about idea + team + execution. You could do it in Rails or Python or PHP (or Arc or Lisp).. :)
In my opinion, a startup is about idea + team + execution. You could do it in Rails or Python or PHP (or Arc or Lisp).. :)
Tough call right now, but certainly within 6 months, it will be a viable option.
I'd most recommend GAE if you are in any way interested in open sourcing. It's a great platform for quickly sharing and re-deploying applications.
I'd most recommend GAE if you are in any way interested in open sourcing. It's a great platform for quickly sharing and re-deploying applications.
Sure if you never need more than 100K page views a day.
Surely this depends entirely on what data you want to store and how you want to query it?
Your question is meaningless without that.
Your question is meaningless without that.
Well, not meaningless. If the answer were a big 'NO', then the question would be just fine, even if it's missing important details.
You're right that the details count, though:-)
You're right that the details count, though:-)
The App Engine is just that - a tool to build apps, not startups. Not even programs. Just apps.
Look for a startup engine.
Look for a startup engine.
What about storage limitations?
I haven't used App Engine, but the BigTable thing looks pretty cool and I think a lot of us are starting to realize that SQL relational databases aren't very good for a lot of the things we do. That said, I think the main concern is lock in. The question I'd be asking is how easy is it to get an established application off App Engine.
The main lock-in issue to me is the datastore. If you are worried about lock-in, you could try to use vanilla Django as much as possible. I think it's likely that an opensource datastore equivalent will appear, or at least something that's close enough like HBase or Hypertable.
Yeah, that's what I was getting at. As soon as you need any persistent storage you're chained to BigTable. That leaves a lot of code that needs to be rewritten if you need to move to a new storage model.
This isn't unlike the storage problems PHP had quite a few years ago. There was a separate module for every database that was supported and they lacked consistent naming conventions as well as consistent functionality. It was literally one set of functions you used to talk to MySQL and a completely different set of functions for PostgreSQL or SQL Server. This is still reflected in the documentation: http://www.php.net/manual/en/refs.database.php
For a long time there wasn't a common wrapper for all of these modules so if you started with MySQL and needed to move to Oracle or something you had to rewrite a bunch of code. I had to do that once and I never used PHP again.
It seems that's where we're at with App Engine and BigTable. I'm sure there's going to eventually be something equivalent or some sort of abstraction, but I wouldn't want to be holding my breath.
This isn't unlike the storage problems PHP had quite a few years ago. There was a separate module for every database that was supported and they lacked consistent naming conventions as well as consistent functionality. It was literally one set of functions you used to talk to MySQL and a completely different set of functions for PostgreSQL or SQL Server. This is still reflected in the documentation: http://www.php.net/manual/en/refs.database.php
For a long time there wasn't a common wrapper for all of these modules so if you started with MySQL and needed to move to Oracle or something you had to rewrite a bunch of code. I had to do that once and I never used PHP again.
It seems that's where we're at with App Engine and BigTable. I'm sure there's going to eventually be something equivalent or some sort of abstraction, but I wouldn't want to be holding my breath.
Isn't there already a decent abstraction, namely the Datastore API (http://code.google.com/appengine/docs/datastore/)? Current apps are written against that API and it's not a stretch to see an opensource BigTable on EC2 with a Datastore API. (See appdrop.com for an even simpler datastore replacement.)
Yes, that's BigTable specific. My point was that right now you have no option but to rewrite code if you need to move off App Engine and BigTable.
How is it not a stretch to see an open source BigTable on EC2? As far as I can tell BigTable has, up to this point, given Google a competitive edge. They are merely letting us come along for a ride and Amazon is a direct competitor in this market. Why in the world would they allow it to run on EC2?
How is it not a stretch to see an open source BigTable on EC2? As far as I can tell BigTable has, up to this point, given Google a competitive edge. They are merely letting us come along for a ride and Amazon is a direct competitor in this market. Why in the world would they allow it to run on EC2?
If you are coding to the Datastore API, you won't have to rewrite that code when moving to another provider who offers the Datastore API. AppDrop.com, for example, was a proof-of-concept that let you run apps on EC2 unchanged. He simply replaced the implementation below the datastore stubs.
Google may not open-source BigTable code, but as I mentioned above, there are a number of open-source projects that provide similar features. Any of these can run on EC2 together with the open-source AppEngine SDK.
Google may not open-source BigTable code, but as I mentioned above, there are a number of open-source projects that provide similar features. Any of these can run on EC2 together with the open-source AppEngine SDK.
"The question I'd be asking is how easy is it to get an established application off App Engine." ... this is the most important question.
Do you see reasons why a startup wouldn't be successful using Gapp engine? The main issue I see is the lack of proper database layer (only Gdata store), but do you think this is something that desqualifies Gapp engine?
I would be glad to hear your comments.