Y Combinator’s FathomDB Takes The Hassle Out Of Managing Your Database(techcrunch.com)
techcrunch.com
Y Combinator’s FathomDB Takes The Hassle Out Of Managing Your Database
http://www.techcrunch.com/2009/02/27/y-combinators-fathomdb-takes-the-hassle-out-of-managing-your-database/
45 comments
I would say that there's am essential piece of information that's not answered in your FAQ: What happens to my data if you go out of business?
That is a dealbreaker for me. If you go out of business (which I hope you don't) I need to get my data out, or I'll be out of business too.
With that said, I've signed up for your service, and hope to use it. It's a great idea.
That is a dealbreaker for me. If you go out of business (which I hope you don't) I need to get my data out, or I'll be out of business too.
With that said, I've signed up for your service, and hope to use it. It's a great idea.
Understand the concern. We'll have options to dump a backup to various destinations under your control. Your S3 account would be a good choice within EC2 (no bandwidth charges), but we could also do FTP.
Great, that would definitely make me sleep better at night. Particularly if it could be automated so that I would have a dump delivered every night.
And good luck with it :-)
And good luck with it :-)
I'd be a lot happier if I saw hints that FathomDB would also handle slave/mastering situations as well. Ideally it would be really easy to set those up, add new slaves, etc. I didn't see anything on this in the FAQ-- nor a clarification on the backups. Are the backups daily, are they EBS snapshots, etc?
Right now, the only thing that looked overly compelling (versus our existing RightScale/free+MySQL tools) was some of the performance analysis reports, but we get hints at that via other mechanisms.
Promising, especially if it grows. I'm hopeful because we use so much EC2 and this could help us speed things up if it adds more features.
Right now, the only thing that looked overly compelling (versus our existing RightScale/free+MySQL tools) was some of the performance analysis reports, but we get hints at that via other mechanisms.
Promising, especially if it grows. I'm hopeful because we use so much EC2 and this could help us speed things up if it adds more features.
Well they're just starting out and I guess that's where the obvious potential is.
The screencast didn't get me so excited either. Mostly because of the guy raving off in that annoying marketing slang over the most basic features, without answering any of the interesting questions.
For example he praises the ability to make queries from the web frontend and the visualization of the mysql logfiles as "enterprise features" - Oh c'mon boy.
More interesting would have been to learn more about that magic "server size" slider that goes from 1-256. Will that select the amazon instance type, mysql configuration, EBS size, or what does it do?
From what I can see they built a pretty frontend that lets you launch customized AMI images with a mysql installation. I sure hope there's more about it because this alone would take little more than 4 weeks to clone...
(We built a similar frontend for our own EC2 postgres cluster, so this is a bit more than an educated guess).
The screencast didn't get me so excited either. Mostly because of the guy raving off in that annoying marketing slang over the most basic features, without answering any of the interesting questions.
For example he praises the ability to make queries from the web frontend and the visualization of the mysql logfiles as "enterprise features" - Oh c'mon boy.
More interesting would have been to learn more about that magic "server size" slider that goes from 1-256. Will that select the amazon instance type, mysql configuration, EBS size, or what does it do?
From what I can see they built a pretty frontend that lets you launch customized AMI images with a mysql installation. I sure hope there's more about it because this alone would take little more than 4 weeks to clone...
(We built a similar frontend for our own EC2 postgres cluster, so this is a bit more than an educated guess).
This is a really great idea. I'm excited to try it out. Congratulations!
I agree completely - what a great idea! - I will definitely give these guys a try. Rightscale is in the same space, but they charge an arm and a leg.
I'd think running sql queries over the internet would slow down response time. Or is it not significant?
We're starting by targeting EC2 customers and ourselves running within EC2 to avoid this issue.
In future, we'll work with more cloud/hosting providers so that we can stay 'close' to the client, and we do have some tech that could be applied here also.
In future, we'll work with more cloud/hosting providers so that we can stay 'close' to the client, and we do have some tech that could be applied here also.
Congrats on release.
People interested in FantomDB should also consider RightScale who also offers managing and scaling MySQL on EC2 - http://blog.rightscale.com/2007/08/20/redundant-mysql-set-up...
RightScale doesn't offer MySQL-specific services such as reporting etc. but can help with treating your entire application stack "as a service"
People interested in FantomDB should also consider RightScale who also offers managing and scaling MySQL on EC2 - http://blog.rightscale.com/2007/08/20/redundant-mysql-set-up...
RightScale doesn't offer MySQL-specific services such as reporting etc. but can help with treating your entire application stack "as a service"
Congrats on the launch and good press! I wish you the best of luck. There is probably a decent number of people who would like to avoid dealing with the database infrastructure directly.
However, this launch raises a question for me, with regards to funding and YCombinator.
Obviously I'm only seeing what's in the video, but it looks like you have a provisioning system, and a web interface (like an enhanced phpMyAdmin). I assume there's backups, failover, monitoring, etc... in the background as well.
Why did you need funding for this? What did you gain from YCombinator? Perhaps I'm downplaying your efforts due to ignorance on how your system really works, but it seems pretty straightforward to build. I don't use EC2, but I'm pretty sure I could build a web app that creates new databases, an enhanced version of phpMyAdmin, and my standard monitoring/backup stuff that I do already, in a couple of days. My understanding is the "cloud" aspect of EC2 makes scaling the new db instances out much easier (than if I did it myself using dedicated servers). So I'm not sure why you felt you needed investors, months of dedicated full time work, exposure to VCs, etc...?
My intent isn't to be snarky or put down your accomplishments, but clearly I'm missing either the complexity and effort to build your solution, or I'm missing some key ingredient that YC provided that made your startup easier/possible/etc...
However, this launch raises a question for me, with regards to funding and YCombinator.
Obviously I'm only seeing what's in the video, but it looks like you have a provisioning system, and a web interface (like an enhanced phpMyAdmin). I assume there's backups, failover, monitoring, etc... in the background as well.
Why did you need funding for this? What did you gain from YCombinator? Perhaps I'm downplaying your efforts due to ignorance on how your system really works, but it seems pretty straightforward to build. I don't use EC2, but I'm pretty sure I could build a web app that creates new databases, an enhanced version of phpMyAdmin, and my standard monitoring/backup stuff that I do already, in a couple of days. My understanding is the "cloud" aspect of EC2 makes scaling the new db instances out much easier (than if I did it myself using dedicated servers). So I'm not sure why you felt you needed investors, months of dedicated full time work, exposure to VCs, etc...?
My intent isn't to be snarky or put down your accomplishments, but clearly I'm missing either the complexity and effort to build your solution, or I'm missing some key ingredient that YC provided that made your startup easier/possible/etc...
If you really could have built it in a few days, you should come work for FathomDB!
I would point out that doing this properly on EC2 is an order of magnitude harder than doing it on dedicated hardware; the assumption on EC2 has to be that things will fail and you have to build accordingly.
The simplicity of the web-based GUI and the offering hides the complexity going on under the surface. But that's the whole point of FathomDB: you just use the database, and you don't see (or care about) the hoops we're jumping through to make it all work.
I would point out that doing this properly on EC2 is an order of magnitude harder than doing it on dedicated hardware; the assumption on EC2 has to be that things will fail and you have to build accordingly.
The simplicity of the web-based GUI and the offering hides the complexity going on under the surface. But that's the whole point of FathomDB: you just use the database, and you don't see (or care about) the hoops we're jumping through to make it all work.
"doing this properly on EC2 is an order of magnitude harder than doing it on dedicated hardware; the assumption on EC2 has to be that things will fail and you have to build accordingly"
I don't follow, how is that different from dedicated hardware where you also have to assume failure will happen?
If anything I'd argue it's the opposite (but ultimately note that I think both take some work). But I say that it's perhaps the opposite because on EC2, you have EBS:
"volumes that operate with 20 GB or less of modified data since their most recent Amazon EBS snapshot can expect an annual failure rate (AFR) of between 0.1% – 0.5%, where failure refers to a complete loss of the volume. This compares with commodity hard disks that will typically fail with an AFR of around 4%"
And neither failure rate involves RAID which you can of course do with both EBS (mdadm) and disks to go for lower AFR projections.
I don't follow, how is that different from dedicated hardware where you also have to assume failure will happen?
If anything I'd argue it's the opposite (but ultimately note that I think both take some work). But I say that it's perhaps the opposite because on EC2, you have EBS:
"volumes that operate with 20 GB or less of modified data since their most recent Amazon EBS snapshot can expect an annual failure rate (AFR) of between 0.1% – 0.5%, where failure refers to a complete loss of the volume. This compares with commodity hard disks that will typically fail with an AFR of around 4%"
And neither failure rate involves RAID which you can of course do with both EBS (mdadm) and disks to go for lower AFR projections.
It sounds like my ignorance of EC2 led me to believe that your service was simpler than it really is. I meant no offense.
YC gives you some money, moral support, and connections galore. Why not YC?
Also see http://www.paulgraham.com/equity.html
Also see http://www.paulgraham.com/equity.html
Nice and simple.
Question: Is it theoretically possible to build a cloud hash-database that uses mysql on the front-end? Any joins would automatically be handled for you and the system would watch what queries you are running to dynamically balance/distribute/reindex data as needed. SimpleDB/Couch/etc work, but they require a steep learning curve for people used to working with relational databases.
At the end of the day you are just saving/getting data, I don't understand why the database service can't deal with the pain.
Question: Is it theoretically possible to build a cloud hash-database that uses mysql on the front-end? Any joins would automatically be handled for you and the system would watch what queries you are running to dynamically balance/distribute/reindex data as needed. SimpleDB/Couch/etc work, but they require a steep learning curve for people used to working with relational databases.
At the end of the day you are just saving/getting data, I don't understand why the database service can't deal with the pain.
I think it's definitely possible to front-end a cloud database with MySQL, but the question is: would it be faster or better in some other way? Relational databases are very functional, reasonably efficient and powerful, and give you some nice guarantees (ACID).
The 'alternative' databases trade off some of those properties for scalability; reintroduce them with a layer on top and I'd guess you'd lose those benefits.
The 'alternative' databases trade off some of those properties for scalability; reintroduce them with a layer on top and I'd guess you'd lose those benefits.
I would go further than just saying "reasonably efficient and powerful". More like "extremely powerful and optimized".
No offense to the grandparent, but that comment strikes me as a direct result of the hype around these key-value store DBs. The idea that you could build out a relational database using a key-value store as the backend and the end result would somehow have the underlying system's scalability properties needs to be shot down with extreme prejudice.
The philosophy behind a key-value store DB is that, if we don't need 99% of the powerful features and ad-hoc queries of a relational database, we can build something that scales much better. There is some (growing) subset of relational functionality that can easily be implemented on top of a key-value store, but the developer better know what that is.
Emulating a full-on relational database is not going to perform at anywhere near the level of a real relational database. You'd be better off working on the problem of better ways of replicating and sharding regular relational databases rather than conflating two different models with competing assumptions.
No offense to the grandparent, but that comment strikes me as a direct result of the hype around these key-value store DBs. The idea that you could build out a relational database using a key-value store as the backend and the end result would somehow have the underlying system's scalability properties needs to be shot down with extreme prejudice.
The philosophy behind a key-value store DB is that, if we don't need 99% of the powerful features and ad-hoc queries of a relational database, we can build something that scales much better. There is some (growing) subset of relational functionality that can easily be implemented on top of a key-value store, but the developer better know what that is.
Emulating a full-on relational database is not going to perform at anywhere near the level of a real relational database. You'd be better off working on the problem of better ways of replicating and sharding regular relational databases rather than conflating two different models with competing assumptions.
This seems to be the holy grail for designing a new database: something with the reliable scalability of a cloud hash DB with the front of a regular SQL.
IMO it'll never have a regular SQL front-end but would require additional instructions or statements, because it needs to know more about how to shard your particular application.
IMO it'll never have a regular SQL front-end but would require additional instructions or statements, because it needs to know more about how to shard your particular application.
how much did it cost you to sponsor that roundtable thing?
I don't think I should say, but I would say that the TechCrunch team were fantastic, put together an amazing event, and helped FathomDB get great launch exposure. I would definitely do it again, and would recommend it to others where the event's theme is relevant to your business, as 'cloud' is to FathomDB.
understandable...how about letting us know how you went about getting in touch with techcrunch, did pg phone Arrington? called yourself? emailed?
Sorry to disappoint: there is no great tale of working back channels, I just emailed them on the address they provided in the conference info.
Don't do a magnolia,
Never hire a DBA,
Effortless, reliable backup,
Get FathomDB today
Never hire a DBA,
Effortless, reliable backup,
Get FathomDB today
Are there plans for adding PostgreSQL?
Is there a benefit to using EC/2 for mysql if the majority of the work is reads rather than writes (Something like 90/10)? A large site I work (non-profit) could move all our mysql DBs to EC/2 with something like FathomDB, but I worry that it will be more expensive (and not that better performance-wise) than our current dedicated server at Rackspace.
Could anyone comment on the feasibility and speed of using this service as a database for a non-EC2 dedicated box?
That will depend greatly on how you use your database (number of queries over time, type of queries, size of returned data, etc...).
I worked on a large DB intensive eCommerce site, with a large number of writes and un-cachable data. While it worked great locally, trying to point a cluster at the DB in the backup data center (50 ms away on a dedicated backbone line) absolutely killed performance.
Also, for another eCommerce site, the local dev instance on my laptop boots in about ~ 1 minute when running against a local database. It takes about ~10 minutes when running against the corporate dev db over the VPN connection (~50ms added latency).
However, for an application with more cacheable data, and less queries per second, it would matter less.
I worked on a large DB intensive eCommerce site, with a large number of writes and un-cachable data. While it worked great locally, trying to point a cluster at the DB in the backup data center (50 ms away on a dedicated backbone line) absolutely killed performance.
Also, for another eCommerce site, the local dev instance on my laptop boots in about ~ 1 minute when running against a local database. It takes about ~10 minutes when running against the corporate dev db over the VPN connection (~50ms added latency).
However, for an application with more cacheable data, and less queries per second, it would matter less.
I guess it would be the standard constraints for anything over a network, particularly a WAN: latency and throughput.
If you have a lot of small queries latency might kill you, and if you have large result sets throughput might kill you. (Also, I don't know how safe the mysql connection is outside a firewalled LAN. Maybe that's not an issue.)
If you have a lot of small queries latency might kill you, and if you have large result sets throughput might kill you. (Also, I don't know how safe the mysql connection is outside a firewalled LAN. Maybe that's not an issue.)
Take a look at http://blog.rightscale.com/2007/11/13/mysql-performance-on-a... - RightScale Blog
MySQL performance on Amazon EC2
Nice! if you can take the guesswork out of backups and slaves, you'll save me (and a lot of others) a ton of work!
Is this something you could replicate your existing MySQL database to in order to have a hot spare? That might be a way to provide service to people who aren't on EC2.
Definitely possible - you get MySQL root on the box. If this is a popular scenario we can also make it simple from our GUI..
[deleted]
Now if FathomDB had a rails plugin that made it stupidly easy to shard, they'd have a new customer.
when will pricing be available? will there be a free tier?
please let me into the beta! [email protected]. My site opensourcerails.com is going down every day because of dreamhosts shitty mysql server
[deleted]
oh man, that is really a great idea! If this service was around 6 months ago, ma.gnolia wouldn't have to go down.
Congrats Justin. Looks really impressive
Great work Justin!
Congratulations Justin! Great job, looks super slick.
Nice quote. The last company I worked for developed trading software for foreign exchange and money markets. Only you couldn't ever sell it to traders; they were too afraid of being put out of a job. You had to go to the Head of Desk and sell him on the benefits.
As it turns out few of them actually were put out of work, many really were able to move up into trading more volume and higher-value instruments. But if you say that up front, no-one will believe you.