Amazon Relational Database Service(aws.amazon.com)
aws.amazon.com
Amazon Relational Database Service
http://aws.amazon.com/rds/
16 comments
It's been said here many times before: if you are an AWS value-add startup, watch out. Their march up the value chain is relentless.
AWS brings so much value to their target space that I'm willing to accept a certain amount of inherent lock-in.
I've honestly never thought this about any other service, and it's a bit disturbing, but in my opinion this reflects their successful march up that value chain.
I've honestly never thought this about any other service, and it's a bit disturbing, but in my opinion this reflects their successful march up that value chain.
The "march up the value chain" isn't about lock-in. It's about the fact there are a lot of business providing services between AWS and other services running on AWS (eg, by providing pre-packaged AMIs containing ready-to-run MySQL databases, and automatic management thereof) This is because in the early days of AWS, it offered you lots of moving parts to tweak, but you had to learn a lot to make best use of it. There was quite a lot of room in the market for paying people to take away some of the headaches in managing AWS yourself.
This is becoming less & less true as Amazon provide more services - if you're still in that space, then you ought to be looking worried.
This is becoming less & less true as Amazon provide more services - if you're still in that space, then you ought to be looking worried.
What does that mean?
In particular, what does that mean for everyone's favorite baby Dropbox?
In particular, what does that mean for everyone's favorite baby Dropbox?
It means Amazon AWS is going to keep moving from bare VMs and storage all the way up through every component required to build on the web. And I should have been more clear. When I said relentless, I mean one should consider the possibility that they will move way beyond traditional definitions of hosting.
Is it hard to imagine Amazon providing utility-priced anycast DNS?
How about high volume email delivery?
Why not a Ruby, PHP, or python hosting stack?
Whole platforms, similar to Google Wave?
Where's this leave startups? Mostly unchanged if they're looking at their markets with a very critical, paranoid eye.
Is it hard to imagine Amazon providing utility-priced anycast DNS?
How about high volume email delivery?
Why not a Ruby, PHP, or python hosting stack?
Whole platforms, similar to Google Wave?
Where's this leave startups? Mostly unchanged if they're looking at their markets with a very critical, paranoid eye.
It means that companies offering services for AWS users need to be aware that AWS is growing vertically and could easily knock them out.
AWS is something I've been meaning to look at for a while now but from what I've seen, the value add services provide a easier experience. I'm not sure how else services are going to be able to differentiate. And at that, you're only going to attract small customers as larger customers are more likely to deal with any technical issues to the lower cost. Although, at that point, they may be willing to ditch AWS altogether. How AWS continues to evolve will be really interesting.
AWS is something I've been meaning to look at for a while now but from what I've seen, the value add services provide a easier experience. I'm not sure how else services are going to be able to differentiate. And at that, you're only going to attract small customers as larger customers are more likely to deal with any technical issues to the lower cost. Although, at that point, they may be willing to ditch AWS altogether. How AWS continues to evolve will be really interesting.
Client services should be safe for a while - that's the place to add value.
This is pretty cool, but for a startup at scale it still makes way more sense to own / operate on your own. For instance for a quadruple extra large instance (68GB) it would cost about $27k a year. Considering you'll want to do master-master for uptime, that's $60k out the door every year for two of these bad boys.
Vs owning it... can get a decent dual master setup for a fraction of that.
Vs owning it... can get a decent dual master setup for a fraction of that.
Run the numbers when you consider bandwidth, spare parts, cost of employees who run out to colos in the middle of the night, flexibility. Compared to our current hosting (SoftLayer) for a site that does several million pageviews a day, the spreadsheet doesn't flip until you have hundreds of servers. One big difference though is that bandwidth at SoftLayer is rolled in, this would make Amazon hosting quite a bit more expensive.
Owning it is only part of the equation, think about hosting (you dont want to host your servers in your house on a DSL line), power usage (those bad boys will use a lot more then your desktop PC), cooling (they tend to get warm), backup solutions (an extra server, tape-libraries etc), maintenance (your super servers will break down at times) etc.
$60K a year is peanuts for most companies.
$60K a year is peanuts for most companies.
I'm facing this right now.
I just bought 6 1U servers for almost nothing for one of the companies I work with, and now we have to figure out where to put them. We had a free 1U rack space before, but we're unsure if we'll be able to get 6U of space for free.
If not, CoLo spots aren't all that cheap. Putting them in someone's house isn't an option and we don't have much money.
Sure we had the money to buy the servers (some were $100/each used from another company that was switching to EC2)- but I don't have 100's/month for racking each of them nearby. We're a super-small company without much capital and zero investment right now. Totally bootstrapped and paying for stuff via consulting. There's no way in hell we have a 60K budget and if we bring in that much total in the next 9 months I'd be happy.
I just bought 6 1U servers for almost nothing for one of the companies I work with, and now we have to figure out where to put them. We had a free 1U rack space before, but we're unsure if we'll be able to get 6U of space for free.
If not, CoLo spots aren't all that cheap. Putting them in someone's house isn't an option and we don't have much money.
Sure we had the money to buy the servers (some were $100/each used from another company that was switching to EC2)- but I don't have 100's/month for racking each of them nearby. We're a super-small company without much capital and zero investment right now. Totally bootstrapped and paying for stuff via consulting. There's no way in hell we have a 60K budget and if we bring in that much total in the next 9 months I'd be happy.
Go to webhostingtalk.com and look around. There are places that will host 1U for $50 each including bandwidth and remote reboot support.
Thanks! I was talking to a friend that runs a small CoLo and it was looking like $200-400/month for each server, depending on options.
> hosting, power usage, cooling
These are all the same thing, in the sense that they're all included by a colo provider, and even high-end colocation space will save you a ton over AWS.
> backup solutions (an extra server, tape-libraries etc)
Grandparent talked about server-level redundancy.
> maintenance (your super servers will break down at times)
24h on-site service is a standard option on server-class hardware, and not all that expensive.
> $60K a year is peanuts for most companies.
It's peanuts for large companies, but they already have their own ops departments and won't shy away from the details of handling a little hardware. Smaller companies will be acutely aware that $60k/yr is a good fraction of the cost of hiring another developer (or marketer).
These are all the same thing, in the sense that they're all included by a colo provider, and even high-end colocation space will save you a ton over AWS.
> backup solutions (an extra server, tape-libraries etc)
Grandparent talked about server-level redundancy.
> maintenance (your super servers will break down at times)
24h on-site service is a standard option on server-class hardware, and not all that expensive.
> $60K a year is peanuts for most companies.
It's peanuts for large companies, but they already have their own ops departments and won't shy away from the details of handling a little hardware. Smaller companies will be acutely aware that $60k/yr is a good fraction of the cost of hiring another developer (or marketer).
Server level redundancy isn't backup, just as RAID isn't backup. If there's a mistake, or software-caused data corruption then you still lose.
I don't really agree with you unless you work for a very large company that has an economy of scale.
Think you can compete with AppEngine for cost per computation/bandwidth/storage? Developing for AppEngine is a bit restrictive, and Amazon provides much more flexibility for not too much more money for computation/bandwidth/storage.
Also, Amazon associated services like S3 and SimpleDB really are inexpensive for what you get.
I used to base my consulting business on deploying customer aps to VPSs or cheap servers. I think that Amazon and Google AppEngine are the future.
Think you can compete with AppEngine for cost per computation/bandwidth/storage? Developing for AppEngine is a bit restrictive, and Amazon provides much more flexibility for not too much more money for computation/bandwidth/storage.
Also, Amazon associated services like S3 and SimpleDB really are inexpensive for what you get.
I used to base my consulting business on deploying customer aps to VPSs or cheap servers. I think that Amazon and Google AppEngine are the future.
I am really impressed by all those nifty moves Amazon has made in the last three years to become the major cloud infrastructure provider (I would have expected it from Google). Every step makes completely sense.
They were able to reinvent themselves, something very rare for economic giants...
What an awesome step forward. Just the thought of less and less System/Database Admin stuff, and more time to create apps is very exciting.
How much less db admin stuff, other than backup and patching, are you actually getting out of here?
I tend to spend most of my time as an application dba (doing schema design and SQL programming using lots of analytical functions for data warehouse apps). So I spend more time doing dba type work analyzing access plans and examining physical storage than mucking around with backups - of course, data warehouses are somewhat forgiving in the backup space since you can usually (if somewhat painfully) reload staged source data if things go completely south.
Looks like a cool service (although maybe postgres would have made me happier than MySQL here) other than that.
Besides, I like all that DBA stuff - it is part of creating apps.
I tend to spend most of my time as an application dba (doing schema design and SQL programming using lots of analytical functions for data warehouse apps). So I spend more time doing dba type work analyzing access plans and examining physical storage than mucking around with backups - of course, data warehouses are somewhat forgiving in the backup space since you can usually (if somewhat painfully) reload staged source data if things go completely south.
Looks like a cool service (although maybe postgres would have made me happier than MySQL here) other than that.
Besides, I like all that DBA stuff - it is part of creating apps.
I wonder how:
http://fathomdb.com/about/home
will respond.
http://fathomdb.com/about/home
will respond.
Benchmarks. Show themselves significantly faster and more configurable (finely tuned via software) and people will pay.
I suspect most people would rather use an official thing directly from Amazon, rather than a marginally (or even significantly) better thing from someone they've never heard of.
Amazon constantly impresses me with their AWS offerings and improvements. It's really refreshing to see them continue to build and improve their services so rapidly. Sometimes it seems there is a new feature or announcement every week.
Now they just need to open that west coast datacenter. :)
Now they just need to open that west coast datacenter. :)
I spent 2 ours this morning playing with RDS - not really a waste of time since I deploy most of my customer's projects to EC2, etc. Sort-of kicking the tires.
Anyway, it is easy to use, and not having to deal yourself with failure recover, managing a EBS volume if you run MySQL yourself, etc. makes it look like a good service once it comes out of beta.
A few things: right now, you can't pay a reservation fee to get the really cheap EC2 rates. (Basically RDS is a managed-for-your EC2+EBS.) Also, it would be great if they also support PostgreSQL.
Anyway, it is easy to use, and not having to deal yourself with failure recover, managing a EBS volume if you run MySQL yourself, etc. makes it look like a good service once it comes out of beta.
A few things: right now, you can't pay a reservation fee to get the really cheap EC2 rates. (Basically RDS is a managed-for-your EC2+EBS.) Also, it would be great if they also support PostgreSQL.
Does anyone have a sense of what the ACID properties of this setup are, especially under replication?
[deleted]
this is kind of a big deal.
It's not as big as it sounds -- this isn't a scalable relational database service. As far as I can see, this is just EC2 instances with mysql preinstalled.
While it does run as an instance (looks like you don't get shell access), they provide new native API calls for adding and removing nodes on demand, and they will maintain the stack and backups for you. And in the future, API calls for automatically putting replicated DBs across availability zones.
I would be less interested in it if it was not as a standalone instance per customer. Can you imagine having someone with a valid account into a shared MySQL server (and a stolen credit card) poking around all day? (especially on MySQL vulnerability announcement days)
I would be less interested in it if it was not as a standalone instance per customer. Can you imagine having someone with a valid account into a shared MySQL server (and a stolen credit card) poking around all day? (especially on MySQL vulnerability announcement days)
I would be less interested in it if it was not as a standalone instance per customer.
I absolutely agree. My point was just that this seems to be Amazon wrapping up their existing service in a way which is easier to use and more marketable, rather than actually releasing something new. (Not that there is anything wrong with this, of course -- I'm sure there are many people who don't want to be bothered with figuring out how to get MySQL configured on EC2.)
I absolutely agree. My point was just that this seems to be Amazon wrapping up their existing service in a way which is easier to use and more marketable, rather than actually releasing something new. (Not that there is anything wrong with this, of course -- I'm sure there are many people who don't want to be bothered with figuring out how to get MySQL configured on EC2.)
I agree RDS is on the lower end of novelty and excitement... And once you have VMs and a network, what could be considered truly new? Was EBS new? We can create a clustered filesystem with GNBD exports without Amazon's help, too.
ah right.
that said, it does remove one layer of complexity for the people who just want cloud-based DBs. before now we would have to either sign up with a 3rd party who would pipe data to Amazon, or manually configure EC2 instances - neither of which I particularly wanted to mess with.
I definitely want to give this a whirl.
that said, it does remove one layer of complexity for the people who just want cloud-based DBs. before now we would have to either sign up with a 3rd party who would pipe data to Amazon, or manually configure EC2 instances - neither of which I particularly wanted to mess with.
I definitely want to give this a whirl.
Agreed. When I read the headline I nearly creamed my jeans: finally! A scalable database in the cloud! Bring it!
But no, it's just MySQL in the cloud. That's great and all, but it's not going to solve the scalability problems of MySQL.
Does anybody do memcache-in-the-cloud yet?
But no, it's just MySQL in the cloud. That's great and all, but it's not going to solve the scalability problems of MySQL.
Does anybody do memcache-in-the-cloud yet?
> Does anybody do memcache-in-the-cloud yet?
You might want to look at what Northscale is up to: http://www.northscale.com/
Related: http://blog.northscale.com/northscale-blog/2009/08/mrroboto-...
You might want to look at what Northscale is up to: http://www.northscale.com/
Related: http://blog.northscale.com/northscale-blog/2009/08/mrroboto-...
I can see why they chose MySQL, but I would gladly pay for Oracle in the cloud as well. Plus, Oracle can scale up to a much bigger machine before you have to start scaling horizontally.
I'm not sure why they chose MySQL. Does mysql still scale linearly up to 8 cores? I figured most people who need 8 cores for mysql would have already gone to sharding or partitioning.
I think it has more to do with user demand, familiarity, and also the ability to migrate over, as many users already use MySQL.
5.1 scales better, and I would bet they're looking into XtraDB (if not already using it) which will go even further on multi-core machines.
edit - 5.1 (recent releases) scales better mostly due to InnoDB 1.0.4 which incorporates a number of patches from Google (Mark Callagan), Percona and more.
edit - 5.1 (recent releases) scales better mostly due to InnoDB 1.0.4 which incorporates a number of patches from Google (Mark Callagan), Percona and more.
but still, it doesn't scale as linearly as other databases like Oracle or PostgreSQL. Even compared against MySQL 5.1, PostgreSQL scales linearly all the way up to 128 cores -- so it seems like for big databases, you'd want to use something that can actually utilize all 128 cores.
I'm pretty sure they're using MySQL because it's popular, not because it scales. I figure when people hit 10 million rows in a MySQL table, they start looking at sharding/partitioning instead of buying a bigger machine.
I'm pretty sure they're using MySQL because it's popular, not because it scales. I figure when people hit 10 million rows in a MySQL table, they start looking at sharding/partitioning instead of buying a bigger machine.
MySQL scales for reads pretty well since it has a well understood and mature replication system. In contrast, the PostgreSQL replication landscape is somewhat muddy. Scaling writes is a challenge of course, and PostgreSQL might be the better choice there for getting the most out of big hardware. Scaling up can really cost though, as the price-to-performance ratio gets outside of the sweet spot when the hardware you're buying gets more exotic.
I'm not debating which database scales better. I'm questioning why anyone would use a "big" server for mysql. If you're going to do replication with mysql, you don't need 10 servers with 68 gigs of memory -- you'd probably be better off with 100 servers with 4 gigs of memory.
However, if you're going to go with "big" database servers, one should use Oracle or PostgreSQL since they can actually make use of the memory & cpu cores.
However, if you're going to go with "big" database servers, one should use Oracle or PostgreSQL since they can actually make use of the memory & cpu cores.
Any 64-bit MySQL will do fine with big memory. It is with > 16 cores that modern MySQL suffers from concurrency problems with mutex contention. These AMZN big memory instances have 4/8 cores, well within the area where MySQL performs well.
Is 10 million rows really the number? I have log files on my disk that are much bigger than that, and I manage them fine with "grep". Inside a database, I would consider this amount of data "trivial".
The price is right and their customers use it frequently. They were probably asked about MySQL a ton of times.
They can give MySQL away for free
That's why - http://www.google.com/trends?q=MySQL,+PostgreSQL
More people are googling the word MySQL means it scales better?
It means it sells better.
Does it sell better after all the "I tried Amazon's MySQL and its lack of $FEATURE killed my business" blog posts?
How would Postgresql fare?
It's already $27k/yr for their largest mysql instance. I'd hate to see what they'd charge for an even larger machine, with oracle license fees on top. :)
Actually they do offer Oracle in the cloud. All the features of Oracle full with Grid Control access, etc.
Amazon did a demo at the Oracle Open World a few weeks ago and it looked promising.
http://aws.amazon.com/solutions/global-solution-providers/or...
Amazon did a demo at the Oracle Open World a few weeks ago and it looked promising.
http://aws.amazon.com/solutions/global-solution-providers/or...
I've been playing with this while it's been in private beta. It's good, and works just like EC2, but for DB
Looking forward to some benchmarks. If its fast enough, this'll make me switch over to EC2.
Particularly latency benchmarks. Quick enough for web-commerce isn't the same as quick enough for collaboration for instance.
Doesn't Amazon realize we don't need no SQL? Spluh! (sarcasm - this is cool)
Too expensive for me.
If you don't value your time, it is expensive. If you already have some bricks-and-mortar, it is certainly expensive as an increment to what you have.
But to me and my startup, its heaven-sent, because it saves two of my commodities that I have to strictly ration: my effort, and calendar time.