Ask HN: Just replace my DBA: painless db admin and scale without the cloud?
14 comments
There is no easy out, if you really need to scale, regardless of infrastructure you are going to need someone who understands databases. If you're lucky you might be able to find someone who can do more than one job. e.g. dba+sys admin. Just make sure that they really do know both things.
Yeah, this kind of question seems really close to "I'm a business guy, all I need is a tech guy to implement my idea", which implies there's some way to avoid having to know how to do something that is core to the business.
I want to make it very clear that I'm not a business guy.
Think of it in the context of building a database-backed enterprise app that customers run in house. You don't want to ship a DBA to every customer...
Think of it in the context of building a database-backed enterprise app that customers run in house. You don't want to ship a DBA to every customer...
For that you use an embedded database system. Many enterprise apps come with embedded database systems, or require a major database system to be installed in tandem, and these things are targeted at medium sized companies that don't have DBA specific to the the department that is using the app.
or explicitly tell them how to configure their system, and what kind of a system to run it on. this doesn't help you avoid a dba, it merely avoids them having one.
It seems like I asked the question in the wrong context. Sorry about that. Given that scale is hard, how about monitoring, or management?
If it helps, I asked a more concrete version of this question focusing on PostgreSQL on stackoverflow: http://stackoverflow.com/questions/556325/postgresql-managem...
The state of the art _is_ hiring a DBA. They get paid to be up on all the database monitoring, scaling, managing and management tools. This is why DB consultants charge $200+ an hour. It's pretty arcane knowledge, and for a business of any size, the database quickly becomes the lifeblood of the organization.
This is also the reason that databases like Couch DB or Amazon Simple DB or Google's App Engine are so appealing. They all have the promise of reducing db management headaches, at the cost of sacrificing features.
This is also the reason that databases like Couch DB or Amazon Simple DB or Google's App Engine are so appealing. They all have the promise of reducing db management headaches, at the cost of sacrificing features.
Try a commerical database, such as Microsoft SQL Server (FD: I work there as a developer). To bring up a recent example, it can build indexes without locking the whole table so you wouldn't need to jump throught the hoops like the FriendFeed guys just did with MySQL. Unless your time is free it will cost you less to buy SQL Server than to write your own "online index thingie" on top of MySQL.
And if your time is free, then you are probably a startup and you can get SQL Server and bunch of other MS tools for free via http://www.microsoft.com/BizSpark/
MySpace runs on SQL Server: http://www.sqlservercentral.com/blogs/steve_jones/archive/20... and if it's good enough for them it will likely be good enough for your site as well. I'm not sure if the scale-out story is any good (there are "partitioned views" which are basically "sharding" but I never used those), but scale-up story is as good as they come - it can efficiently utilize 64-cores of the HP superdome.
I don't know much about Oracle and IBM DB2, but I know that the latter has a free version without too many restrictions on it, so give them a spin as well. If ease of use is importnat to you, common wisdom has it that Microsoft's is the easiest to use among the top commercial vendors. NB: I did not verify the common wisdom, YMMV :-)
And if your time is free, then you are probably a startup and you can get SQL Server and bunch of other MS tools for free via http://www.microsoft.com/BizSpark/
MySpace runs on SQL Server: http://www.sqlservercentral.com/blogs/steve_jones/archive/20... and if it's good enough for them it will likely be good enough for your site as well. I'm not sure if the scale-out story is any good (there are "partitioned views" which are basically "sharding" but I never used those), but scale-up story is as good as they come - it can efficiently utilize 64-cores of the HP superdome.
I don't know much about Oracle and IBM DB2, but I know that the latter has a free version without too many restrictions on it, so give them a spin as well. If ease of use is importnat to you, common wisdom has it that Microsoft's is the easiest to use among the top commercial vendors. NB: I did not verify the common wisdom, YMMV :-)
You're suggesting to use a commercial database but don't provide any reasons beyond what free databases can do.
PostgreSQL is pretty much on par in terms of features and performance with SQLServer and Oracle. So if you're looking for a free solution to roll your own then look no further.
The only reason I can see for going with a commercial DB is when you not only want commercial support (you can buy that for Postgres, too) but a shrinkwrapped "don't bother me"- solution with SLAs etc. Be prepared to pay through your nose for that, though.
In terms of usability there is no big difference between any of the commercial or non-commercial offerings. They all have their special peculiarities (which is why you always hire a DBA who is familar with your RDBMS, not just with "any" RDBMS) and the shiny GUIs that some of them provide are not normally of interest in day-to-day work.
PostgreSQL is pretty much on par in terms of features and performance with SQLServer and Oracle. So if you're looking for a free solution to roll your own then look no further.
The only reason I can see for going with a commercial DB is when you not only want commercial support (you can buy that for Postgres, too) but a shrinkwrapped "don't bother me"- solution with SLAs etc. Be prepared to pay through your nose for that, though.
In terms of usability there is no big difference between any of the commercial or non-commercial offerings. They all have their special peculiarities (which is why you always hire a DBA who is familar with your RDBMS, not just with "any" RDBMS) and the shiny GUIs that some of them provide are not normally of interest in day-to-day work.
Managed hosting works for us (Engine Yard), it is significantly cheaper than hiring your own DBA.
having managed hosting doesn't mean you don't need to a dba. In my experience all the host does (worked for a managed hosting company) is set up the servers and keep them running, their help really can't help you with tweaking queries or tuning the db. I can't speak for on engine yard specifically. But I wouldn't bank on your host knowing anything beyond the basics of databases.
Engine Yard has several excellent DBAs on staff. They tell you what queries are slow and tune the db to optimize for your type of traffic. That's why they are worth the 30% premium on top of EC2.
engine yard may be an exceptional exception to the rule. I was generalizing, and speaking of managed hosting in general. Like I said, I can't speak of engine yard specifically, I haven't worked for or with them.
What is the state of the art -- in terms of scalability, management, monitoring -- for people who need to keep their data in house? Maybe X where X:FathomDB as Xobni:Gmail? Or are we stuck hiring a DBA?