Are Commercial Databases Worth It?(codingthewheel.com)
codingthewheel.com
Are Commercial Databases Worth It?
http://www.codingthewheel.com/archives/are-commercial-databases-worth-it
12 comments
Isn't this the angle that Redhat, IBM, and now Oracle are taking? I.E. the software is free, but the support contract is gonna cost you.
I recently started working with a Fortune 1000 that has a support agreement with RedHat for, among other things, JBoss. The developers and admins rave about the support provided but disparage other vendors to whom much more was paid.
Yes. My instinct is that for companies that need that "warm fuzzy" of support their contracts make sense.
The difference being that Oracle getting itself into an unrecoverable state is really really rare whereas MySQL corrupts data at the drop of a hat. Seriously, even Monty himself has blogged about it.
Supporting your own database effectively means writing your own database. Once you start hacking at it to fix a problem it's a fork and you're on your own. Any organization that puts its eggs in one person like this is bonkers.
Supporting your own database effectively means writing your own database. Once you start hacking at it to fix a problem it's a fork and you're on your own. Any organization that puts its eggs in one person like this is bonkers.
There are tens of thousands of organizations that disagree with you. I bet that most of them don't fork MySQL when they hit bugs (assuming they do hit bugs).
I'm not claiming that MySQL is perfect (or even good), only that many organizations think it's good enough, and are willing to take that risk.
I'm not claiming that MySQL is perfect (or even good), only that many organizations think it's good enough, and are willing to take that risk.
No, they buy support from MySQL AB, which isn't cheap.
IBM used to market in exactly this way: "Nobody ever got fired for choosing IBM".
That's actually an excellent idea. It solves a clear problem that everyone sees, but noone can do amything about due to company politics, plausible deniability, and career moves.
If you're not working on it you should :-)
If you're not working on it you should :-)
Is it really possible to audit complex systems in a reasonable amount of time, & would they have to call whenever they make any small changes?
I can't wait for the day when such systems become plug & play
I can't wait for the day when such systems become plug & play
I worked at HP in IT. We had literally hundreds of stupid little internal projects. Some were giant, some were 3 aspx pages. All of them used SQL Server or Oracle. But none of them came anywhere close to leveraging the databases.
I can think of only one app that was large enough and complex enough to really require a "commercial" db, and it was talking with SAP.... ohh, and it ran into constant problems.
What I don't understand is that other than features, the other argument is "Support". I have never seen microsoft get on the phone with a guy for free (yes you can pay...) and help out. But you know what, you can pay for a guy to help you out with postgres.
And you know, by going with proprietary databases, that cost so much money, there were severe restrictions on getting a database. They were in the hands of another team, who took weeks or months to get us setup. I know a part of that is server management, but another part was cost cutting on licenses. Remember that a small cause can have a very large effect in a corporate culture.
I can think of only one app that was large enough and complex enough to really require a "commercial" db, and it was talking with SAP.... ohh, and it ran into constant problems.
What I don't understand is that other than features, the other argument is "Support". I have never seen microsoft get on the phone with a guy for free (yes you can pay...) and help out. But you know what, you can pay for a guy to help you out with postgres.
And you know, by going with proprietary databases, that cost so much money, there were severe restrictions on getting a database. They were in the hands of another team, who took weeks or months to get us setup. I know a part of that is server management, but another part was cost cutting on licenses. Remember that a small cause can have a very large effect in a corporate culture.
The "support" argument is almost always a bad one. Support for infrastructure products is never free; it's sometimes bundled with the license fee, but it's never free. Support is available in the form of paid consultants for any popular piece of open-source infrastructure. What commercial products DO have is official support, which provides some degree of consistency. Even that isn't a strong argument though; it's easy to tell good consultants from bad by asking for references.
My limited experience with "official" support channels is that they're manned by people trained to be "tech support" rather than actual experts on the subject.
Actually, I've talked with quite a few techs in Microsoft Professional Support that know what they are talking about. But you do have to pay a few hundred dollars per incident for that.
It's nice to know they are there when you're doing a hairy Exchange migration...
It's nice to know they are there when you're doing a hairy Exchange migration...
Where a lot of people make their mistake is in spending tens or hundreds of thousands of dollars on commercial databases and then doing nothing more than simple SELECT, UPDATE, INSERT type operations with them, bury them under an ORM and try to pretend they aren't there.
Can you get your money's worth out of a database like Oracle? I think you absolutely can if you USE the features it comes with. If you don't want to do that, or don't need those features, then you are throwing your money away.
Can you get your money's worth out of a database like Oracle? I think you absolutely can if you USE the features it comes with. If you don't want to do that, or don't need those features, then you are throwing your money away.
CONNECT BY PRIOR on the other hand can be a life saver.
There are projects where I really love using Oracle (and to a lesser degree SQL Server). It has everything to do with features that solve problems.
I loathe DB2 though. Also, there are some weird smaller commercial databases like Pervasive that seem to add nothing but difficult interfaces.
There are projects where I really love using Oracle (and to a lesser degree SQL Server). It has everything to do with features that solve problems.
I loathe DB2 though. Also, there are some weird smaller commercial databases like Pervasive that seem to add nothing but difficult interfaces.
> CONNECT BY PRIOR on the other hand can be a life saver.
FWIW, the latest beta release of PostgreSQL supports recursive queries (using the SQL standard WITH RECURSIVE rather than CONNECT BY, but still).
FWIW, the latest beta release of PostgreSQL supports recursive queries (using the SQL standard WITH RECURSIVE rather than CONNECT BY, but still).
Thanks for the tip!
The list of features that justify using a commercial database keeps getting shorter each year.
The list of features that justify using a commercial database keeps getting shorter each year.
Enterprise world is so full of insanity that this is just a non-issue. The company is going to waste a lot of money but who cares? You, as a developer, are lucky enough to work with Oracle, which is an excellent db.
Here, I am forced to fill excel sheets for deployments (I have to write "compile this package here" or "copy/paste the executable here" -- no kidding), which are checked by a committee on tuesday and executed on the next thursday. No deployments are allowed in other days of week, except for very critical stuff.
So, the insanity is everywhere, and choosing the wrong database vendor is just the top of the iceberg.
Just add TPS reports and you're working at Initech.
Have you ever been on a day-long 7-person email conversation to change one string to another in a text file, and 6 of those people were IT people? I feel your pain.
Have you ever been on a day-long 7-person email conversation to change one string to another in a text file, and 6 of those people were IT people? I feel your pain.
I've used MySQL extensively for personal projects (all of mine have been at a pretty small scale). I've also used both Oracle and SQL Server for projects at large companies.
There's probably not much of an advantage in the database engine itself, but the SQL Server suite shined in 2 areas: analytics and reporting. The 2 add-on components made large-scale data mining & model-building much simple, and also automated the generation & delivery of complex reports tracking many metrics. I could probably do both with MySQL, but only with much more effort.
There's probably not much of an advantage in the database engine itself, but the SQL Server suite shined in 2 areas: analytics and reporting. The 2 add-on components made large-scale data mining & model-building much simple, and also automated the generation & delivery of complex reports tracking many metrics. I could probably do both with MySQL, but only with much more effort.
"I could probably do both with MySQL, but only with much more effort."
I know you could. You know you could. Most of the people here know you could.
But your boss doesn't want to bet on you. That would require him to manage you. Easier to spend a bunch of money through a capital expenditure request to get a piece of commercial software that already does it. When your boss(es) don't get the job done, it's easier to blame a vendor than an employee.
I know you could. You know you could. Most of the people here know you could.
But your boss doesn't want to bet on you. That would require him to manage you. Easier to spend a bunch of money through a capital expenditure request to get a piece of commercial software that already does it. When your boss(es) don't get the job done, it's easier to blame a vendor than an employee.
Or maybe said boss would spend less just buying software than paying an employee to duplicate an maintain the functionality. For a large enough company, one software license is not that big of an expense.
In tandem with the entire "Buying a $BigCompany database never hurt anyone" and "software vs support" debate, a couple of specific points I'd like to make..
- Security
Securing data in a database is pretty important if you're dealing with any kind of sensitive data. Levels of sensitivity may depend on what you do and may vary over SSNs to sales numbers for next quarter. AFAIK (correct me if I'm wrong) - PGSql/MySQL don't really offer concrete solutions to protect data - apart from common sense like patches, secure passwords, et al. Now look at offerings from the big co's. Companies of any kind will feel more comfortable with these.
- Auditing/Compliance
Are you a public corporation? Semi-large? Do you have payroll or other info in a database? Enter Sarbanes Oxley! Easier to have solutions built by the db vendors rather than deploy 3rd party stuff.
- Security
Securing data in a database is pretty important if you're dealing with any kind of sensitive data. Levels of sensitivity may depend on what you do and may vary over SSNs to sales numbers for next quarter. AFAIK (correct me if I'm wrong) - PGSql/MySQL don't really offer concrete solutions to protect data - apart from common sense like patches, secure passwords, et al. Now look at offerings from the big co's. Companies of any kind will feel more comfortable with these.
- Auditing/Compliance
Are you a public corporation? Semi-large? Do you have payroll or other info in a database? Enter Sarbanes Oxley! Easier to have solutions built by the db vendors rather than deploy 3rd party stuff.
PGSql/MySQL don't really offer concrete solutions to protect data - apart from common sense like patches, secure passwords, et al. Now look at offerings from the big co's.
You're comparing commercial databases favorably to Open Source databases on the grounds of security? This differs vastly from my experience. Oracle is an impenetrable mess:
- do or don't use TNSLSNR
- if you use TNSLSNR, do or do not secure it with password protection
- my experience with Oracle has been that port allocation is rather nondeterministic. I've attempted to perform identical installations several times and ended up with TNS and the databases on weird ports for reasons I can't explain.
- user management of oracle behaves differently on different platforms, i think. IIRC it integrates with NT credentials on Windows, and UNIX credentials for certain connection situations, but there's also a username/password file that controls who can connect
- you issue alter password commands in unquoted SQL. Because of this you don't get to use things like mixed case (?) or spaces or most punctuation characters
- but you CAN use quotes around the password, in which case it behaves sanely. I didn't even know this until I looked it up it's so uncommon.
- Oracle attempted to market Oracle 9 as "Unbreakable": http://news.cnet.com/8301-10784_3-5808928-7.html . They had enjoyed a pass from hackers since systems/network folks tend to bury the DB at the back of the architecture, rarely accessible on the public internet. After those shenanigans security minded DBAs had not the best time of their careers as they had dozens of security patches per quarter for a while.
Perhaps the commercial guys have security "features", such as row based security or encryption, that the free products don't have. While it's surely better to have those than not, I'm not convinced applying those proprietary features to an otherwise insecure design is satisfactory "defense in depth". You can, however, have sufficient defense in depth without those features.
Sarbanes Oxley isn't too bad: the crux of it is that you have to enumerate your controls and stick to them. I understand PCI compliance is another matter. I don't know much about it, though.
You're comparing commercial databases favorably to Open Source databases on the grounds of security? This differs vastly from my experience. Oracle is an impenetrable mess:
- do or don't use TNSLSNR
- if you use TNSLSNR, do or do not secure it with password protection
- my experience with Oracle has been that port allocation is rather nondeterministic. I've attempted to perform identical installations several times and ended up with TNS and the databases on weird ports for reasons I can't explain.
- user management of oracle behaves differently on different platforms, i think. IIRC it integrates with NT credentials on Windows, and UNIX credentials for certain connection situations, but there's also a username/password file that controls who can connect
- you issue alter password commands in unquoted SQL. Because of this you don't get to use things like mixed case (?) or spaces or most punctuation characters
- but you CAN use quotes around the password, in which case it behaves sanely. I didn't even know this until I looked it up it's so uncommon.
- Oracle attempted to market Oracle 9 as "Unbreakable": http://news.cnet.com/8301-10784_3-5808928-7.html . They had enjoyed a pass from hackers since systems/network folks tend to bury the DB at the back of the architecture, rarely accessible on the public internet. After those shenanigans security minded DBAs had not the best time of their careers as they had dozens of security patches per quarter for a while.
Perhaps the commercial guys have security "features", such as row based security or encryption, that the free products don't have. While it's surely better to have those than not, I'm not convinced applying those proprietary features to an otherwise insecure design is satisfactory "defense in depth". You can, however, have sufficient defense in depth without those features.
Sarbanes Oxley isn't too bad: the crux of it is that you have to enumerate your controls and stick to them. I understand PCI compliance is another matter. I don't know much about it, though.
Is the issue really commercial DBs? It seems like most projects that use a (SQL) DB are solving a problem they shouldn't have, and doing it in an overly complicated manner.
If a project truly requires ACID to the point there's a sensible worry about the disk holding the right data, I'm pretty sure I'd save money (and hair) paying for a medium server and a license to deal with that (particular) data. It's really rare to run into a problem that requires both that level of reliability and needs performance beyond what a medium server can reasonably handle. I think most projects are far far more likely to screw the data in some other way, so one could question if it's worthwhile.
I'm saying this having had to implement true ACID transactions (and replication) myself. It's a ton of work getting it right through all of the layers of libraries, os, filesystems, controllers, and drives. It's hard to trust user-level programs to be able to do it given an arbitrary stack below them. I place more trust in a system that can cut through some of those layers (or call across campus and control how some of those layers work) and work with particular hardware.
I place even more trust in someone screwing the data up at a higher level and it really not mattering nearly as much as people make it out to anyway. Even in "critical" cases like storing financial transactions, it usually just results in people having to call around or get in and fixing a few things.. which is WAY cheaper than the huge cost to lower the odds by 90%, and you've got to be prepared to do that anyway.
If a project truly requires ACID to the point there's a sensible worry about the disk holding the right data, I'm pretty sure I'd save money (and hair) paying for a medium server and a license to deal with that (particular) data. It's really rare to run into a problem that requires both that level of reliability and needs performance beyond what a medium server can reasonably handle. I think most projects are far far more likely to screw the data in some other way, so one could question if it's worthwhile.
I'm saying this having had to implement true ACID transactions (and replication) myself. It's a ton of work getting it right through all of the layers of libraries, os, filesystems, controllers, and drives. It's hard to trust user-level programs to be able to do it given an arbitrary stack below them. I place more trust in a system that can cut through some of those layers (or call across campus and control how some of those layers work) and work with particular hardware.
I place even more trust in someone screwing the data up at a higher level and it really not mattering nearly as much as people make it out to anyway. Even in "critical" cases like storing financial transactions, it usually just results in people having to call around or get in and fixing a few things.. which is WAY cheaper than the huge cost to lower the odds by 90%, and you've got to be prepared to do that anyway.
Not sure where he dug up that price chart. We run Small Business Server and it gets us the whole OS, along with Enterprise Edition on 4 processors for like $2,000 all in. That's what, 10 billable hours to pay for it? Cheaper than the box it's running on.
SQL Server is cheap, and like he said, it's miles better than the free options.
SQL Server is cheap, and like he said, it's miles better than the free options.
That price chart is consistent with what I've seen in projects at big companies I've worked with. SQL Server costs of over $20k/box are normal - and that applies to UAT and test boxes also.
The real danger of using expensive proprietary databases is that because of the cost per box you tend to design your projects to use one big expensive server instead of multiple small, cheap servers. The big server approach eventually hits a scalability ceiling and then you have to pay for X more big expensive servers and re-architect a bit. This may sound absurd but I've seen it happen more than once. The killer feature of the free, open-source databases is as much the "free" part as it is the "open source" part.
The real danger of using expensive proprietary databases is that because of the cost per box you tend to design your projects to use one big expensive server instead of multiple small, cheap servers. The big server approach eventually hits a scalability ceiling and then you have to pay for X more big expensive servers and re-architect a bit. This may sound absurd but I've seen it happen more than once. The killer feature of the free, open-source databases is as much the "free" part as it is the "open source" part.
Are you honestly arguing that $2,000 in software costs is a big enough price tag to sway your technology decision?
We run a box that cost ~$4k all in ($2k hardware + the aforementioned software costs), and stash it in a cage that costs $400/month to keep it connected power and a fat pipe. Assuming it lasts us 4 years, that's $23,200 in server costs over the life of the box.
Now assume we'd skimped on a LAMP stack and pocked ourself a cool two grand in software costs. At the end of that 4 years, we'd still have spent $21,200 keeping our servers alive.
And it scales out exactly the same way. We can throw hardware at the problem every bit as easily as you. It's just, what? 9% more expensive over time, depending on your math. And that's on top of a ridiculously low price tag anyway.
Web servers are dirt cheap, regardless of which stack you go with. The price difference between commercial and free databases are just noise. In the end, it's certainly not something you should consider when choosing a technology.
We run a box that cost ~$4k all in ($2k hardware + the aforementioned software costs), and stash it in a cage that costs $400/month to keep it connected power and a fat pipe. Assuming it lasts us 4 years, that's $23,200 in server costs over the life of the box.
Now assume we'd skimped on a LAMP stack and pocked ourself a cool two grand in software costs. At the end of that 4 years, we'd still have spent $21,200 keeping our servers alive.
And it scales out exactly the same way. We can throw hardware at the problem every bit as easily as you. It's just, what? 9% more expensive over time, depending on your math. And that's on top of a ridiculously low price tag anyway.
Web servers are dirt cheap, regardless of which stack you go with. The price difference between commercial and free databases are just noise. In the end, it's certainly not something you should consider when choosing a technology.
No, you miss my point in two distinct ways. $20k per server is middle of the road for a MS SQL in a big enterprise. It's on the low side for Oracle. It happens all the time; I've seen it (and argued against it.)
The other point, though, is that it's not the $2k or $20k in software costs. Really, to a big enterprise, $20k is nothing. The real problem is that it ties you down to making architectural decisions because of the cost of the software. Say you're right and you can license MS SQL for only $2k. What happens when you start partitioning your data and end up with 50 servers? Suddenly that $2k is $100k... or if you're on the enterprise plan it's $20k * 50 = $1,000,000. And I'm not speculating. I've lived thorough this scenario more than once.
Also, I object to the phrase "skimped on a LAMP stack." It's not skimping. Cost factors aside, the LAMP stack is better.
The other point, though, is that it's not the $2k or $20k in software costs. Really, to a big enterprise, $20k is nothing. The real problem is that it ties you down to making architectural decisions because of the cost of the software. Say you're right and you can license MS SQL for only $2k. What happens when you start partitioning your data and end up with 50 servers? Suddenly that $2k is $100k... or if you're on the enterprise plan it's $20k * 50 = $1,000,000. And I'm not speculating. I've lived thorough this scenario more than once.
Also, I object to the phrase "skimped on a LAMP stack." It's not skimping. Cost factors aside, the LAMP stack is better.
I actually do get your point. Let me paraphrase it to ensure we're on the same page:
- you have seen organizations pay $20k for SQL Server
- 50 boxes like that will cost a lot of money.
- SQL Server is actually worse than (at least one of) the free options
- I've actually purchased SQL server, bundled with the OS and all other necessary software for $2k.
- 50 boxes like that won't cost significantly more than they would with free software, if you consider the dominant cost of operating it, which is hosting fees.
- SQL Server is actually really good.
Reading comprehension failure. He wrote $20k. You read $2,000.
Ah, but he's speculating. I actually spent $2,000. That's what it costs to provision a production box with Windows Server and SQL Server.
1) No-one actually charges per-box but per-CPU or per user.
2) Dev licenses are usually free - the development edition of MSSQL is a free download from MSDN
3) Far more absurd is "sharding", which IBM invented in the 80s and all the major vendors abandoned in the 90s.
2) Dev licenses are usually free - the development edition of MSSQL is a free download from MSDN
3) Far more absurd is "sharding", which IBM invented in the 80s and all the major vendors abandoned in the 90s.
My favorite phrase in this article was "four-dimensional enraged-leprechaun hypercube visualization".
"Well sure, you spend a ridiculous amount of money on IT. Ever wonder why?"
So you have someone to blame, who is tough to identify with open source software.
Never forget, in many large enterprises, these decisions are not technical, they are political.
So you have someone to blame, who is tough to identify with open source software.
Never forget, in many large enterprises, these decisions are not technical, they are political.
these decisions are not technical, they are political.
Wrong, they're financial. The only factor is what the auditors will sign off on.
Wrong, they're financial. The only factor is what the auditors will sign off on.
Yes. If you are a large organization and have the money for licenses and DBAs, they're definitely worth it.
Not everyone is a startup. Not everyone cares about clever hacks and rag-tag scalability and bragging about how they scaled up to 10k transactions per second on YouTube. Sometimes you just want to throw $10M at the fucking thing and have it work so everyone in your company gets the data they need.
Not everyone is a startup. Not everyone cares about clever hacks and rag-tag scalability and bragging about how they scaled up to 10k transactions per second on YouTube. Sometimes you just want to throw $10M at the fucking thing and have it work so everyone in your company gets the data they need.
I also think there are established communities (and codebases) that have existed for decades built around some of these products (like Oracle or DB2 on the mainframe). This is also a consideration for many organizations large and small.
I'm not saying that the open source solutions can't be competitive from a technological standpoint, simply that they have to win on cost, availability of experienced people for dev and admin, and overcome what amounts to a late start in the market.
Plus, on the low end of the market, aren't there express editions of SQL Server and Oracle available for free? For lightweight use these would seem to be reasonable competitors to open source.
I'm not saying that the open source solutions can't be competitive from a technological standpoint, simply that they have to win on cost, availability of experienced people for dev and admin, and overcome what amounts to a late start in the market.
Plus, on the low end of the market, aren't there express editions of SQL Server and Oracle available for free? For lightweight use these would seem to be reasonable competitors to open source.
It also means non-techies in big organisations can hire people by certification and have a reasonable idea of what they're getting. "We run Oracle."
I think this is a huge part of the issue. I have worked professionally on the big three commercial rdbms (DB2 distributed - not mainframe -, SQL Server, and Oracle). I consider myself a SQL hacker with some DBA experience and knowledge, mainly around DB design and performance tuning.
I simply don't care what rdbms I coding to, I can tweak to leverage quirkiness in T-SQL or PL/SQL if needed or advantageous to a project. And, <whispering>, PL/pgSQL.
A common experience for me when job searching is to apply for a job where, for example, they are primarily looking for Oracle PL/SQL development. I've been doing SQL coding for roughly 8 years, the last 2 years or so have been primarily in Oracle PL/SQL. I have been turned down for interviews with feedback that says "oh, we need someone with 5 years of Oracle development work."
If it's techies doing the resume reviews, I always get at least an interview. Sometimes even then, I have to sell the fact that SQL skills transfer nicely between rdbms harder than I should.
I simply don't care what rdbms I coding to, I can tweak to leverage quirkiness in T-SQL or PL/SQL if needed or advantageous to a project. And, <whispering>, PL/pgSQL.
A common experience for me when job searching is to apply for a job where, for example, they are primarily looking for Oracle PL/SQL development. I've been doing SQL coding for roughly 8 years, the last 2 years or so have been primarily in Oracle PL/SQL. I have been turned down for interviews with feedback that says "oh, we need someone with 5 years of Oracle development work."
If it's techies doing the resume reviews, I always get at least an interview. Sometimes even then, I have to sell the fact that SQL skills transfer nicely between rdbms harder than I should.
IT and most departments have a budget, the more you spend the more budget you get next year. Nobody tries to reduce budget by saving here and there. Au contraire, you have to come up with ways to increase headcounts and needed resources so you get more budget next year.
Unfortunately that's how the corporate world works.
Unfortunately that's how the corporate world works.
I've always called this plausible deniability in corporate culture. "It's not my fault the DB is down. I'm on the phone with Oracle right now, they'll have a fix soon". The OSS solution is much riskier. "I chose this free DB, and now it's broken." In large corporations, the first is much safer for the employee, but much more expensive for the corporation.
As a half-serious idea for a company, I thought of IT insurance. Rather than paying support to Oracle, we will audit your solution and charge premiums based on the riskiness of your setup. In case your OSS DB goes down, the insurance company would pay the downtime.
The company would provide two services. 1) I expect our "plausible deniability" would be cheaper than paying Oracle, and 2) we put a pricetag on the reliability of your setup.