Migrating Graphite from SQLite to PostgreSQL(obfuscurity.com)
obfuscurity.com
Migrating Graphite from SQLite to PostgreSQL
http://obfuscurity.com/2013/12/Migrating-Graphite-from-SQLite-to-PostgreSQL
3 comments
There is no reason why one would use SQLite in real production. Even during testing and dev, I am a firm believer of testing against real database that you are going to use in production. The problem "it's hard to setup a new database" is a result of not willing to invest in re-provisionable system.
Except that while Graphite is a "real web application", it's generally internal-facing only. As such, I never looked at it as needing a true "production-ready" RDBMS with real concurrency. And to be honest, we got by just fine with it in production at Heroku and GitHub, at heavy volume, running in a single server on SQLite. It never bit us, and all things considered, was a reasonable choice.
That said, knowing what I know now, I would've never done that. ;-)
That said, knowing what I know now, I would've never done that. ;-)
I disagree because it is internal facing lol. If the software is capable of running on a true production-ready DBMS, there is no point why it can't be done already.
If SQLite is cheap to use because it's a single file, well, a sudo apt-get install PostgreSQL and then a few commands to set up the database and user password won't take you longer than 10 minutes. That's what I am arguing. There is no reason why we would use SQLite for heavy loading application such as logging. And I think we can now agree that's indeed true and I think we should promote people to use production-ready DBMS even during dev and testing, because those are the one you are going to use in production :)
If SQLite is cheap to use because it's a single file, well, a sudo apt-get install PostgreSQL and then a few commands to set up the database and user password won't take you longer than 10 minutes. That's what I am arguing. There is no reason why we would use SQLite for heavy loading application such as logging. And I think we can now agree that's indeed true and I think we should promote people to use production-ready DBMS even during dev and testing, because those are the one you are going to use in production :)
Nobody ever said it /can't/ be done. I would never have argued against using a real RDBMS. But I've often been guilty of telling folks that SQLite was "good enough".
Installing PostgreSQL as you described is not "good enough". If you're going to bother with that, you need to understand the database enough to tune it properly, setup backups, test restores, etc. There are more considerations than simply installing it from apt-get and then forgetting about it.
Installing PostgreSQL as you described is not "good enough". If you're going to bother with that, you need to understand the database enough to tune it properly, setup backups, test restores, etc. There are more considerations than simply installing it from apt-get and then forgetting about it.
Yes that's true, but by default they come with good configurations which should enable better write and read in most cases against SQLite :)
[deleted]
There are plenty of reasons.
TL;DR - graphite devs don't understand postgresql, graphite user misdiagnoses the problem and blames sqlite for it.
* Single file to back up
* Unix fs permissions sufficient for security
* Simplicity valuable over write concurrency
* Ability to write functions/aggregates/collations in the language you're embedding sqlite in
* No need for maintaining and securing a system daemon
TL;DR - graphite devs don't understand postgresql, graphite user misdiagnoses the problem and blames sqlite for it.
The way I read this, this _is_ a problem with SQLite (somewhat). Both schemas limit the length of graph names, but SQLite ignores that part of the schema.
One can discuss whether it is a good idea to have that arbitrary, low limit, but that is not what this is about.
I agree that there are plenty of reasons to use SQLite in production. IMO, the conclusion should have been that one should run the same database in development, test, and production.
One can discuss whether it is a good idea to have that arbitrary, low limit, but that is not what this is about.
I agree that there are plenty of reasons to use SQLite in production. IMO, the conclusion should have been that one should run the same database in development, test, and production.
Excuse my french, but you're an idiot. I don't know if you simply didn't read the article, or are incapable of grasping the facts within.
There is no reason why one would use SQLite in real production.
Yes there is. Small storage for a local app? SQLite is ideal.
Yes there is. Small storage for a local app? SQLite is ideal.
As a fan of both databases, I just have to say, you don't always need all the constraints.
Also, at the very least, graphite should have been able to enforce them at the app level in addition to at the DB level.
Also, at the very least, graphite should have been able to enforce them at the app level in addition to at the DB level.
I completely agree. I was rather disappointed to find that it didn't.
Is it by default it ships with SQLite or just Doc?
It is configured to use SQLite by default, but it doesn't ship /embed/vendorize it. You're required to install the sqlite bindings yourself.
I'm not sure what you're asking here. Graphite uses SQLite by default. What is "Doc"?
So it ships with SQLite. Doc as in documentation - telling users to get started let's try SQLite.
Meh, SQLite is awesome. So is PostgreSQL, but this rant is uncalled for.
How is the article a rant? Or are you referring to the one of the comments above?