Comparing Mongo DB and Couch DB(mongodb.org)
mongodb.org
Comparing Mongo DB and Couch DB
http://www.mongodb.org/display/DOCS/Comparing+Mongo+DB+and+Couch+DB
6 comments
I'm new to both (and noSQL in general), first heard about CouchDB a week or two ago. Would Couch be a good fit for building a logging system, or is there something else I should look into?
I wouldn't use it for a write-heavy logging system ( and I am a big fan of it).
Well actually it depends on what type of logging. If you plan on pouring megabytes of data/sec second into it continuously you might want to find something else. I don't think CouchDB would be a good candidate for that. It might be if you plan on archiving your post-processed logged data for a long term storage but you have to be smart about its format.
Yeah 'logging' can mean a lot of things that it is hard to give a better advice without more information.
Other DBs/tools you might want to look at:
* rrdtool http://oss.oetiker.ch/rrdtool/
* MongoDB http://www.mongodb.org/
* Redis http://redis.io/
* Custom format + plain text files
Well actually it depends on what type of logging. If you plan on pouring megabytes of data/sec second into it continuously you might want to find something else. I don't think CouchDB would be a good candidate for that. It might be if you plan on archiving your post-processed logged data for a long term storage but you have to be smart about its format.
Yeah 'logging' can mean a lot of things that it is hard to give a better advice without more information.
Other DBs/tools you might want to look at:
* rrdtool http://oss.oetiker.ch/rrdtool/
* MongoDB http://www.mongodb.org/
* Redis http://redis.io/
* Custom format + plain text files
So, both MongoDB and Redis are DB's optimized for keeping stuff heavily in memory. I wouldn't use either for heavy logging.
RRDTool is nice, but only for data where you don't mind losing records.
For straight logging, SQL is actually a better solution than a lot of those options. If you need weird queries, something like Elastic Search or Solr on top of your SQL DB can buy you quite a bit of freedom. As can denormalizing your SQL.
One other thing to consider of course, is something like syslog.
RRDTool is nice, but only for data where you don't mind losing records.
For straight logging, SQL is actually a better solution than a lot of those options. If you need weird queries, something like Elastic Search or Solr on top of your SQL DB can buy you quite a bit of freedom. As can denormalizing your SQL.
One other thing to consider of course, is something like syslog.
HBase might be a good solution, too, but I don't know a whole lot about it. CouchDB would work fine but you'd want to add some operational automation and a proxy to roll over between temporary databases since deleting old logs from a single database won't fully reduce its size and, as stated above, compaction isn't terribly fun or fast but you can avoid it by just flat out deleting old log databases.
I'm not an expert on CouchDB (and I would love someone to correct where my approach is wrong).
I could see having a CouchDB cluster per co-location, recording a document per logging event. Then back haul through replication to a central service for analysis/dashboard use.
The only tricky part I see is to delete old logging data requires a compaction period and according to http://wiki.apache.org/couchdb/Compaction it's entirely possible for the compaction process to be slower then write data and never actually finish.
Edit: Also the back end server doing analysis may need to be a BigCouch cluster to handle the load (depending on size) https://github.com/cloudant/bigcouch
I could see having a CouchDB cluster per co-location, recording a document per logging event. Then back haul through replication to a central service for analysis/dashboard use.
The only tricky part I see is to delete old logging data requires a compaction period and according to http://wiki.apache.org/couchdb/Compaction it's entirely possible for the compaction process to be slower then write data and never actually finish.
Edit: Also the back end server doing analysis may need to be a BigCouch cluster to handle the load (depending on size) https://github.com/cloudant/bigcouch
Absolutely it would be great for logging. Assuming that you don't need atomicity for your logs (eg, I want to see that log line reflect in my reports _right_ _now_) Couch is a perfect way to dump a huge amount of data really quickly. For reporting, the map/reduce system of Couch is a great fit. There's an initial learning curve -- as with anything, really -- but you'll find that Couch would be a great way to get logged data (which can be pretty unstructured) and report on it easily and quickly.
no, it wouldn't. It would be a horrible fit, imo.
You don't need versioning info for logging, and the extra compacting of the database is another aspect, since with logging heavy applications you will write all the time...
I love CouchDB but i definitely wouldn't use it for logging.
I love CouchDB but i definitely wouldn't use it for logging.
You don't need to worry about the versioning for a document that you never update. You can just blast the logfiles into the database and not worry about conflicts. Similarly, database compaction is really the biggest concern when you're updating documents, not just dumping new ones into the database. I would expect a very linear growth for a logging app.
The problem doesn't come with the inserts, it comes with the deletes, where CouchDB will keep some metadata and stubs around to support its replication use cases. In a pure logging setup this is perhaps less than ideal.
ok, my initial thought was one document per logfile, containing an increasing amount of loglines, but that's probably the worst approach possible. One document per logline which is appended to the database should be much better.
Couch's map-reduce is quite slow for ad-hoc stuff. It's excellent for frequently viewed data (as you can get stale views, and new views are sort of incrementally done), but it's slow to set up.
atomicity doesn't mean, or imply, instantaneous.
However, a document update in CouchDB is fully ACID (including the A of Atomicity). People often confuse transactions around multiple changes with ACID for reasons that I cannot fathom. CouchDB does not have support the former (they don't scale well in clusters) but absolutely supports the latter.
However, a document update in CouchDB is fully ACID (including the A of Atomicity). People often confuse transactions around multiple changes with ACID for reasons that I cannot fathom. CouchDB does not have support the former (they don't scale well in clusters) but absolutely supports the latter.
No. CouchDB is good for storing important data, slow at writing, and bad at ad-hoc querying.
MongoDB is lightning fast at writing (provided you haven't got indexes, or your indexes fit in RAM), good at ad-hoc stuff, and a little more durable than /dev/null. Just remember to back up occasionally.
MongoDB is lightning fast at writing (provided you haven't got indexes, or your indexes fit in RAM), good at ad-hoc stuff, and a little more durable than /dev/null. Just remember to back up occasionally.
Cassandra is a good fit for write heavy workload, as it is optimized for write...
I am a fan of both and decide from case to case which one to use. Both are under heavy development. I am often surprised about new features when checking the newest versions.
Basically I think this is pointing to mongo for replacing traditional website databases and couch for only special cases. Do you guys think this is fair? I haven't spent too much time with either but I'm a little bit suspicious because this is from mongodb.
Anybody know of any other direct comparisons between these?
Anybody know of any other direct comparisons between these?
For my part (and I'm biased) a database that doesn't consider durability paramount shouldn't be considered a database at all.
I don't consider redis durable for similar reasons as mongo - yet redis doesn't hide what it is or what it lacks in terms of durability... so it doesn't bother me that it isn't.
but mongo often downplays it's lack of durability, which makes me wary of using it or recommending it because people don't know how to use it.
but mongo often downplays it's lack of durability, which makes me wary of using it or recommending it because people don't know how to use it.
This is exactly the problem with MongoDB. It's practical use is a semi-durable, in-memory data store, but they advertise it as a data store of record that can be used for large, >RAM sets of any conceivable use case.
It seemed a bit dishonest seeing benchmarks that showed how MongoDB beats CouchDB in write-performance tests when there was no acknowledgement of write requests comming back to the client.
Not saying that MongoDB wouldn't be faster with full commits turn on, it is just that their design decision to pick that as the default and calling their product a 'database' seemed very strange at best, and dishonest at worst.
Not saying that MongoDB wouldn't be faster with full commits turn on, it is just that their design decision to pick that as the default and calling their product a 'database' seemed very strange at best, and dishonest at worst.
For the record: we (MongoDB) have a policy of not posting official Benchmarks. Could you clarify what benchmarks you are referring to that were posted and "dishonest"?
The only benchmarks we have ever done have been for internal testing comparisons between different versions.
The "Benchmarks" page on our site clarifies this policy: http://www.mongodb.org/display/DOCS/Benchmarks
Previously there were several third party benchmarks on this page, which we recently took down to stand firmer on our benchmark policy. In the interest of full disclosure however, you may examine the page history to verify that we have always clarified these as unaffiliated third party sourced.
The only benchmarks we have ever done have been for internal testing comparisons between different versions.
The "Benchmarks" page on our site clarifies this policy: http://www.mongodb.org/display/DOCS/Benchmarks
Previously there were several third party benchmarks on this page, which we recently took down to stand firmer on our benchmark policy. In the interest of full disclosure however, you may examine the page history to verify that we have always clarified these as unaffiliated third party sourced.
[deleted]
I am inclined to agree. I don't trust Mongo with my data, although I would consider it for things I don't mind losing. That is, if redis didn't exist.
Apart from that, I actually found the comparison fair and accurate.
Apart from that, I actually found the comparison fair and accurate.
I'm a little biased, being a pretty heavy CouchDB user, but it seems like Mongo is claiming that their auto-sharding is the defining reason to use Mongo. The forthcoming release of Couchbase 2.0 is going to have the same type of functionality, with buckets distributed across the cluster rather than required replication of the full dataset. Couchbase 3.0 should (if I remember correctly) have the functionality for both the key/value store and for view results.
And BigCouch has sharding too (edited to add: for data and views)
--dur
Here are two relevant posts from a startup that migrated from CouchDB to MongoDB:
http://seancoates.com/blogs/gimme-bar-no-longer-on-couchdb
http://seancoates.com/blogs/gimme-bar-on-mongodb
http://seancoates.com/blogs/gimme-bar-no-longer-on-couchdb
http://seancoates.com/blogs/gimme-bar-on-mongodb
I don't see how these posts are relevant for a general purpose comparison. It is one use case which doesn't fit CouchDB. Such articles give users a wrong impression. Fact is that MongoDB doesn't fit this use case either. Good solutions handle the following graph separate from the main implementation using specialized graph databases.