At high concurrency, I'd argue you will probably end up being more cpu efficient as well. The cost of context switching effectively larger frames and getting into and out of privileged mode can get expensive.
I agree with you in theory, in practice, my experience has been a bit different. Specifically,
1. Since Mongo has a database (or maybe collection now) level lock, doing rebalancing under a heavy write load is impossible.
3. Mongos creates one thread per connection. This means that if you've got to be very careful about the number of clients you start up at any given time (or in total).
I'll agree they are not operational nightmares, but now that we're set up with Riak we can do things that I'm pretty sure your Postgres/MySQL/etc. setup cannot.
1. Add individual nodes to the cluster and have it automatically rebalance over the new nodes.
2. Have it function without intervention in the face of node failure and network partitions.
3. Query any node in the cluster for any data point (even if it doesn't have that data locally).
I'm sure there's other things I'm missing, but the point made by timdoug is the key one. We're at a scale now where it's worth trading up-front application code for reliability and stability down the line.
In a nutshell, gevent monkey patches the socket library, whereas diesel doesn't. This means that you can use any (previously) blocking libraries with gevent, whereas, in diesel you have to write them again. The upside of the rewriting is that it creates a more coherent (and opinionated) ecosystem.
I added an implementation [1] in diesel [2][3], which uses select.epoll (or libev, on non-Linux systems) and got a around 150x speedup [4]. I only repeated the tests a few times (but they were all close) and didn't install the Go compiler so I could test against Go (I'd be interested to see how this stacks up on your machine). Like you say in your post, it's nice to have something wrap up the bother of epoll for you.
Agreed, I should have clarified. We don't just use IRC, but IRC along with other tools to handle pasting text and file uploads. I think what has always bothered me about Campfire (which I'm sure can be solved) is that it's not easy to do thing from the command line. I want to be able to pipe the output of git diff to my pastebin, or upload a log file from a remote machine.
At Bump we tried Campfire, but ended up using IRC (and use an open source IRC bot, https://bitbucket.org/yougov/pmxbot/src). I'm curious why you decided to go with Campfire instead of IRC.
We (Bump) have found two bugs in Redis and both times we've worked closely with Salvatore and had them fixed in less than 48 hours. He's a great programmer who really cares about the product.
One thing you don't note in your post (but made a big difference for us) is that 2.4 uses jemalloc. This reduced memory fragmentation for us by around 25%.
Just as a word of warning, Redis 2.2 cannot read rdb or aof files created by 2.4 (meaning I'm pretty sure 2.2 can't be a slave of 2.4. So, if you have a fail over scenario, you might be forced to upgrade your master to 2.4.
Bitcask can guarantee one disk seek, whereas LevelDB will do one disk seek per level, so at least from that perspective, it can't be better.
Level also has to look down the entire tree if a key is missing. This means inserts end up being more expensive than reads or updates (which are all just a hash lookup in Bitcask).
Mathias, what Riak (or other distributed databases provide) that Mongo doesn't is "just-works" scaling characteristics beyond one machine.
* If I want to ensure that my data is written to three machines, all my writes will stop working in MongoDB if one machine goes down. With Riak, it will start issuing the writes to another node in the cluster and rebalance when the missing node comes back online.
* If I'm running MongoDB in a sharded configuration, if one of the shards cannot be reached, all writes will stop. With Riak, any node will accept the writes and, once the network issues are resolved, move them to the appropriate node.
That said, conflict resolution is hard and there's no real way to get around it when you're using a distributed database like Riak. With Riak, as Chad says you get "increased development complexity for massively decreased deployment complexity." There's no silver bullet, it's important to look at the trade-offs of each option.
I wrote the post and to clarify, I didn't mean to suggest that interfacing with Android was impossible with Clojure, simply that is was easier with Scala. As rickmode points out, there are some things that make Clojure more difficult.
Agreed, a very cool idea. I believe you are right on the parallelism front, cloud.call just offloads the processing on their servers, but you'd have to call it multiple times to get any parallelism.