Riak at Braintree(basho.com)
basho.com
Riak at Braintree
http://basho.com/blog/technical/2012/08/14/riak-at-braintree/
2 comments
If anyone from basho is here anyway to find the video/slides of a talk that openx(http://blog.openx.org/07/openx-hosts-riak-meet-up/) did?
thanks
That video should be available within a few weeks, so stay tuned.
Also, Anthony Molinaro from OpenX will be speaking at RICON, the distributed systems developer conference we're hosting i October. Tickets are still available :)
http://basho.com/community/ricon2012/
Also, Anthony Molinaro from OpenX will be speaking at RICON, the distributed systems developer conference we're hosting i October. Tickets are still available :)
http://basho.com/community/ricon2012/
Also do you have an equivalent page to this (http://www.10gen.com/presentations) . Basically a page that lists every presentation and videos that speak about riak.
we're in the process of a complete overhaul of our website--not to sound like a broken record, but...stay tuned :)
This seems to be it something like it http://basho.com/company/production-users/
Scroll down and many companies have slides.
Braintree sucks, perfect match!
Care to elaborate on your obviously educated opinion of Riak?
Seriously, in my experience Riak is absolutely nothing but AWESOME. If you actually understand CAP and know what you want out of a system, Riak is a great for what it does. Probably best in class.
Yes, they do have some strengths like predictable performance and easy administration but they still have to improve on things like:
- weak data model
- difficult query API
- poor single node performance
- small community/weak adoption
If you want just a single node, use Couch or Mongo - Riak wasn't built to work reliably OR quickly on a single node; Riak gets all of its fault-tolerance, low-latency, and great performance from > 3 node clusters (preferably > 5).
Weak data model is by design. It's a Key/Value store - if you need something more complex then it isn't the right fit for you; Riak is not meant to replace relational databases.
Small community? Certainly not. Weak adoption? Definitely not. Adobe is using Riak, Braintree, and hundreds of other LARGE companies.
The query API is pretty easy to use from the clients in my experience. If you want REAL performance you can write Erlang map/reduce workers - which admittedly gets more complicated but you get far more performance out of that than doing it in JS (which is limited on ANY platform, no just Riak).
Your gripes really aren't legitimate, purely because you're not understanding the use-case for Riak.
We use riak for ONLY TWO pieces of data in our app (out of hundreds of pieces). Everything else is in a relational database. Riak is perfect for FAST GROWING data that maps to the Key/Value model very easily; I can't even begin to describe how perfect Riak is for that. Everything else fits into MySQL/PostgreSQL perfectly and without a hitch. Our fast growing data is handled purely by Riak and Map/Reduce using Erlang workers is FAST.
Weak data model is by design. It's a Key/Value store - if you need something more complex then it isn't the right fit for you; Riak is not meant to replace relational databases.
Small community? Certainly not. Weak adoption? Definitely not. Adobe is using Riak, Braintree, and hundreds of other LARGE companies.
The query API is pretty easy to use from the clients in my experience. If you want REAL performance you can write Erlang map/reduce workers - which admittedly gets more complicated but you get far more performance out of that than doing it in JS (which is limited on ANY platform, no just Riak).
Your gripes really aren't legitimate, purely because you're not understanding the use-case for Riak.
We use riak for ONLY TWO pieces of data in our app (out of hundreds of pieces). Everything else is in a relational database. Riak is perfect for FAST GROWING data that maps to the Key/Value model very easily; I can't even begin to describe how perfect Riak is for that. Everything else fits into MySQL/PostgreSQL perfectly and without a hitch. Our fast growing data is handled purely by Riak and Map/Reduce using Erlang workers is FAST.
> - poor single node performance
SERIOUSLY! Riak is NOT MEANT for single-node clusters!!! It gets faster THE MORE NODES YOU ADD.
The minimum recommended number of nodes is 3; the "ideal" minimum number of nodes is 5.
The minimum recommended number of nodes is 3; the "ideal" minimum number of nodes is 5.
yes, with n_val=3 (default) 3 nodes is the minimum. I was merely testing (actually had 3 riak instances on a single box as a "cluster") but with r,w=1 and n_val=1 it was about the same (slow), which was disconcerting. cpu was very high in one instance (others were nearly idle, and IO was low. I did not try dw=0 though (if that would even work). Might be worth a try.
I am going to try bitcask too, to see if eleveldb was the culprit and could be tuned.
I am going to try bitcask too, to see if eleveldb was the culprit and could be tuned.
Your test isn't valid if it was all on one box - the disk I/O is going to be really nasty with that. Your test would be better on three different machines.
The disk io was pretty low. It is serial get/update/delete (as I mentioned earlier). I even tried moving the data dirs to ramdisk (was plenty of ram), but it didn't help much at all. i/o was certainly not the problem.
It very well could be a configuration issue. I am still trying a few things to see if I can get an improvement.
It very well could be a configuration issue. I am still trying a few things to see if I can get an improvement.
If you haven't done so already, I would highly-recommend taking a look at Basho Bench, our open source benchmarking tool for Riak.
http://wiki.basho.com/Benchmarking.html https://github.com/basho/basho_bench
http://wiki.basho.com/Benchmarking.html https://github.com/basho/basho_bench
thanks. Will do!
Interesting. I can vouch for the speed of leveldb on a five node cluster - I'm using it in our startup on 5 dedicated machines with around 12GB of RAM, running on FreeBSD. Riak version is 1.1.4; Riak 1.2 is supposed to have a lot of leveldb refinements.
Good to hear. I will keep poking at things. I probably have something misconfigured or an issue with how I am using the python client.
Thanks.
Thanks.
What did you find slow? writes? retrieving by key? map-reduce?
From what I've heard erlang map-reduce is much performant than the javascript one.
It was a sequence simple operations (simple as in no 2i or mapred) of read an existing key A, check if a nonexistent key exists B, write a previously nonexistent key C, delete an existing key D, overwrite an existing key A.
All performed in serial over protobuf (python riak lib).
All performed in serial over protobuf (python riak lib).
LevelDB is known to have heavily degrading performance when you ask for a key that doesn't exist.
And what about Braintree? Why do they suck? Pretty well thought of with some great referencable customers.
Because we were a startup using it for about one year and paying the minimum price (I believe around $50/mo) plus another $15 fee/mo, plus they kept 6% of our transactions for 6 months (which we haven't seen any of it yet). Thankfully Stripe came into play (just as easy to implement) with zero monthly fees, and not holding any of our money. At that point we contacted Braintree to cancel our account and just then they offered a lower monthly fee (I believe around $20) and when I said no, they offered to match Stripe at zero cost. Why wouldn't they have offered this from the get go is beyond me and bad practice.
Kristi from Braintree here.
First, I want to apologize that we weren't able to reach out about lower fees sooner than we did. High fixed monthly fees have been imposed by the underlying banks that provide merchant accounts for years. Some banks have recently relaxed those and Braintree, Stripe and others in the industry are now able to take advantage of it.
The reason we were able to waive the fixed fee when you contacted us was not because it was in our discretion the entire time, but because we had only recently been able to get some of those fixed fees removed by the underlying banks. Since that time, we have been proactively contacting hundreds of our customers that would benefit from the new bank structure to offer them lower pricing.
The 6% reserve on your transactions wasn't imposed by Braintree but rather the bank that was providing your merchant account (Braintree is not holding back anyone's money). We work extremely hard to prevent reserves being imposed on our merchants and when it does happen, our intent is to get it removed as soon as possible. Please reach out to us so that we can work on getting your funds released asap.
We could have handled your situation better, and we're sorry that we weren't able to get lower fees to you faster. We've been browbeating banks for years on behalf of startups to get simpler, better fee structures. We've made a lot of progress and think we've had an impact on the industry with things like simpler pricing and card data portability. That said, we recognize there is a lot more work to do, and we will continue to work hard to provide an exceptional payments experience for startups.
First, I want to apologize that we weren't able to reach out about lower fees sooner than we did. High fixed monthly fees have been imposed by the underlying banks that provide merchant accounts for years. Some banks have recently relaxed those and Braintree, Stripe and others in the industry are now able to take advantage of it.
The reason we were able to waive the fixed fee when you contacted us was not because it was in our discretion the entire time, but because we had only recently been able to get some of those fixed fees removed by the underlying banks. Since that time, we have been proactively contacting hundreds of our customers that would benefit from the new bank structure to offer them lower pricing.
The 6% reserve on your transactions wasn't imposed by Braintree but rather the bank that was providing your merchant account (Braintree is not holding back anyone's money). We work extremely hard to prevent reserves being imposed on our merchants and when it does happen, our intent is to get it removed as soon as possible. Please reach out to us so that we can work on getting your funds released asap.
We could have handled your situation better, and we're sorry that we weren't able to get lower fees to you faster. We've been browbeating banks for years on behalf of startups to get simpler, better fee structures. We've made a lot of progress and think we've had an impact on the industry with things like simpler pricing and card data portability. That said, we recognize there is a lot more work to do, and we will continue to work hard to provide an exceptional payments experience for startups.
Ok that's a valid negative experience.
Really loving braintree, haven't checked out stripe, but I think for what we are doing braintree are totally awesome. There has to be a reason why basecamp and others are using them for example. I guess everyone has their reasons to like or dislike something, and valid to have both opinions but we love it.