Some fun with Redis Cluster testing(antirez.com)
antirez.com
Some fun with Redis Cluster testing
http://antirez.com/news/69
4 comments
I think this is a great lesson on testing. I'm excited to see what else he finds.
Why is this a great lesson?
The test is not repeatable. There was a power outage. Random machine configurations on random network conditions. These are all great ways to stress redis-cluster, which is still young, but they are not repeatable tests.
They are not great tests. Great tests would be a script that can run every 24 hours over a group of servers, simulating or creating similar events and network conditions. Then when a code change is made, you can actually observe if it made things better.
Driving a car across the Sahara Desert can be a good event, but it shouldn't replace Crash Tests in a Lab environment.
The test is not repeatable. There was a power outage. Random machine configurations on random network conditions. These are all great ways to stress redis-cluster, which is still young, but they are not repeatable tests.
They are not great tests. Great tests would be a script that can run every 24 hours over a group of servers, simulating or creating similar events and network conditions. Then when a code change is made, you can actually observe if it made things better.
Driving a car across the Sahara Desert can be a good event, but it shouldn't replace Crash Tests in a Lab environment.
Hi pquerna, what you describe is exactly what the Redis Cluster standard test will be (the one that will implement unit tests, not just fuzzy testing like the one I did now). Probably to make it easy to run in different platforms, what is needed is direct partitions simulation support into Redis Cluster, what I mean is, an explicit API to filter at higher level specific nodes and clients.
However I believe that I'll still do the kind of tests described in the blog post since the fact itself that these tests are so "home brew" and cheap make the conditions extremely random and are good at spotting actual real-world deficiencies.
However I believe that I'll still do the kind of tests described in the blog post since the fact itself that these tests are so "home brew" and cheap make the conditions extremely random and are good at spotting actual real-world deficiencies.
Sorry, the word I should say is "experiments". I know it's not a good test case, you're right 100%. The lesson I was leaning toward was to always playing with software in new ways because you'll often find bugs where you'd least expect them.