Show HN: Bandit Algorithm for iOS, Android and Unity(splitforce.com)
splitforce.com
Show HN: Bandit Algorithm for iOS, Android and Unity
http://splitforce.com/resources/auto-optimization/
5 comments
Have seen a lot of these services, but this is definitely worth checking out. I think Multi-armed bandits are the way to go with optimisation, and just won Startup Asia's hackathon using this technique for a tinder-style app. A lot of people talk about learning from their customers, but few know how to really do it - understanding the math behind what's going on gives me much better confidence when deploying this type of service... Great work sharing it in this way.
As a splitforce user this is pretty cool.. biggest issue is having to wait for conclusive results, whatever speeds this up and auto prunes away the losing options is welcome.
Thanks, we think it's pretty cool too! And you've totally hit on why this is awesome - by automatically funneling users towards better-performing variations you can focus your resources on validating what matters and cut down on the deadweight loss associated with a traditional A/B. For more on this here's a recommended read: http://stevehanov.ca/blog/index.php?id=132?utm_medium=referr...
Do you guys do any sort of logistic regression for optimizing numeric values? That would be really cool.
Right now auto-optimization only supports binary goal types. But we support A/B testing with Time and Quanity (numeric) goal types as well, so working on a solution to automate those types of experiments.
Getting Splitforce already at the test stages of development is one of the smartest things a game studio can do.
True, but why? Activision spends over $10M annually on employing an analytics team of about a dozen PhDs to build Rubin causal models to be able to show the right thing at the right time to players. That kind of budget makes sense for a $1 billion title like Call of Duty, but where does that leave the other 99%? The fact is that even for the companies that don't employ a full-time team of statisticians, if you're running your app or game like a true business you should be leveraging data to make better business decisions. Bandit algorithms like those we're proposing just let you do that at a fraction of the cost ;-)
That custom scroll handler which overrides the native ios mobilesafari scroll handler is quite the annoying distraction. :-/
Thanks for the heads up - looking into it...
I used the Javascript Bayesian Bandit library found here:
https://github.com/omphalos/bayesian-bandit.js/blob/master/b...
I then used it to create a job that runs every 5 minutes on parse.com that updates an 'order' field based on the 5 minute sample.
Then, from the client side, I simply do a query sorted by the 'order' field and I get the top variant for that 5 minute interval. Works like a charm!