A/Bingo split testing now on App Engine, built for Khan Academy(bjk5.com)
bjk5.com
A/Bingo split testing now on App Engine, built for Khan Academy
http://bjk5.com/post/10171483254/a-bingo-split-testing-now-on-app-engine-built-for-khan
4 comments
Hearing about this made my week.
Hearing about this make me sad because my App Engine is Java-based :(
Some options:
1) The code is available under MIT and the docs are CCed and specifically designed to make it easy to clone this. At least three people have successfully done so, in a variety of programming languages, web frameworks, and technology stacks. If programming Java is not difficult for you, block off about a week or two and bam problem solved.
2) If you have access to vast amounts of cash but your programmers are just too busy delighting people to write an A/B testing framework, you could hire me to rewrite the whole codebase in Java. This would only cost you low five figures.
3) If you don't want to program and don't have tens of thousands lying around but do have tens lying around, Visual Website Optimizer is a great option for $CHEAP.
4) If you have neither programming skills nor tens of dollars then you may want to reevaluate whether A/B testing is a huge need at this point in your life.
1) The code is available under MIT and the docs are CCed and specifically designed to make it easy to clone this. At least three people have successfully done so, in a variety of programming languages, web frameworks, and technology stacks. If programming Java is not difficult for you, block off about a week or two and bam problem solved.
2) If you have access to vast amounts of cash but your programmers are just too busy delighting people to write an A/B testing framework, you could hire me to rewrite the whole codebase in Java. This would only cost you low five figures.
3) If you don't want to program and don't have tens of thousands lying around but do have tens lying around, Visual Website Optimizer is a great option for $CHEAP.
4) If you have neither programming skills nor tens of dollars then you may want to reevaluate whether A/B testing is a huge need at this point in your life.
Here's my C# port:
http://www.fairtutor.com/fairlycertain/
I'd be surprised if you managed to take more than 5 minutes translating the core so that it compiled in Java. From there, it's just a matter of removing the ASP.NET usercontrols from the package and (optionally) wrapping it in your Java framework's equivalent.
http://www.fairtutor.com/fairlycertain/
I'd be surprised if you managed to take more than 5 minutes translating the core so that it compiled in Java. From there, it's just a matter of removing the ASP.NET usercontrols from the package and (optionally) wrapping it in your Java framework's equivalent.
[deleted]
Huge congrats on this!
I do not think you can just stop the test when you find something you think is significant
http://www.evanmiller.org/how-not-to-run-an-ab-test.html 'If instead of deciding ahead of time, “this experiment will collect exactly 1,000 observations,” you say, “we’ll run it until we see a significant difference,” all the reported significance levels become meaningless. '
Reading the stats.py it looks like you might be doing this.
http://www.evanmiller.org/how-not-to-run-an-ab-test.html 'If instead of deciding ahead of time, “this experiment will collect exactly 1,000 observations,” you say, “we’ll run it until we see a significant difference,” all the reported significance levels become meaningless. '
Reading the stats.py it looks like you might be doing this.
Wow... this is awesome! I was literally looking around this week for exactly this kind of framework. Fantastic work, and definitely plan on using it.
Thanks to Ben and Khan Academy!
Thanks to Ben and Khan Academy!
Troy Goode wrote a port for ASP.NET MVC, ABsolutelyMaybe. I've had good luck with it. It wasn't too difficult to migrate from the default storage engine, which is serialization.
https://github.com/TroyGoode/ABsoluteMaybe
https://github.com/TroyGoode/ABsoluteMaybe