Running a startup on Haskell(bos.github.com)
bos.github.com
Running a startup on Haskell
http://bos.github.com/strange-loop-2011/talk/talk.html#(1)
16 comments
Can anyone comment on the claim "QuickCheck is shockingly more effective at finding bugs than unit tests"? I'd be interested in hearing other opinions.
The claim is true: QuickCheck is surprisingly more effective at finding problems (and documenting expected behavior) than unit tests. When I first tried it, not long after the QuickCheck paper came out at ICFP 2000, it found problems that I never would have thought to test for. [1] (That's why I end up writing something like QuickCheck when I code in environments that don't already have one. [2] It's worth it.)
The key to its effectiveness is that it doesn't suffer from limited human imagination. When I write test cases by hand, it's up to me to think of all the things that could go wrong and write test cases for them. But, being human, I have blind spots. Even when I try to be systematic about detecting edge cases and writing tests for them, I miss some. But when QuickCheck-like tools write the test cases for me, they can dream up corner cases like you wouldn't believe.
And, even better, when I use QuickCheck, my test code doesn't end up looking like an enumeration of corner cases. Instead, it becomes clear, concise, and formal documentation. (For a good example of QuickCheck properties as documentation, see [3].) I just specify the intended general properties of my code, and QuickCheck generates the messy test cases behind the scenes, where they don't become visible noise.
[1] http://blog.moertel.com/pages/seven-lessons-from-the-icfp-pr...
[2] http://search.cpan.org/dist/Test-LectroTest/lib/Test/LectroT...
[3] http://haskell.org/ghc/docs/latest/html/libraries/base/Text-...
The key to its effectiveness is that it doesn't suffer from limited human imagination. When I write test cases by hand, it's up to me to think of all the things that could go wrong and write test cases for them. But, being human, I have blind spots. Even when I try to be systematic about detecting edge cases and writing tests for them, I miss some. But when QuickCheck-like tools write the test cases for me, they can dream up corner cases like you wouldn't believe.
And, even better, when I use QuickCheck, my test code doesn't end up looking like an enumeration of corner cases. Instead, it becomes clear, concise, and formal documentation. (For a good example of QuickCheck properties as documentation, see [3].) I just specify the intended general properties of my code, and QuickCheck generates the messy test cases behind the scenes, where they don't become visible noise.
[1] http://blog.moertel.com/pages/seven-lessons-from-the-icfp-pr...
[2] http://search.cpan.org/dist/Test-LectroTest/lib/Test/LectroT...
[3] http://haskell.org/ghc/docs/latest/html/libraries/base/Text-...
I haven't used QuickCheck, but read about it. So take this with a grain of salt, but it seems to me that specifying properties that should hold for your functions and then letting QuickCheck throw all kinds of data at them would be more efficient at least, if not more effective, than manually writing a bunch of unit tests and coming up with your own test data.
It's surprising how often people overlook weird edge cases that QuickCheck finds, because it doesn't have as many preconceptions about what kinds of inputs are sensible.
It tends to be best for mathematical code because you can quickly think of invariances.
In any situation where you can state a powerful invariance your code must always be consistent with it's both fast and thorough. The common example is that (= str (reverse (reverse sequences))) for all sequences.
Note that this only tests invertibility. There are lots of ways reversal can be broken without violating this invariance. It's just really easy to write down this invariance and use it to rapidly check gross violations.
In any situation where you can state a powerful invariance your code must always be consistent with it's both fast and thorough. The common example is that (= str (reverse (reverse sequences))) for all sequences.
Note that this only tests invertibility. There are lots of ways reversal can be broken without violating this invariance. It's just really easy to write down this invariance and use it to rapidly check gross violations.
It's essentially a unit-fuzzing framework, so it can find holes in boundary conditions people generally forget to check, because it's in their blind spot or because "nobody would ever do that".
QC does not care, and since it's injecting fuzzed data at the unit leve (~function, generally) it makes it quite easy to see precisely where the failure happens.
QC does not care, and since it's injecting fuzzed data at the unit leve (~function, generally) it makes it quite easy to see precisely where the failure happens.
Also writing up the laws that the fuzzers checks, is a good exercise in thinking about your API. Read http://donsbot.wordpress.com/2007/05/01/roll-your-own-window... and http://donsbot.wordpress.com/2007/05/17/roll-your-own-window... for some examples.
Quick also not only gives you the failing cases, but tries to simplify and shrink them first.
Quick also not only gives you the failing cases, but tries to simplify and shrink them first.
It's worth noting that QuickCheck is used with functional code only. For non-functional code with side effects (e.g. database transactions), Haskell has unit test frameworks that are not too different from unit testing frameworks in other languages.
What distingishes QuickCheck from regular unit testing is that you give it a function to test, some invariants that must hold and an input generator. Some of this is automated with compiler magic (where possible) to make simple tests really simple to write. QuickCheck will then grind through a bunch of random inputs and see if the invariants hold, finally showing you the failed cases.
What distingishes QuickCheck from regular unit testing is that you give it a function to test, some invariants that must hold and an input generator. Some of this is automated with compiler magic (where possible) to make simple tests really simple to write. QuickCheck will then grind through a bunch of random inputs and see if the invariants hold, finally showing you the failed cases.
> It's worth noting that QuickCheck is used with functional code only.
That's actually not true. You can use QuickCheck with "normal" code that has side effects, too, and it's every bit as useful there.
That's actually not true. You can use QuickCheck with "normal" code that has side effects, too, and it's every bit as useful there.
I really like the idea of QuickCheck. Presumably it is -best- used in conjunction with unit testing. It seems to me though that it would be suitable mainly for numeric type code, given the random nature of the inputs?
Remember that Haskell datatypes are very rich. QuickCheck will generate random instances of any datatype you define. Because Haskell datatypes are so expressive,. t's normal for Haskell programmers to define complex types for their functions to process. Hence you can easily get good coverage of any idiomatic Haskell code with QuickCheck.
If you're working in a datatype poor language, or simply one where the syntax makes defining complex datatypes painful then you're much more likely to pass around datatypes which are in implicitly defined. In this case you'll have to write a QuickCheck function to generate suitable instances in order to test your code. (This is also true for Haskell: there are often a few functions where you have to explain to QuickCheck how to generate appropriate test cases.)
If you're working in a datatype poor language, or simply one where the syntax makes defining complex datatypes painful then you're much more likely to pass around datatypes which are in implicitly defined. In this case you'll have to write a QuickCheck function to generate suitable instances in order to test your code. (This is also true for Haskell: there are often a few functions where you have to explain to QuickCheck how to generate appropriate test cases.)
Any reason to use C# instead of F#, considering F# is more functional?
Wonder if it has something to do with COM...
Wonder if it has something to do with COM...
Hackers News is boring because of this type of post. What is the purpose of your link?
Did you know that that was a slideshow? You're seeing the title page, click on it to advance to the next page. repeat.
Except that, depending on your browser, it is impossible to navigate to the second slide. In opera mini, the slide itself is not clickable, there are no arrows visible, arrow keys are not available, and the "contents?" link points to a trunkated list starting at slide 15. I'd really like to read the presentation, but it is completely inaccessible.
Arrow keys also work.
That said, this is pretty poorly designed. I also had no clue what to do for the first few seconds after I clicked the link. It was only at slide 38/41 that I noticed the tiny, hidden controls on the very bottom blending into my status bar.
I didn't even notice those. Fortunately, I've worked with systems like S5 slideshows, and this looked a lot like a slideshow (no scroll bar, large text, numeric fragment in the URL).
Usability could definitely be better, but I will admit to writing up similarly boring slides in the past.
Usability could definitely be better, but I will admit to writing up similarly boring slides in the past.
Compare this slideshow to the basic S5 slideshow at http://meyerweb.com/eric/tools/s5/s5-intro.html . A simple CSS template could solve the usability problem here.
And on an iPad, nothing works.
I don't know if it's Opera being smart, but space works for me as well.
For background, Bryan gave this talk at the excellent Strange Loop 2011 conference (https://thestrangeloop.com/).
Abstract: https://thestrangeloop.com/sessions/running-a-startup-on-has...
He also gave a workshop on Sunday (which I unfortunately was not able to attend): https://thestrangeloop.com/sessions/haskell-functional-progr...
Abstract: https://thestrangeloop.com/sessions/running-a-startup-on-has...
He also gave a workshop on Sunday (which I unfortunately was not able to attend): https://thestrangeloop.com/sessions/haskell-functional-progr...
For a different type of background, Bryan O'Sullivan is the author of some of the best Haskell libraries out there. The de facto standard JSON parser/serializer is his doing, for example, and his Riak client is the best I've seen in any language. This guy really knows what he's talking about.
I'm very impressed with the amount and quality of libraries they have released on Github [1].
Going to use the following in my "webmachine in Haskell" project.
Going to use the following in my "webmachine in Haskell" project.
- aeson: Fast json library [2]
- pronk: load test webservers [3]
[deleted]
So what is "the best compiler bug ever" (slide 31)?
There was a certain condition (~ 10 years ago) where if the source file didn't pass the type checker it would be deleted.
Bryan quickly assured everyone that it had since been resolved. =)
Bryan quickly assured everyone that it had since been resolved. =)
agreed. heart-broken when i realized the next slide wasn't a continuation of that one.
Interesting that he comes down in favor of Snap. I've been looking at the Snap vs. Yesod question for my own projects, lately, and there's interesting arguments both ways. Yesod is big and comprehensive, with four or five custom DSLs that have compilers implemented in Template Haskell. Snap is tiny, really just a routing engine and server with a small template library tacked on.
Yesod reminds me of Rails, while Snap reminds me of Servlets + JSP.
Yesod reminds me of Rails, while Snap reminds me of Servlets + JSP.
Yesod is a web framework. It is designed to display web pages with some sort of datastore backing them. If you had a CRUD-type app with a few different data models then Yesod will make it really easy to generate the appropriate HTML and forms and make them type safe. It extends the Haskell all the way down to the application layer, assuming standard HTML as your primary response and a certain data representation and persistence layer.
Snap (I have only looked at it. Please correct me) seems like it only goes as far as the HTTP layer. It's designed so you can implement web services. Those services might return HTML, but they might return JSON or XML. It doesn't care or help you with your data representation. You have to do your own data persistence and display layers.
Snap (I have only looked at it. Please correct me) seems like it only goes as far as the HTTP layer. It's designed so you can implement web services. Those services might return HTML, but they might return JSON or XML. It doesn't care or help you with your data representation. You have to do your own data persistence and display layers.
I must hunt down and learn to use a QuickCheck-like library for my language.
If I can't find one, I should write one.
For Ruby: http://rushcheck.rubyforge.org/about.html
Check out the citations on http://en.wikipedia.org/wiki/QuickCheck for QuickChecks in other languages.
Check out the citations on http://en.wikipedia.org/wiki/QuickCheck for QuickChecks in other languages.
Scala has scala check: http://code.google.com/p/scalacheck/
Ruby, because of non static types, can't be as magical, but people have given their best: https://github.com/IKEGAMIDaisuke/rushcheck
Ruby, because of non static types, can't be as magical, but people have given their best: https://github.com/IKEGAMIDaisuke/rushcheck
I'm not sure about one that can generate arbitrary data, but Heckle[0] will break your Ruby tests in extravagant ways.
[0] http://ruby.sadi.st/Heckle.html
[0] http://ruby.sadi.st/Heckle.html
A quick google search returned this:
* qc for python (http://dan.bravender.us/2009/6/21/Simple_Quickcheck_implemen...)
* QuickCheck Wikipedia page with links ot implementation for many languages (http://en.wikipedia.org/wiki/QuickCheck)
Personally I am looking forward to trying out qc for python. Has anyone here used it before?
* qc for python (http://dan.bravender.us/2009/6/21/Simple_Quickcheck_implemen...)
* QuickCheck Wikipedia page with links ot implementation for many languages (http://en.wikipedia.org/wiki/QuickCheck)
Personally I am looking forward to trying out qc for python. Has anyone here used it before?
Go has a QuickCheck-like package distributed with it. It's under testing/quick.
http://golang.org/pkg/testing/quick/
http://golang.org/pkg/testing/quick/
For C# I in fact thought of what is in the above quote and wrote my own: https://github.com/ane/DotCheck
It's not yet very advanced (sometimes the library is a bit messy), but it provides the basic QuickCheck functionality.
It's not yet very advanced (sometimes the library is a bit messy), but it provides the basic QuickCheck functionality.
There's qc [1] for Javascript.
[1] http://willowbend.cx/2009/12/05/qc-js-quickcheck-javascript/
[1] http://willowbend.cx/2009/12/05/qc-js-quickcheck-javascript/
rushcheck is a bit moribund, or was last time i checked. https://github.com/hayeah/rantly is pretty nice.
For Perl it's a module called Test::LectroTest
* https://metacpan.org/module/Test::LectroTest
* http://community.moertel.com/ss/space/LectroTest
* https://metacpan.org/module/Test::LectroTest
* http://community.moertel.com/ss/space/LectroTest
Very cool, I didn't realize there was a generic version of Pex (the unit test generator for C#). Now if only I could figure out some way to unit test drivers...
It appears he's teaching a course at Stanford on Haskell starting next week, MW later in the afternoons.
Taking a class just to learn a language is a bit dodgy, but Haskell is probably a forgivable exception.
Taking a class just to learn a language is a bit dodgy, but Haskell is probably a forgivable exception.
I'll actually be sitting in on this class if not enrolled in it. I look forward to another good forum where I can improve my Haskell and functional programming skills, particularly with a good instructor like Brian O'Sullivan.
I think the main advantage of a course at Stanford is access to the instructors. Sure, I could just read Real World Haskell and do all the exercises, but with the course I have a space where I can directly talk to Brian or David at least once a week if I have questions.
I think the main advantage of a course at Stanford is access to the instructors. Sure, I could just read Real World Haskell and do all the exercises, but with the course I have a space where I can directly talk to Brian or David at least once a week if I have questions.
It's a class in Haskell, not on Haskell.
That is great. In my school we only had a class with Ocaml, not quite as mind blowing since it didn't have lazy evaluation, monads, type classes, or any of the other crazy distinctive things in Haskell that don't exist in ML-style languages.
Hopefully they cover the theory behind Haskell as much as the practical side of it.
Hopefully they cover the theory behind Haskell as much as the practical side of it.
ML's first-class modules let you achieve many of the same things as type classes. In fact, a lot of people in the Haskell community would like something a bit closer to ML (in other words, more powerful) in this regard.
Monads can be formulated in any language with first-class functions. Monadic I/O is more distinctively Haskellish, and of course Haskell has syntactic sugar to make using them easier, but as a tool and a concept they're far from unique to Haskell.
Monads can be formulated in any language with first-class functions. Monadic I/O is more distinctively Haskellish, and of course Haskell has syntactic sugar to make using them easier, but as a tool and a concept they're far from unique to Haskell.
> Monads can be formulated in any language with first-class functions.
Yes, though they only really make sense if you also have static typing, and polymorphism by return type also helps.
Yes, though they only really make sense if you also have static typing, and polymorphism by return type also helps.
It is pretty disappointing that they decided to go with c# for the local client. This pretty much guarantees that it will not work on Linux.
C# lets you make very powerful UIs with very little work (thanks to WPF) which most importantly feel quite native. WPF is also the only widespread framework that I know of that lets you have pretty much any kind of control inside another control (well, other than HTML that is -- the problem with HTML is that you lose the native feeling).
That being said Mono supports System.Windows.Forms (the poor predater to WPF) so there's always that. Or they can code against the (now dying?) Silverlight which actually works on Linux ala Moonlight.
But really, the best user experience is the one that feels the most native. So on Mac that would mean building a Cocoa client, on Windows WPF, on Linux Gtk/Qt (but then again who actually pays for Linux apps?)
Going with a one tech that supports it all just has a higher probability of having something that supports everything poorly, instead of something that supports one thing very well.
That being said Mono supports System.Windows.Forms (the poor predater to WPF) so there's always that. Or they can code against the (now dying?) Silverlight which actually works on Linux ala Moonlight.
But really, the best user experience is the one that feels the most native. So on Mac that would mean building a Cocoa client, on Windows WPF, on Linux Gtk/Qt (but then again who actually pays for Linux apps?)
Going with a one tech that supports it all just has a higher probability of having something that supports everything poorly, instead of something that supports one thing very well.
> but then again who actually pays for Linux apps?
Despite this oft repeated statement, I have seen no evidence that Linux users don't pay for apps. The only thing saying it does is scare developers away from a perfectly good source of income.
Despite this oft repeated statement, I have seen no evidence that Linux users don't pay for apps. The only thing saying it does is scare developers away from a perfectly good source of income.
It looks like he built an outlook extension. I don't know of any applications in the same cadre on Linux for which users pay. Oracle and RHEL itself are two examples where users pay for the software; I can't recall any other significant examples from the top of my head.
He revealed during the talk (after a very small amount of prodding) that it was an Outlook extension (since people on Windows are willing to pay for software... out of company budgets).
The brains are in the cloud though, apparently.
The brains are in the cloud though, apparently.
Not really, as long as they don't use Windows API directly and rely on .NET Framework they should be able to run it on MONO.
He said in the Q&A that they used C# specifically to target integrating with Outlook, so that wouldn't apply in this case.
Mono's not an acronym, as far as I know.
See Mono.
See: actual Mono apps.
I only have a handful of experiences with GUI-intensive C# apps on Mac and Linux. They were all flat-out bad, though.
I only have a handful of experiences with GUI-intensive C# apps on Mac and Linux. They were all flat-out bad, though.
[deleted]
A related talk: http://vimeo.com/27192476 (video)