What it's like to use Haskell(engineering.imvu.com)
engineering.imvu.com
What it's like to use Haskell
http://engineering.imvu.com/2014/03/24/what-its-like-to-use-haskell
303 comments
We deal with Haskell resource leaks the same way you would in C++ or Java.
We have production monitors on every host that show basic metrics like memory, disk, and CPU utilization. Atop that, we added a tracker for the number of suspended Haskell threads. (that is, threads which are not blocked on I/O, but are also not running)
We found that the machines are usually able to handle requests as soon as they come in, so if the number of Haskell threads goes above 0 for any length of time, the machine is about an hour away from melting down.
We can restart the process without losing any connections, so this leaves us a very comfortable margin of error.
Once we know we have a problem, it's usually pretty simple to run the heap profiler on the process and look at recent commits. We continuously deploy, so there's only about a 10 minute delay before a particular commit is running in front of customers. This makes tracking regressions down really fast.
Even in cases where we can't figure out why a bit of code is leaking, we can almost always identify it and revert it until we understand what's going on.
We have production monitors on every host that show basic metrics like memory, disk, and CPU utilization. Atop that, we added a tracker for the number of suspended Haskell threads. (that is, threads which are not blocked on I/O, but are also not running)
We found that the machines are usually able to handle requests as soon as they come in, so if the number of Haskell threads goes above 0 for any length of time, the machine is about an hour away from melting down.
We can restart the process without losing any connections, so this leaves us a very comfortable margin of error.
Once we know we have a problem, it's usually pretty simple to run the heap profiler on the process and look at recent commits. We continuously deploy, so there's only about a 10 minute delay before a particular commit is running in front of customers. This makes tracking regressions down really fast.
Even in cases where we can't figure out why a bit of code is leaking, we can almost always identify it and revert it until we understand what's going on.
> We can restart the process without losing any connections
Would you mind expanding on this a bit? I'm not too familiar with Haskell, but I am familiar with various was of blocking new connections while allowing existing connections to complete, either at the load-balancer level or built-in each individual process.
What Haskell stack are you using, and how are graceful restarts accomplished?
Thanks.
Would you mind expanding on this a bit? I'm not too familiar with Haskell, but I am familiar with various was of blocking new connections while allowing existing connections to complete, either at the load-balancer level or built-in each individual process.
What Haskell stack are you using, and how are graceful restarts accomplished?
Thanks.
One of my coworkers wrote a really cool bit of software to do this. I want him to open source it.
Basically, you can share a single socket amongst many servers. The OS ensures that just one process accepts each connection.
You can therefore have a manager process that owns the socket and passes it on to application processes.
To update, start new processes, then politely tell the old ones to go away.
Basically, you can share a single socket amongst many servers. The OS ensures that just one process accepts each connection.
You can therefore have a manager process that owns the socket and passes it on to application processes.
To update, start new processes, then politely tell the old ones to go away.
One really cool thing in Linux is that you can actually pass file descriptors between processes over unix domain sockets.
Windows has supported this for ~14 years too.
Good to know. Does it work for everything that's an fd in Linux? I know you've got to treat sockets and files differently in some cases (or at least did once)...
It works for most kernel handles, sockets might be a little more normal starting with Win7 but I stopped doing Windows development around then.
Here are the official docs: http://msdn.microsoft.com/en-us/library/windows/desktop/ms72...
Here are the official docs: http://msdn.microsoft.com/en-us/library/windows/desktop/ms72...
Looks like there's a separate function for sockets.
Still, cool stuff there too.
Still, cool stuff there too.
einhorn [1] implements this model and is pretty effective. Used in production at Stripe and other places. (It's written in Ruby, but can run application processes in any language.)
[1] https://github.com/stripe/einhorn
[1] https://github.com/stripe/einhorn
Basically, catch SIGINT, then stop listening to a socket/port. Finish all current requests and exit. The "watcher" parent process will restart the process with the new executable. Repeat for all other processes listening to the socket/port.
I can't answer for grandparent, but you should check out https://github.com/notogawa/graceful
Except in Haskell you can build ekg right into your server. http://ocharles.org.uk/blog/posts/2012-12-11-24-day-of-hacka...
"we added a tracker for the number of suspended Haskell threads" - would you mind sharing how you did that? I couldn't see any obvious GHC APIs for it.
It looks like you're right. I misspoke.
We track total threads, working or not. It works great as an indicator because it tends to stay below the number of CPU cores on the server.
We track total threads, working or not. It works great as an indicator because it tends to stay below the number of CPU cores on the server.
I take it you mean OS-level threads then?
We track Haskell threads.
edit: Found the code. :)
We rolled our own implementation. Our WAI application action increments a counter and decrements it again whenever an HTTP request is received and completed.
It doesn't track threads created as a part of HTTP request handling, but we don't allow those actions to forkIO anyway. There hasn't been any demand for it.
edit: Found the code. :)
We rolled our own implementation. Our WAI application action increments a counter and decrements it again whenever an HTTP request is received and completed.
It doesn't track threads created as a part of HTTP request handling, but we don't allow those actions to forkIO anyway. There hasn't been any demand for it.
Ah, right, that makes sense - thanks.
... so, how? Isn't that what he was asking?
I don't believe sandboxes fix the particular problem the OP was describing, which is that of reproducible builds across multiple environments (different team members / CI / prod).
A cabal sandbox means once you've got your dependencies to resolve and your app to build, it'll continue to build and use the same versions of its dependencies, when building from that sandbox (which pretty much means "when building in that working directory"). But it gives you no guarantee that if your dev build got version 0.1.2 of a transitive dependency, then your CI server will also get version 0.1.2, and not 0.1.3.
If it turns out that your app works with 0.1.2 but not with 0.1.3, then your dev machine will reproducibly produce working builds, while your CI server will reproducibly produce broken builds.
What's really needed is an analogue to the Gemfile.lock used by Ruby or npm-shrinkwrap.json in the Node world, which is checked into version control, and freezes the exact versions of all transitive dependencies until explicitly updated. I think there's a "cabal freeze" command in development, but I'm not sure what the status is.
A cabal sandbox means once you've got your dependencies to resolve and your app to build, it'll continue to build and use the same versions of its dependencies, when building from that sandbox (which pretty much means "when building in that working directory"). But it gives you no guarantee that if your dev build got version 0.1.2 of a transitive dependency, then your CI server will also get version 0.1.2, and not 0.1.3.
If it turns out that your app works with 0.1.2 but not with 0.1.3, then your dev machine will reproducibly produce working builds, while your CI server will reproducibly produce broken builds.
What's really needed is an analogue to the Gemfile.lock used by Ruby or npm-shrinkwrap.json in the Node world, which is checked into version control, and freezes the exact versions of all transitive dependencies until explicitly updated. I think there's a "cabal freeze" command in development, but I'm not sure what the status is.
You can easily lock to a specific dependency version in the cabal file if that is your desire.
Sure - if you can reliably identify the exact required versions of all of your transitive dependencies. That's infeasible for nontrivial applications. (And even that's only if the set of exact versions you find manage to not have conflicting requirements with each other.)
The reason Gemfile.lock works is because it lets you achieve that the same way you create working code - figure out what works in dev, using a combination of skill and trial and error, then lock it down in version control and deploy exactly that to CI/prod/other devs.
People have written shell scripts to scan your sandbox for installed package versions and update your cabal file to require those versions, but it's an inherently approximate process - e.g. if you upgraded a transitive dependency but still have the previous version in the sandbox, the shell script has to guess which one you want, because there's no explicit relationship between your code and a particular version.
There's a more fundamental problem with that approach - it ignores the difference between "my app semantically requires package X at version y" and "I have tested my app with package X at version y". The cabal file expresses the former - which is why it doesn't include transitive dependencies, and why it's more idiomatic to specify broad version ranges than exact version constraints. "cabal freeze", if it existed, would express the latter. Reliable engineering requires both.
The reason Gemfile.lock works is because it lets you achieve that the same way you create working code - figure out what works in dev, using a combination of skill and trial and error, then lock it down in version control and deploy exactly that to CI/prod/other devs.
People have written shell scripts to scan your sandbox for installed package versions and update your cabal file to require those versions, but it's an inherently approximate process - e.g. if you upgraded a transitive dependency but still have the previous version in the sandbox, the shell script has to guess which one you want, because there's no explicit relationship between your code and a particular version.
There's a more fundamental problem with that approach - it ignores the difference between "my app semantically requires package X at version y" and "I have tested my app with package X at version y". The cabal file expresses the former - which is why it doesn't include transitive dependencies, and why it's more idiomatic to specify broad version ranges than exact version constraints. "cabal freeze", if it existed, would express the latter. Reliable engineering requires both.
We manage all deps in our cabal file, and have scripts to make sure that what is in package database matches the cabal file (this way we can upgrade versions without manually unregistering and reinstalling).
Like you said, there is nothing seamless that is part of cabal ... yet. I would like improve our workflow and integrate it into cabal.
Like you said, there is nothing seamless that is part of cabal ... yet. I would like improve our workflow and integrate it into cabal.
One Haskell resource leak I've encountered a couple of times has to do with opening large numbers of files combined with non-strict semantics. By default Haskell will open IO handles but not consume them until the contents are needed, and thus, not close them. To read the contents of many files in a directory, the result is opening thousands of concurrent file handles and exhausting the OS's IO handle pool. The solution is to add strictness annotations to force evaluation and relinquish the handles, which isn't fun and isn't pretty.
This problem is being addressed by a number of packages like conduit, pipes, and (at a lower level) io-streams. These are second-generation solutions to the problem that was pioneered by the iteratee and enumerator packages.
Seconding. I've used conduit before, and it was a delight to use something so carefully designed. The blog posts about conduit are in themselves an insight into how to think with Haskell.
http://www.yesodweb.com/blog/2013/10/core-flaw-pipes-conduit http://www.yesodweb.com/blog/2013/10/simpler-conduit-core
http://www.yesodweb.com/blog/2013/10/core-flaw-pipes-conduit http://www.yesodweb.com/blog/2013/10/simpler-conduit-core
The library in question was using unsafePerformIO to open a file handle. It was just a bug.
There's apparently a particular instance you're referring to?
But the general issue can be encountered with lazy IO without any use of unsafePerformIO. There has been a lot of discussion about this around the various enumerator-like libraries - in particular, Snoyman has many posts about ensuring timely release of resources.
But the general issue can be encountered with lazy IO without any use of unsafePerformIO. There has been a lot of discussion about this around the various enumerator-like libraries - in particular, Snoyman has many posts about ensuring timely release of resources.
As for resource leaks, the particular example in the post was a bit unfortunate. The problem in general have a solution though. You can either use functions that limit a resource inside a limited scope, and then have it clean up automatically when done (like 'withFile'). And if you want to do more complicated things there are the resourcet and pipes-safe libraries.
Switching a programming language is a major decision for a company. It requires changing the runtime library and possibly re-writing a lot of in-house infrastructure. While personal affinity to a language is important, making a decision for a company might have substantial business consequences.
For a company, overall language adoption, availability of libraries, tools and talent (and I'm not talking about training someone to be productive; sooner or later you need real experts and training an expert is expensive in any language) are extremely important. That's why when choosing between two languages, assuming both are well suited to the job at hand, a company should always pick the one with the wider adoption unless the other is so significantly "better" to trump lesser adoption. And the smaller the adoption, the "better" the language needs to be.
There's no doubt Haskell is an excellent programming language. I'm learning it and enjoying it. But because it has such low adoption rates, it needs to provide benefits that are orders of magnitude higher than other, more popular languages. I guess that for some very particular use cases it does, but I just can't see it in the vast majority of cases.
Hobbyists can afford playing with different languages, and can (and perhaps should) jump from one to the next. Companies can't (or most certainly shouldn't, unless for toy projects); they should pick (wisely) a popular language that's right for them, and just stick with it.
BTW, I think that if a company does want to try a good, though not-so-popular language, it should pick a JVM language, as the interoperability among JVM languages, and the availability of libraries that can be shared among the popular and less-so JVM languages, reduces much of the risk.
For a company, overall language adoption, availability of libraries, tools and talent (and I'm not talking about training someone to be productive; sooner or later you need real experts and training an expert is expensive in any language) are extremely important. That's why when choosing between two languages, assuming both are well suited to the job at hand, a company should always pick the one with the wider adoption unless the other is so significantly "better" to trump lesser adoption. And the smaller the adoption, the "better" the language needs to be.
There's no doubt Haskell is an excellent programming language. I'm learning it and enjoying it. But because it has such low adoption rates, it needs to provide benefits that are orders of magnitude higher than other, more popular languages. I guess that for some very particular use cases it does, but I just can't see it in the vast majority of cases.
Hobbyists can afford playing with different languages, and can (and perhaps should) jump from one to the next. Companies can't (or most certainly shouldn't, unless for toy projects); they should pick (wisely) a popular language that's right for them, and just stick with it.
BTW, I think that if a company does want to try a good, though not-so-popular language, it should pick a JVM language, as the interoperability among JVM languages, and the availability of libraries that can be shared among the popular and less-so JVM languages, reduces much of the risk.
As the executive sponsor for this effort, I'd like to point out that we considered a wide variety of options. Java (and other JVM languages) as well as node.js, ruby, python, C++, and others. We also already had in-house Erlang experience, but chose not to go with that for web services for various reasons.
Haskell really is special, in that it represents a very strong point very far out on the "statically typed" spectrum. Among the options, it had the best bang for the buck. Erlang was also functional, but not statically typed. C++ was statically typed, but not functional. The others were largely not sufficiently better than our existing PHP solution, and/or were beat out by Haskell in the consideration.
It's also important to understand that Haskell is not replacing PHP; it's augmenting PHP. We have many millions of lines of PHP running, serving many millions of active users, and there would be very little value in porting the "long tail" of that business logic. Meanwhile, there is huge value to be un-locked in gaining tight error checking and great performance on the "important core."
I'd be really curious on why you chose not to go with Erlang. Could you expand on that a little?
Hi, I'm the author of the article.
I think you're right, and I think Haskell delivers on your definition of "better." I chose to try Haskell because I've been through a lot of agonizing maintenance work and I'd much rather do it in an environment that has mandatory, pervasive static analysis and tests that are by default isolated from all possible nondeterminism.
We don't have any 8-year-old Haskell yet, so I can't fairly compare it to refactoring our PHP codebase, but I have made a few sweeping changes to our 60k lines of Haskell over the past year and a half. It tends to be mostly effortless, no matter how much code needs adjusting.
I think you're right, and I think Haskell delivers on your definition of "better." I chose to try Haskell because I've been through a lot of agonizing maintenance work and I'd much rather do it in an environment that has mandatory, pervasive static analysis and tests that are by default isolated from all possible nondeterminism.
We don't have any 8-year-old Haskell yet, so I can't fairly compare it to refactoring our PHP codebase, but I have made a few sweeping changes to our 60k lines of Haskell over the past year and a half. It tends to be mostly effortless, no matter how much code needs adjusting.
I'm profoundly puzzled by your statement. Most of the really high quality developers I've seen have at least poked Haskell - the really lousy developers don't even touch it.
One way to phrase my objection is that popularity is a really good measure of how many really bad developers exist for a language. I am not interested in being replacable by a really bad developer, working with them, or hiring them.
Haskell (and a few other languages) occupy a very nice space where entire classes of errors are wiped out by language design (null pointers); other classes are strongly limited by the type inferencing. This bonus in particular scales non-linearly with the LoC of your codebase due to the interaction of multiple components of the codebase all having to be type safe.
One way to phrase my objection is that popularity is a really good measure of how many really bad developers exist for a language. I am not interested in being replacable by a really bad developer, working with them, or hiring them.
Haskell (and a few other languages) occupy a very nice space where entire classes of errors are wiped out by language design (null pointers); other classes are strongly limited by the type inferencing. This bonus in particular scales non-linearly with the LoC of your codebase due to the interaction of multiple components of the codebase all having to be type safe.
I about 50% agree with you here. The factor that you're forgetting is the filter effect: Haskell is not adopted, it's learned after a thankless (there are no jobs) process of deep study. This means that the quality of available talent is of a much higher standard. If you adopt a technology with these characteristics, and you're one of the few who are hiring in this pool then you have a significant advantage!
The only risk in my mind is when the big boys figure it out and start gobbling up Haskellers! :-)
The only risk in my mind is when the big boys figure it out and start gobbling up Haskellers! :-)
Author here.
We've trained a few people up on Haskell at IMVU now, and I'm starting to think that learning Haskell is at its very hardest when you're building tiny toy applications and all you have for answers are google searches and books.
It seems to help a lot to root the learning process in a specific concrete goal, in an existing codebase that already has idioms and patterns in-place, and to offer direct access to people who have answers to newbie questions.
We've trained a few people up on Haskell at IMVU now, and I'm starting to think that learning Haskell is at its very hardest when you're building tiny toy applications and all you have for answers are google searches and books.
It seems to help a lot to root the learning process in a specific concrete goal, in an existing codebase that already has idioms and patterns in-place, and to offer direct access to people who have answers to newbie questions.
I'm really enamored with haskell, and you're hitting on exactly the problem I'm having.
Any advise on learning haskell out in the wild?
Any advise on learning haskell out in the wild?
I learned a lot about writing good Haskell code from working with Yesod. Since it's a framework it structures your code for you and exposes you to a lot of well-designed patterns. Yesod and Conduit are two of the best written packages I've come into contact with (though there are many many more) and simply following their guides has given me new insight into what high-quality Haskell code should look like.
Hard work and practice in your spare time. Probably not what you wanted to hear, but that's the way most people do it.
Find a project to work on and someone to mentor you, talk to people at on #haskell and the mailing lists and at meetups.
I think this is a bad filter. I'd rather filter prospective employees based on an ambitious algorithm or a complex project they've done. Most of the people I know that like solving hard problems with novel data structures, algorithms, or mechanically-sympathetic implementations are averse to spending time on learning new programming languages. It seems to them (and to me) like focusing on bling rather than on substance.
Of course, this is a crude generalization, and obviously, anyone who's interested in learning any new, perhaps unappreciated, and difficult skill would make a good hire, and Haskellers are no exception. But I wouldn't filter based on programming language preference.
Of course, this is a crude generalization, and obviously, anyone who's interested in learning any new, perhaps unappreciated, and difficult skill would make a good hire, and Haskellers are no exception. But I wouldn't filter based on programming language preference.
I have to disagree on two points.
First, the implicit point you are making that ambitious algorithms or novel data structures are useful in industry. For the vast majority of applications this simply isn't the case.
Second, the argument that people interested in learning advanced computer science aren't interested in learning new programming languages. This goes against all of my experience; my friends who are most interested in advanced computer science concepts are exactly the ones who spend the most time exploring different programming paradigms as well.
I have one idea of what the distinction might be, though: learning different programming paradigms is very different from learning different programming languages. I certainly wouldn't want to waste time learning yet another Java or yet another Python, since as you say that would be focusing on bling rather than substance. If this is the distinction, Haskell comes out fine, as it is clearly a different paradigm from the mainstream languages.
First, the implicit point you are making that ambitious algorithms or novel data structures are useful in industry. For the vast majority of applications this simply isn't the case.
Second, the argument that people interested in learning advanced computer science aren't interested in learning new programming languages. This goes against all of my experience; my friends who are most interested in advanced computer science concepts are exactly the ones who spend the most time exploring different programming paradigms as well.
I have one idea of what the distinction might be, though: learning different programming paradigms is very different from learning different programming languages. I certainly wouldn't want to waste time learning yet another Java or yet another Python, since as you say that would be focusing on bling rather than substance. If this is the distinction, Haskell comes out fine, as it is clearly a different paradigm from the mainstream languages.
[deleted]
Wow, completely the opposite experience here. The majority of the prog. lang. enthusiasts I know are deeply into other aspects of computer science. They love high level languages because they can express their ideas so cleanly. Lots of devs are drooling over Rust because it allows us clean code with C-level control over memory layout and allocation.
The majority of PL enthusiasts might well be interested in other aspects of computer science, but I was pointing out the the converse isn't true: the majority of people interested in the more challenging aspects of comp sci are not necessarily interested in programming languages, which is why this would be a bad filter.
I drool over Rust, too, but I wouldn't switch my company's (little) C++ code to Rust tomorrow. Also, Rust appeals mostly to C/C++ programmers, who don't usually need as many OSS/OTS libraries as application-level programmers (and those, usually specialized, libraries they do need, either work with Rust or don't; if they don't, that alone would be a deal breaker).
People who risk their company's investment (or make any important decision) based on stuff they drool over, might need to get their priorities straight.
I drool over Rust, too, but I wouldn't switch my company's (little) C++ code to Rust tomorrow. Also, Rust appeals mostly to C/C++ programmers, who don't usually need as many OSS/OTS libraries as application-level programmers (and those, usually specialized, libraries they do need, either work with Rust or don't; if they don't, that alone would be a deal breaker).
People who risk their company's investment (or make any important decision) based on stuff they drool over, might need to get their priorities straight.
Many of the super programmers at google have PL research backgrounds! Like Jeff Dean. Most of the others are in systems, which are fairly aligned, and often both. Such PL enthusiasts are the last ones to try and switch the language being used.
> Such PL enthusiasts are the last ones to try and switch the language being used.
As a PL enthusiast myself, I agree with this.
By the time I had my first job out of college, I'd written programs in some twenty-odd different PLs, and felt comfortable with many of them. My employer needed me to learn SAS, so I did. They wanted some code written in Java, so I wrote it. Neither SAS nor Java is particularly fun to program in the way Haskell et al. are, but that's what they wanted, and I could do it.
There was a whole team of Java guys, and when asked to learn SAS they sat around bemoaning it and whining about how Java is so much better. "Why not just do it in Java?" they asked.
I've run into similar experiences at every company since. If people don't want Java, they want C++ or Perl. If someone writes a utility in Python, there are complainers. Some programmers will refuse to even use a particular program if it's not written in their language of choice.
In my experience, it's usually the people who aren't PL enthusiasts that are more opinionated about language choice, simply because they're less flexible.
As a PL enthusiast myself, I agree with this.
By the time I had my first job out of college, I'd written programs in some twenty-odd different PLs, and felt comfortable with many of them. My employer needed me to learn SAS, so I did. They wanted some code written in Java, so I wrote it. Neither SAS nor Java is particularly fun to program in the way Haskell et al. are, but that's what they wanted, and I could do it.
There was a whole team of Java guys, and when asked to learn SAS they sat around bemoaning it and whining about how Java is so much better. "Why not just do it in Java?" they asked.
I've run into similar experiences at every company since. If people don't want Java, they want C++ or Perl. If someone writes a utility in Python, there are complainers. Some programmers will refuse to even use a particular program if it's not written in their language of choice.
In my experience, it's usually the people who aren't PL enthusiasts that are more opinionated about language choice, simply because they're less flexible.
> Rust appeals mostly to C/C++ programmers,
There's actually a lot of people who aren't coming from C++ too. A whole bunch of Rubyists are checking out Rust, for example.
That said, you're totally right that production Rust is probably a bad idea. You'd be user #4!
There's actually a lot of people who aren't coming from C++ too. A whole bunch of Rubyists are checking out Rust, for example.
That said, you're totally right that production Rust is probably a bad idea. You'd be user #4!
That means it's a great filter, though, in one way—it has high precision but low recall. If you're building a smallish team on a lowish budget then precision is vital and recall isn't.
Assuming the remaining population is sufficient (which, for Haskell, for a smallish team, in a job that can be done remotely or is in a tech hot-spot, it probably is).
I'm a Haskell user and I find Rust really interesting. I know a goodly number of other Haskell users who are also interested in Rust too.
I don't think you're speaking from a lot of knowledge here.
I don't think you're speaking from a lot of knowledge here.
"Rust appeals mostly to C/C++ programmers"
As primarily a Scala programmer, Rust appeals greatly to me. It's basically the same language without cruft inherited from Java, and its much more amenable to optimisation that Java. For example, large heaps have unacceptable GC pauses, so projects like Cassandra invest a lot of effort into "off-heap" memory management (i.e. manual memory management). With Rust that just works a whole heap better (pun intended).
People tend to talk about "expressiveness" when discussing programming languages, which usually mean concision. I'm much more interested in what I call "expressive width", which means how high- and low-level can get in a language. PHP is little expressive width, because its runtime is so poor. Scala has good expressive width because I can play with threads, CAS operations, mem-mapped files, and even manual memory management (which is going quite far off-piste, but possible with sun.Unsafe). Rust has even more expressive width than Scala.
[Rust needs higher-kinded types before it will truly make me happy. I believe they'll arrive soonish.]
For your other points:
- P(interested in PL | interested in CS) != P(interested in CS | interested in PL).
Agreed, but I don't think it matters. Anyone interested in discrete maths (so, CS) will pick up Haskell quickly.
- "People who risk their company's investment (or make any important decision) based on stuff they drool over, might need to get their priorities straight."
Depends on your company's size and ethos. I'd rather have a few great people than a room full of monkeys, but I understand arguments that go the other way.
As primarily a Scala programmer, Rust appeals greatly to me. It's basically the same language without cruft inherited from Java, and its much more amenable to optimisation that Java. For example, large heaps have unacceptable GC pauses, so projects like Cassandra invest a lot of effort into "off-heap" memory management (i.e. manual memory management). With Rust that just works a whole heap better (pun intended).
People tend to talk about "expressiveness" when discussing programming languages, which usually mean concision. I'm much more interested in what I call "expressive width", which means how high- and low-level can get in a language. PHP is little expressive width, because its runtime is so poor. Scala has good expressive width because I can play with threads, CAS operations, mem-mapped files, and even manual memory management (which is going quite far off-piste, but possible with sun.Unsafe). Rust has even more expressive width than Scala.
[Rust needs higher-kinded types before it will truly make me happy. I believe they'll arrive soonish.]
For your other points:
- P(interested in PL | interested in CS) != P(interested in CS | interested in PL).
Agreed, but I don't think it matters. Anyone interested in discrete maths (so, CS) will pick up Haskell quickly.
- "People who risk their company's investment (or make any important decision) based on stuff they drool over, might need to get their priorities straight."
Depends on your company's size and ethos. I'd rather have a few great people than a room full of monkeys, but I understand arguments that go the other way.
> I believe they'll arrive soonish
Depends what you mean by soonish... there's certainly no concrete plan for how to implement them yet (or even a vaguely sensible proposed syntax).
Depends what you mean by soonish... there's certainly no concrete plan for how to implement them yet (or even a vaguely sensible proposed syntax).
[deleted]
Haskell was inflicted on my whole CS class, and for that class P(avid problem solver|likes haskell) was mighty high.
As long as you can sample from a decent talent pool, I think it's a good filter. (Provided you only care about your outcome, not fairness to job seekers.)
As long as you can sample from a decent talent pool, I think it's a good filter. (Provided you only care about your outcome, not fairness to job seekers.)
> I'd rather filter prospective employees based on an ambitious algorithm or a complex project they've done
I've never needed the former, and only have a little bit of the latter. I wish I had more of both, but I find I have to do hard stuff on the side.
What do you do, exactly?
I've never needed the former, and only have a little bit of the latter. I wish I had more of both, but I find I have to do hard stuff on the side.
What do you do, exactly?
learning another script language is totally different from learning Haskell. For the latter it requires much more than syntax translation, which is implied in your comment.
This is my worry with Python. Its a good language, but it is too easy to learn.
I ofter read that Python developers are of a higher quality than PHP developers.
So downvoters, how do you differentiate a good coder who chose the language based on features and reputation, over a poor coder, who chose it "because its easier than Perl"?
The point I am trying to make is that I think there are benefits to having a higher bar to entry.
So downvoters, how do you differentiate a good coder who chose the language based on features and reputation, over a poor coder, who chose it "because its easier than Perl"?
The point I am trying to make is that I think there are benefits to having a higher bar to entry.
If you hang out a lot on reddit, this particular site is written in python, so there might be a bias. Python is also often chosen for entry courses in CS, while PHP is probably more picked up by people who need to know it for work, so it is not necessarly well understood by many of its newly professional users.
My guess is that python is a more well rounded language, with a nicer syntax, which enforces indentation rules (that's a plus for teachers). PHP only shows its merits after a while, and through good practices.
However, looking at the charts over at langpop.com, I don't see how people would choose PHP over perl for its easiness, given the wide difference of popularity. They just don't sit in the same ballpark (except for maybe a few specific markets).
My guess is that python is a more well rounded language, with a nicer syntax, which enforces indentation rules (that's a plus for teachers). PHP only shows its merits after a while, and through good practices.
However, looking at the charts over at langpop.com, I don't see how people would choose PHP over perl for its easiness, given the wide difference of popularity. They just don't sit in the same ballpark (except for maybe a few specific markets).
I usually look at their code, which is often a good indication of whether they are a good coder or not.
I think it is pretty safe to say that the average quality of PHP developers is lower than any other language except possibly javascript. There's a huge amount of very low knowledge people in both camps dragging the average down. Comparing python to ruby or perl things seem pretty similar.
I've been hiring both Python and PHP devs. It's absolutely night and day difference.
I don't think you need to worry... YET.
1. Python is simply a superior language.
2. There isn't currently a "WordPress" type of project that is bringing in droves of awful Python coders.
Will it eventually turn sour? If things continue with Python, I think so. But the language and community will do a good job keeping things slightly sane. I think the Python community understands the importance on both fronts. Sure, bad programmers are everywhere. But bad contributing programmers need to be kept at bay and out of our favorite GitHub repo's.
In my experiences as a solo founder needing junior-level programmers, Python is currently in the perfect sweet spot for adoption vs. quality. It's largely up to us in the community as to how long that will last.
I don't think you need to worry... YET.
1. Python is simply a superior language.
2. There isn't currently a "WordPress" type of project that is bringing in droves of awful Python coders.
Will it eventually turn sour? If things continue with Python, I think so. But the language and community will do a good job keeping things slightly sane. I think the Python community understands the importance on both fronts. Sure, bad programmers are everywhere. But bad contributing programmers need to be kept at bay and out of our favorite GitHub repo's.
In my experiences as a solo founder needing junior-level programmers, Python is currently in the perfect sweet spot for adoption vs. quality. It's largely up to us in the community as to how long that will last.
Python tries to make it easier to avoid pitfalls. In contrast to PHP or Perl, Python believes that
> There should be one--and preferably only one--obvious way to do it.
Even something like significant whitespace forces everyone to format their code similarly.
That's not to say an incompetent programmer can't write horrible code in Python, just that Python tries to make it easier to do the right thing than the wrong thing.
> There should be one--and preferably only one--obvious way to do it.
Even something like significant whitespace forces everyone to format their code similarly.
That's not to say an incompetent programmer can't write horrible code in Python, just that Python tries to make it easier to do the right thing than the wrong thing.
Which makes it quite ironic pg dubbed this phenomenon the "Python Paradox".
http://paulgraham.com/pypar.html
http://paulgraham.com/pypar.html
I see it the other way around. It's easier to find experts in esoteric languages because you don't have to sift through the same piles of dullards that use main stream languages. A person who has chosen to use a language like Haskell is probably not your average brogrammer.
Also, PHP in particular is so bad, to become an expert in it suggests a willing blindness to its problems, or a general attitude of "live with problems" that converges to a complete shit sandwich over time. When one could have just switched to something else so long ago, PHP is a point against any claim of being a software expert.
It is just like the traffic in DC: whatever your arguments against doing anything about a bad situation because "we don't have the time" or "we don't have the money", it ain't getting any better in the future. If you don't fix it now, it won't ever get fixed.
Also, PHP in particular is so bad, to become an expert in it suggests a willing blindness to its problems, or a general attitude of "live with problems" that converges to a complete shit sandwich over time. When one could have just switched to something else so long ago, PHP is a point against any claim of being a software expert.
It is just like the traffic in DC: whatever your arguments against doing anything about a bad situation because "we don't have the time" or "we don't have the money", it ain't getting any better in the future. If you don't fix it now, it won't ever get fixed.
It is a moral crime to recruit the Haskell pool for its precertified geniuses, and then force hek to use Java and Python.
Well, in general, I think it's stupid for managers to tell programmers what programming languages to use, anyway.
My experience is that popularity does matter a lot, but that the value of popularity is logistic rather than e.g. linear. Not enough popularity does make using a language in production unrealistic, but there is a threshold of usability after which increased popularity seems to have rapidly diminishing returns. I would argue that Haskell passed this threshold a couple years ago for most use cases. There is still a lot of work to do on cabal and some of the core libraries, but we're happily using Haskell in production, and for us the benefits very clearly outweigh the occasional difficulties.
I don't think you need orders of magnitude improvements to get a large adopted base [Note: I'm specifically not saying "rate"]. Just wait for new projects (or new companies) who're looking for which one is just marginally a better/funner/cooler choice over the rest. They'll accrue credibility for the language, and that'll add to it's case for usage in organizations that have already selected languages, and are looking for new ones. I think an organization that's been paying for a prior PHP choice would be quite happy to jump into Haskell, but is just waiting for a little validation of their choice with some more success stories.
And that's not really a function of metrics, it just needs a few well-spoken cheerleaders with a few good stories.
The largest issue Haskell has with adoption is its intentional avoidance of (at least, mainstream) cargo-cult programming culture. The "avoid success at all costs" attitude makes cheerleading Haskell as a proper, valid selection for betting your project less cool. The best you can do is "look, I got away with using Haskell!"
And that's not really a function of metrics, it just needs a few well-spoken cheerleaders with a few good stories.
The largest issue Haskell has with adoption is its intentional avoidance of (at least, mainstream) cargo-cult programming culture. The "avoid success at all costs" attitude makes cheerleading Haskell as a proper, valid selection for betting your project less cool. The best you can do is "look, I got away with using Haskell!"
I think it's fine to cheerlead for Haskell, as we're all happy to increase the size of the community. What we don't want is for the community values to be diluted by that size increase, which is the sense in which "avoid success at all costs" seems to be meant. For example, I'm always happy to help someone understand monads, but I would be very unhappy if that involved renaming them to "workflows".
Yes, it is important to maintain the shibboleths that protect the community from riffraff.
The notion that terms like "monad" are nothing but shibboleths is very common and completely misguided. Instead, these terms are used for a couple of very good reasons.
First, the simple reason that they are accurate terms for which there are no good replacements. The English language does not have the necessary scope to cover the relevant abstractions. This leaves you with two options: use the accurate term from mathematics, or make up some new term. Either way the term will be something new to most people, and making something up does not provide any value.
This leads me to the second reason: the mathematical terms actually do provide additional value. If you are working with monoids and do some Googling, you can discover a wealth of information that is directly applicable in practice, for example you can learn that List is the initial monoid [1]. You don't have to learn any of this, it's completely optional, but if you do choose to explore it you can get real benefits. Renaming monads "workflows" would prevent you from gaining those benefits, and would not get you anything in exchange other than making the people who are scared of math more comfortable.
[1] An example of why this is useful to know: http://byorgey.wordpress.com/2012/11/05/foldr-is-made-of-mon...
Also this post from the wiki, though it's a bit heavier on the theory: http://www.haskell.org/haskellwiki/Free_structure
First, the simple reason that they are accurate terms for which there are no good replacements. The English language does not have the necessary scope to cover the relevant abstractions. This leaves you with two options: use the accurate term from mathematics, or make up some new term. Either way the term will be something new to most people, and making something up does not provide any value.
This leads me to the second reason: the mathematical terms actually do provide additional value. If you are working with monoids and do some Googling, you can discover a wealth of information that is directly applicable in practice, for example you can learn that List is the initial monoid [1]. You don't have to learn any of this, it's completely optional, but if you do choose to explore it you can get real benefits. Renaming monads "workflows" would prevent you from gaining those benefits, and would not get you anything in exchange other than making the people who are scared of math more comfortable.
[1] An example of why this is useful to know: http://byorgey.wordpress.com/2012/11/05/foldr-is-made-of-mon...
Also this post from the wiki, though it's a bit heavier on the theory: http://www.haskell.org/haskellwiki/Free_structure
Making something up would have the advantage of hiding all of the various monad tutorials...
> That's why when choosing between two languages, assuming both are well suited to the job at hand, a company should always pick the one with the wider adoption unless the other is so significantly "better" to trump lesser adoption.
I think it plateaus at a certain point. You don't need the largest community, you just need critical mass so you don't have to build everything yourself. In fact, at some point being the biggest ends up diluting the talent pool because of the number of people getting into it for the money, and this is becoming a much bigger problem as the traditional job economy dries up and the demand for programmers increases.
As to being "significantly better", I think Haskell has that in spades. In fact the things that make Haskell better are probably difficult to appreciate by a lot of younger programmers who are using relatively new languages and frameworks like Node and Rails. When you start seeing the effects of code rot and programmer turnover on a codebase over time, the types of static checks that are a couple orders more powerful and simultaneously less verbose and restrictive than what most people think of when they hear "static language" (ie. Java), then Haskell really starts to shine.
Personally I think companies that invest in Haskell are going to start seeing major dividends in terms of productivity and agility over the lifetime of the company.
I think it plateaus at a certain point. You don't need the largest community, you just need critical mass so you don't have to build everything yourself. In fact, at some point being the biggest ends up diluting the talent pool because of the number of people getting into it for the money, and this is becoming a much bigger problem as the traditional job economy dries up and the demand for programmers increases.
As to being "significantly better", I think Haskell has that in spades. In fact the things that make Haskell better are probably difficult to appreciate by a lot of younger programmers who are using relatively new languages and frameworks like Node and Rails. When you start seeing the effects of code rot and programmer turnover on a codebase over time, the types of static checks that are a couple orders more powerful and simultaneously less verbose and restrictive than what most people think of when they hear "static language" (ie. Java), then Haskell really starts to shine.
Personally I think companies that invest in Haskell are going to start seeing major dividends in terms of productivity and agility over the lifetime of the company.
I don't disagree with your major point - "companies that invest in [[functional programming languages with a strong typing system]] are going to start seeing major dividends in terms of productivity and agility over the lifetime of the company." Haskell in particular I'm not sure of - the 'code rot' could easily manifest itself in performance. Haskell's laziness makes slow code and code that use up a lot of memory hard to locate, and at times require significant refactoring, to correct, though this is offset by haskell's type system somewhat...
You can see this manifesting in the project of the article's author under the "Not All Sunshine and Rainbows" section.
What I'm getting at - Bad code can be written in any language. IMO what's more important with regards to the language is how easy does it make to fix it.
I've only used haskell for my thesis project as well as having toyed with it in my free time now and then over the past two years, and have no commercial Haskell experience, so perhaps someone with more experience can chime in...
EDIT: A Haskell project I was working on: http://stackoverflow.com/questions/21654081/how-do-i-optimis...
You can see this manifesting in the project of the article's author under the "Not All Sunshine and Rainbows" section.
What I'm getting at - Bad code can be written in any language. IMO what's more important with regards to the language is how easy does it make to fix it.
I've only used haskell for my thesis project as well as having toyed with it in my free time now and then over the past two years, and have no commercial Haskell experience, so perhaps someone with more experience can chime in...
EDIT: A Haskell project I was working on: http://stackoverflow.com/questions/21654081/how-do-i-optimis...
So far as code leaks go I think the professional story is divided. Everyone runs into them, some people learn to fix them, those that do tend to think of them as a little annoying but not significantly more so than profiling and tuning strict code.
Fortunately, outside of space leaks the Haskell runtime is actually really quite fast. At the end of the day most of your code which isn't leaky (buggy) will be really fast almost for free.
But that second step can be painful right now. The tools exist to profile and debug space leaks, but they're not cohesive and require a fair amount of black knowledge. It's also fairly murky as to how to escalate: if you've identified your problem area, strictified it a bunch, and it's still broken... well what then?
The answer seems to be growing that internal expertise. There are enough people out there writing enough stuff that a dedicated student of the runtime can learn all of the tricks needed to really attack failures like that—profiling, unboxing, reading core, figuring out the inliner, and a variety of style guide hints to making things all work smoothly (e.g. avoid the Writer monad).
Fortunately, outside of space leaks the Haskell runtime is actually really quite fast. At the end of the day most of your code which isn't leaky (buggy) will be really fast almost for free.
But that second step can be painful right now. The tools exist to profile and debug space leaks, but they're not cohesive and require a fair amount of black knowledge. It's also fairly murky as to how to escalate: if you've identified your problem area, strictified it a bunch, and it's still broken... well what then?
The answer seems to be growing that internal expertise. There are enough people out there writing enough stuff that a dedicated student of the runtime can learn all of the tricks needed to really attack failures like that—profiling, unboxing, reading core, figuring out the inliner, and a variety of style guide hints to making things all work smoothly (e.g. avoid the Writer monad).
Speaking as a part time GHC contributor (and alleged "core haskeller"), theres A LOT of exciting work going on right now for improving debuggability wrt correctness bugs and performance tuning.
GHC 7.10 is slated to (hopefully have): 1. exceptions will come with stack traces! a. Some of this same work will also allow for sampling based based performance tools b. mind you, a lots still in flux 2. theres a non zero chance you'll have in langauge facilities to create threads that get killed automatically if they exceed certain programaticaly set CPU/Memory/other resource limits! 3. Other things which are a combination of hard work and clever research grade engineering
I myself (as Tel knows) have plans for making GHC haskell AMAZING for numerical computing, and my wee ways of contributing to GHC are guided by that goal
GHC 7.10 is slated to (hopefully have): 1. exceptions will come with stack traces! a. Some of this same work will also allow for sampling based based performance tools b. mind you, a lots still in flux 2. theres a non zero chance you'll have in langauge facilities to create threads that get killed automatically if they exceed certain programaticaly set CPU/Memory/other resource limits! 3. Other things which are a combination of hard work and clever research grade engineering
I myself (as Tel knows) have plans for making GHC haskell AMAZING for numerical computing, and my wee ways of contributing to GHC are guided by that goal
Upvoted.
"The answer seems to be growing that internal expertise."
Or you can go with OCaml. :)
"The answer seems to be growing that internal expertise."
Or you can go with OCaml. :)
Sure, or any strict language. There are a lot of benefits to laziness though even before you start to consider other Haskell/OCaml deltas. Laziness provides a much better platform for separation of concerns which has enabled Haskell programs to have an unheard of degree of decomposition and reuse.
As always, it's a tradeoff. At this point, I don't personally feel afraid of debugging core or profiling space leaks. I don't feel like I'm much more than 50% up to date on techniques to fix them... but when they arise I have little trouble eliminating them.
Lazy resource deallocation is a problem of lazy IO. The solution is to not use lazy IO and that's tremendously tenable today due to libraries like pipes and conduit.
As always, it's a tradeoff. At this point, I don't personally feel afraid of debugging core or profiling space leaks. I don't feel like I'm much more than 50% up to date on techniques to fix them... but when they arise I have little trouble eliminating them.
Lazy resource deallocation is a problem of lazy IO. The solution is to not use lazy IO and that's tremendously tenable today due to libraries like pipes and conduit.
I agree.
Would like to note I've spent more time trying to optimize this[1] than writing code, however.
http://stackoverflow.com/questions/21654081/how-do-i-optimis...
Would like to note I've spent more time trying to optimize this[1] than writing code, however.
http://stackoverflow.com/questions/21654081/how-do-i-optimis...
On a quick glance I'd suggest (in addition to the responses already given) to (a) use a more efficient structure for holding events (Data.Sequence or maybe Pipes) since (++) has bad performance when the left argument is large (as is definitely your case), (b) for all of your "static" types, add a strictness annotation and possibly an {-# UNBOX #-}, (c) consider replacing System.Random with mwc-random, it's more efficient and has better randomness.
Diving into processOrder since it's the big time/space hog I think strictness annotations might help uniformly since there's a lot of packing/unpacking going on. To really dig in you can add some manual SCC annotations or dump core on it and see what's doing all the indirection.
These are all pretty general, high-level Haskell performance notes, by which I mean if you learn them once you'll be able to apply them much more quickly in the future whenever you find bottlenecks.
Diving into processOrder since it's the big time/space hog I think strictness annotations might help uniformly since there's a lot of packing/unpacking going on. To really dig in you can add some manual SCC annotations or dump core on it and see what's doing all the indirection.
These are all pretty general, high-level Haskell performance notes, by which I mean if you learn them once you'll be able to apply them much more quickly in the future whenever you find bottlenecks.
Yeah, a good starting point is that if something is running slow, and if there's lists involved, look at how you're using them. Also, as a special case, replace String with (strict or lazy) Text or Bytestring, as appropriate.
Unfortunately OCaml leaves a lot to be desired after working with Haskell for a significant amount of time. As far as I know GHC's extensions makes the type system much more powerful in non trivial ways. For example things like type families, data kinds, kind polymorphism, type level naturals, RankNTypes, and so on. That being said OCaml is still orders of magnitude better than most other options. As well, a least for me, it is a lot more work introduce explicit laziness all over the place into eager language, than it is to go the other way and apply selective strictness.
Is it fair to say OCaml is a more practical choice (as of now)?
That depends on the domain.
Would appreciate it if you would expand on that. (My comment was general in nature.)
Well, most important is probably availability and stability of domain relevant libraries in either language. There are areas Haskell has well covered and areas where it's thinner...
It's also been the case historically that Haskell had much better concurrency support - I'm not sure whether that's still true - in which case a more parallel workload might push Haskell-wards (if it's not so parallel that simply spinning up N entirely separate processes make sense, in which case concurrency support doesn't matter).
I'm not familiar enough with the current state of OCaml, or with Haskell libraries outside the domains I've been playing in, to really give a thorough run-down of specific domains, sadly. I'd be interested to see it if someone else took a swing.
It's also been the case historically that Haskell had much better concurrency support - I'm not sure whether that's still true - in which case a more parallel workload might push Haskell-wards (if it's not so parallel that simply spinning up N entirely separate processes make sense, in which case concurrency support doesn't matter).
I'm not familiar enough with the current state of OCaml, or with Haskell libraries outside the domains I've been playing in, to really give a thorough run-down of specific domains, sadly. I'd be interested to see it if someone else took a swing.
Or F#, if you are using .net or mono. It's pretty well integrated into Visual Studio these days, and quite good in Xamarin Studio.
Sadly, although the performance of mono is good enough for most things i.e. GUI, it's several factors slower in performance-critical applications.
Although I'm excited about F#, the experience in Visual Studio is really great.
Although I'm excited about F#, the experience in Visual Studio is really great.
> What I'm getting at - Bad code can be written in any language. IMO what's more important with regards to the language is how easy does it make to fix it.
Hm, not sure I agree with this. As a professional rubyist I am aware of the wide variety of bugs that can and do happen because of things that Haskell simply wouldn't allow you to do (at least without pathological abuse of the language that would never pass a code review). The thing that makes Haskell special to me is just how much the commitment to functional purity and a powerful type system buy you in terms of decreasing the surface are of potential problems.
You've rightly pointed out that Haskell has its own memory and performance pitfalls that are very difficult to debug, but my instinct is that those things, while perhaps being showstoppers for embedded and realtime systems, are not intractable in a typical server environment, and that it's orders magnitude easier to improve the tooling around detecting and solving these problems than it is to ever make a language like Ruby code safer and require fewer tests.
Of course I'm an utter beginner at Haskell, so I don't have a real sense of its downsides at scale, so take my opinion with a teaspoon of salt.
Hm, not sure I agree with this. As a professional rubyist I am aware of the wide variety of bugs that can and do happen because of things that Haskell simply wouldn't allow you to do (at least without pathological abuse of the language that would never pass a code review). The thing that makes Haskell special to me is just how much the commitment to functional purity and a powerful type system buy you in terms of decreasing the surface are of potential problems.
You've rightly pointed out that Haskell has its own memory and performance pitfalls that are very difficult to debug, but my instinct is that those things, while perhaps being showstoppers for embedded and realtime systems, are not intractable in a typical server environment, and that it's orders magnitude easier to improve the tooling around detecting and solving these problems than it is to ever make a language like Ruby code safer and require fewer tests.
Of course I'm an utter beginner at Haskell, so I don't have a real sense of its downsides at scale, so take my opinion with a teaspoon of salt.
I really wish Haskell weren't lazy by default. Laziness is great in places, allows you to write very declaratively, and even allows for some computations that would otherwise be impossible or less efficient. However, most of the time, you envision and write code as a series of steps (even though theoretically in a functional language, you're just writing an expression). In a practical matter, the cases in which lazy evaluation becomes desirable exist, but are limited. Most of the time, you want, and expect when you write it, your code to be evaluated strictly. Of course you can enforce strictness manually with `seq` and bang notations and the like, but this clutters up your code and might not even matter if you're using library code which doesn't do the same. IMO Haskell would greatly served by having laziness be opt-in, not opt-out. But its pretty ingrained at this point, and there doesn't seem to be too much of a push to change it (who knows how much extant code would be broken by such a change).
> However, most of the time, you envision and write code as a series of steps
Perhaps counterintuitively, I disagree here. I think using laziness enough will eventually change your mind that most code must be thought of in such discrete steps. Instead, I tend to keep in mind and think constantly about how a computation might be partially delivered and how it might be consumed. This is part of the declarative promise of laziness playing out, I feel.
Ultimately I don't think there's a "right" decision. I'm comfortable in strict and lazy languages. I do think everyone should experience both strict and lazy evaluations enough to no longer be worried about them and instead just see each as another, separate form of computation. At the very least it'll make you much more aware of what evaluation order and normalization feel like.
(I'd also highly recommend everyone plays with a total language for a while, too. That feel is quite distinct.)
At the end of the day, I feel like having laziness by default and picking around for places which ought to be strict is very dual to having strictness be default and picking around for places to make lazy. Both are pretty annoying.
I do think people are generally more sensitive to problems of excessive laziness than they are to problems of excessive strictness. Manually wiring around stopping criteria and consumption control through all of your (no longer decomposable) loops/recursion is the price of strictness. Both problems are helped by tooling, as well.
Perhaps counterintuitively, I disagree here. I think using laziness enough will eventually change your mind that most code must be thought of in such discrete steps. Instead, I tend to keep in mind and think constantly about how a computation might be partially delivered and how it might be consumed. This is part of the declarative promise of laziness playing out, I feel.
Ultimately I don't think there's a "right" decision. I'm comfortable in strict and lazy languages. I do think everyone should experience both strict and lazy evaluations enough to no longer be worried about them and instead just see each as another, separate form of computation. At the very least it'll make you much more aware of what evaluation order and normalization feel like.
(I'd also highly recommend everyone plays with a total language for a while, too. That feel is quite distinct.)
At the end of the day, I feel like having laziness by default and picking around for places which ought to be strict is very dual to having strictness be default and picking around for places to make lazy. Both are pretty annoying.
I do think people are generally more sensitive to problems of excessive laziness than they are to problems of excessive strictness. Manually wiring around stopping criteria and consumption control through all of your (no longer decomposable) loops/recursion is the price of strictness. Both problems are helped by tooling, as well.
It certainly doesn't need to be thought of as a series of discrete steps, and laziness supports that view. Laziness is part of the Haskell philosophy of staying high-level: so high level that you're not even telling the machine in what order to perform its computations. However I stand by my contention that most of the time, when we as programmers write code to be executed, we're envisioning it as a series of steps: this is a major reason why monadic code is so popular and why Haskell's designers invented do-syntax to imitate imperative code. You say that it would be just as annoying to specify the individual places to make lazy, but I struggle to agree with that. After all, the vast majority of programming languages have strictness by default (even other functional languages like SML or Idris) and don't present any difficulty to the programmer. On the other hand, getting around Haskell's default laziness is a constant issue for performance-minded code (from what I've seen; certainly it's a constant topic of conversation), and makes stack traces and debugging hard.
To give a specific example, strictness is a pain in the butt when you try to write a combinator library. I have once tried to write a Parsec clone for Ocaml, and sometimes, I had to write functions in eta-expanded form, lest I enter an infinite recursion.
tel is right. We are desensitized to the problems of strict evaluation. It's the default, so we accept it as a fact of life. Lazy evaluation on the other hand, is "new", so whatever problems it have are magnified by our unwillingness to handle new problems. This is made even worse when we try to use lazy evaluation the same way we use strict evaluation: you get even more problems, and none of the advantages.
tel is right. We are desensitized to the problems of strict evaluation. It's the default, so we accept it as a fact of life. Lazy evaluation on the other hand, is "new", so whatever problems it have are magnified by our unwillingness to handle new problems. This is made even worse when we try to use lazy evaluation the same way we use strict evaluation: you get even more problems, and none of the advantages.
I think people are desensitized to the downsides of strict code. I don't think that either is "better", but I appreciate having spent a lot of time working in a lazy-by-default language.
I think the best option is total-by-default, honestly. I really like how Idris works, for instance. If you must work with some code which is non-normalizing then your choice of laziness/strictness can be a major architectural one—like organizing the entire pipeline of your program around a lazy consumer stream/set of coroutines.
I think the best option is total-by-default, honestly. I really like how Idris works, for instance. If you must work with some code which is non-normalizing then your choice of laziness/strictness can be a major architectural one—like organizing the entire pipeline of your program around a lazy consumer stream/set of coroutines.
Idris, one of the candidates to be "the next Haskell", is strict by default. It's not a mature language (+ecosystem) yet, but certainly usable and interesting. If you like exploring new cool concepts, you should have a look at it! (Plus it's very easy to learn when you know Haskell already.)
http://www.idris-lang.org/
http://www.idris-lang.org/
Idris also makes it trivial to annotate laziness, something most languages are lacking, also because it is total by default the evaluation strategy doesn't actually matter for all the total code.
Yeah, Idris really seems like a cool language. I wish it was better documented, at both the usage- and implementation-level. The source code for example is woefully under-commented; a quick grep of the source shows about one line of comments for 15 lines of code.
I think you are generalizing a bit too much. You may very well write code that way, but I don't. Having come to haskell from several years with ocaml, I found the switch to lazy by default to be entirely irrelevant and inconsequential. As long as I have the ability to be strict when I need to, and lazy when I need to (which both languages give me), I don't care which is the default.
In practice, I've found space leaks in Haskell code to be a bit analogous to extraneous copies in C++ code or NullPointerExceptions in Java: They're usually not a huge problem, but sometimes you get it wrong even if you're experienced and know what you're doing.
We've only had one really disasterous space leak and it was caught and reverted right away. The hardest ones are generally the slow ones where the servers take 3-7 days to run out of RAM and fall over. If you never go more than a few hours without a code push, you might not even notice it.
We've only had one really disasterous space leak and it was caught and reverted right away. The hardest ones are generally the slow ones where the servers take 3-7 days to run out of RAM and fall over. If you never go more than a few hours without a code push, you might not even notice it.
FWIW one reason the service mentioned in the blog was so fast was because of laziness. Because it didn't need most of the request, the work required to parse it was mostly avoided.
>Haskell's laziness makes slow code and code that use up a lot of memory hard to locate, and at times require significant refactoring, to correct
This is largely mythology built up around a misunderstanding of the grain of truth at the core of it. Lazyness can create pathological performance issues, just as strictness can. 90% of the time neither matters, 5% of the time you need strictness, 5% of the time you need laziness. Using strictness in haskell is very simple, and knowing when you need to is very easy to learn. The profiling tools make it incredibly obvious when laziness is causing a problem. And I can't even imagine a case where it requires significant refactoring. It is generally adding a couple of "!"s or importing Module.Strict instead of Module.Lazy.
This is largely mythology built up around a misunderstanding of the grain of truth at the core of it. Lazyness can create pathological performance issues, just as strictness can. 90% of the time neither matters, 5% of the time you need strictness, 5% of the time you need laziness. Using strictness in haskell is very simple, and knowing when you need to is very easy to learn. The profiling tools make it incredibly obvious when laziness is causing a problem. And I can't even imagine a case where it requires significant refactoring. It is generally adding a couple of "!"s or importing Module.Strict instead of Module.Lazy.
> This is largely mythology built up around a misunderstanding of the grain of truth at the core of it.
I agree. I've encountered a few space leaks in my Haskell code, but it's always been pretty easy to track down: profile the memory usage and look for a conspicuously huge peak. It will usually be fixable by forcing an intermediate result, or using a different recursion strategy (eg. a fold instead of explicit recursion). Haskell functions tend to be so small and single-purpose that there are no knock-on effects of doing this refactoring.
I think there's a perception that 'Haskell is difficult to debug', whereas the reality is more like 'Haskell will reject all of the easy bugs'. In other words, the average difficulty of a Haskell bug may be higher, but the density is less.
I agree. I've encountered a few space leaks in my Haskell code, but it's always been pretty easy to track down: profile the memory usage and look for a conspicuously huge peak. It will usually be fixable by forcing an intermediate result, or using a different recursion strategy (eg. a fold instead of explicit recursion). Haskell functions tend to be so small and single-purpose that there are no knock-on effects of doing this refactoring.
I think there's a perception that 'Haskell is difficult to debug', whereas the reality is more like 'Haskell will reject all of the easy bugs'. In other words, the average difficulty of a Haskell bug may be higher, but the density is less.
> eg. a fold instead of explicit recursion
Wouldn't adding some bangs in the patterns kind of do the same thing as using foldl'?
Wouldn't adding some bangs in the patterns kind of do the same thing as using foldl'?
Yes. Most recommendations for implementing recursion in Haskell that I've seen have amounted to writing it in terms of folds. You gain the benefit of experts vetting the code, and novices being able to understand what it is doing it a glance.
I'd like to add that for the 90% of the time it doesn't matter, 95% of that time you could use a total language. This is one of the joys of using a total language: seeing what's still easy!
I see no reason why strictness would have pathological performance problems. If you don't need to compute a value, dinner compute it. No performance problems.
I also see no reason you would need laziness for 5% of programming problems. That seems absurd given that the overwhelming majority of programmers have gotten along without it for 50 years.
I also see no reason you would need laziness for 5% of programming problems. That seems absurd given that the overwhelming majority of programmers have gotten along without it for 50 years.
Most programmers use laziness all over the place, they unfortunately have few tools for introducing it, and thus don't recognize it as such.
For example in C/C++ your only ability to introduce laziness is via manual thunking, if, or the logical conjunctive (&&), and disjunction (||) which have non-strict semantics. For example you often see the idiom of:
For example in C/C++ your only ability to introduce laziness is via manual thunking, if, or the logical conjunctive (&&), and disjunction (||) which have non-strict semantics. For example you often see the idiom of:
if (x != NULL && x.foo == bar) { ... }
until :: Bool -> a -> a -> a
until b t f = case (not b)
True -> t
False -> f>I see no reason why strictness would have pathological performance problems. If you don't need to compute a value, dinner compute it. No performance problems.
You just answered your own question. "If you don't need to compute a value, then don't compute it" is laziness.
>That seems absurd given that the overwhelming majority of programmers have gotten along without it for 50 years.
No they haven't. Most programmers don't understand it, but they tend to use it on a regular basis.
You just answered your own question. "If you don't need to compute a value, then don't compute it" is laziness.
>That seems absurd given that the overwhelming majority of programmers have gotten along without it for 50 years.
No they haven't. Most programmers don't understand it, but they tend to use it on a regular basis.
> You just answered your own question. "If you don't need to compute a value, then don't compute it" is laziness.
No -- "laziness" is saying "the programmer wants us to compute this value, but we don't need it just yet, so we're going to create a thunk that will sit on the heap until it's evaluated or garbage collected". Strictness is doing what the programmer says, which involves the programmer deciding what does and does not have to be computed while he's writing the program.
No -- "laziness" is saying "the programmer wants us to compute this value, but we don't need it just yet, so we're going to create a thunk that will sit on the heap until it's evaluated or garbage collected". Strictness is doing what the programmer says, which involves the programmer deciding what does and does not have to be computed while he's writing the program.
Perhaps I was not clear. You stated the very reason for laziness: to not compute things you don't need computed. There are many cases where it is very beneficial to be able to determine this at evaluation time, rather than when writing the code. Hence laziness, and hence it being used all the time in most mainstream languages.
You say it's used all the time -- so how about you give an example of a "lazy" solution to a problem in, say, C?
I implement laziness "manually" all the time in Objective-C.
For example, imagine something happens which invalidates a property which must be calculated (say, a bounding box for a graphical object).
Instead of immediately recomputing it, just set it to nil, and only recompute it the next time the property is actually accessed.
This way you can harmlessly invalidate it multiple times without doing a potentially expensive calculation (which just gets thrown out by subsequent invalidations).
For example, imagine something happens which invalidates a property which must be calculated (say, a bounding box for a graphical object).
Instead of immediately recomputing it, just set it to nil, and only recompute it the next time the property is actually accessed.
This way you can harmlessly invalidate it multiple times without doing a potentially expensive calculation (which just gets thrown out by subsequent invalidations).
That's not laziness, that's just cache invalidation.
It's fair to say that it's laziness, in my view. That sort of thing is just a manual implementation of what laziness would do in a particular situation. The closure that the thunk would hold is essentially diffused into the object that contains the lazy property.
A fast sorting algorithm in Haskell degrades gracefully to a (faster) selection algorithm when you don't ask for the whole thing.
In other words, "sort lst" gives you a sorted list, but "take 5 (sort lst)" gives you the first five elements, without sorting the rest of the list.
In other words, "sort lst" gives you a sorted list, but "take 5 (sort lst)" gives you the first five elements, without sorting the rest of the list.
Java example:
if (logger.isDebugEnabled())
logger.debug("..." + ...);SomeObjec *getObject(){
if (!theObject) theObject = createObject();
return theObject; }
return theObject; }
foo || bar
Every year I read on HN that this year is Haskell's year. What's so different about 2014? The whole 'only for REAL PROGRAMMERS ' argument still hasn't gotten any traction, my dudes, lisper's been trying it since the beginning of time.
Anyone got a link to a good todo-mvc implementation or something like that?
Anyone got a link to a good todo-mvc implementation or something like that?
>Every year I read on HN that this year is Haskell's year
Really? Every year I read on haskell a bunch of uniformed dismissal of haskell from people who seem to have a deep seated hatred of learning. How do you get "this year is haskell's year" from his comment at all?
>Anyone got a link to a good todo-mvc implementation
A what? MVC is an object oriented pattern, I would certainly hope nobody tried to do it in haskell.
Really? Every year I read on haskell a bunch of uniformed dismissal of haskell from people who seem to have a deep seated hatred of learning. How do you get "this year is haskell's year" from his comment at all?
>Anyone got a link to a good todo-mvc implementation
A what? MVC is an object oriented pattern, I would certainly hope nobody tried to do it in haskell.
Every year, on a Haskell thread, someone calls me a noob for not knowing it. Then you get like the same 3 or 4 dudes talking about how it's only a matter of time until the enterprise picks it up and all the dumb people go away. The some stuff about static analysis, and I'm like, that's it?
I just want to see a working to-do list application, or anything resembling that size.
I just want to see a working to-do list application, or anything resembling that size.
>Every year, on a Haskell thread, someone calls me a noob for not knowing it.
That sounds a lot like you putting words in other people's mouths. I've never seen anything like that.
>Then you get like the same 3 or 4 dudes talking about how it's only a matter of time until the enterprise picks it up and all the dumb people go away.
Or that.
>I just want to see a working to-do list application, or anything resembling that size before we start the name-calling.
Now I am even more confused. Why do you want to call people names, who do you want to call names, and why do you want to see some random toy app first?
That sounds a lot like you putting words in other people's mouths. I've never seen anything like that.
>Then you get like the same 3 or 4 dudes talking about how it's only a matter of time until the enterprise picks it up and all the dumb people go away.
Or that.
>I just want to see a working to-do list application, or anything resembling that size before we start the name-calling.
Now I am even more confused. Why do you want to call people names, who do you want to call names, and why do you want to see some random toy app first?
"In fact the things that make Haskell better are probably difficult to appreciate by a lot of younger programmers who are using relatively new languages and frameworks like Node and Rails"
Are you reading this stuff? I'm not going to talk to you if you're just being a dick. I know you can read, so stop being obtuse. Come up with something, quick.
Are you reading this stuff? I'm not going to talk to you if you're just being a dick. I know you can read, so stop being obtuse. Come up with something, quick.
Yes, I am reading it. That does not say anything even remotely close to "ulisesrmzroche is a noob for not knowing haskell". I am not being a dick, I am being polite and trying to understand where you are coming from. I don't see how insulting me helps either of us, only how it makes your claims that "haskell people keep insulting me" rather hypocritical.
Ok, nevermind dude. You're either not being honest or really can't understand nuance in the English language, which, either or, doesn't give us a lot of room for arguing. Saying 'a lot of younger programmers probably don't appreciate the best parts of haskell' is clear Ageism. I think it's annoying.
Since I said that, let me be perfectly clear. It's not ageism, it's experience-ism. There are things that you learn when you work on a code base that is 10, 20, 30 years old. There are things you learn when you've been a programmer for 10, 20, 30 years. A lot of the pain Haskell solves isn't apparent in a lot of Node or Rails apps simply because they are so new. That's it, don't read too much into it.
Omg, that's exactly the same thing but more PC. Whatever, I hate you HN. I'm taking my ball and going home.
Hey don't be the white person complaining about racism or the man complaining about sexism. If young people weren't getting a shot in the tech industry because of discrimination then I'd be a tad more sensitive to your protestations. But as it is, you're irrationally sticking your head in the sand. If you don't think that learn things and come to new insights that take years to develop then there's nothing I can say to convince you. But if you stick with it for a while, come back to me in 10 years and let's do a code review on what you wrote today and then we'll discuss if you still think I'm ageist for saying there are particular lessons learned through experience.
Hey, don't take it too bad. If you say anything negative about certain topics you will get downvotes. It's mostly because people not in love with certain topics have learned to just leave them alone and it's left a big echo chamber where anything positive will be upvoted and anything negative will be downvoted.
You'll notice this by seeing that in non-Haskell threads, critical remarks about Haskell are upvoted, while in 'Haskell threads' such as this one where an echo chamber forms, anything critical about Haskell will be downvoted. It's an interesting behavior, but it says a lot more about certain cliques than it does about HN.
Don't worry about it too much and get on with actually using practical tools to solve real problems and leave the echo chamber in their own peaceful world. Everyone is much better off.
You'll notice this by seeing that in non-Haskell threads, critical remarks about Haskell are upvoted, while in 'Haskell threads' such as this one where an echo chamber forms, anything critical about Haskell will be downvoted. It's an interesting behavior, but it says a lot more about certain cliques than it does about HN.
Don't worry about it too much and get on with actually using practical tools to solve real problems and leave the echo chamber in their own peaceful world. Everyone is much better off.
Reverse-agism is part of an egalitarian mindset of many HN commenters, who believe experience is more important than individual variation in skill. For them, skill and ability increase with experience, so it is an objective fact that young developers as a group are going to be worse by most measures.
So when someone out of college makes more than someone with 10 years experience, they assume this must be due to agism, since the 10 years experience must, in their view, be more important than being one of the top graduates of that year.
On the original comment, I usually associate Haskell with younger programmers, since it is taught in many schools like MIT and UW. So I found the comment a bit grating since I thought Haskell was the one thing we could have.
So when someone out of college makes more than someone with 10 years experience, they assume this must be due to agism, since the 10 years experience must, in their view, be more important than being one of the top graduates of that year.
On the original comment, I usually associate Haskell with younger programmers, since it is taught in many schools like MIT and UW. So I found the comment a bit grating since I thought Haskell was the one thing we could have.
"On the original comment, I usually associate Haskell with younger programmers"
Huh. FWIW, I associate it more with older programmers. Certainly, some of the more interesting and visible members of the Bay Area meetup are older than me and I'm not fresh out of school.
Huh. FWIW, I associate it more with older programmers. Certainly, some of the more interesting and visible members of the Bay Area meetup are older than me and I'm not fresh out of school.
What a ponderous false dichotomy. No one believes experience matters more than individual variation in skill. Don't put words in my mouth.
>Saying 'a lot of younger programmers probably don't appreciate the best parts of haskell' is clear age-ism
No it isn't.
>I think it's annoying.
Ok, but that doesn't support the claim you made and I questioned.
No it isn't.
>I think it's annoying.
Ok, but that doesn't support the claim you made and I questioned.
As a 21 year old this is not ageism in the slightest, it is reality that as you see more and more problems the applicability of some solutions becomes more apparent. For example, when talking to seasoned engineers who have written a lot of concurrent code they immediately understand the value of purity, and immutable data because they have been burned by the problems that crop up when you try to write threaded programs using shared mutable memory.
On the other hand if you take a 20 year old student still doing their undergraduate degree (like many of the people in the classes I TA), and try to explain the value of immutable data, many of them just don't get it. It has nothing to do with their intelligence or age, but the fact that they haven't written enough code to develop a sense of why such a thing would be useful.
If you want example apps their are plenty littering the internet all covering the various web frameworks, for example here is one covering Happstack: http://happstack.com/docs/crashcourse/index.html
On the other hand if you take a 20 year old student still doing their undergraduate degree (like many of the people in the classes I TA), and try to explain the value of immutable data, many of them just don't get it. It has nothing to do with their intelligence or age, but the fact that they haven't written enough code to develop a sense of why such a thing would be useful.
If you want example apps their are plenty littering the internet all covering the various web frameworks, for example here is one covering Happstack: http://happstack.com/docs/crashcourse/index.html
Are you for real? It's like deja-vu. I guess I should know better by now. There's probably some guy in a lisp thread who has been on my end of this argument for the last 20 years or so.
http://en.wikipedia.org/wiki/Ageism#Ageist_prejudice
http://en.wikipedia.org/wiki/Ageism#Ageist_prejudice
I know what ageism is. You quoted something that is not an example of it. Given that this is entirely pointless, would you mind if we went back to the actual discussion?
BTW, I think that if a company does want to try a good, though not-so-popular language, it should pick a JVM language, as the interoperability among JVM languages, and the availability of libraries that can be shared among the popular and less-so JVM languages, reduces much of the risk.
I used to agree with that, but I think the interoperability of JVM languages is overstated. For instance, using a Scala library from Java or Kotlin is a painful experience, unless its developers invested time to export a Java-friendly API (e.g. to some extend Akka).
The situation for Haskell/C is comparable to that of Scala/Java. It is easy to use Java libraries from Scala, but hard to use Scala libraries from Java. In a similar fashion, it is easy to use C libraries from Haskell, but hard to use Haskell libraries from C.
My experience with large projects is that it is more important to avoid using a large mixture of languages without clearly defined boundaries. E.g. nowadays it's popular to use a mixture of C, C++, Python, Cython, and perhaps even a little Prolog on the side. And while it all links, it is very hard to maintain, debug or for a new developer to get into. Pick one language/ecosystem and stick to it.
I used to agree with that, but I think the interoperability of JVM languages is overstated. For instance, using a Scala library from Java or Kotlin is a painful experience, unless its developers invested time to export a Java-friendly API (e.g. to some extend Akka).
The situation for Haskell/C is comparable to that of Scala/Java. It is easy to use Java libraries from Scala, but hard to use Scala libraries from Java. In a similar fashion, it is easy to use C libraries from Haskell, but hard to use Haskell libraries from C.
My experience with large projects is that it is more important to avoid using a large mixture of languages without clearly defined boundaries. E.g. nowadays it's popular to use a mixture of C, C++, Python, Cython, and perhaps even a little Prolog on the side. And while it all links, it is very hard to maintain, debug or for a new developer to get into. Pick one language/ecosystem and stick to it.
> Pick one language/ecosystem and stick to it.
For a given executable, you mean. One exec, one team, one language ecosystem ("C + Lua" would count as one ecosystem).
By the time two different team work on two different executables, the boundaries you speak of are pretty clear: the two pieces of program will communicate through inter-processes means: files, pipes, sockets… At that point, you don't really care about programming language mismatch: the only common logic will be parsing and generating the common formats. If those are simple enough, you simply won't care. And if they are not, you can always use a compiler generator that outputs C code, and use that in both programs (most ecosystems have a C FFI, so that shouldn't be too hard).
For a given executable, you mean. One exec, one team, one language ecosystem ("C + Lua" would count as one ecosystem).
By the time two different team work on two different executables, the boundaries you speak of are pretty clear: the two pieces of program will communicate through inter-processes means: files, pipes, sockets… At that point, you don't really care about programming language mismatch: the only common logic will be parsing and generating the common formats. If those are simple enough, you simply won't care. And if they are not, you can always use a compiler generator that outputs C code, and use that in both programs (most ecosystems have a C FFI, so that shouldn't be too hard).
You still do care, you just care less. Two languages means two ecosystems. If you build a fancy business object library that could be beneficial in two execs... you'll only see that benefit if they happen to share languages or be FFI compatible.
I've worked on projects with single executable deliverables that needed two languages: Python for the interesting bits, and C for the fast bits. FFI is not always fun (I used ctypes) but there was really no other way.
> It requires changing the runtime library and possibly re-writing a lot of in-house infrastructure.
I think this is a bit overstated. (See below.)
> BTW, I think that if a company does want to try a good, though not-so-popular language, it should pick a JVM language, as the interoperability among JVM languages, and the availability of libraries that can be shared among the popular and less-so JVM languages, reduces much of the risk.
For the most part I think this advantage of JVM languages goes away if you design your system as loosely coupled services. Amazon is the canonical example of this, but I've worked in places where C++ and Java were used together easily because of this kind of design.
Also, I'll echo some of the others and say that Haskell has pretty good adoption and a fantastically helpful community. Three or four years ago I would have agreed with you, but the community has come a long way in the past few years.
I think this is a bit overstated. (See below.)
> BTW, I think that if a company does want to try a good, though not-so-popular language, it should pick a JVM language, as the interoperability among JVM languages, and the availability of libraries that can be shared among the popular and less-so JVM languages, reduces much of the risk.
For the most part I think this advantage of JVM languages goes away if you design your system as loosely coupled services. Amazon is the canonical example of this, but I've worked in places where C++ and Java were used together easily because of this kind of design.
Also, I'll echo some of the others and say that Haskell has pretty good adoption and a fantastically helpful community. Three or four years ago I would have agreed with you, but the community has come a long way in the past few years.
This argument is called "Go home and use Java" in latin.
Infact, by this logic it is impossible for any language to be a competitor to java, because it is trumped in community, adoption and programmer replacability.
The diff any language can offer cannot be more than the advantage java has in these three aspects for a guy who wants to use java. Typically, you will find the good java advocate agreeing that the shiny new language is better - but common the net advantage is "trumped" by java's reach.
In case someone has ever convinced the seasoned java advocate, please share your angle of attack and the language of your choice. I don't think there are any.
Infact, by this logic it is impossible for any language to be a competitor to java, because it is trumped in community, adoption and programmer replacability.
The diff any language can offer cannot be more than the advantage java has in these three aspects for a guy who wants to use java. Typically, you will find the good java advocate agreeing that the shiny new language is better - but common the net advantage is "trumped" by java's reach.
In case someone has ever convinced the seasoned java advocate, please share your angle of attack and the language of your choice. I don't think there are any.
I think there are.
Java was not the first popular programming language, and software has been around long enough to know how language adoption works. The secret is incrementalism. If you want to consider the Java world, then Java was an incremental change in syntax over C++ and a revolution in the runtime, while other JVM languages might be an incremental change in use of the runtime but a revolution in syntax/concepts.
So incrementalism is one way. The other is picking a niche. Some languages are widely used in certain industries because they are particularly well suited to some use cases. That kind of success is often enough, but sometimes a language might expand from its original niche after having achieved success there.
But in many cases, the sentiment you're reeling against is actually correct. The fact that language B is better than language A does not always (or even often) mean that the best course of action for the organization is to adopt language B. Switching a language has a very significant cost, and it's a change that a company should never consider more often than once every five years and optimally not more than once a decade, unless language B is so advantageous that it would be stupid not to switch. Well, that's assuming the choice of language A was wise in the first place.
Because the switching cost is high, there's a very high bar to competing successfully, as there well should be.
Java was not the first popular programming language, and software has been around long enough to know how language adoption works. The secret is incrementalism. If you want to consider the Java world, then Java was an incremental change in syntax over C++ and a revolution in the runtime, while other JVM languages might be an incremental change in use of the runtime but a revolution in syntax/concepts.
So incrementalism is one way. The other is picking a niche. Some languages are widely used in certain industries because they are particularly well suited to some use cases. That kind of success is often enough, but sometimes a language might expand from its original niche after having achieved success there.
But in many cases, the sentiment you're reeling against is actually correct. The fact that language B is better than language A does not always (or even often) mean that the best course of action for the organization is to adopt language B. Switching a language has a very significant cost, and it's a change that a company should never consider more often than once every five years and optimally not more than once a decade, unless language B is so advantageous that it would be stupid not to switch. Well, that's assuming the choice of language A was wise in the first place.
Because the switching cost is high, there's a very high bar to competing successfully, as there well should be.
This whole "competing languages" is backwards.
We write our own application code. We often write our own (specialized) library code. Sometimes, we even write our own little frameworks. But we very rarely write our own languages. There's a wall somehow between languages and everything else: the language is something you chose. Everything else is something you might build —or at least adapt to your ends.
Why? This is beyond silly.
Take one example: Closures in Java. Closures are very useful. If you put garbage collection in your language, but forget about closures, you fail at language design forever. Java made this mistake, and forced us to use anonymous inner classes, and other equally horrible workarounds instead. Java need closures and we knew that for years.
What did we do? We waited for the landlord.
We didn't have to. I'm sure some of you have heard, there's this nifty concept called "source to source transformation"[1]. Here's how it translate in Java: you take source code that is like Java, but is not quite Java. Feed that into your compiler, output Java code, and voilà, you have modified the language. In the specific case of closures, it would mean invent a syntax for closures, and compile that into those ugly anonymous inner classes.
Some people probably have done that, and I just don't know about them. Anyway, it seems the technique didn't get any traction. (The only example I know of is here[2]. Skip to page 6 and 7. Then read the whole thing.)
I don't know about you, but the next time someone forces me to use a language that sucks, I'm going to at least write a preprocessor, and modify the damn language. In other words, I can write LISP in any language, and I will. And when I'm done, even COBOL won't suck.
[1]: http://www.youtube.com/watch?v=EGeN2IC7N0Q
[2]: http://www.cs.indiana.edu/~dfried/dfried/dfried/mex.pdf
We write our own application code. We often write our own (specialized) library code. Sometimes, we even write our own little frameworks. But we very rarely write our own languages. There's a wall somehow between languages and everything else: the language is something you chose. Everything else is something you might build —or at least adapt to your ends.
Why? This is beyond silly.
Take one example: Closures in Java. Closures are very useful. If you put garbage collection in your language, but forget about closures, you fail at language design forever. Java made this mistake, and forced us to use anonymous inner classes, and other equally horrible workarounds instead. Java need closures and we knew that for years.
What did we do? We waited for the landlord.
We didn't have to. I'm sure some of you have heard, there's this nifty concept called "source to source transformation"[1]. Here's how it translate in Java: you take source code that is like Java, but is not quite Java. Feed that into your compiler, output Java code, and voilà, you have modified the language. In the specific case of closures, it would mean invent a syntax for closures, and compile that into those ugly anonymous inner classes.
Some people probably have done that, and I just don't know about them. Anyway, it seems the technique didn't get any traction. (The only example I know of is here[2]. Skip to page 6 and 7. Then read the whole thing.)
I don't know about you, but the next time someone forces me to use a language that sucks, I'm going to at least write a preprocessor, and modify the damn language. In other words, I can write LISP in any language, and I will. And when I'm done, even COBOL won't suck.
[1]: http://www.youtube.com/watch?v=EGeN2IC7N0Q
[2]: http://www.cs.indiana.edu/~dfried/dfried/dfried/mex.pdf
The gatekeepers at an organization refuse to allow devs to submit unreadable machine generated source code, and won't allow the fragmentation of multiple original source languages.
But 1000 ad-hoc human written micro a frameworks and spaghetti approximations or workarounds to closures etc? No problem. Massive LoC throughput, solving "complex problems" as demonstrated by code base size...
But 1000 ad-hoc human written micro a frameworks and spaghetti approximations or workarounds to closures etc? No problem. Massive LoC throughput, solving "complex problems" as demonstrated by code base size...
I often find that the people deriding the "gatekeepers" have never been gatekeepers themselves. Obviously, every developer should do whatever the hell they find most productive, because developers are known to have such incredible foresight, they never just pick tools they feel like using for no good reason, and they always consider the grand-scale effect of their decisions on the organization as a whole.
I have a feeling that the "grand-scale effect" you speak of is overrated. Any sufficiently big organization will be fragmented into groups that hardly talk to each other. At that point, if you want code that's usable by several groups, you must publish it as if it were Open Source (and have it compete with actual Open Source code). That is so much effort that most simply won't do it. It may not be even worth the trouble since the different groups are probably working on different software anyway.
While it makes sense to follow some unified standard as a team, it makes much less sense across teams. With few exceptions, there is no "grand scale".
While it makes sense to follow some unified standard as a team, it makes much less sense across teams. With few exceptions, there is no "grand scale".
I am soo going to use that argument.
Languages have a LOT of ecosystem demands that frameworks and libraries do not. DSLs (especially embedded ones) save on those demands and do get written all the time. That said, I'm of the belief that they should get written more, still.
> Languages have a LOT of ecosystem demands that frameworks and libraries do not.
I have the feeling that you know something I don't. What do you mean by "ecosystem demand"? Could you please name three?
I have the feeling that you know something I don't. What do you mean by "ecosystem demand"? Could you please name three?
I'm sure it's more my poor choice of word, sorry about that. I just meant that if you're starting from scratch on a language there's a fairly large amount of non-useful stuff that must be done before it becomes valuable.
I'm basically assuming that the only reason why you'd have enough business value in creating a new language comes from it either modeling something interesting or having a different semantic focus. Everything else is just wasted time then.
Some examples:
1. Documentation. How do people know what your language means? How do they know what happens when it breaks?
2. Shaking out edge cases (much harder to do than debugging a program!)
3. Syntax. Picking it. Parsing it. Ensuring it's complete and doesn't have weird edge cases.
4. Compiler. Runtime. Parallel (!) runtime? Platform independence? Efficiency?
5. Static analysis (type checking, memory analysis, nulls)
6. Managing modules/files/packages
7. Foreign calls
8. Dependency management
All of these are fun in their own right, but most businesses would have a hard time arguing that they're worth working on unless they've already got a language tied into their core value proposition. It's also important to mention that there exist tools to make some of them easier (bnfc, llvm, &c).
But if you just start with an embedded DSL you can get right to the modeling or semantic issues by piggy-backing on the host language for (most of) 2, 3, 4, 5, 6, 7, and 8. If you prove that there's something valuable there and the host language is slowing you down then you can start to unembed it by building the "whole ecosystem".
I'm basically assuming that the only reason why you'd have enough business value in creating a new language comes from it either modeling something interesting or having a different semantic focus. Everything else is just wasted time then.
Some examples:
1. Documentation. How do people know what your language means? How do they know what happens when it breaks?
2. Shaking out edge cases (much harder to do than debugging a program!)
3. Syntax. Picking it. Parsing it. Ensuring it's complete and doesn't have weird edge cases.
4. Compiler. Runtime. Parallel (!) runtime? Platform independence? Efficiency?
5. Static analysis (type checking, memory analysis, nulls)
6. Managing modules/files/packages
7. Foreign calls
8. Dependency management
All of these are fun in their own right, but most businesses would have a hard time arguing that they're worth working on unless they've already got a language tied into their core value proposition. It's also important to mention that there exist tools to make some of them easier (bnfc, llvm, &c).
But if you just start with an embedded DSL you can get right to the modeling or semantic issues by piggy-backing on the host language for (most of) 2, 3, 4, 5, 6, 7, and 8. If you prove that there's something valuable there and the host language is slowing you down then you can start to unembed it by building the "whole ecosystem".
We don't have to chose between embedded DSL (meaning, a kind of fancy library), and a fully fledged external DSL. I was advocating source-to-source transformation, which I think is a nice compromise between the two: it feels almost like an external DSL, but doesn't require much more work than an internal one.
The way I see it, source-to-source transformation helps a lot with 2, 4, 5, and 7: we can rely on the host language for these. Actually, 5 is even better than an embedded DSL: you can implement additional checks. As for 6 and 8, you will generally avoid the issue altogether by letting your host language deal with them.
An example of source-to-source transformation at work is OMeta. The thing compiles itself to JavaScript, and its source code takes about 400 lines of JavaScript and OMeta. I'm currently working on a clone for Lua, and my prototypes so far are under 300 lines.
I agree that picking up a syntax is hard (parsing it is trivial). But with enough feedback, you can make a decent one. Just make sure to tell everyone that the first 5-10 versions of your language are all pre-alpha, and totally incompatible with each other. And source to source transformation isn't about building a whole new language. Most of the time, it is about extending the host language. This means much less syntax to worry about in the first place.
Actually, if you're smart about it, you can often write most of the semantics of your language in a library, then add a light layer of syntax sugar on top of it. That's what I did when implementing a PEG syntax for Haskell[1]. It's just syntax sugar over Parsec.
[1]: http://loup-vaillant.fr/projects/metacompilers/ (bottom of the page)
---
Now back to my Java example: It was just a matter of adding a nice syntax sugar over anonymous inner classes, and maybe allow a syntax for calling objects as if they were functions, C++ `operator()` style (Even so, writing obj.app(args) isn't that cumbersome). Not much work, really.
The way I see it, source-to-source transformation helps a lot with 2, 4, 5, and 7: we can rely on the host language for these. Actually, 5 is even better than an embedded DSL: you can implement additional checks. As for 6 and 8, you will generally avoid the issue altogether by letting your host language deal with them.
An example of source-to-source transformation at work is OMeta. The thing compiles itself to JavaScript, and its source code takes about 400 lines of JavaScript and OMeta. I'm currently working on a clone for Lua, and my prototypes so far are under 300 lines.
I agree that picking up a syntax is hard (parsing it is trivial). But with enough feedback, you can make a decent one. Just make sure to tell everyone that the first 5-10 versions of your language are all pre-alpha, and totally incompatible with each other. And source to source transformation isn't about building a whole new language. Most of the time, it is about extending the host language. This means much less syntax to worry about in the first place.
Actually, if you're smart about it, you can often write most of the semantics of your language in a library, then add a light layer of syntax sugar on top of it. That's what I did when implementing a PEG syntax for Haskell[1]. It's just syntax sugar over Parsec.
[1]: http://loup-vaillant.fr/projects/metacompilers/ (bottom of the page)
---
Now back to my Java example: It was just a matter of adding a nice syntax sugar over anonymous inner classes, and maybe allow a syntax for calling objects as if they were functions, C++ `operator()` style (Even so, writing obj.app(args) isn't that cumbersome). Not much work, really.
Source-to-source is an interesting example. I think Coffeescript has proven it's pretty possible to get a long way there, though it's undoubtable that it still has some syntactic and semantic design issues.
I think another important example of source-to-source is Lisp macros. Super powerful and when used well and marketed well they can really transform what Lisp means. I also think that when marketed poorly they lead to an explosion of competing "languages".
I think another important example of source-to-source is Lisp macros. Super powerful and when used well and marketed well they can really transform what Lisp means. I also think that when marketed poorly they lead to an explosion of competing "languages".
I think you are vastly overstating things. You need an acceptable level of adoption. One where there are high quality libraries for common needs. One where you can hire experts. Beyond that level, increasing popularity doesn't really get you anything. Haskell is already beyond that level.
Haskeller at IMVU.
Haskell could still benefit from a larger community. We have had to write some key low level tech, but we have been able to do so at an incredible pace.
The /dev/urandom bug mentioned was caught by us because we are one of the few places using Haskell that also has a fairly large user base. If the community was larger we would have run into less issues.
That said, we have run into a surprisingly few amount of issues :)
Haskell could still benefit from a larger community. We have had to write some key low level tech, but we have been able to do so at an incredible pace.
The /dev/urandom bug mentioned was caught by us because we are one of the few places using Haskell that also has a fairly large user base. If the community was larger we would have run into less issues.
That said, we have run into a surprisingly few amount of issues :)
>If the community was larger we would have run into less issues.
I'm not so sure that's the case. For example, debian's user base is vastly larger than openbsd's, yet debian has bugs like "all the keys you ever generated are garbage" and openbsd doesn't. The "many eyes" hypothesis is bogus. Bugs are found and fixed more through deliberate effort than just large numbers of people.
I'm not so sure that's the case. For example, debian's user base is vastly larger than openbsd's, yet debian has bugs like "all the keys you ever generated are garbage" and openbsd doesn't. The "many eyes" hypothesis is bogus. Bugs are found and fixed more through deliberate effort than just large numbers of people.
So always choose PHP?
Hi pron, you're absolutely right. Transitioning to a new language and runtime is a huge undertaking. I work with Andy, the author, and he took exactly the right steps in getting Haskell adopted. Nothing was forced through, and he got both engineering and executive buy-in for the transition.
We do not use Haskell exclusively -- we also have a pile of PHP and some Python -- but it's awesome to have such a great tool at our disposal. I've been very impressed at how mature GHC's runtime system is. The Haskell services have run perfectly for months with no problems.
The Haskell type system even lets us say, for example, that you cannot talk to MySQL within a Redis transaction, preventing entire classes of bugs.
(I have many complaints about Haskell too, but net net, it's an enormous win over PHP and Python.)
We do not use Haskell exclusively -- we also have a pile of PHP and some Python -- but it's awesome to have such a great tool at our disposal. I've been very impressed at how mature GHC's runtime system is. The Haskell services have run perfectly for months with no problems.
The Haskell type system even lets us say, for example, that you cannot talk to MySQL within a Redis transaction, preventing entire classes of bugs.
(I have many complaints about Haskell too, but net net, it's an enormous win over PHP and Python.)
The Haskell type system does not let you do that by default. What kind of monads do you use? Are they part of some library or did you develop them in house?
Do you have a policy around the use of I/O monads versus specialized monads for controlling access like that?
Do you have a policy around the use of I/O monads versus specialized monads for controlling access like that?
I can't remember the exact monad names, but Redis transactions are in a specific Monad that does not give you access to IO. I don't know whether that's functionality provided by Hedis or whether we wrote that.
We have a general Monad typeclass called "World" which gives access to MySQL, Redis, HTTP, Memcache, and so on. There are more specific Monad typeclasses if you have a function that, for example, should only ever talk to Memcache and nothing else. I think it's called SupportsMemcache.
My point about the Haskell type system is that the _ability_ to limit the operations in a context is a capability that few languages have.
We have a general Monad typeclass called "World" which gives access to MySQL, Redis, HTTP, Memcache, and so on. There are more specific Monad typeclasses if you have a function that, for example, should only ever talk to Memcache and nothing else. I think it's called SupportsMemcache.
My point about the Haskell type system is that the _ability_ to limit the operations in a context is a capability that few languages have.
I'll expand on the details.
We have a simple mechanism for controlling access to IO.
There are a series of type classes that provide access to all of the IO based services (mysql, redis, memcache, etc.). All of the request handlers are written to use these type classes not IO. There are two instances of the type classes, one pure for tests using fakes, and one real using IO.
We have a simple mechanism for controlling access to IO.
There are a series of type classes that provide access to all of the IO based services (mysql, redis, memcache, etc.). All of the request handlers are written to use these type classes not IO. There are two instances of the type classes, one pure for tests using fakes, and one real using IO.
Yes of course, but it's a next step to actually use that ability. Very cool that you do :) It takes some tough developer discipline to deprive yourself from the power or the IO monad.
It might take some developer discipline to set things up that way initially, but the great thing about Haskell is that once you have it set up, it does not require any developer discipline. If your system runs in the Foo monad, you can only do operations that have been made available to you. This is the big problem that other languages have...you simply cannot enforce some of these guarantees.
The type system allows you to say "you cannot use that here". It doesn't say it itself, and particular libraries may or may not exploit the ability. A classic example is STM, which relies on IO internally but doesn't let you use IO inside it (without resorting to aptly named and justly avoided unsafe... functions) because that would be dumb.
Always use the best tool for the job. As long as it is compiled, statically typed, runs on the JVM, and blends functional and OO programming seamlessly.
What do you mean, the job ad was tailored to this specific person?
Wait, IMVU? As in those terrible banner-ads with the Bratz-dolls-looking avatars? They use Haskell under the hood?
Seriously, this is like finding out that McDonalds' restaurants are powered by nuclear fusion.
Although this does make me very curious about learning Haskell - I've been pretty stagnant about learning new languages.
Seriously, this is like finding out that McDonalds' restaurants are powered by nuclear fusion.
Although this does make me very curious about learning Haskell - I've been pretty stagnant about learning new languages.
I'm curious about the claim that it only takes a few days to get people productive working in Haskell. I like the language a lot, but learning it has definitely been slow compared to any other language I've learned. It just seems like there are a lot of changes to how you think.
Is this a feature of the way your code is structured that simplifies what people have to understand about the language before they can contribute?
Is this a feature of the way your code is structured that simplifies what people have to understand about the language before they can contribute?
To be fair to the OP, the original claim was: "People who have prior functional programming knowledge seem to find their stride in just a few days." I would be somewhat surprised if someone with no prior functional programming experience could be productive in a few days in Haskell, because as you say, there are a lot of changes to how you think about programming. It's a lot easier for people with experience in a Lisp, and even easier yet for people with experience in another ML descendant.
All that said, the structure of the code definitely does help a lot in lowering the bar for contributions. When Don Stewart was on the Haskell Cast [1], he mentioned that the way they use Haskell allows contributions from people who not only have no prior functional experience but who have little CS background at all past basic programming literacy. Much of the ability to do this comes from Haskell's rigorous isolation of side effects.
[1]: http://www.haskellcast.com/episode/002-don-stewart-on-real-w... around 42:40 or so
All that said, the structure of the code definitely does help a lot in lowering the bar for contributions. When Don Stewart was on the Haskell Cast [1], he mentioned that the way they use Haskell allows contributions from people who not only have no prior functional experience but who have little CS background at all past basic programming literacy. Much of the ability to do this comes from Haskell's rigorous isolation of side effects.
[1]: http://www.haskellcast.com/episode/002-don-stewart-on-real-w... around 42:40 or so
I probably should have been more explicit: I use higher order functions and recursion all over the place in higher level languages that I use, so I'm familiar with parts of functional programming. But structuring impure programs in Haskell still has taken a long time to get used to (and I'm not great at it yet).
I work with Andy, the author. I watched the Haskell push from the sidelines and did not learn it until well after it had already gained momentum. My intend behind not learning the language was to retain empathy with people who hadn't yet gone through the spin-up.
When I finally got around to learning Haskell, I found myself able to productively make small changes within a few days. A lot of the more... idiomatic? uses of Functor and Applicative took another week or so to learn. The books Learn You a Haskell and Real World Haskell were helpful there.
I am by no means an expert at Haskell now, but within a few weeks I was able to contribute to most areas of the stack.
When I finally got around to learning Haskell, I found myself able to productively make small changes within a few days. A lot of the more... idiomatic? uses of Functor and Applicative took another week or so to learn. The books Learn You a Haskell and Real World Haskell were helpful there.
I am by no means an expert at Haskell now, but within a few weeks I was able to contribute to most areas of the stack.
I think it's easier in general to work on an application that's up and running . Architecting a new system in a language you're new to would be much more time consuming. But in the case where the groundwork is laid you get by with much less language depth. Recent someone posted on here about how financial analysts were able to use a subset of Haskell in the field without really having to think of it as programming at all, as all the plumbing under the DSL is covered up.
I'm the author of the article.
This is very true in every language, and doubly so in Haskell. If you just take the language as it is without any context, the language leaves whole classes of problems with a ridiculous overabundance of potentially viable solutions.
The one that springs to mind most prominently is error handling. What's the right way to do this? Option types? Exceptions? EitherT? We even explored a library that used type classes to provide Java-style checked exceptions.
We sorted this out early, when the "team" was just 2 people working in extra hours, but if we'd still been in that state when we scaled up, the project would have been a disaster.
FYI, the solution we came up with was to use option types for all recoverable errors. We only use exceptions for nonrecoverable failures that should trigger production alerts.
This is very true in every language, and doubly so in Haskell. If you just take the language as it is without any context, the language leaves whole classes of problems with a ridiculous overabundance of potentially viable solutions.
The one that springs to mind most prominently is error handling. What's the right way to do this? Option types? Exceptions? EitherT? We even explored a library that used type classes to provide Java-style checked exceptions.
We sorted this out early, when the "team" was just 2 people working in extra hours, but if we'd still been in that state when we scaled up, the project would have been a disaster.
FYI, the solution we came up with was to use option types for all recoverable errors. We only use exceptions for nonrecoverable failures that should trigger production alerts.
Haskeller at IMVU.
I was surprised how fast people were picking up the language when I started
It is way different learning the language surrounded by teachers and a working environment. That was my takeaway.
I was surprised how fast people were picking up the language when I started
It is way different learning the language surrounded by teachers and a working environment. That was my takeaway.
I've been learning haskell for the better part of 8 years. I say learning, because I spend far too much time trying to study the patterns and math from the abstract point of view. Every few months I print off a research paper and struggle with it for a few days until I get it. I never get to code large projects in it because no one uses it in industry up in the north.
I can code in the language no problem but sometimes I feel that idiomatic ideal is always just out of reach to all but the Simons.
I can code in the language no problem but sometimes I feel that idiomatic ideal is always just out of reach to all but the Simons.
I highly suggest reading Richard Bird's book "Pearls of Functional Algorithm Design". It's a really great way to get some insight into top-style FP.
[deleted]
> I wouldn’t write an OS kernel in it, but Haskell is way better than PHP as a systems language.
What's the definition of "systems language"? If PHP could even remotely be considered an option, the definition is clearly not what I think it is...
What's the definition of "systems language"? If PHP could even remotely be considered an option, the definition is clearly not what I think it is...
Haskell is a terrible choice for a production system.
I know well over a dozen programming languages. I have a stronger theoretical background in mathematics, computer science, and compiler design than most web developers. But Haskell has always been utterly mystifying to me.
I've attempted to learn Haskell several times, and I've had monads explained to me several times right here on HN -- but I simply don't seem to be able to wrap my mind around the concept.
To be sure, if I had a year or so to devote to studying the theory behind Haskell's type system, I might indeed become more productive writing web applications in Haskell. And I don't doubt that there are certain problems in logic programming or language theory for which Haskell provides useful concepts and powerful tools, to the point where the solutions to important problems may have trivial implementations.
I've always thought that a major problem with Haskell for web development is: (A) You'll have difficulty finding people who both know Haskell and are interested in web applications, and (B) you'll have difficulty training web developers in Haskell. The author of the article apparently has a training process that gets around (B), and I'd kind of like to know what it is -- a way to learn Haskell without a ton of study.
I know well over a dozen programming languages. I have a stronger theoretical background in mathematics, computer science, and compiler design than most web developers. But Haskell has always been utterly mystifying to me.
I've attempted to learn Haskell several times, and I've had monads explained to me several times right here on HN -- but I simply don't seem to be able to wrap my mind around the concept.
To be sure, if I had a year or so to devote to studying the theory behind Haskell's type system, I might indeed become more productive writing web applications in Haskell. And I don't doubt that there are certain problems in logic programming or language theory for which Haskell provides useful concepts and powerful tools, to the point where the solutions to important problems may have trivial implementations.
I've always thought that a major problem with Haskell for web development is: (A) You'll have difficulty finding people who both know Haskell and are interested in web applications, and (B) you'll have difficulty training web developers in Haskell. The author of the article apparently has a training process that gets around (B), and I'd kind of like to know what it is -- a way to learn Haskell without a ton of study.
I've seen experienced programmers failing at learning Haskell for the same reason experienced programmers fail at learning Vim, the initial learning curve is very steep, and they don't see the value on climbing that wall. Now, if you know Vim, ask yourself, was it worth it? I'm convinced 99% of people would say yes. It is the same with Haskell, all those seemingly complex concepts come with a big reward.
Learning Haskell is easier with a strong functional programming background; sadly CS students don't learn much about FP, so having lots of experience in CS and compilers won't necessarily put you in a better position than somebody who has no formal background in CS but lots of programming experience.
A math background on the other hand should help you. Learn about how category theory is applied in Haskell, and you're already half way there. Monads are just a piece of the puzzle.
I do agree with your last three points. I haven't used Haskell in production, so my experience in "real world Haskell" is limited, but it has helped me immensely in my daily work with other languages. Haskell forever changes the way you think and approach problems, just like Vim changes the way you write and edit code; it will make you a better programmer.
Learning Haskell is easier with a strong functional programming background; sadly CS students don't learn much about FP, so having lots of experience in CS and compilers won't necessarily put you in a better position than somebody who has no formal background in CS but lots of programming experience.
A math background on the other hand should help you. Learn about how category theory is applied in Haskell, and you're already half way there. Monads are just a piece of the puzzle.
I do agree with your last three points. I haven't used Haskell in production, so my experience in "real world Haskell" is limited, but it has helped me immensely in my daily work with other languages. Haskell forever changes the way you think and approach problems, just like Vim changes the way you write and edit code; it will make you a better programmer.
> the same reason experienced programmers fail at learning Vim, the initial learning curve is very steep, and they don't see the value on climbing that wall.
This is definitely me. I only got really into Linux with Debian-like systems after Nano became those distributions' standard editor. I know the bare minimum about vim for occasional, minimal editing of configuration files. I only ever use it when I'm stuck with a rescue prompt or some third-rate distro that has nothing else. Usually one of my first goals is to get either a GUI, or file transfer capability, or a package manager up and running; then I stop using it.
This is definitely me. I only got really into Linux with Debian-like systems after Nano became those distributions' standard editor. I know the bare minimum about vim for occasional, minimal editing of configuration files. I only ever use it when I'm stuck with a rescue prompt or some third-rate distro that has nothing else. Usually one of my first goals is to get either a GUI, or file transfer capability, or a package manager up and running; then I stop using it.
"I only got really into Linux with Debian-like systems after Nano became those distributions' standard editor."
I'd recommend learning an editor that has any sort of power - vim, emacs, or something lesser known. Trying to get anything done in nano is like pulling teeth. Yeesh.
I'd recommend learning an editor that has any sort of power - vim, emacs, or something lesser known. Trying to get anything done in nano is like pulling teeth. Yeesh.
Vim is multi-platform, and there are GUIs for Vim, particularly GVim and Macvim, where you can use the mouse, and copy/paste, etc, just have to do it the Vim way. You gave up too early, Nano isn't even close; it's like comparing Notepad to Notepad++. Give both Haskell and Vim another chance.
Haskell is something that really needs to be used in order to "learn" it. I didn't learn monads by study/reading, I learned them by just using Haskell for real-world projects.
We use Haskell for all our production systems and it is great. Maintenance is MUCH easier in Haskell and it's usually quite simple for new people (with Haskell experience) to get up to speed with a project. We even had a situation where one guy was able to make a highly nontrivial refactoring to a several thousand line project that he had no prior involvement with. It took him a few hours to do the refactoring and once he got it compiling it worked the first time. We are convinced that this feat would have been impossible in any other mainstream language.
We use Haskell for all our production systems and it is great. Maintenance is MUCH easier in Haskell and it's usually quite simple for new people (with Haskell experience) to get up to speed with a project. We even had a situation where one guy was able to make a highly nontrivial refactoring to a several thousand line project that he had no prior involvement with. It took him a few hours to do the refactoring and once he got it compiling it worked the first time. We are convinced that this feat would have been impossible in any other mainstream language.
It's a huge help to ask a newcomer to start off by make changes to an already-established project that has best practices and idioms already sorted.
If you just start with the full language, there are just too many ways to solve every problem and it's not clear which is better for your particular use case. It's a bit like Perl, except moreso: even basic flow control is a library.
If you just start with the full language, there are just too many ways to solve every problem and it's not clear which is better for your particular use case. It's a bit like Perl, except moreso: even basic flow control is a library.
I found Haskell was not that hard. But I did try to learn by reading Learn You a Haskell twice and gave up.
When I really learnt it, I treated it like I would any other language: I tried to write nontrivial programs in it, in my case the 2048 game. I learnt Monads by adding real randomness to the game.
So I would say the main problem with Haskell is that people treat it like something special, and not like any other language.
When I really learnt it, I treated it like I would any other language: I tried to write nontrivial programs in it, in my case the 2048 game. I learnt Monads by adding real randomness to the game.
So I would say the main problem with Haskell is that people treat it like something special, and not like any other language.
"I've had monads explained to me several times right here on HN"
Well, let's add one to the pile, though I second the "use some monads" recommendation. This is one perspective; there are others:
Monad is an interface. It is used when reifying certain kinds of context to give a standard way of chaining those contexts. That reification means 1) you can talk explicitly about those contexts, and 2) the type system can enforce more rules around those contexts. The abstraction provided by the interface means we can hide away a lot of the plumbing.
The Monad interface is defined in terms of a type constructor; that is, a type-level function that takes a type as an argument and produces another type - like a C++ template with one argument. There is a sense in which the monad "wraps" the "inner" type, though for some monads this is metaphorical at best. Still, it provides a vocabulary.
Well, let's add one to the pile, though I second the "use some monads" recommendation. This is one perspective; there are others:
Monad is an interface. It is used when reifying certain kinds of context to give a standard way of chaining those contexts. That reification means 1) you can talk explicitly about those contexts, and 2) the type system can enforce more rules around those contexts. The abstraction provided by the interface means we can hide away a lot of the plumbing.
The Monad interface is defined in terms of a type constructor; that is, a type-level function that takes a type as an argument and produces another type - like a C++ template with one argument. There is a sense in which the monad "wraps" the "inner" type, though for some monads this is metaphorical at best. Still, it provides a vocabulary.
I'm sorry you're only now realizing that just having a bunch of credentials doesn't mean you're a genius. It just means you took the time to study a few specific things in-depth.
Sadly there is no way around actually taking the time to try and understand something in order to understand it. I guess we need to perfect the art of brain transplants.
Sadly there is no way around actually taking the time to try and understand something in order to understand it. I guess we need to perfect the art of brain transplants.
It's good to hear stories about how FP is being used in industry and I'd like to read more.
For those who have an interest in this, I recommend the Commercial Users of Functional Programming workshop (http://cufp.org). It's where I first heard about Facebook's use of OCaml to create Hack (http://www.youtube.com/watch?v=gKWNjFagR9k)
For those who have an interest in this, I recommend the Commercial Users of Functional Programming workshop (http://cufp.org). It's where I first heard about Facebook's use of OCaml to create Hack (http://www.youtube.com/watch?v=gKWNjFagR9k)
While visiting your link (http://cufp.org), I saw Chester Bennin of Linkin Park fame there. I didn't know that he's also a programmer! Programming in a functional manner. Another Wow!
Isn't the Linkin Park vocalist named Chester Bennington? I think you're off by a gton.
Chester Bennin is what the website says. Click his picture in the upper-right corner, you'll be then taken to the page where it says his name is 'chester Bennin.'
I agree there is a "chester Bennin" listed on CUFP, there is also a person named "Chester Bennington" who was the lead singer of Linkin Park. What I disagree with is your claim that these people with different last names are the same person.
Haven't you seen the picture? It looks like Chester. I was not the one who claimed they are the same person. If they are indeed different then the CUFP website is at fault for misinformation.
Oh, sorry, I didn't realize that CUFP was saying this. I searched for "chester bennin site:cufp.org" on Google and the only result returned was this page: http://cufp.org/users/genie
Apology accepted :)
But you're kind of right. I can't really find information on Chester (from Linkin Park) doing functional programming.
It's either the guy on the picture really looks like Chester, or the said person (programmer) used Chester's pic as his avatar (unlikely, though, since they are a reputable sight).
But you're kind of right. I can't really find information on Chester (from Linkin Park) doing functional programming.
It's either the guy on the picture really looks like Chester, or the said person (programmer) used Chester's pic as his avatar (unlikely, though, since they are a reputable sight).
It always surprises me to see these imvu.com posts on HN. They are often insightful and interesting, but when you see the website you wonder who pays money for this. Do they sell virtual goods for their own virtual world, similar to secondlife?
Here is the shop: http://www.imvu.com/shop/
Clothing, Furniture, Rooms, Avatars.
I've never heard of imvu before but I don't see how it's any different than micro-transactions within a MMO. If anyone plans to make the argument that MMO's are different because they have "gameplay", then you're probably someone that has never played a very social MMO before.
As an ex-EVE player, I can say with extreme confidence that at least 80% of our alliance ("guild"/social group) was not there to actually play the game. It's pretty common that after a certain amount of game time, you just see the "game" as a glorified chat room. It was an ongoing joke to point out how most of us paid $15 to CCP every month for the ability to talk to each other over XMPP.
Clothing, Furniture, Rooms, Avatars.
I've never heard of imvu before but I don't see how it's any different than micro-transactions within a MMO. If anyone plans to make the argument that MMO's are different because they have "gameplay", then you're probably someone that has never played a very social MMO before.
As an ex-EVE player, I can say with extreme confidence that at least 80% of our alliance ("guild"/social group) was not there to actually play the game. It's pretty common that after a certain amount of game time, you just see the "game" as a glorified chat room. It was an ongoing joke to point out how most of us paid $15 to CCP every month for the ability to talk to each other over XMPP.
They do. I was once an active IMVU user. Outward appearances can be deceiving, but believe me they are profitable. If you read Eric Ries book 'The Lean Startup' you can have a glimpse of IMVU's back story there.
Yeah, IMVU is a company behind a product which most people know about because of the "Lean Startup" book, not because they used the product itself. I must say I read the book with great interest but I never found the product itself interesting enough to even have a look at at :)
The target market is pretty clearly teenaged/tweener girls. I'm not surprised to learn that Hacker News doesn't have many of those.
i'd like to hear more about employers' perspectives on hiring for "an uncommon language without a comparatively sparse industrial track record." there was some debate about it in the clojure 1.6 release announcement. the author of the article didn't find it to be a problem, but then he was apparently only hiring one person.
good post, Tikhon!
good post, Tikhon!
When I left my last Haskell job, my employer put out ads to the Haskell community and quickly got a pool of 5-10 solid applications. A good number of those were from people outside the U.S. which was a no-go for them. After waiting a few weeks they decided to open it up and not require Haskell experience. I looked through a number of these resumes and there was a massive drop in quality. It was like night and day. None of those applicants were even remotely interesting IMO. They ended up waiting a few more months and finally found a Haskell applicant in the U.S. that they were happy with. If you're willing to accept people working remotely your chances of finding someone are much better.
Not all uncommon languages are made equal. It's very different to hire someone to work on Scala than it is to hire someone to work in PICK.
I believe than when push comes to shove, the language your team uses is not the primary determination of the quality of your software. I've actually seen good programs in PICK, an ancient abomination that nobody should be using today. Your major difference is people. Good developers with a good direction will give you good software. Bad developers with the same toolset will give you bad software. So, if anything, your language selection is important because it gives you a very different talent pool.
Hiring a Scala developer, for instance, isn't all that easy, but it's definitely not impossible. For the extra pain you pay in finding someone that will use Scala instead of Java, you get, in exchange, a major filter on your applicants, which tends to be a good thing.
With an antique like PICK, the filter works against you, because what you get is the very old that have not moved on, and the desperate.
I do not think that getting more and more exclusive gets you a better filter, even when you are looking for relatively hip, uncommon languages. Is your Clojure developer naturally better than your Scala developer? How about Haskell? It's not really an issue of every filtering picking the best, but picking people that care about their tools.
So if someone told me that they are hiring developers, and that they completely lack a network to pull them from, it'll be easier to get a good core of developers in one of those uncommon languages than if you just place an ad for people that have 10 years of Java experience. This situation is uncommon though: If you are hiring, you typically have a network. Then what you do is to target some of the best developers you can get a hold of, and hire around the technologies they want to use. You'd be surprised by how much money a competitor has to pay a developer to poach them out of you if they really like your current toolset, but aren't really all that fond of the one the competitor uses.
We decided on Scala because our seed team wanted to use it, and just went out and hired locals that had similar technology interest. We also brought in a few major independent consultants that made sure our core team really understood the language, instead of just staring with good developers with little experience in their toolset. Overall, our experience was very positive, and we have a happy Scala crew.
Of course, YMMV.
I believe than when push comes to shove, the language your team uses is not the primary determination of the quality of your software. I've actually seen good programs in PICK, an ancient abomination that nobody should be using today. Your major difference is people. Good developers with a good direction will give you good software. Bad developers with the same toolset will give you bad software. So, if anything, your language selection is important because it gives you a very different talent pool.
Hiring a Scala developer, for instance, isn't all that easy, but it's definitely not impossible. For the extra pain you pay in finding someone that will use Scala instead of Java, you get, in exchange, a major filter on your applicants, which tends to be a good thing.
With an antique like PICK, the filter works against you, because what you get is the very old that have not moved on, and the desperate.
I do not think that getting more and more exclusive gets you a better filter, even when you are looking for relatively hip, uncommon languages. Is your Clojure developer naturally better than your Scala developer? How about Haskell? It's not really an issue of every filtering picking the best, but picking people that care about their tools.
So if someone told me that they are hiring developers, and that they completely lack a network to pull them from, it'll be easier to get a good core of developers in one of those uncommon languages than if you just place an ad for people that have 10 years of Java experience. This situation is uncommon though: If you are hiring, you typically have a network. Then what you do is to target some of the best developers you can get a hold of, and hire around the technologies they want to use. You'd be surprised by how much money a competitor has to pay a developer to poach them out of you if they really like your current toolset, but aren't really all that fond of the one the competitor uses.
We decided on Scala because our seed team wanted to use it, and just went out and hired locals that had similar technology interest. We also brought in a few major independent consultants that made sure our core team really understood the language, instead of just staring with good developers with little experience in their toolset. Overall, our experience was very positive, and we have a happy Scala crew.
Of course, YMMV.
I've had experience on both sides, plus know some other people who have been recently hiring haskellers. I have a huge problem hiring haskell people because the company I work for kinda sucks. I don't want to work here, so I can't really be surprised that nobody else does either. Haskell users seem to be more picky about where they work and what they work on, so if you've got micromanagement and "do this wrong because I said so" going on, you'll have a hard time. Where as you can still easily get hordes of PHP programmers willing to tolerate anything.
On the other hand, someone hiring remote haskell programmers for a company where the culture values quality had 100+ applications for half a dozen openings and really has their pick of a lot of top notch people. I think if you can offer a good job, finding haskell people isn't a problem. But if you can't, then I would stick to popular_language_x.
On the other hand, someone hiring remote haskell programmers for a company where the culture values quality had 100+ applications for half a dozen openings and really has their pick of a lot of top notch people. I think if you can offer a good job, finding haskell people isn't a problem. But if you can't, then I would stick to popular_language_x.
I can't like this post enough. It affirms everything I believe to be true about developing software using modern languages. I wish more developers would invest more time in learning better tools.
It's funny. I want to either give your comment a "+1 Me too!" response, or a huge rant on software quality and the language quagmire we're in with weak (C++/Java) or missing (JS/Python) type systems.
To throw a statement out there without substantiation, and then cowardly running away, I'll say that Haskell feels as easy as Python, where in Python you have to do a little thinking about what types of objects get passed where, often while debugging, in Haskell you have to figure out the types of data/functions while compiling. The result, I feel, is about the same amount of work. The difference is that the Haskell result is substantially, /substantially/ easier to use in larger projects.
Again, that's without substantiation, and with me immediately running away in cowardice!
To throw a statement out there without substantiation, and then cowardly running away, I'll say that Haskell feels as easy as Python, where in Python you have to do a little thinking about what types of objects get passed where, often while debugging, in Haskell you have to figure out the types of data/functions while compiling. The result, I feel, is about the same amount of work. The difference is that the Haskell result is substantially, /substantially/ easier to use in larger projects.
Again, that's without substantiation, and with me immediately running away in cowardice!
Haskell is definitely not as "easy" as Python but I actually think it's more straight forward than Python is. There's more cognitive work up-front but the pay off is very big in terms of stability, maintainability, etc... in the long run. I actually think it costs less in amount of work to do something in Haskell than in Python (and Erlang, even).
When I have to drop into Python or Javascript I feel like I'm pulling out my MiniTonka toys vs. the actual dump truck out back by the barn. When I went from PHP to Python it felt like that too.
As it stands though, I write systems critical services in HS now and keep the web app stuff in Python. I have written two production web apps in Snap and I loved every minute of it.
When I have to drop into Python or Javascript I feel like I'm pulling out my MiniTonka toys vs. the actual dump truck out back by the barn. When I went from PHP to Python it felt like that too.
As it stands though, I write systems critical services in HS now and keep the web app stuff in Python. I have written two production web apps in Snap and I loved every minute of it.
I guess saying "This." didn't seem verbose enough.
Let me propose an incredibly controversial idea: Haskell isn't a better tool, it's just yet another tool. Which is aggressively promoted not so much with actual benefits, but by trying to shame people who prefer other tools by calling them dullards and telling them that they don't know what they are doing.
Let me propose an incredibly controversial idea: Haskell isn't a better tool, it's just yet another tool. Which is aggressively promoted not so much with actual benefits, but by trying to shame people who prefer other tools by calling them dullards and telling them that they don't know what they are doing.
> by trying to shame people who prefer other tools by calling them dullards and telling them that they don't know what they are doing.
I think this a minority but I have seen this. Generally with languages that have their roots in the more science side of things there is an air of elitism. For example quite often a venn diagram is posted which implies programs (or programmers, can't remember) not made/using type systems are worse.
The Haskell community aren't too bad though
I think this a minority but I have seen this. Generally with languages that have their roots in the more science side of things there is an air of elitism. For example quite often a venn diagram is posted which implies programs (or programmers, can't remember) not made/using type systems are worse.
The Haskell community aren't too bad though
The problem I had with Haskell when trying it out was that every time I wanted to use a package with dependencies I ran into "cabal: Error: some packages failed to install. The exception was: ExitFailure 1.". The solution was most often trying earlier versions of the package until it worked or not use the package at all. Cabal just seemed so broken.
Not to diminish the Cabal problems here, but you run into similar problems with Maven as well. Version management is a real hard problem. Especially when there is little awareness in producing compatible successors. Especially this problem made my experiences with Haskell somewhat bitter. But I have the same problem (in really productive code) with Maven. Writing a working Maven plugin can be made pretty challenging by that as especially the stuff you want to use in Maven plugins is burdened with incompatibilities.
So, Cabal is in good company in that regard.
So, Cabal is in good company in that regard.
Haskell packages like to specify requirements using both a lower bound and an upper bound. So package A depends on B being version 0.55 <= B < 0.99. Package C depends on B being version 0.90 <= B <= 1.35. So if you installed C before A you will get version 1.35 of B and will now need to either downgrade B or have two versions installed. Chaos ensues.
Why Hackage packagers like to specify upper bounds I'm not sure of. Backwards compatibility either is very hard to achieve in Haskell or maintainers just don't care that much about it. I don't know about Maven so I can't say whether that is as bad. But at least pip, npm and gems are a breeze compared to cabal.
Why Hackage packagers like to specify upper bounds I'm not sure of. Backwards compatibility either is very hard to achieve in Haskell or maintainers just don't care that much about it. I don't know about Maven so I can't say whether that is as bad. But at least pip, npm and gems are a breeze compared to cabal.
Packages are supposed to specify upper bounds because without them you can end up with spontaneous breakages when a transitive dependency several steps away changes in a backwards-incompatible way. Haskell has the PVP which can prevent this. It is pretty similar to Semantic Versioning (http://semver.org/). The upper bounds are not specified willy-nilly. They are specified in a way that includes backwards-compatible revisions, but excludes backwards-incompatible ones.
I've been using Haskell for more than 6 years and in my experience the worst build problems happen with parts of the ecosystem that don't specify upper bounds. There's been a lot of discussion about this in the community. We're aware of the issues and we're working on solving them. One attractive possibility being discussed is adding a flag to cabal that allows one to ignore upper bounds. In theory this seems like it can preserve the benefits we get from the added information in an upper bound while eliminating the pain. An extension to this idea is to make two operators (say < and <!) that specify a "soft" upper bound that is just the highest you've been able to test against and a "hard" upper bound where you're telling cabal that your package is known to break. This extra information would make the solver much better when you tell it to ignore upper bounds because it would only ignore the ones that are safe to ignore.
I suspect that this problem is more visible in Haskell than in other languages. Haskell's type system gives you a way to bound the number of things that you must understand about an API in order to use it. The confidence that we get from having these guarantees makes code reuse orders of magnitude easier in Haskell than it is in pretty much any other language in mainstream use today. Others have made this same observation. Here (https://www.youtube.com/watch?v=BveDrw9CwEg#t=903) is a presentation given at CUFP last year by a company using Haskell in production that gives the actual reuse numbers that they've seen for several different languages.
I've been using Haskell for more than 6 years and in my experience the worst build problems happen with parts of the ecosystem that don't specify upper bounds. There's been a lot of discussion about this in the community. We're aware of the issues and we're working on solving them. One attractive possibility being discussed is adding a flag to cabal that allows one to ignore upper bounds. In theory this seems like it can preserve the benefits we get from the added information in an upper bound while eliminating the pain. An extension to this idea is to make two operators (say < and <!) that specify a "soft" upper bound that is just the highest you've been able to test against and a "hard" upper bound where you're telling cabal that your package is known to break. This extra information would make the solver much better when you tell it to ignore upper bounds because it would only ignore the ones that are safe to ignore.
I suspect that this problem is more visible in Haskell than in other languages. Haskell's type system gives you a way to bound the number of things that you must understand about an API in order to use it. The confidence that we get from having these guarantees makes code reuse orders of magnitude easier in Haskell than it is in pretty much any other language in mainstream use today. Others have made this same observation. Here (https://www.youtube.com/watch?v=BveDrw9CwEg#t=903) is a presentation given at CUFP last year by a company using Haskell in production that gives the actual reuse numbers that they've seen for several different languages.
"Why Hackage packagers like to specify upper bounds I'm not sure of."
There is a policy that says you're supposed to do this:
http://www.haskell.org/haskellwiki/Package_versioning_policy
The idea is that by specifying upper bounds, the package is more likely to build after the dependencies change.
This idea is subject to considerable controversy.
http://www.haskell.org/pipermail/haskell-cafe/2012-August/10...
There's also been more recent mailing list traffic on it, but my Google skills are failing me here and the arguments haven't changed all that much.
My opinion is that we are relying too heavily on upper bounds to solve a whole bevy of interrelated, but separate, problems. Other solutions are evolving to help deal with some of the problems. cabal sandboxes allow you to build a package separately from your others, reducing dependency issues. cabal freeze will help you specify exact dependencies, including transitive ones. Another project, Stackage, aims to maintain a whole slew of packages in a buildable state:
https://github.com/fpco/stackage
There is a policy that says you're supposed to do this:
http://www.haskell.org/haskellwiki/Package_versioning_policy
The idea is that by specifying upper bounds, the package is more likely to build after the dependencies change.
This idea is subject to considerable controversy.
http://www.haskell.org/pipermail/haskell-cafe/2012-August/10...
There's also been more recent mailing list traffic on it, but my Google skills are failing me here and the arguments haven't changed all that much.
My opinion is that we are relying too heavily on upper bounds to solve a whole bevy of interrelated, but separate, problems. Other solutions are evolving to help deal with some of the problems. cabal sandboxes allow you to build a package separately from your others, reducing dependency issues. cabal freeze will help you specify exact dependencies, including transitive ones. Another project, Stackage, aims to maintain a whole slew of packages in a buildable state:
https://github.com/fpco/stackage
It's an incredibly stupid policy. As others have noted, other package managers also has the capability of setting upper bounds on package versions and pip also has the ability to freeze your dependencies to specific versions.
However those features aren't necessary to productively use pip for example. Because Python package maintainers are very good at maintaining backwards compatibility (or using deprecation cycles when those are needed) because people that depend on their packages get upset with them. The rules are different for beta releases because it's your own fault if you depend on them and their api changes.
Precisely because specifying upper bounds is encouraged, Haskell maintainers seem to act as if they have a free card when it comes to backwards incompatible changes. In the long run that causes much more packaging troubles then specifying upper bounds saves. Perhaps there's nothing wrong with Haskell nor Cabal, but the Haskell community's way to handle backwards compatibility is wrong imo.
However those features aren't necessary to productively use pip for example. Because Python package maintainers are very good at maintaining backwards compatibility (or using deprecation cycles when those are needed) because people that depend on their packages get upset with them. The rules are different for beta releases because it's your own fault if you depend on them and their api changes.
Precisely because specifying upper bounds is encouraged, Haskell maintainers seem to act as if they have a free card when it comes to backwards incompatible changes. In the long run that causes much more packaging troubles then specifying upper bounds saves. Perhaps there's nothing wrong with Haskell nor Cabal, but the Haskell community's way to handle backwards compatibility is wrong imo.
Upper bounds are information about what you know your package works with. It makes no sense to throw this away. What does make sense is improving our tooling to allow this information to be ignored in certain circumstances.
I've been on the other side of the fence with a large app written a long time ago that did not specify upper bounds. Now that app is essentially unbuildable given the time I'm willing to put in. If I and all my dependencies had followed the PVP and specified correct upper bounds everywhere, I would still be able to build my app today. No, I wouldn't be able to build it with the latest version of other libraries, but that's not what I want to do.
If you don't specify upper bounds, the probability of your package building goes to ZERO in the long term. Not epsilon...zero. That's unacceptable for me. If it works today, I want it to work for years to come.
I've been on the other side of the fence with a large app written a long time ago that did not specify upper bounds. Now that app is essentially unbuildable given the time I'm willing to put in. If I and all my dependencies had followed the PVP and specified correct upper bounds everywhere, I would still be able to build my app today. No, I wouldn't be able to build it with the latest version of other libraries, but that's not what I want to do.
If you don't specify upper bounds, the probability of your package building goes to ZERO in the long term. Not epsilon...zero. That's unacceptable for me. If it works today, I want it to work for years to come.
"Upper bounds are information about what you know your package works with."
Not really; the upper bounds generally are speculative. They specify that the package will not work with particular versions when typically those versions have not yet been released. Often when those versions are released they work fine, which prompts a useless flurry of dependency bumping.
"If I and all my dependencies had followed the PVP and specified correct upper bounds everywhere, I would still be able to build my app today."
I now distribute my packages with the output of the ghc-pkg command so you can see the exact versions the library builds with. That solves your problem without using speculative upper bounds.
I wouldn't expect it to work for "years to come" in any event, though: Haskell changes too much. You probably won't be able to get today's package to work with the compiler and base package that will exist years hence. It's hard to even get an old GHC to build: today's GHC won't build it because of...dependency problems.
Not really; the upper bounds generally are speculative. They specify that the package will not work with particular versions when typically those versions have not yet been released. Often when those versions are released they work fine, which prompts a useless flurry of dependency bumping.
"If I and all my dependencies had followed the PVP and specified correct upper bounds everywhere, I would still be able to build my app today."
I now distribute my packages with the output of the ghc-pkg command so you can see the exact versions the library builds with. That solves your problem without using speculative upper bounds.
I wouldn't expect it to work for "years to come" in any event, though: Haskell changes too much. You probably won't be able to get today's package to work with the compiler and base package that will exist years hence. It's hard to even get an old GHC to build: today's GHC won't build it because of...dependency problems.
> Not really; the upper bounds generally are speculative.
Yes, an upper bound is still useful information. It says that a package is known to work with a certain range of dependency versions. This is why I think cabal should support something like < for speculative upper bounds and <! for known failure upper bounds. Then we could have a flag for optionally ignoring speculative upper bounds when desired.
> I wouldn't expect it to work for "years to come" in any event, though
I can still download a binary of GHC 6.12.3. Hackage also keeps the entire package history so there's no reason things shouldn't build for years to come. I'm all for the fast moving nature of the Haskell ecosystem, but we also need things to be able to build over the long term and there's no reason we shouldn't be able to do that.
Yes, an upper bound is still useful information. It says that a package is known to work with a certain range of dependency versions. This is why I think cabal should support something like < for speculative upper bounds and <! for known failure upper bounds. Then we could have a flag for optionally ignoring speculative upper bounds when desired.
> I wouldn't expect it to work for "years to come" in any event, though
I can still download a binary of GHC 6.12.3. Hackage also keeps the entire package history so there's no reason things shouldn't build for years to come. I'm all for the fast moving nature of the Haskell ecosystem, but we also need things to be able to build over the long term and there's no reason we shouldn't be able to do that.
"I can still download a binary of GHC 6.12.3."
Which won't even work on current versions of Debian GNU/Linux because the soname for libgmp changed.
"Hackage also keeps the entire package history so there's no reason things shouldn't build for years to come."
No it doesn't. I had specified a dependency on a particular version of text, and it summarily vanished from Hackage.
Even assuming that it's a worthwhile goal to make sure old software builds with current tools, upper bounds do nothing to get you there.
Which won't even work on current versions of Debian GNU/Linux because the soname for libgmp changed.
"Hackage also keeps the entire package history so there's no reason things shouldn't build for years to come."
No it doesn't. I had specified a dependency on a particular version of text, and it summarily vanished from Hackage.
Even assuming that it's a worthwhile goal to make sure old software builds with current tools, upper bounds do nothing to get you there.
> Which won't even work on current versions of Debian GNU/Linux because the soname for libgmp changed.
Again, my argument assumes that you're not upgrading to current versions of things. Enterprise production-grade deployments often fix their OS version because of this. There's a reason RHEL is so far behind the most recent versions. If you are upgrading, then you'll be fixing these problems incrementally as you go along and it won't get out of hand.
> No it doesn't. I had specified a dependency on a particular version of text
You're making my point for me. It looks like you were probably depending on 0.11.1.4, which doesn't exist now. Your problem would not have happened if you had used an upper bound of "< 0.12" like the PVP says instead of locking down to one specific set of versions. If you had done that, cabal would have picked up subsequent 0.11 point releases which should have worked fine. Also, I can still download text-0.1 which is more than 5 years old.
Again, my argument assumes that you're not upgrading to current versions of things. Enterprise production-grade deployments often fix their OS version because of this. There's a reason RHEL is so far behind the most recent versions. If you are upgrading, then you'll be fixing these problems incrementally as you go along and it won't get out of hand.
> No it doesn't. I had specified a dependency on a particular version of text
You're making my point for me. It looks like you were probably depending on 0.11.1.4, which doesn't exist now. Your problem would not have happened if you had used an upper bound of "< 0.12" like the PVP says instead of locking down to one specific set of versions. If you had done that, cabal would have picked up subsequent 0.11 point releases which should have worked fine. Also, I can still download text-0.1 which is more than 5 years old.
The old adage "Haskellers doesn't know how it feels to shot themselves in the foot because they are walking around with a bleeding flesh wound in their feet" fits aptly. The problem upper bounds tries to solve just does not exist in other languages because packagers are expected not to break backwards compatibility. I can almost without exception run the same apps I developed for Django 1.0 (csrf protection introduced in 1.2 caused backwards incompatible changes but that was an exception) almost ten years ago with current software versions.
In fact, if any of the packages demanded an upper bound it would suck because I don't want to use a legacy Django web server containing tons of exploits which no one is ever going to fix because it's not maintained anymore.
In fact, if any of the packages demanded an upper bound it would suck because I don't want to use a legacy Django web server containing tons of exploits which no one is ever going to fix because it's not maintained anymore.
> The problem upper bounds tries to solve just does not exist in other languages because packagers are expected not to break backwards compatibility.
I'll agree that Haskell moves faster and has more breaking changes, but that statement is just wrong. Look at http://semver.org/ (not related to Haskell at all) and you'll see that the very first point on the page is about incompatible API changes. So clearly this issue exists outside of Haskell and people have developed methods for managing it with version bound schemes. You are arguing against rapidly changing software, not against upper bounds. In your example where the Django server is not making backwards-incompatible changes, upper bounds wouldn't hurt you at all because the bounds are on the major version number, but exploit fixes that don't break backwards compatibility will only bump the minor version number.
Comparing Haskell to any other mainstream language in this discussion is invalid because the other languages have been around a lot longer and have reached a more stable state. Python appeared in 1991. The first Haskell standard appeared in 1998. So that means Python has at least 7 years of stability on Haskell. I would argue that Haskell gained adoption much more slowly because it is much less similar to any mainstream language that came before it, so the actual number should be larger. Paul Graham's essay "The Python Paradox" came out in 2004. I would suggest that Haskell is just now getting close to the point that Python was at when PG wrote that essay. That means that Python has at least 10 years on Haskell. So if you're comparing breaking changes in Haskell today with Python, you need to compare it with Python as it was 10 years ago. If you think the breaking changes are not worth that much pain for you, then don't use Haskell right now. But you shouldn't make that decision without educating yourself about the benefits the language has to offer. For me, it is a small price to pay compared to the benefits I get from Haskell.
Only time will tell, but I predict that companies based solely on Haskell will emerge in a few years dominating their competition because they can write better quality software, iterate faster, with fewer people, more reuse, fewer bugs, and easier maintenance than companies not using Haskell.
I'll agree that Haskell moves faster and has more breaking changes, but that statement is just wrong. Look at http://semver.org/ (not related to Haskell at all) and you'll see that the very first point on the page is about incompatible API changes. So clearly this issue exists outside of Haskell and people have developed methods for managing it with version bound schemes. You are arguing against rapidly changing software, not against upper bounds. In your example where the Django server is not making backwards-incompatible changes, upper bounds wouldn't hurt you at all because the bounds are on the major version number, but exploit fixes that don't break backwards compatibility will only bump the minor version number.
Comparing Haskell to any other mainstream language in this discussion is invalid because the other languages have been around a lot longer and have reached a more stable state. Python appeared in 1991. The first Haskell standard appeared in 1998. So that means Python has at least 7 years of stability on Haskell. I would argue that Haskell gained adoption much more slowly because it is much less similar to any mainstream language that came before it, so the actual number should be larger. Paul Graham's essay "The Python Paradox" came out in 2004. I would suggest that Haskell is just now getting close to the point that Python was at when PG wrote that essay. That means that Python has at least 10 years on Haskell. So if you're comparing breaking changes in Haskell today with Python, you need to compare it with Python as it was 10 years ago. If you think the breaking changes are not worth that much pain for you, then don't use Haskell right now. But you shouldn't make that decision without educating yourself about the benefits the language has to offer. For me, it is a small price to pay compared to the benefits I get from Haskell.
Only time will tell, but I predict that companies based solely on Haskell will emerge in a few years dominating their competition because they can write better quality software, iterate faster, with fewer people, more reuse, fewer bugs, and easier maintenance than companies not using Haskell.
I think the more recent issue you're referring to is this?
http://www.haskell.org/pipermail/libraries/2014-February/022...
> But at least pip, npm and gems are a breeze compared to cabal.
I've not really used the others, but Gemfiles have the concept of upper bounds: http://stackoverflow.com/questions/5170547/what-does-tilde-g...
Docs: http://guides.rubygems.org/patterns/#pessimistic_version_con...
I've not really used the others, but Gemfiles have the concept of upper bounds: http://stackoverflow.com/questions/5170547/what-does-tilde-g...
Docs: http://guides.rubygems.org/patterns/#pessimistic_version_con...
>Why Hackage packagers like to specify upper bounds I'm not sure of
Because having working code spontaneously break is very bad. You can't just blindly pretend any future versions of all your dependencies will work without updating your code. APIs change. So you specify the version range your code works with.
>Backwards compatibility either is very hard to achieve in Haskell
Backwards compatibility is the whole point of upper bounds.
>But at least pip, npm and gems are a breeze compared to cabal.
I had constant problems with pip. Because it will blindly install the latest 'foo' dependency even though the API changed and now my app is broken.
Because having working code spontaneously break is very bad. You can't just blindly pretend any future versions of all your dependencies will work without updating your code. APIs change. So you specify the version range your code works with.
>Backwards compatibility either is very hard to achieve in Haskell
Backwards compatibility is the whole point of upper bounds.
>But at least pip, npm and gems are a breeze compared to cabal.
I had constant problems with pip. Because it will blindly install the latest 'foo' dependency even though the API changed and now my app is broken.
You can't just blindly pretend any future versions of all your dependencies will work without updating your code. APIs change.
If you have a risk of arbitrary changes in APIs happening in any minor update, your infrastructure and dependency management are fundamentally flawed and ongoing development will always be difficult. There is nothing unique to Haskell about this, it's just software development 101, separating interface from implementation and all that jazz.
There does seem to be a surprising... I'm not sure how to describe it, maybe a lack of respect?... for this principle in parts of the Haskell community, given the same community's obvious general preference for correctness and being sure everything fits together in a controlled way.
Of course sometimes you want to make major changes to an API and maybe they break backward compatibility, but usually that is a Big Deal in software development, it doesn't normally just get done in some quick point release along with a couple of bug fixes and a security patch.
I had constant problems with pip. Because it will blindly install the latest 'foo' dependency even though the API changed and now my app is broken.
Not that I'm doubting you, but I do find that experience surprising. I work with Python all the time, on different kinds of projects but typically always using pip for package management, and I can't immediately think of a single time that's happened to me. Of course, you can also tell pip to install only within a specific version range if you ever need to.
If you have a risk of arbitrary changes in APIs happening in any minor update, your infrastructure and dependency management are fundamentally flawed and ongoing development will always be difficult. There is nothing unique to Haskell about this, it's just software development 101, separating interface from implementation and all that jazz.
There does seem to be a surprising... I'm not sure how to describe it, maybe a lack of respect?... for this principle in parts of the Haskell community, given the same community's obvious general preference for correctness and being sure everything fits together in a controlled way.
Of course sometimes you want to make major changes to an API and maybe they break backward compatibility, but usually that is a Big Deal in software development, it doesn't normally just get done in some quick point release along with a couple of bug fixes and a security patch.
I had constant problems with pip. Because it will blindly install the latest 'foo' dependency even though the API changed and now my app is broken.
Not that I'm doubting you, but I do find that experience surprising. I work with Python all the time, on different kinds of projects but typically always using pip for package management, and I can't immediately think of a single time that's happened to me. Of course, you can also tell pip to install only within a specific version range if you ever need to.
"If you have a risk of arbitrary changes in APIs happening in any minor update"
Upper bounds typically restrict major version, not minor version numbers which would represent non-breaking changes.
Upper bounds typically restrict major version, not minor version numbers which would represent non-breaking changes.
Perhaps my initial choice was words was poor. Allow me to clarify: I don't think it matters very much what level of version numbering indicates a breaking change, because the main problem is how often those breaking changes happen. I've used libraries that have had a total of zero breaking API changes over a period of years in professional production code written in many other languages. In Haskell, I seem to have run into one ever-changing package or another via Cabal every time I've tried to do something moderately serious in the language. Given the emphasis on relatively small, general and highly reusable parts in Haskell code, I find this surprising, and unless I've been remarkably unlucky in the projects/packages I've tried, it must be a significant drag on development efficiency.
There are some parts of the ecosystem that are incredibly stable (base, containers, parsec) and some aren't (lens). Of course, a lot of new hotness winds up being somewhat experimental, and so we're stuck with bad decisions made early on or wind up faced with the choice of when to cause breakage - in which case "earlier is better" might not be wrong... there have been some efforts to isolate more stable subsets, where that's wanted.
Fair point, though I'd suggest that it's not just general/utilitarian/language support areas like lenses where rapid evolution happens, but also quite a few of the practical, real world things like dealing with comms protocols or user interfaces. This is true to some extent for every mainstream language I know as well, but I think perhaps because Haskell's ecosystem isn't as large and there aren't as many (or sometimes any) ready-made alternatives, you feel the effect more when you run into it. For better or worse, I think this rapid evolution does lend some weight to claims that Haskell isn't ready for use on most mainstream projects yet and to the idea that it's more of an academic's or theoretician's language, even if a few such people happen to work in industrial software development rather than in research now.
I suppose it's a similar situation to what we see in web development. There are always people who want to push the envelope with the latest HTML7/CSS5/SomethingScript technologies, and it's helpful for advancing the state of the art for some people to do that. At the same time, if you're trying to build real web sites for paying clients, what you really need is solid foundations that are as standardised and portable as possible.
I suppose it's a similar situation to what we see in web development. There are always people who want to push the envelope with the latest HTML7/CSS5/SomethingScript technologies, and it's helpful for advancing the state of the art for some people to do that. At the same time, if you're trying to build real web sites for paying clients, what you really need is solid foundations that are as standardised and portable as possible.
"general/utilitarian/language support areas like lenses where rapid evolution happens"
Oh, no question. It just was an example I had off the top of my head of somewhere there has been a lot of instability recently.
"For better or worse, I think this rapid evolution does lend some weight to claims that Haskell isn't ready for use on most mainstream projects yet and to the idea that it's more of an academic's or theoretician's language, even if a few such people happen to work in industrial software development rather than in research now."
To an extent, but I'm not sure I accept the dichotomy. The great thing about Haskell over HTML7/CSS5/SomethingScript is that the people doing experimental stuff are doing it in the same language you're working in, and it will bleed directly back into "things that are stable enough to use" as they get stable. The thing that Haskell-in-industry has over academia is that those experimental innovations scratch itches found by people working in industry. The presence of Haskell in academia means those industrial innovators have more things to reach for. Where it works, it works very well. Because the Haskell community is small, we don't have the large mass of stable libraries that something like Python has, but how much that matters depends both on how stable you need things and on what you need in the first place.
"At the same time, if you're trying to build real web sites for paying clients, what you really need is solid foundations that are as standardised and portable as possible."
Absolutely. If you need things stable, stick to stable packages. There are efforts (stackage, Haskell Platform) toward making this easy. If you need things stable, and you have needs that stable libraries don't meet, and it doesn't make sense for you to get involved in maintaining less stable libraries, then Haskell is probably not a good fit for your project.
Oh, no question. It just was an example I had off the top of my head of somewhere there has been a lot of instability recently.
"For better or worse, I think this rapid evolution does lend some weight to claims that Haskell isn't ready for use on most mainstream projects yet and to the idea that it's more of an academic's or theoretician's language, even if a few such people happen to work in industrial software development rather than in research now."
To an extent, but I'm not sure I accept the dichotomy. The great thing about Haskell over HTML7/CSS5/SomethingScript is that the people doing experimental stuff are doing it in the same language you're working in, and it will bleed directly back into "things that are stable enough to use" as they get stable. The thing that Haskell-in-industry has over academia is that those experimental innovations scratch itches found by people working in industry. The presence of Haskell in academia means those industrial innovators have more things to reach for. Where it works, it works very well. Because the Haskell community is small, we don't have the large mass of stable libraries that something like Python has, but how much that matters depends both on how stable you need things and on what you need in the first place.
"At the same time, if you're trying to build real web sites for paying clients, what you really need is solid foundations that are as standardised and portable as possible."
Absolutely. If you need things stable, stick to stable packages. There are efforts (stackage, Haskell Platform) toward making this easy. If you need things stable, and you have needs that stable libraries don't meet, and it doesn't make sense for you to get involved in maintaining less stable libraries, then Haskell is probably not a good fit for your project.
That is the most elaborate strawman I have seen in a while. You are now literally saying "haskell is dumb because they do what I think they should do. They should do what they already do instead."
>If you have a risk of arbitrary changes in APIs happening in any minor update
You don't. API changes require changing the major version. Read the PVP.
>There is nothing unique to Haskell about this
Precisely my point.
>Of course, you can also tell pip to install only within a specific version range if you ever need to.
Same with cabal. So what is the problem again?
>If you have a risk of arbitrary changes in APIs happening in any minor update
You don't. API changes require changing the major version. Read the PVP.
>There is nothing unique to Haskell about this
Precisely my point.
>Of course, you can also tell pip to install only within a specific version range if you ever need to.
Same with cabal. So what is the problem again?
That is the most elaborate strawman I have seen in a while. You are now literally saying "haskell is dumb because they do what I think they should do. They should do what they already do instead."
No, I'm not literally saying any such thing.
There are two points here. One is whether or not your version numbering scheme and package/dependency management tools systematically track breaking changes. The other is how often breaking changes happen.
As far as I'm aware, Haskell doesn't generally have a big problem with the first point. As you say, PVP takes care of that as long as people follow it.
But breaking changes, even if properly acknowledged, seem to happen with amazing (not in a good way) frequency in the Haskell ecosystem. No doubt there's some variation between packages, but I'm another example of someone whose early experiments with the language proved very frustrating because of the constant maintenance headaches with Cabal etc.
No, I'm not literally saying any such thing.
There are two points here. One is whether or not your version numbering scheme and package/dependency management tools systematically track breaking changes. The other is how often breaking changes happen.
As far as I'm aware, Haskell doesn't generally have a big problem with the first point. As you say, PVP takes care of that as long as people follow it.
But breaking changes, even if properly acknowledged, seem to happen with amazing (not in a good way) frequency in the Haskell ecosystem. No doubt there's some variation between packages, but I'm another example of someone whose early experiments with the language proved very frustrating because of the constant maintenance headaches with Cabal etc.
If your complaint is "some haskell libraries move too fast for my liking" then you should say that in the first place. If you frame your complaint as "cabal sux and everything else solves this wtf is wrong with you guys" then you have to reasonably expect that is what people will respond to.
Please stop claiming I wrote things that are nothing like what I wrote. It's impossible to have an intelligent conversation while you're doing that.
It is very like what you wrote, try reading your post again.
My point is that the Haskell community seems (or at least parts of it seem) more tolerant of frequent API changes, including backwards-incompatible ones, than we would normally see in mainstream programming languages. I think this is a drag on development efficiency.
You wrote yourself, "having working code spontaneously break is very bad". You can protect against dependencies breaking by adding upper limits on version numbers for your dependencies, but that doesn't fix the problem, it just conceals it, and it has a cost: every time someone makes a breaking change in a library API, everyone using that library needs to check through what actually changed to see whether it affects their particular situation and then update their dependency management again.
In my original post, I never even mentioned version numbering schemes, and in two different posts now I've reiterated that this isn't what I'm concerned about.
I wasn't having a dig at Cabal for how it supports controlled dependencies either, nor suggesting that other package managers do that job better. That would be like blaming the UI for every bug in software because the UI is where the consequences of bugs are visible to the user. Cabal isn't the problem, the implied need to use certain features of Cabal routinely because otherwise things will break is the problem.
I wasn't even objecting to Haskell libraries developing "too fast for my liking". I think the innovations coming out of the Haskell community are some of the most interesting and promising ideas in the programming world right now, and in general I'm a big fan. I'm just saying that, other things being equal, breaking changes in APIs are undesirable, and that the Haskell ecosystem seems a lot more tolerant of such changes than most, whether because of that desire to move forward quickly or otherwise. That's great in a research language, but I don't think it is helpful for mainstream industrial development, because it does have practical consequences particularly in terms of maintenance overheads.
You wrote yourself, "having working code spontaneously break is very bad". You can protect against dependencies breaking by adding upper limits on version numbers for your dependencies, but that doesn't fix the problem, it just conceals it, and it has a cost: every time someone makes a breaking change in a library API, everyone using that library needs to check through what actually changed to see whether it affects their particular situation and then update their dependency management again.
In my original post, I never even mentioned version numbering schemes, and in two different posts now I've reiterated that this isn't what I'm concerned about.
I wasn't having a dig at Cabal for how it supports controlled dependencies either, nor suggesting that other package managers do that job better. That would be like blaming the UI for every bug in software because the UI is where the consequences of bugs are visible to the user. Cabal isn't the problem, the implied need to use certain features of Cabal routinely because otherwise things will break is the problem.
I wasn't even objecting to Haskell libraries developing "too fast for my liking". I think the innovations coming out of the Haskell community are some of the most interesting and promising ideas in the programming world right now, and in general I'm a big fan. I'm just saying that, other things being equal, breaking changes in APIs are undesirable, and that the Haskell ecosystem seems a lot more tolerant of such changes than most, whether because of that desire to move forward quickly or otherwise. That's great in a research language, but I don't think it is helpful for mainstream industrial development, because it does have practical consequences particularly in terms of maintenance overheads.
>You can protect against dependencies breaking by adding upper limits
As every sane system does. So again, why are you acting like this is a haskell thing?
>but that doesn't fix the problem
Yes it does.
>every time someone makes a breaking change in a library API, everyone using that library needs to check through what actually changed to see whether it affects their particular situation
That is the case no matter what. If I write an SSL application in C, I need to pay attention if a new version of openssl comes out. Once again, this isn't even remotely a haskell issue.
>but I don't think it is helpful for mainstream industrial development
You are not required to update to every new version that is released. If you do not want to track the bleeding edge then don't. That's why we have upper bounds.
As every sane system does. So again, why are you acting like this is a haskell thing?
>but that doesn't fix the problem
Yes it does.
>every time someone makes a breaking change in a library API, everyone using that library needs to check through what actually changed to see whether it affects their particular situation
That is the case no matter what. If I write an SSL application in C, I need to pay attention if a new version of openssl comes out. Once again, this isn't even remotely a haskell issue.
>but I don't think it is helpful for mainstream industrial development
You are not required to update to every new version that is released. If you do not want to track the bleeding edge then don't. That's why we have upper bounds.
It's not really cabal but libraries that are broken. They tend to define dependencies too liberally with >=, which obviously breaks things when the depended library makes non-backwards compatible changes.
Also, the primary language implementation itself makes rather large changes sometimes.
Also, the primary language implementation itself makes rather large changes sometimes.
Cabal sandboxes have fixed a lot of these problems, usually this results from having installed all packages globally, and then cabal tries to find a satisfying version between all global packages and fails. You can also take Ed Kmett's approach and just do --force --reinstall every time. I prefer the sandboxes though, when something breaks you can always nuke the sandbox and quickly reinstall. I would recommend upgrading to Cabal 1.18.x and try the sandboxes out.
This has been my experience exactly. It was such a problem that it really turned me off of trying to use Haskell for a serious project.
Cabal has been the focus of a ton of work lately, much of which is still ongoing. It has been improving rapidly, notably with sandboxes in 1.18. It still has a long way to go, admittedly, but I find myself having trouble with it way less frequently now.
so a dumb little thing that fixes that a lot:
--max-backjumps 10000
Also the latest HEAD of cabal has the freeze command which is remniscient of pip freeze.
--max-backjumps 10000
Also the latest HEAD of cabal has the freeze command which is remniscient of pip freeze.
Using cabal sandbox'ing helps avoid this issue.
I didn't know what Haskell code looked like so I went to the official website and there was this code example[1]:
qsort (p:xs) = qsort [x | x<-xs, x<p] ++ [p] ++ qsort [x | x<-xs, x>=p]
Is this a good example of good Haskell code?
[1] http://www.haskell.org/haskellwiki/Introduction#Brevity
qsort (p:xs) = qsort [x | x<-xs, x<p] ++ [p] ++ qsort [x | x<-xs, x>=p]
Is this a good example of good Haskell code?
[1] http://www.haskell.org/haskellwiki/Introduction#Brevity
I think that's a bit extreme for an example - I'd go for the following version, which is less concise but easier to read:
Then you have the base case - if the list is empty, you'll return an empty list.
After that you have the case "item++[list of items]" (note that the second one can be an empty list), where you return [list of smaller items]++item++[list of larger items] (the two lists are defined under the "where" - filter removes elements from a list).
Note however that, instead of "(lesser) ++ item ++ (larger)" you have "(quicksort lesser) ++ item ++ (quicksort larger)" - this will call the algorithm recursively over smaller lists, but I bet you already knew that.
Both pieces of code do the same thing, but I think mine is what you'd expect from a typical piece of Haskell code - the one you have is closer to "see how small I can make this function".
quicksort :: Ord a => [a] -> [a]
quicksort [] = []
quicksort (p:xs) = (quicksort lesser) ++ [p] ++ (quicksort greater)
where
lesser = filter (< p) xs
greater = filter (>= p) xs
Then you have the base case - if the list is empty, you'll return an empty list.
After that you have the case "item++[list of items]" (note that the second one can be an empty list), where you return [list of smaller items]++item++[list of larger items] (the two lists are defined under the "where" - filter removes elements from a list).
Note however that, instead of "(lesser) ++ item ++ (larger)" you have "(quicksort lesser) ++ item ++ (quicksort larger)" - this will call the algorithm recursively over smaller lists, but I bet you already knew that.
Both pieces of code do the same thing, but I think mine is what you'd expect from a typical piece of Haskell code - the one you have is closer to "see how small I can make this function".
This is a nice example what Haskell can do, but the sort itself is quite inefficient and it's not a true quicksort.
It's absolutely a true quicksort.
Is it in place? If not, then it is not a true quicksort.
Interesting question -- in Haskell that means something else (some would say "nothing"). The language is designed around making sure that you only specify the function that the code is supposed to accomplish, and all implementation decisions are left to the compiler -- just as it would be for the RHS of a statement like `x = 3 + 4*(5 + 1);` in C.
In this case, as in all others, the Haskell compiler would decide what is the optimal way to implement it, given whatever other constraints you've placed on the program.
You can add additional constraints that ensure the compiled code turns out to be in-place, at a cost of verbosity.
In this case, as in all others, the Haskell compiler would decide what is the optimal way to implement it, given whatever other constraints you've placed on the program.
You can add additional constraints that ensure the compiled code turns out to be in-place, at a cost of verbosity.
If there is an example where such an optimization has actually been implemented and correctly chosen, that is news to me.
I mean, to a very large extent, this is the reason why classical imperative languages are so easy to understand and reason about. The programmers directions are much more straight forwardly executed than in examples like these hypotheticals.
Oh, I just got to your last sentence. Basically, that this would be a lot more verbose if written in such a way that it would be in place. Which is kind of the point of this criticism. Isn't it?
Don't get me wrong, the power of Haskell is not really in question. Just that example.
I mean, to a very large extent, this is the reason why classical imperative languages are so easy to understand and reason about. The programmers directions are much more straight forwardly executed than in examples like these hypotheticals.
Oh, I just got to your last sentence. Basically, that this would be a lot more verbose if written in such a way that it would be in place. Which is kind of the point of this criticism. Isn't it?
Don't get me wrong, the power of Haskell is not really in question. Just that example.
Your criticisms are all from the perspective of "it keeps me from telling it how to implement the functionality", which is true, but not what Haskell's design (or pure FP language design in general) optimizes for.
Sometimes, you really do want to make commands to the bare metal regarding exactly how you want the program implemented. For general use, however, you don't: compilers are (at least these days) a lot better at finding optimizations than the typical user.
It comes down to abstraction levels: pick the one that matches the problem you're working on. The FP philosophy is "speaking about specific memory cells is way too low" in most cases, like telling it you want a sorted list.
In fact, it would probably be even better (from that selfsame philosophy) to define a sort in terms of the conditions the final list would meet (as opposed to nested, recursing conditions like in the example). Something like, "return a list where every element is <= the one after it, and where every element is also in the original list, as many times as it occurs there".
If you want to see the examples of quicksort in Haskell to enforce in-place and other requirements, see this discussion here, with links:
http://www.haskell.org/haskellwiki/Introduction/Direct_Trans...
tl;dr: Yeah, Haskell is worse at letting you force the sort to be in-place, but that can be a very good thing.
Sometimes, you really do want to make commands to the bare metal regarding exactly how you want the program implemented. For general use, however, you don't: compilers are (at least these days) a lot better at finding optimizations than the typical user.
It comes down to abstraction levels: pick the one that matches the problem you're working on. The FP philosophy is "speaking about specific memory cells is way too low" in most cases, like telling it you want a sorted list.
In fact, it would probably be even better (from that selfsame philosophy) to define a sort in terms of the conditions the final list would meet (as opposed to nested, recursing conditions like in the example). Something like, "return a list where every element is <= the one after it, and where every element is also in the original list, as many times as it occurs there".
If you want to see the examples of quicksort in Haskell to enforce in-place and other requirements, see this discussion here, with links:
http://www.haskell.org/haskellwiki/Introduction/Direct_Trans...
tl;dr: Yeah, Haskell is worse at letting you force the sort to be in-place, but that can be a very good thing.
I understand what you are saying. I'm arguing that when quick sort was first contrived, the speed aspect that you can achieve with the in place variant was not just a lucky by product. It was the bloody point. To such an extent that if you are lucky enough -- which most of us are -- to be in a place where that is not a consideration, then sure, go for the higher level abstractions.
That said, it is highly telling that none of the actual implementations of quick sort in any of these languages are that succinct. To the point that it is still a truism to "use a library" for something such as sorting. Because that isn't easy. Do not be fooled by such a quaint example.
That said, it is highly telling that none of the actual implementations of quick sort in any of these languages are that succinct. To the point that it is still a truism to "use a library" for something such as sorting. Because that isn't easy. Do not be fooled by such a quaint example.
Haskell:
http://www.haskell.org/pipermail/haskell-cafe/2009-August/06...
Twice as slow as C but without also in-place like C and extremely similar (see the sort function here):
http://en.wikibooks.org/wiki/Algorithm_Implementation/Sortin...
It's interesting to see is how very similar Haskel and C sources fundamentally become once they really solve the same problem and not the different ones.
http://www.haskell.org/pipermail/haskell-cafe/2009-August/06...
Twice as slow as C but without also in-place like C and extremely similar (see the sort function here):
http://en.wikibooks.org/wiki/Algorithm_Implementation/Sortin...
It's interesting to see is how very similar Haskel and C sources fundamentally become once they really solve the same problem and not the different ones.
The "direct compare" in Haskell isn't. The Ord constraint effectively passes a dictionary, and even if that were inlined in a particular case you could always apply a newtype for a different ordering.
I'd really like to learn, can you please explain where the Ord constraint is, and where the dictionary is passed in the link I've given (with the STUArray -- isn't that really an array in a C sense)? To give you an idea how little I know: I understand the basic ideas of Haskell (that is, what I as a C programmer who made some compilers understand as basic: the type inference and the illusion of the infinite memory by not doing things in place but lazily with the GC unless unsafe or when it is possible to optimize the laziness out) but I don't really know the language. Thanks in advance.
Ah, my bad, the STUArray example is specialized and probably does not pass an Ord around (though there's implicitly one present because of the less-then). Certainly, the function does not meaningfully allow the user to pick it.
Has anyone ever bothered to ask Tony Hoare what he thinks on this topic? I know SPJ and him both work for MS Research, maybe he can get on that and settle this for us.
I would be curious to know the answer. Though, I should have added in my original post that I am far from the originator of this view. I see a sibling post elaborated. I think the crux is simply that without the "in place" nature, it loses a lot of its speed advantage. Cache localities and such.
Quicksort can be implemented in an in-place fashion, but that's not a requirement by any means.
If I understand correctly, this code has a bug.
The list is the sorted list of lesser items, then the pivot, then the sorted list of greater items. But since the list of greater items is defined with ">=" rather than just ">", the list of greater items will also contain the pivot. That will put the pivot in the sorted list twice (for each recursive call).
The list is the sorted list of lesser items, then the pivot, then the sorted list of greater items. But since the list of greater items is defined with ">=" rather than just ">", the list of greater items will also contain the pivot. That will put the pivot in the sorted list twice (for each recursive call).
Note that the argument, the list to be sorted, is pattern matched as
(p:xs)
greater = filter (>= p) xs
Ah, I see. Thanks!
If I'm reading the code correctly: given the list [5,5,2,3] and the match (p:xs), p will match to the first 5 and xs will match to [5,2,3].
Once the matching is done, the pivot is no longer in the list xs, and therefore it won't be included twice.
Once the matching is done, the pivot is no longer in the list xs, and therefore it won't be included twice.
Thanks for the insightful reply! I can understand most of the code but I still do not understand how should I know what "p" and "xs" is. Is it something you get when you declare the function as being "possible" to order (I believe that's what "Ord" means based on your explanation)?
One last thing I'll add to the other replies, the example shown is a bit unusual, like indexing a for loop with "q" instead of "i". The traditional pattern is this:
Also, Haskellers often use "x" here because they write a lot of functions that operate on "anything"; for instance, in this case, you don't really know what you're sorting, so, call it x because what other name is any better? There's a set of about 10 of these conventional one-letter variable names. (A few more than conventional imperative languages, but not obscenely so; imperative has i, j, and k for iteration, a and b in sorting, sometimes f for function, and p for pointer.)
(x:xs)
Also, Haskellers often use "x" here because they write a lot of functions that operate on "anything"; for instance, in this case, you don't really know what you're sorting, so, call it x because what other name is any better? There's a set of about 10 of these conventional one-letter variable names. (A few more than conventional imperative languages, but not obscenely so; imperative has i, j, and k for iteration, a and b in sorting, sometimes f for function, and p for pointer.)
I assume here it's p for pivot, xs because the rest of the list is just items, not pivots (as ps would imply). I'm not sure I love the naming choice, but it has a logic to it.
Those are pattern matches [1]. In Haskell, you have a very powerful pattern matching system for describing and deconstructing function arguments. So quicksort here is defined as
This is a valid pattern match because AFAIK the colon (:) is a type constructor in Haskell, and it's used to create lists. For instance, 1 : [] == [1], and 1 : 2 : [3, 4] = [1, 2, 3, 4].
Because (:) is a type constructor, and not an operator (unlike (+) for instance), you can use it in pattern matching expressions to deconstruct lists. This is quite useful for many things, as you might imagine.
[1]: https://en.wikibooks.org/wiki/Haskell/Pattern_matching
quicksort [] = []
quicksort (p:xs) = ...
This is a valid pattern match because AFAIK the colon (:) is a type constructor in Haskell, and it's used to create lists. For instance, 1 : [] == [1], and 1 : 2 : [3, 4] = [1, 2, 3, 4].
Because (:) is a type constructor, and not an operator (unlike (+) for instance), you can use it in pattern matching expressions to deconstruct lists. This is quite useful for many things, as you might imagine.
[1]: https://en.wikibooks.org/wiki/Haskell/Pattern_matching
(:) is a data constructor, not a type constructor, and its type is a -> [a] -> [a]. Examples of type constructors: [], Maybe, Either.
To be clear: [] can refer to two different entities, a data constructor (empty list) and a type constructor (the list type).
[deleted]
Thanks for the clarification. You might be able to tell, that I am only in the first stages of trying to learn Haskell.
No, it's pattern matching (it's a bit warty because that's the only case I can think of where the pattern matching part doesn't look like the way you build the data structure).
Normal pattern matching:
Normal pattern matching:
-- a Foo record, with a data constructor Foo and a single field
data Foo = Foo { bar :: String }
-- takes an instance of Foo, returns "prefix:" + the value of the bar field
prefixBarInFoo :: Foo -> String
-- Pattern matches on Foo, tells Haskell to bind 'valueOfBar' to the 'bar' field of the instance of Foo
prefixBarInFoo (Foo valueOfBar) = "prefix:" ++ valueOfBar
myList = [1,2,3,4,5]
addOne :: [Integer] -> [Integer]
addOne [] = []
-- when called with 'myList', binds x to 1 and xs to [2,3,4,5] (the rest of the list)
addOne (x:xs) = (x+1) : (addOne xs)As dllthomas commented, 'p' would be better named "pivot".
xs is just convention. A variable containing an arbitrary number is often called 'x', right? Well, a list of such things is plural, so xs.
So pattern match a list, say the first item is the pivot, the remainder is just a list of arbitrary numbers, 'xs'.
xs is just convention. A variable containing an arbitrary number is often called 'x', right? Well, a list of such things is plural, so xs.
So pattern match a list, say the first item is the pivot, the remainder is just a list of arbitrary numbers, 'xs'.
That is how you destructure the list, with the syntax (x:xs) where x is the first element of the list, and xs is the remainder of the list.
[deleted]
Absolutely not. I'm actually very frustrated that they give that code as an example. First of all, it misses the most important of quicksort - the fact that it can be implemented in place (and yes, you can write an pure in-place quicksort using STArrays).
Second of all, it's bad because it runs through the list xs twice. Instead, they should use the partition function to get both the left-hand and right-hand sides at once.
And thirdly, it's inefficient by use of the (++) operator, which takes time proportional to the length of the left-hand list. A better (out-of-place) implementation of quicksort would use an accumulator so this wouldn't happen (and actually, this means that we should do the partitioning ourselves rather than using the partition function as mentioned above).
Here is an example that makes these improvements with quicksort [1].
[1] http://en.literateprograms.org/Quicksort_(Haskell)#Using_an_...
Second of all, it's bad because it runs through the list xs twice. Instead, they should use the partition function to get both the left-hand and right-hand sides at once.
And thirdly, it's inefficient by use of the (++) operator, which takes time proportional to the length of the left-hand list. A better (out-of-place) implementation of quicksort would use an accumulator so this wouldn't happen (and actually, this means that we should do the partitioning ourselves rather than using the partition function as mentioned above).
Here is an example that makes these improvements with quicksort [1].
[1] http://en.literateprograms.org/Quicksort_(Haskell)#Using_an_...
This is a typical pitfall of many FP code chuck examples. trading efficiency for superficial succinctness.
Eh, a lot of FP is used for teaching. I don't think this example is as much about succinctness as it is about clarity. The idea of partitioning on a pivot and recursing is definitely key to quicksort, if not its totality.
It indeeds looks a bit magical, like the infamous fibonacci example that uses lazy lists.
I don't think that list comprehensions are very idiomatic in Haskell, one would probably use Data.List.partition instead:
I don't think that list comprehensions are very idiomatic in Haskell, one would probably use Data.List.partition instead:
qsort [] = []
qsort (p:xs) = low ++ [p] ++ high
where (low, high) = partition (< p) xsI think you are missing the recursive calls ;).
Indeed!
Hey, who said that static typing caught all bugs ? :)
Hey, who said that static typing caught all bugs ? :)
Sure it does, you just need Quickcheck property based testing:
forall $ \xs -> sorted (qsort xs :: [Int])It's fairly natural "teaching code". I'd say it's a concise example of one of the ideas behind quicksort, though, as many are quick to point out, it ignores a lot of other very important aspects.
That said, it's not very good practicing Haskell code. We can look at the Data.List module from the base package to see some optimized list operations with a well-designed module API [0] though some of the optimization methods will be non-obvious and kludgy looking. Data.Complex shows some fairly natural data structure work [1] if you factor out the CPP headers which are there so it'll compile on both GHC and Hugs while using all of the newest GHC features.
[0] http://hackage.haskell.org/package/base-4.6.0.1/docs/src/Dat...
[1] http://hackage.haskell.org/package/base-4.6.0.1/docs/src/Dat...
That said, it's not very good practicing Haskell code. We can look at the Data.List module from the base package to see some optimized list operations with a well-designed module API [0] though some of the optimization methods will be non-obvious and kludgy looking. Data.Complex shows some fairly natural data structure work [1] if you factor out the CPP headers which are there so it'll compile on both GHC and Hugs while using all of the newest GHC features.
[0] http://hackage.haskell.org/package/base-4.6.0.1/docs/src/Dat...
[1] http://hackage.haskell.org/package/base-4.6.0.1/docs/src/Dat...
List comprehensions actually don't tend to get used very frequently, so I would not point to that as a good example of what Haskell code looks like. It's hard to point to one example, as different parts of the code look very different. That is, pure functions look very different from monadic functions, which both look very different from datatype declarations and typeclass instances. I would recommend poking around a whole codebase, one example of which can be found here: https://github.com/xmonad/xmonad
This post may also be useful: http://blog.ezyang.com/2011/11/how-to-read-haskell/
This post may also be useful: http://blog.ezyang.com/2011/11/how-to-read-haskell/
It's a good example of how quickly you can express an algorithm, but in terms or readability, some additional whitespace, and possibly some better naming would make it nicer.
Once you understand what bits like "p:xs", "++" and "[x|y,z]" mean though, it will make much more sense.
Once you understand what bits like "p:xs", "++" and "[x|y,z]" mean though, it will make much more sense.
As good as sort is a good example of real world problem.
It does give you a broad idea, but does not in any way show what a complex program looks like.
It does give you a broad idea, but does not in any way show what a complex program looks like.
Resource leaks in Haskell perhaps a bit trickier to track than in other languages and I would appreciate hearing more about the issue you were experiencing and how you solved the problem. In many blog posts there have been warnings against long running Haskell processes but you guys seem to have fairly successful with it.
Also, the problems you were experiencing with Cabal might be fixed in newer versions with the sandboxes feature which is now built-in with later versions of Cabal.