OCaml: Polymorphism(haifengl.wordpress.com)
haifengl.wordpress.com
OCaml: Polymorphism
http://haifengl.wordpress.com/2014/07/08/ocaml-polymorphism/
4 comments
I'm quite happy to see that every week one or two quality OCaml articles are posted on HN and lobste.rs. OCaml doesn't have the general appeal of JavaScript, the innovation factor of Rust, the get-shit-done reputation of Go, but it deserves to be more widely known.
It doesn't have the get-shit-done reputation of Go, but it should. I personally find that OCaml is a very pragmatic language.
What's the story for building a little HTTP server or command-line tool ? Go has a great batteries-included standard library.
> building a little HTTP server
OCaml doesn't, but there are good stdlib replacements, and its package manager is really nice.
opam install cohttp
opam install cmdliner
OCaml doesn't, but there are good stdlib replacements, and its package manager is really nice.
I tried to use Opam on Debian Unstable a few months ago. It was 1.1.0.
When requested to install a second package with overlapping dependencies as a previous package, it would try reinstalling all dependencies, including previously installed ones, and would fail.
That is, in my book, an epic fail for a package manager, let alone a supposedly >1.0 version (if that even means anything today). I'm still annoyed about that. Searching online found recommendations to get the latest version from Github.
1.2.0 has since been released and included in Debian. I keep meaning to give it another go, but I must admit being wary after that first experience.
When requested to install a second package with overlapping dependencies as a previous package, it would try reinstalling all dependencies, including previously installed ones, and would fail.
That is, in my book, an epic fail for a package manager, let alone a supposedly >1.0 version (if that even means anything today). I'm still annoyed about that. Searching online found recommendations to get the latest version from Github.
1.2.0 has since been released and included in Debian. I keep meaning to give it another go, but I must admit being wary after that first experience.
> When requested to install a second package with overlapping dependencies as a previous package, it would try reinstalling all dependencies
This was unfortunately a mistake made by upstream Debian. Rather than reimplement the constraint solver badly, we use the Aspcud external solver. An upgrade of Aspcud changed its command-line interface, and OPAM 1.1.0 wasn't upgraded to OPAM 1.1.1 at the same time (which detected the new version of Aspcud and worked with it).
A similar issue (to do with the libdose interface) has been in the Ubuntu package ages and not addressed by upstream despite a detailed bug report: https://bugs.launchpad.net/ubuntu/+source/opam/+bug/1401346
> Searching online found recommendations to get the latest version from Github
What else could we recommend?
It's really frustrating to know that a source-compiled OPAM works fine, but that the packaged binary versions are broken and our only recourse is to wait six months for the next OS release of Ubuntu. On the bright side, the OPAM 1.2.x series has been very stable and is now making its way upstream, so this is hopefully a temporary issue.
This was unfortunately a mistake made by upstream Debian. Rather than reimplement the constraint solver badly, we use the Aspcud external solver. An upgrade of Aspcud changed its command-line interface, and OPAM 1.1.0 wasn't upgraded to OPAM 1.1.1 at the same time (which detected the new version of Aspcud and worked with it).
A similar issue (to do with the libdose interface) has been in the Ubuntu package ages and not addressed by upstream despite a detailed bug report: https://bugs.launchpad.net/ubuntu/+source/opam/+bug/1401346
> Searching online found recommendations to get the latest version from Github
What else could we recommend?
It's really frustrating to know that a source-compiled OPAM works fine, but that the packaged binary versions are broken and our only recourse is to wait six months for the next OS release of Ubuntu. On the bright side, the OPAM 1.2.x series has been very stable and is now making its way upstream, so this is hopefully a temporary issue.
That explains a lot. I thought opam was behaving better now, than I recall it did at some random point in the past.
I'm really grateful to everyone doing all this work. This is a typical example of the kind of "excitement" mixing distro package managers and language package managers can lead to. The only recourse is to file a bug, maybe provide a backport if feasible, and move on -- and have everyone (package maintainers, distros) be a little wiser, and (hopefully) everything working on the next release.
Anyway, I see I have opam in my ~/bin -- so clearly I've compiled it myself at some point. No wonder, as I'm still running stable on this laptop:
https://packages.debian.org/search?keywords=opam&searchon=na...
Maybe there should be a note in the wiki about it, although it should be resolved when jessie is out?:
https://wiki.debian.org/OCaml
(Yeah, I know, it's a wiki. I could just add it. I don't feel quite confident enough to go and clutter such a short little page, though. Different if there's a page where "someone is wrong on the internet".)
I'm really grateful to everyone doing all this work. This is a typical example of the kind of "excitement" mixing distro package managers and language package managers can lead to. The only recourse is to file a bug, maybe provide a backport if feasible, and move on -- and have everyone (package maintainers, distros) be a little wiser, and (hopefully) everything working on the next release.
Anyway, I see I have opam in my ~/bin -- so clearly I've compiled it myself at some point. No wonder, as I'm still running stable on this laptop:
https://packages.debian.org/search?keywords=opam&searchon=na...
Maybe there should be a note in the wiki about it, although it should be resolved when jessie is out?:
https://wiki.debian.org/OCaml
(Yeah, I know, it's a wiki. I could just add it. I don't feel quite confident enough to go and clutter such a short little page, though. Different if there's a page where "someone is wrong on the internet".)
Thanks for your work!
In that particular timeframe (Sept/Oct 2014), the Debian maintainer just hadn't uploaded a newer version of the package: https://tracker.debian.org/pkg/opam
It's a pity, because I think Debian's Experimental section would've been ideal for this.
I've been using Debian long enough that I'm a bit ashamed not to be able to package things on my own yet...
In that particular timeframe (Sept/Oct 2014), the Debian maintainer just hadn't uploaded a newer version of the package: https://tracker.debian.org/pkg/opam
It's a pity, because I think Debian's Experimental section would've been ideal for this.
I've been using Debian long enough that I'm a bit ashamed not to be able to package things on my own yet...
I agree, opam was rough a while ago - not sure about now. Fwiw both erlang and haskell have also been rather painful on Debian. As far as I know, ruby still is, to a certain extent. Go is sort of in between by virtue of building mostly stand-alone binaries, if you can get the program to compile.
Haskell's cabal has gotten better; it's still easier to run xmonad/mobar from Debian packages, but at least cabal with packages/cache in ~/.cabal kinda-sorta works.
It seems the path towards package managers that both work well and work well with stable/distro packages is long, ardous and filled with repeating the mistakes of others. Npm seems to have learned a bit from python - if you stay away from -g(lobal), it tends to mostly just work (not sure about c/c++ dependencies - for better or worse a lot of npm packages are mostly pure js).
Python with virtualenv works very well, except for a few hairy c-heavy packages (pygame, sci-py) - but running "sudo pip install ..." is a recipe for disaster (or a mongrel, unpatchable mess). Keep to virtualenvs and you can stay mostly sane (or use the isolation provided by buildout -- preferably in concert with virtualenv).
Haskell's cabal has gotten better; it's still easier to run xmonad/mobar from Debian packages, but at least cabal with packages/cache in ~/.cabal kinda-sorta works.
It seems the path towards package managers that both work well and work well with stable/distro packages is long, ardous and filled with repeating the mistakes of others. Npm seems to have learned a bit from python - if you stay away from -g(lobal), it tends to mostly just work (not sure about c/c++ dependencies - for better or worse a lot of npm packages are mostly pure js).
Python with virtualenv works very well, except for a few hairy c-heavy packages (pygame, sci-py) - but running "sudo pip install ..." is a recipe for disaster (or a mongrel, unpatchable mess). Keep to virtualenvs and you can stay mostly sane (or use the isolation provided by buildout -- preferably in concert with virtualenv).
I'm not sure we're talking about the same thing. Are you lamenting the difficult interface between distro package managers and language package managers? Or lamenting that language package managers haven't learned the lessons of previous package managers, distro and language alike?
I was lamenting that Opam 1.1 wasn't dealing with a trivial package installation situation. I mentioned Debian to indicate that was the source of the version I was using, implying that there may have already been a newer version out there that I could've been using, but then explicitly saying that 1.1 should've dealt with the situation in the first place.
FWIW, I definitely agree with the terrible situation of language package managers in general. I use Python sporadically, and I'm always scared of running pip. Virtualenv I'm very unhappy about because, while it definitely fulfills it's task marvelously, I think it shouldn't even have to exist in the first place! Why doesn't core Python deal with the problem itself, searching for dependencies in the main script's subdirs, the user's libdir, and lastly the system libdir? Admittedly, I don't know the intricacies of the Python ecosystem that prevent that.
I was lamenting that Opam 1.1 wasn't dealing with a trivial package installation situation. I mentioned Debian to indicate that was the source of the version I was using, implying that there may have already been a newer version out there that I could've been using, but then explicitly saying that 1.1 should've dealt with the situation in the first place.
FWIW, I definitely agree with the terrible situation of language package managers in general. I use Python sporadically, and I'm always scared of running pip. Virtualenv I'm very unhappy about because, while it definitely fulfills it's task marvelously, I think it shouldn't even have to exist in the first place! Why doesn't core Python deal with the problem itself, searching for dependencies in the main script's subdirs, the user's libdir, and lastly the system libdir? Admittedly, I don't know the intricacies of the Python ecosystem that prevent that.
> I was lamenting that Opam 1.1 wasn't dealing with a trivial package installation situation.
Well, it's all connected. A package will need a certain version of ocaml itself, it's standard-lib (or alternative) other libs it needs. These will all depend on various c libraries to a certain degree. So if opam started re-downloading a lot of dependencies that should be available, that sounds like either a) some part needed a newer version, or needed to recompile with different flags, or b) opam didn't find the dependencies installed by dpkg.
Python and Debian go pretty well together, probably partly because a few core utilities use python. So with some creative use of "apt-get build-dep" you can usually get the needed c libraries that things depend on, and so have a better time installing things that need c-code in virtualenvs.
I'm not sure how you think linking to (possibly rapidly moving) c-libraries from a language that runs across a wide range of operating systems and versions should work, if not with something like virtualenv (or buildout). There will, I think, always be a need for packages that are moving faster than stable distributions can/should keep up with -- and there needs to be a sane way to deal with that.
With c code, you can stuff things in /usr/local -- but keeping that in sync as you patch the main os (and do dist-upgrades) and as you need to patch the various libraries/dependencies... is challenging. And that's just for c. When you put another layer on top (ruby, python etc) -- it doesn't really get any easier.
One option is to go the way of bsd ports/gentoo -- but there are at least two issues/trade-offs with recompiling everything from source: 1) if you're mixing and matching versions and compile-options, you quickly end up with unique binaries. Which is to say, binaries that no-one else have tested before you. Granted any bugs you encounter will be in the program and/or the compiler -- but you'll have the joy of finding them. In production, most like. 2) It really doesn't make much sense to recompile "all the things" if the goal is to end up with the same binaries anyway.
Well, it's all connected. A package will need a certain version of ocaml itself, it's standard-lib (or alternative) other libs it needs. These will all depend on various c libraries to a certain degree. So if opam started re-downloading a lot of dependencies that should be available, that sounds like either a) some part needed a newer version, or needed to recompile with different flags, or b) opam didn't find the dependencies installed by dpkg.
Python and Debian go pretty well together, probably partly because a few core utilities use python. So with some creative use of "apt-get build-dep" you can usually get the needed c libraries that things depend on, and so have a better time installing things that need c-code in virtualenvs.
I'm not sure how you think linking to (possibly rapidly moving) c-libraries from a language that runs across a wide range of operating systems and versions should work, if not with something like virtualenv (or buildout). There will, I think, always be a need for packages that are moving faster than stable distributions can/should keep up with -- and there needs to be a sane way to deal with that.
With c code, you can stuff things in /usr/local -- but keeping that in sync as you patch the main os (and do dist-upgrades) and as you need to patch the various libraries/dependencies... is challenging. And that's just for c. When you put another layer on top (ruby, python etc) -- it doesn't really get any easier.
One option is to go the way of bsd ports/gentoo -- but there are at least two issues/trade-offs with recompiling everything from source: 1) if you're mixing and matching versions and compile-options, you quickly end up with unique binaries. Which is to say, binaries that no-one else have tested before you. Granted any bugs you encounter will be in the program and/or the compiler -- but you'll have the joy of finding them. In production, most like. 2) It really doesn't make much sense to recompile "all the things" if the goal is to end up with the same binaries anyway.
FWIW, with opam 1.2.0 installed on Debian jessie (no system ocaml), after doing "opam update;opam upgrade" -- cmdline installed fine.
cohttp wouldn't install, due to a dependency, conduit, not installing properly[1] -- possibly due to the old libssl-version (with backported fixes) in Debian stable. I do have a sid and a jessie chroot -- but I've yet to try running it there -- part of the point of chroot is automounting of the home folder in the chroot(s) -- and that'd give me a .opam that wouldn't work under stable; or I'd have to jocky around with altearnative .opam folders...
[1] An error along the lines of:
File "lib/conduit_async_ssl.ml", line 28, characters 18-108: # Error: This expression has type # Ssl.Connection.t Deferred.Or_error.t = # Ssl.Connection.t Core_kernel.Std.Or_error.t Deferred.t # but an expression was expected of type unit Deferred.t # Type # Ssl.Connection.t Core_kernel.Std.Or_error.t = # (Ssl.Connection.t, Error.t) Result.t # is not compatible with type unit # Command exited with code 2. ### stderr ###
cohttp wouldn't install, due to a dependency, conduit, not installing properly[1] -- possibly due to the old libssl-version (with backported fixes) in Debian stable. I do have a sid and a jessie chroot -- but I've yet to try running it there -- part of the point of chroot is automounting of the home folder in the chroot(s) -- and that'd give me a .opam that wouldn't work under stable; or I'd have to jocky around with altearnative .opam folders...
[1] An error along the lines of:
File "lib/conduit_async_ssl.ml", line 28, characters 18-108: # Error: This expression has type # Ssl.Connection.t Deferred.Or_error.t = # Ssl.Connection.t Core_kernel.Std.Or_error.t Deferred.t # but an expression was expected of type unit Deferred.t # Type # Ssl.Connection.t Core_kernel.Std.Or_error.t = # (Ssl.Connection.t, Error.t) Result.t # is not compatible with type unit # Command exited with code 2. ### stderr ###
If anyone would like to do things with Cmdliner, then there are few libs that would be great to have as stand-alone executables (it's part of the MirageOS Pioneer Projects) [1].
https://github.com/mirage/mirage-www/wiki/Pioneer-Projects#c...
https://github.com/mirage/mirage-www/wiki/Pioneer-Projects#c...
nice list! was this always on the mirageos site? i remember spending some time poking around specifically looking for something like this, and coming up blank. (suggestion: link to it from the "contributing to mirage" page as well)
Thanks for the pointer to `cmdliner`. I've built a handful of command line tools using Command from the Core library (as described by Real World OCaml [1]), but I always have to have the book open to use it. This looks very nice and it looks like it might fix an annoyance I have with Core's `Command`, which is that it won't parse flags without spaces (i.e., `-r2` failes to parse as equivalent to `-r 2`).
[1] https://realworldocaml.org/v1/en/html/command-line-parsing.h...
[1] https://realworldocaml.org/v1/en/html/command-line-parsing.h...
cmdliner's applicative-functor interface has a bit of a learning curve but the end result is some really clean programs. It can also generate manpages!
The Ocaml standard library is pretty shitty (you certaily want to use one of the many stdlib replacements) but Opam is pretty neat for 3rd party libraries (albeit some areas are a bit fragmented for historical reasons).
> The Ocaml standard library is pretty shitty
Oh come on, it's not that bad, it's just minimal. Usually I can get pretty far without having to reach for something like Batteries. That's a good thing, especially when writing reusable libraries where you don't necessarily want to pull in an entire stdlib replacement.
Oh come on, it's not that bad, it's just minimal. Usually I can get pretty far without having to reach for something like Batteries. That's a good thing, especially when writing reusable libraries where you don't necessarily want to pull in an entire stdlib replacement.
I'm really hoping as I've explored it more and more that the lightweight standard lib is a boon more than a weakness... but to achieve this there needs to be major force to drive useful functionality down into canonical library packages.
> "there needs to be major force to drive useful functionality down into canonical library packages"
This tends to happen over time, as the community naturally converges on certain libraries, but it's rather opaque for a new user as to what those are. In some cases, you end up with two options and then life is a little more complicated (cf. async and lwt).
This tends to happen over time, as the community naturally converges on certain libraries, but it's rather opaque for a new user as to what those are. In some cases, you end up with two options and then life is a little more complicated (cf. async and lwt).
Oh, absolutely!
It's funny because ml languages are a lot less alien to the mainstream than before.
Things people are used to:
Things people are used to:
- `let` keyword (rust, ES6)
- pattern matching (generalization of ES6 destructuring)
- function first, HOF combinators (ES5 functional literature)
- tuples (python)
You might find the intro to Real World OCaml interesting as these ideas have percolated out from languages such as OCaml. It's great if that helps lower the barrier for people to try it out.
https://realworldocaml.org/v1/en/html/prologue.html#why-ocam...
https://realworldocaml.org/v1/en/html/prologue.html#why-ocam...
Thanks for bring Rust into the topic. I know that it is very new. But still wonder if some real life projects are using it (except rustc).
Mozilla Servo is a browser engine written in Rust; skylight.io is a Ruby Gem written in Rust. Then there are several smaller projects for game engines, web server stuff, cryptography, etc.
The Rust language itself has a noticeably heavy ML influence.
All of them.
But the one I like the most is Parametric polymorphism because it tends to make algorithms generically available to anything.
And the one I like the least is subtype polymorphism due to it's reliance on inheritance with all it's attendant problems. The exception being for the sub-genre of "interface" polymorphism. Which doesn't really exist in C++. However when it does exist such as in languages like Go, Java, or C# then it fixes almost all of the problems that tradional subtype polymorphism has by taking inheritance completely out of the picture.
But the one I like the most is Parametric polymorphism because it tends to make algorithms generically available to anything.
And the one I like the least is subtype polymorphism due to it's reliance on inheritance with all it's attendant problems. The exception being for the sub-genre of "interface" polymorphism. Which doesn't really exist in C++. However when it does exist such as in languages like Go, Java, or C# then it fixes almost all of the problems that tradional subtype polymorphism has by taking inheritance completely out of the picture.
OCaml models "interface" polymorphism using an extension to parametric polymorphism called row polymorphism (https://realworldocaml.org/v1/en/html/objects.html), which lets you avoid subtyping even for interface types.
Agree. I like most parametric polymorphism too. The drawbacks are:
1. In C++, the compile error messages with templates are so hard to navigate. 2. In OCaml, sometimes we need pay attentions on the performance.
1. In C++, the compile error messages with templates are so hard to navigate. 2. In OCaml, sometimes we need pay attentions on the performance.
parametric polymorphism also give you "theorems for free" if you choose the right places to put your abstractions.
Another kind of polymorphism that's way underappreciated is what Qt calls "static polymorphism" (http://doc.qt.digia.com/qq/qq13-apis.html#staticpolymorphism).
This is just a fancy word for naming stuff consistently instead of trying to shoehorn constructs to have some kind of formal language-aware relationship via abstract methods, overloaded methods, etc., which sometimes just increases coupling/complexity with no real benefit.
This is just a fancy word for naming stuff consistently instead of trying to shoehorn constructs to have some kind of formal language-aware relationship via abstract methods, overloaded methods, etc., which sometimes just increases coupling/complexity with no real benefit.
This happens pretty naturally in OCaml actually. It arises from the structural subtyping relationship on module signatures. You don't really shoehorn anything at all...
Interesting. It is more a design pattern rather than a language thing. But it is always good to design a consistent API across similar but not related classes.
Agreed, with all languages I'm aware of this is purely a design concern, outside the scope of language specification. However I'm currently working on a library that relies heavily on this so-called static polymorphism, so you can imagine what a pain it is whenever I do a major refactor. Without formal relationships between classes, methods, etc., it's much more of a manual and error-prone process, as far as introducing inconsistencies in the API. This has led me to ponder some language-level (or even just IDE-level) ways to specify loose relationships between disparate constructs that would make such iterative API development easier. No brilliant insights yet though. :)
Structural subtyping are mostly close to static polymorphism. After all, we just don't like inheritance, not dynamic polymorphism. Besides, interface and trait are not really inheritance although closely related.
It would be cool to see a "layered" language. By this, I mean a language that has no typing at its lowest level, but you can add it as a library on top of the base level. Then you can add stuff like automatic resolution of calls, etc., as another abstraction level. Of course, you could also add other rules instead of typing. For example, you could pepper your programs with arbitrary proofs, etc., and this could be directed by yet another library on top of your program.
Perhaps I'm dreaming. But it sounds like useful to me, as there are so many languages that are almost similar, and yet are slightly different in the way they handle typing and things like polymorphism. It would seem to me that those languages could all share a common base.
Perhaps I'm dreaming. But it sounds like useful to me, as there are so many languages that are almost similar, and yet are slightly different in the way they handle typing and things like polymorphism. It would seem to me that those languages could all share a common base.
Perl is dynamic enough to support this.
See http://search.cpan.org/~ether/Moose-2.1403/lib/Moose/Manual/... and http://search.cpan.org/~ether/Moose-2.1403/lib/Moose/Manual....
Competing type systems, competing Object systems, and enough internal run-time rewriting to do whatever it is you want.
See also http://search.cpan.org/~smueller/Filter-Simple-0.91/lib/Filt... for a simple rewriting filter, and the Acme namespace for all kinds of fancy experiments.
See http://search.cpan.org/~ether/Moose-2.1403/lib/Moose/Manual/... and http://search.cpan.org/~ether/Moose-2.1403/lib/Moose/Manual....
Competing type systems, competing Object systems, and enough internal run-time rewriting to do whatever it is you want.
See also http://search.cpan.org/~smueller/Filter-Simple-0.91/lib/Filt... for a simple rewriting filter, and the Acme namespace for all kinds of fancy experiments.
Noether is a "layered" language like you describe, as it's actually a set of seven languages arranged in a hierarchy where each layer adds a feature to the one below it, with the tradeoff being that as you add features you lessen your ability to reason about aspects of your code. See https://github.com/noether-lang/noether/blob/master/doc/pres... and skip down the slide "Sublanguages and their symmetries".
Optionally typed programming languages are still a big research area and we are just now starting to see some of that research go more mainstream, with things such as Typescript.
That said, I don't expect to see a language that does all the things you listed at the same time, since type systems are all about tradeoffs between expressivity and complexity and there is an upper bound in how much complexity a programmer can handle. There are also a lot of other tradeoffs that you need to think about: do unproved assertions get converted to runtime checks or to compile errors? Can you implement that efficiently? How will type inference work? Is it OK to design an unsound type system? etc etc.
I can give you some pointers if you want to look for something more specific.
That said, I don't expect to see a language that does all the things you listed at the same time, since type systems are all about tradeoffs between expressivity and complexity and there is an upper bound in how much complexity a programmer can handle. There are also a lot of other tradeoffs that you need to think about: do unproved assertions get converted to runtime checks or to compile errors? Can you implement that efficiently? How will type inference work? Is it OK to design an unsound type system? etc etc.
I can give you some pointers if you want to look for something more specific.
I am not sure if typing can be added as a library. In terms of typed lambda calculus vs untyped lambda calculus, can we do this way?
But the idea of "layered" language is nice. In Scala, many features are not really from language itself, but from libraries.
But the idea of "layered" language is nice. In Scala, many features are not really from language itself, but from libraries.