Unix system programming in OCaml(ocamlunix.forge.ocamlcore.org)
ocamlunix.forge.ocamlcore.org
Unix system programming in OCaml
http://ocamlunix.forge.ocamlcore.org/ocamlunix.html
2 comments
> 2. No true parallelism due to runtime limitations (single runtime lock)
I've thought about this, and it's not a big deal in the multicore/SMP sense. For I/O concurrency, there is LWT or Async. It does fall short for "small-scale" parallelism though, eg map-reduce over an array, multiplying big-but-not-huge matrices, etc. At least OCaml doesn't pretend to support threading, unlike most other GIL-bound languages.
For high-concurrency or large parallel problems, multiple processes are a better option since that naturally scales beyond a single machine. OCaml's memory footprint is easily an order of magnitude smaller than eg, Ruby's, so I could run 10x the number of workers using the same resources. And then they'd complete tasks faster, thanks to excellent single-thread performance.
In some sense, OCaml indirectly enables the best of the Unix tool tradition. The maintainers haven't tried to throw the kitchen sink at the language, which practically encourages composable designs where distributed/parallel systems are concerned.
I've thought about this, and it's not a big deal in the multicore/SMP sense. For I/O concurrency, there is LWT or Async. It does fall short for "small-scale" parallelism though, eg map-reduce over an array, multiplying big-but-not-huge matrices, etc. At least OCaml doesn't pretend to support threading, unlike most other GIL-bound languages.
For high-concurrency or large parallel problems, multiple processes are a better option since that naturally scales beyond a single machine. OCaml's memory footprint is easily an order of magnitude smaller than eg, Ruby's, so I could run 10x the number of workers using the same resources. And then they'd complete tasks faster, thanks to excellent single-thread performance.
In some sense, OCaml indirectly enables the best of the Unix tool tradition. The maintainers haven't tried to throw the kitchen sink at the language, which practically encourages composable designs where distributed/parallel systems are concerned.
> At least OCaml doesn't pretend to support threading, unlike most other GIL-bound languages.
OCaml does support threading, you don't need LWT of Async for I/O concurrency.
http://caml.inria.fr/pub/docs/manual-ocaml-4.00/manual039.ht...
OCaml does support threading, you don't need LWT of Async for I/O concurrency.
http://caml.inria.fr/pub/docs/manual-ocaml-4.00/manual039.ht...
And the community isn't as vibrant as the communities of other functional languages such as Clojure or Haskell
...good luck having a trader with very basic coding knowledge "review" Haskell or Clojure code (the process JS is "famous" for) - even if they understand the value of it and are willing to learn it, it will always be a huge mental effort handling monads or reading code "inside our" lisp-style. And coders will not resist doing "smart stuff" in languages like Clojure or Haskell, which will make everything worse. Otoh, smart business folks with some technical background can digest oop and functional programming in a language with infix operators quite easily if they know they have a lot to gain from doing it.
Without firsthand knowledge of the Jane Street corporate environment it is naive of me to think that allowing engineering to pick the tools which best let them get their work done need not be incompatible with the requirements of non-expert review of the source code.
I would think that if the company culture is one where it is expected that your code is going to be reviewed by domain experts, but not necessarily language experts, there would be a strong emphasis placed on producing source code which would convey its meaning without a detailed knowledge of the language spec and runtime system.
I would think that if the company culture is one where it is expected that your code is going to be reviewed by domain experts, but not necessarily language experts, there would be a strong emphasis placed on producing source code which would convey its meaning without a detailed knowledge of the language spec and runtime system.
This is all true. On the other hand, it's worth mentioning the major benefits that Jane Street gets from OCaml:
1. Excellent single-core performance. 2. Code is fast to write and can be quickly adapted, so you get the agility of the typical dynamic languages but the benefits of a static type system. 3. Although the community has few members, the quality of engineers in it is very high. Someone who knows OCaml well is much more likely to be a good hire. 4. Legibility of code. Partners review production code. These are very smart guys, but they're traders-- not quants or programmers by trade.
Writing libraries, if you have the resources, is a small cost. It's heavy upfront, but once you have good libraries, that's no longer an issue. That issue depends on the scope of your project. You probably wouldn't use OCaml for a web project you intended to deliver in one month.
The benefit that I think is most interesting is the 4th, because the reading of code is where most software companies seem to fall down. Of course, this involves style as much as the language itself, but OCaml is, in my experience, the language best optimized for the reader's concerns. Haskell can be similarly legible (and is much terser) but one can also go off the rails with it.
1. Excellent single-core performance. 2. Code is fast to write and can be quickly adapted, so you get the agility of the typical dynamic languages but the benefits of a static type system. 3. Although the community has few members, the quality of engineers in it is very high. Someone who knows OCaml well is much more likely to be a good hire. 4. Legibility of code. Partners review production code. These are very smart guys, but they're traders-- not quants or programmers by trade.
Writing libraries, if you have the resources, is a small cost. It's heavy upfront, but once you have good libraries, that's no longer an issue. That issue depends on the scope of your project. You probably wouldn't use OCaml for a web project you intended to deliver in one month.
The benefit that I think is most interesting is the 4th, because the reading of code is where most software companies seem to fall down. Of course, this involves style as much as the language itself, but OCaml is, in my experience, the language best optimized for the reader's concerns. Haskell can be similarly legible (and is much terser) but one can also go off the rails with it.
What I like most about the Jane Street story is that they weren't afraid to rewrite their system and even went through a half-done abandoned Java rewrite. It shows that if you're working with smart enough people you can break away from the "incremental development reality" and discover the hidden powers of all sorts of technologies that others are afraid to try for fear of transition nightmare.
It's more a question of company priority and customer demands. Try rewriting a few hundred thousand lines of code currently in production at several customers, who all want modifications. Unless you have enough manpower to dedicate a team to maintenance, that's not going to happen. Especially if the competition keeps churning out new features.
...if I think about it, the few companies that did rewrite their codebase and benefited from this were the ones that were their own customers of their code as JS and Twitter both fall in here. You're right, it's probably not worth it if you're selling "software as a product", but when you are basically an saas business or even better, you're doing in-house dev, and everything runs on your servers, you are free to even run simultaneously multiple version of your thing for different tasks.
3. The compiler's error messages lack useful detail (e.g. "syntax error" and a line number!)
4. Stack traces are as likely to be wrong as right
5. Too much boilerplate exception handling (e.g Not_found)
6. Clunky type polymorphism (e.g. Set)
4. Stack traces are as likely to be wrong as right
5. Too much boilerplate exception handling (e.g Not_found)
6. Clunky type polymorphism (e.g. Set)
Claims 3 and 6 lack a true understanding of Ocaml. The compiler's error messages are exceptionally good in most cases.
Syntax errors without detail are a product of the parsing technology: its a minor irritant and worst.
The most difficult problem (IMHO) comes from type inference when a type inferred from context is not the one intended, but a conflict is not discovered until code with the intended typing is found: the error reflects the second location which is confusing because there is no error at that point. Of course this doesn't occur if you actually provide type annotations, so it is merely a cost of being able to omit them.
The excess exception handling is a reasonable comment, however it is easy to wrap the exception throwing code in a variant to enforce local error checking, and in any case this style is a property of the library, not the core language.
The type polymorphism in Ocaml is anything but clunky: for basic stuff you don't even need any type annotations. The thing is that Set us NOT type polymorphism. Its module polymorphic, and that requires explicit binding of an instance. There is a simple enough reason: module functors take non-type arguments, particularly function closures, which are values, which do not have unique signatures, so implicit instantiation is not possible.
Ocaml does have some other serious disadvantages. One is that because of the way the optimiser uses information from compiles on which it depends, there are annoying constraints on build order. In addition, recursion cannot span compilation boundaries which is pretty lame for a functional programming language: even C has no problem with that.
Finally, the political situation is probably the biggest problem. The development team is excellent but limited and their focus is on the type system, rather than libraries. Because the community is modest compared to say C++, many attempts to make extended libraries have failed to gain acceptance, because no one wants to depend on an unresourced third party library and the core INRIA team doesn't have the resources or interest to extend the standard distribution.
Despite these difficulties .. I would never want to go back to writing C++ (OMG .. the pain!). As a language .. Ocaml is light years ahead.
Syntax errors without detail are a product of the parsing technology: its a minor irritant and worst.
The most difficult problem (IMHO) comes from type inference when a type inferred from context is not the one intended, but a conflict is not discovered until code with the intended typing is found: the error reflects the second location which is confusing because there is no error at that point. Of course this doesn't occur if you actually provide type annotations, so it is merely a cost of being able to omit them.
The excess exception handling is a reasonable comment, however it is easy to wrap the exception throwing code in a variant to enforce local error checking, and in any case this style is a property of the library, not the core language.
The type polymorphism in Ocaml is anything but clunky: for basic stuff you don't even need any type annotations. The thing is that Set us NOT type polymorphism. Its module polymorphic, and that requires explicit binding of an instance. There is a simple enough reason: module functors take non-type arguments, particularly function closures, which are values, which do not have unique signatures, so implicit instantiation is not possible.
Ocaml does have some other serious disadvantages. One is that because of the way the optimiser uses information from compiles on which it depends, there are annoying constraints on build order. In addition, recursion cannot span compilation boundaries which is pretty lame for a functional programming language: even C has no problem with that.
Finally, the political situation is probably the biggest problem. The development team is excellent but limited and their focus is on the type system, rather than libraries. Because the community is modest compared to say C++, many attempts to make extended libraries have failed to gain acceptance, because no one wants to depend on an unresourced third party library and the core INRIA team doesn't have the resources or interest to extend the standard distribution.
Despite these difficulties .. I would never want to go back to writing C++ (OMG .. the pain!). As a language .. Ocaml is light years ahead.
> Syntax errors without detail are a product of the parsing technology: its a minor irritant and worst.
You may think so, but it infuriates me many times a day. This is the most basic thing which a compiler should be getting right, and yet they fail.
I agree with you on type inference errors, but my solution is to annotate function parameters, which avoids the most confusing problems.
I don't want to wrap my error handling code, or work around it with other libraries, I just want something usable out of the box. Languages need to be evaluated holistically, and the design of the standard library is a critical part of that, look at the C++ STL.
Yes, I meant module polymorphism. I much prefer the way that F# deals with Sets, Maps, etc.
I think Ocaml has had a good run, but it's starting to look dated, F# has really innovated, it's just a shame that it's tied to .NET.
You may think so, but it infuriates me many times a day. This is the most basic thing which a compiler should be getting right, and yet they fail.
I agree with you on type inference errors, but my solution is to annotate function parameters, which avoids the most confusing problems.
I don't want to wrap my error handling code, or work around it with other libraries, I just want something usable out of the box. Languages need to be evaluated holistically, and the design of the standard library is a critical part of that, look at the C++ STL.
Yes, I meant module polymorphism. I much prefer the way that F# deals with Sets, Maps, etc.
I think Ocaml has had a good run, but it's starting to look dated, F# has really innovated, it's just a shame that it's tied to .NET.
I end up using F# more, as it is way easier to introduce in Windows based environments than OCaml is.
From: http://ocamlunix.forge.ocamlcore.org/ocamlunix.html#htoc7 , an example of a file hierarchy: http://ocamlunix.forge.ocamlcore.org/ocamlunix-image1.png
1. Standard library was so weak that they had to come up with in-house replacement (which is now open sourced).
2. No true parallelism due to runtime limitations (single runtime lock) - http://queue.acm.org/detail.cfm?id=2038036