Caml Trading: Experiences in Functional Programming on Wall Street(janestreetcapital.com)
janestreetcapital.com
Caml Trading: Experiences in Functional Programming on Wall Street
http://www.janestreetcapital.com/minsky_weeks-jfp_18.pdf
3 comments
> even though the compiler is really simple, it does a fantastic job at optimization.
The paper also refers to OCaml's simple compiler and optimizer. This is surprising, considering OCaml's fast benchmark showings and functional languages' reputation for slow, memory intensive code. Is the OCaml team researching more advanced optimizations or is "worst case 50% performance of C code" considered good enough (compared to the tradeoff of a more complex implementation)?
The paper also refers to OCaml's simple compiler and optimizer. This is surprising, considering OCaml's fast benchmark showings and functional languages' reputation for slow, memory intensive code. Is the OCaml team researching more advanced optimizations or is "worst case 50% performance of C code" considered good enough (compared to the tradeoff of a more complex implementation)?
You know how it is with benchmarks...it can vary so much. I'm not sure how much work is done these days on the OCaml compiler. I recall there being a big debate a few years back about it, a lot of people cried foul that OCaml benchmarks were close to C/C++ speeds because of the heavy use of OCaml's imperative constructs (rather than writing functional style implementations of the code).
In my experience with OCaml, you can write efficient code that approaches C/C++ speeds if you know how functional language compilers work. The biggest performance killer in functional languages is efficiently implementing closures on a stack based machine (i.e. funarg problem). OCaml is smart in that it implements activation records on the stack if it can tell that a function won't return a closure. If you're writing in a heavily functional style, you'll inherently create a lot of closures and lots of heap-based activation records (i.e. less efficient). At the same time, the compiler can do a lot of interesting optimizations that would be difficult in an imperative language. Pretty much every OCaml book including the official manual describes all the various places where you can have performance penalties and what OCaml is doing underneath the hood. Also like any half decent functional compiler, OCaml performs a lot of inlining which results in huge performance gains.
One thing to note, a big part of the reason why OCaml as well as other ML languages and Haskell have such great performance compared to other functional languages is because of very strict and static typing. For example, one of the interesting restrictions of the OCaml type system when dealing with objects is that you cannot downcast an object once it has been upcast (i.e. like a C++ dynamic_cast). Though this restriction is annoying, the generated code never has to do any runtime type checking - everything is statically verified!
In my experience with OCaml, you can write efficient code that approaches C/C++ speeds if you know how functional language compilers work. The biggest performance killer in functional languages is efficiently implementing closures on a stack based machine (i.e. funarg problem). OCaml is smart in that it implements activation records on the stack if it can tell that a function won't return a closure. If you're writing in a heavily functional style, you'll inherently create a lot of closures and lots of heap-based activation records (i.e. less efficient). At the same time, the compiler can do a lot of interesting optimizations that would be difficult in an imperative language. Pretty much every OCaml book including the official manual describes all the various places where you can have performance penalties and what OCaml is doing underneath the hood. Also like any half decent functional compiler, OCaml performs a lot of inlining which results in huge performance gains.
One thing to note, a big part of the reason why OCaml as well as other ML languages and Haskell have such great performance compared to other functional languages is because of very strict and static typing. For example, one of the interesting restrictions of the OCaml type system when dealing with objects is that you cannot downcast an object once it has been upcast (i.e. like a C++ dynamic_cast). Though this restriction is annoying, the generated code never has to do any runtime type checking - everything is statically verified!
They are a HN favorite but do any HN'ers work there? If yes what's your experience?
Seems like Yaron Minsky (the first author of the paper) has an account here [1]. Not active though.
[1] news.ycombinator.com/user?id=yminsky
[1] news.ycombinator.com/user?id=yminsky
Even better, are there any other trading firms using functional languages?
Some more info: http://quant.stackexchange.com/questions/303/is-f-used-in-tr...
Also, if you want to consider Scala: http://quant.stackexchange.com/questions/298/is-scala-used-i...
Also, if you want to consider Scala: http://quant.stackexchange.com/questions/298/is-scala-used-i...
At Bloomberg we use OCaml and Haskell for certain things (at least on my team). There are lots of people who are into functional programming and experiment with new tools/languages/etc for a variety of tasks.
Standard Chartered use Haskell. Don Stewart, Neil Mitchell and Lennart Augustsson all work there.
Tsuru Capital do FP and have advertised jobs on the haskell-cafe mailing list before.
Tsuru Capital do FP and have advertised jobs on the haskell-cafe mailing list before.
Tsuru Capital claims to be using Haskell almost exclusively.
http://www.tsurucapital.com/en/
http://www.tsurucapital.com/en/
I work at Tsuru Capital. We use Haskell almost exclusively, including trading directly on the market.
Your website mentions low-latency-execution; how well have you found Haskell can cope with this? Have you run into problems with GC pauses etc? From my experience, major GC collection can be quite large and can't really be avoided...
Haskell can cope well enough. We don't focus on ultra-high frequency trading so much, though we'll take any latency advantages we can get. We focus more on high-level trading strategies. GC pauses are not really an issue. Our system is tuned to avoid doing GC at inopportune times.
I'm curious if you benchmarked Haskell implementations against any C or C++ implementations (especially for market connectivity). Any hint about magnitude of difference?
Could you give some details on how you tweaked the GC?
Did you evaluate OCaml vs Haskell? I'm curious to know what criteria would lead trading firms to consider non-traditional languages yet choose different functional languages?
We use F# and Scala at our fund.
I run technology there.
Feel free to fire me off an email if you'd like to know more.
I run technology there.
Feel free to fire me off an email if you'd like to know more.
I'm pretty sure that Credit Suisse is using F# (a derivative of OCaml) extensively.
Also, I've seen quite a few posts on Don Syme's blog about other F# job openings in the financial industry.
Take a look here:
http://www.haskell.org/haskellwiki/Haskell_in_industry
There are a number of banks mentioned such as Bank of America, Barclays Capital and Cedit Suisse. A company called Allston Trading is apparently using Haskell to do some HFT too.
There are a number of banks mentioned such as Bank of America, Barclays Capital and Cedit Suisse. A company called Allston Trading is apparently using Haskell to do some HFT too.
From what I heard, Credit Suisse has replaced most Haskell with F# now.
The Credit trading technology teams at Citi (London & New York) are using Clojure.
EDIT: fixed spelling.
EDIT: fixed spelling.
I know Barclays uses Haskell
I always notice those at Jane Street rip on the OO aspects of OCaml. The place I found classes/objects to come in handy are when you write extensible software frameworks...though I agree, usually you can do without. However, it is worth taking a look at how OCaml approaches objects...quite different from what you would find in your typical C++/Java/C# style language.
The lack of concurrent garbage collector is really becoming a limitation with the trend towards higher number of cores on a single die. However, because the GC and compiler are so simple, OCaml is one of those languages where you can look at object code and understand what the compiler is doing. What's even more interesting is that even though the compiler is really simple, it does a fantastic job at optimization. The creator of the language, Xavier Leroy, is often quoted as saying that "worst case 50% performance of C code".
Oh one other thing...for those that have not tried a functional language, I think OCaml gives the best introduction. You can code imperatively with it if you want, but you'll ween yourself off of it in time in exchange for more declarative, functional techniques. Oh and you won't look back! ;)