Tuning Concurrent Clojure(tbray.org)
tbray.org
Tuning Concurrent Clojure
http://www.tbray.org/ongoing/When/200x/2009/12/08/WF-Tuning-Clojure
4 comments
I would say Tim Bray is making an amazing effort to tackle the hard problem of optimizing a language that he has even at this point only passing familiarity. That said, it's great to have such a well known hacker try to put Clojure through it's paces.
Having optimized my fair share of Clojure programs it's certainly possible to get very close to Java speeds. Also in my experience there's often a point of diminishing return. It's often simpler to just write that tight loop in a tiny Java class and be done with it. Just as Python is extensible in C when you need performance, Clojure is extensible in Java.
With the work in the Clojure new branch, the possibility of writing wicked fast Clojure in Clojure is just around the corner.
I work in Python and JavaScript day in and day out and my comparisons follow:
I can say without a doubt, idiomatic Clojure (not heavily optimized) is generally faster than Python, and more elegant IMHO. Once you get used to pretty much everything returning a value, Python can drive you insane. The only thing I generally miss about Python is that's more usefulf/friendly as a command line scripting language. But for building an app, Clojure all the way.
I use JavaScript even more, and Clojure has all the things I like about JavaScript except even more flexible and useful in contexts other than web programming (OpenGL programming for example). I picked up Clojure precisely because I wanted a fast elegant dynamic and concurrency ready programming language.
As the JDK evolves and our laptops start having 4 cores, I think the case for Clojure will only grow.
Having optimized my fair share of Clojure programs it's certainly possible to get very close to Java speeds. Also in my experience there's often a point of diminishing return. It's often simpler to just write that tight loop in a tiny Java class and be done with it. Just as Python is extensible in C when you need performance, Clojure is extensible in Java.
With the work in the Clojure new branch, the possibility of writing wicked fast Clojure in Clojure is just around the corner.
I work in Python and JavaScript day in and day out and my comparisons follow:
I can say without a doubt, idiomatic Clojure (not heavily optimized) is generally faster than Python, and more elegant IMHO. Once you get used to pretty much everything returning a value, Python can drive you insane. The only thing I generally miss about Python is that's more usefulf/friendly as a command line scripting language. But for building an app, Clojure all the way.
I use JavaScript even more, and Clojure has all the things I like about JavaScript except even more flexible and useful in contexts other than web programming (OpenGL programming for example). I picked up Clojure precisely because I wanted a fast elegant dynamic and concurrency ready programming language.
As the JDK evolves and our laptops start having 4 cores, I think the case for Clojure will only grow.
Haskell is the obvious alternative. It is a very elegant functional language that is, in my opinion, executed well. Does it fit into your list of languages that miss the mark. If so, in what ways?
I also really like Clojure at the moment but have programmed in Haskell in the past and found there was much to recommend it. If you are genuinely looking for a good functional language you would be doing yourself a disservice if you didn't take a serious look at Haskell.
I also really like Clojure at the moment but have programmed in Haskell in the past and found there was much to recommend it. If you are genuinely looking for a good functional language you would be doing yourself a disservice if you didn't take a serious look at Haskell.
As much as I like Haskell and appreciate how it expands a programmer's view, it's one of the last languages I'd describe as "feeling smooth like python" ;)
I think Haskell is very good for robust, complex software, but not something you can pick up quickly like Python.
I think Haskell is very good for robust, complex software, but not something you can pick up quickly like Python.
how is haskell the obvious choice? The parent poster talks about clojure & python -- 2 dynamically typed languages. It's going to be a world of difference moving to a statically typed one.
The change in typing is not really much of a jump when you realise that Haskell's type inference means you don't have to declare too many type signatures up front. A lot of type signatures in Haskell code are not technically necessary and are usual there for documentation.
For example, this is perfectly valid Haskell code and doesn't mention a single type:
For example, this is perfectly valid Haskell code and doesn't mention a single type:
factorial n = if n > 0 then n * factorial (n-1) else 1
I hadn't looked at Haskell yet, actually. It seems to slip off my radar.
I'm really enjoying this series. Definitely getting down and gritty with what Clojure is really capable of (and all the other languages he's tried). Of all the articles I've read on concurrent programming and the languages that promise new ways to handle it, I find the Wide Finder series the most pragmatic and concrete.
I wouldn't say that the article gets down to the nitty gritty of what Clojure is capable at all. It's a great series about a high profile hacker having a go at what I think is a great new programming language.
For a down and gritty analysis of Clojure optimization you're better served reading Christophe Grand's posts. Or even my own fast but butt ugly optimization of Uncle Bob's Jarvis March- http://github.com/swannodette/convex-hull/blob/master/convex... where I use macros to inline everything.
His version ran in 25s, mine in about 0.5.
For a down and gritty analysis of Clojure optimization you're better served reading Christophe Grand's posts. Or even my own fast but butt ugly optimization of Uncle Bob's Jarvis March- http://github.com/swannodette/convex-hull/blob/master/convex... where I use macros to inline everything.
His version ran in 25s, mine in about 0.5.
I'd really like to see a Common Lisp solution to the wide finder sometime (in case any lisper is really bored and feels like cranking one out :D )
I am shaking my head in dismay.
Reduce is a loop. Rewriting all of your loop code as reduce won't get you any performance gains. If anything you will lose performance over looping because you are now passing stuff to a function and it is getting boxed.
Why is he splitting all of the lines with a #"\n" regex? there are perfectly good methods for reading something line-by-line in a number of places (in both java and clojure).
I like Tim Bray and anyone who writes about clojure or lisp, but I feel like I'm tripping on acid here.
Reduce is a loop. Rewriting all of your loop code as reduce won't get you any performance gains. If anything you will lose performance over looping because you are now passing stuff to a function and it is getting boxed.
Why is he splitting all of the lines with a #"\n" regex? there are perfectly good methods for reading something line-by-line in a number of places (in both java and clojure).
I like Tim Bray and anyone who writes about clojure or lisp, but I feel like I'm tripping on acid here.
Reduce is actually faster than using loop/recur normally IME. Probably at least in part due to chuncked seqs which map and reduce among other functions can take advantage of.
Most of the built in clojure functions are better optimized than doing things by hand yourself.
Most of the built in clojure functions are better optimized than doing things by hand yourself.
Reduce /IS/ a loop: (from clojure.core)
([f val coll] (let [s (seq coll)] (if s (if (chunked-seq? s) (recur f (.reduce (chunk-first s) f val) (chunk-next s)) (recur f (f val (first s)) (next s))) val)))
It follows that I can use chunks in a loop... chunking and looping are not mutually exclusive.
It also happens to be the case that reduce is a function call, and (AFAIK) that you can't use transients (like conj!) or make use of unboxed values.
This is an issue, as such, one should still loop for best performance.
([f val coll] (let [s (seq coll)] (if s (if (chunked-seq? s) (recur f (.reduce (chunk-first s) f val) (chunk-next s)) (recur f (f val (first s)) (next s))) val)))
It follows that I can use chunks in a loop... chunking and looping are not mutually exclusive.
It also happens to be the case that reduce is a function call, and (AFAIK) that you can't use transients (like conj!) or make use of unboxed values.
This is an issue, as such, one should still loop for best performance.
If I'm not mistaken chunking is only supported on some of Clojure's data structures. Reduce is certainly the better option with vectors (and maps?) if you write your own loop/recur you'll simply be duplicating the existing implementation.
Also I really can't imagine using loop/recur on anything except for vectors and maps most of the time anyway, so why not just use reduce?
Also I really can't imagine using loop/recur on anything except for vectors and maps most of the time anyway, so why not just use reduce?
Yet I've always seen functional languages as a real fundamental approach that seems really elegant in theory. I have "Programming Clojure" on my Christmas list. Something about functional languages' elegance makes me really want to have a functional language succeed long-term. But every single functional language I've seen so far seems to miss the mark in seemingly important ways: Erlang with its syntax and purported string issues; Common Lisp (et. al) with its fragmented variants (so I hear); etc.
I would really like to see a great functional language executed well. You know, the whole deal: speed, interoperability, diverse libraries, concurrency...
Maybe Clojure is it, and it just has warts to iron out. Maybe the obtuseness that Tim Bray described here is just not "the way to do it". I hope that functional languages aren't inherently more difficult to work with than, say, Python.
I haven't worked with any of these functional languages yet, so granted I probably shouldn't be complaining. I know that with more experience in the languages I'd probably get a better understanding of how they excel -- and, unfortunately, where they don't.
Just... nothing seems to hit the sweet spot as well as Python and Javascript has, for me. I guess that's an unrealistic expectation for not having actually worked with functional languages.
I want a functional language to feel as smooth as Python. (If that just requires more experience and familiarity, fine. But I'd be hard-pressed to believe that the code demonstrated here could seem as clear to a Lisp expert as Python equivalents would feel to a Python programmer.)
Are the issues Tim found in this post actually just caused by lack of experience, or is this really how the language gets down to feel? I'd love to hear what lisp (clojure or not) folks can share.