Why functional programming?
24 comments
You appear to know what you can know from the outside. The only thing left is to try it out. Or choose not to; if you're that deep in embedded programming its utility to you will probably be marginal.
One bit of advice though; "trying it out" means getting to the point where you can actually "do things" with it, not running through two tutorials, filtering a couple of lists and writing a factorial function. That's the equivalent of learning how to write an if statement or a for loop or something, not the payoff. It's likely been a long time since you've actually felt like you don't know what you're doing, but that will be a sign you're on the right track.
You might also be interested in this: http://www.jerf.org/iri/post/2908 Being in the embedded world, you've probably just gotten used to certain things being "just the way it is" after 28 years, but one of the things that FP can show you is that those things aren't "just the way it is"; they are that way as a result of the choices we've collectively made. You've probably independently partially rediscovered a lot of FP's insights about state management over the years; you've probably also not independently pushed it as far as the FP community as a whole has. We don't actually have to live in a world where a flag flipped 25 million instructions ago is now causing a crash in seemingly unrelated code; we chose that world. It is not without its advantages, but it is not all gravy either.
One bit of advice though; "trying it out" means getting to the point where you can actually "do things" with it, not running through two tutorials, filtering a couple of lists and writing a factorial function. That's the equivalent of learning how to write an if statement or a for loop or something, not the payoff. It's likely been a long time since you've actually felt like you don't know what you're doing, but that will be a sign you're on the right track.
You might also be interested in this: http://www.jerf.org/iri/post/2908 Being in the embedded world, you've probably just gotten used to certain things being "just the way it is" after 28 years, but one of the things that FP can show you is that those things aren't "just the way it is"; they are that way as a result of the choices we've collectively made. You've probably independently partially rediscovered a lot of FP's insights about state management over the years; you've probably also not independently pushed it as far as the FP community as a whole has. We don't actually have to live in a world where a flag flipped 25 million instructions ago is now causing a crash in seemingly unrelated code; we chose that world. It is not without its advantages, but it is not all gravy either.
That was really interesting. Thanks.
And even in the embedded world, "constrain the state space like your life depends on it" is still very sound advice, even if you can't go fully functional.
And even in the embedded world, "constrain the state space like your life depends on it" is still very sound advice, even if you can't go fully functional.
Even if you can't go any functional, knowing FP has benefits. As an example, 25 years ago I did work on 8-bit video games. About as far from FP as you can get. Back then, I filled many sheets of paper convincing myself that rearranging my code and data in certain beneficial ways would actually work. I could have eliminated much of this drudgery had I known then what I know today about FP. See [1] for one example in which knowing two things – (1) what an involution was and (2) that applying (reverse ⋅ concat) is the same as applying (concat ⋅ reverse ⋅ map reverse) – would have let me go immediately to the preprocessing solution that I had to painstakingly work out on paper.
[1] http://blog.moertel.com/posts/2013-12-14-great-old-timey-gam...
[1] http://blog.moertel.com/posts/2013-12-14-great-old-timey-gam...
Revisit the Blub Paradox: http://www.paulgraham.com/avg.html
In short: Languages less powerful than Blub are obviously less powerful, because they're missing some feature he's used to. But when our hypothetical Blub programmer looks in the other direction, up the power continuum, he doesn't realize he's looking up. What he sees are merely weird languages. He probably considers them about equivalent in power to Blub, but with all this other hairy stuff thrown in as well. Blub is good enough for him, because he thinks in Blub.
Interestingly enough, I think the Blub Paradox greatly informs how pg views people too. You can be a "normal person" or you can be a "super smart person." Normal people can't understand super smart people because it's just beyond them. (part of the whole "people aren't created equal—some are just better than everybody else" selectivist philosophy.)
In short: Languages less powerful than Blub are obviously less powerful, because they're missing some feature he's used to. But when our hypothetical Blub programmer looks in the other direction, up the power continuum, he doesn't realize he's looking up. What he sees are merely weird languages. He probably considers them about equivalent in power to Blub, but with all this other hairy stuff thrown in as well. Blub is good enough for him, because he thinks in Blub.
Interestingly enough, I think the Blub Paradox greatly informs how pg views people too. You can be a "normal person" or you can be a "super smart person." Normal people can't understand super smart people because it's just beyond them. (part of the whole "people aren't created equal—some are just better than everybody else" selectivist philosophy.)
tl;dr - you don't need to care, you can do perfectly well without it, but you're missing something.
Longer answer ...
There are several issues here, and many people conflate them. They are realted, and I'm not going to be able to do them justice in a quick reply here. It's Christmas Eve, and I have a lot to do.
So, briefly ...
Learning FP, really learning it, gives you a new insight into programming. It can change the way you work, and the way you think. If nothing else, it's mind expanding, and that's always a good thing.
There's a seminal paper "Why Functional Programming Matters"[0] (WFPM). It dates back to 1984, but it's still relevant. In fact it's so relevant that raganwald[1] wrote "Why Why Functional Programming Matters Matters." That's also a really good read, with many useful insights.
fogus[2] has included WFPM in his list of 10 technical papers every programmer should read twice.[3] There's also a StackOverflow discussion[4], probably several.
There are many, many programmers out there, capable, productive, useful programmers getting stuff done, but knowing nothing (significant) about FP. And that's fine. But personally, I think that if you want to be serious about your craft, you should learn about FP, what it is, what it's good for, how to think in it, when to use it, and when to avoid it. It's not the One True Style(tm), but it is
[0] http://www.cse.chalmers.se/~rjmh/Papers/whyfp.html
[1] https://news.ycombinator.com/submitted?id=raganwald
[2] https://news.ycombinator.com/submitted?id=fogus
[3] http://blog.fogus.me/2011/09/08/10-technical-papers-every-pr...
[4] http://stackoverflow.com/questions/36504/why-functional-lang...
Longer answer ...
There are several issues here, and many people conflate them. They are realted, and I'm not going to be able to do them justice in a quick reply here. It's Christmas Eve, and I have a lot to do.
So, briefly ...
Learning FP, really learning it, gives you a new insight into programming. It can change the way you work, and the way you think. If nothing else, it's mind expanding, and that's always a good thing.
There's a seminal paper "Why Functional Programming Matters"[0] (WFPM). It dates back to 1984, but it's still relevant. In fact it's so relevant that raganwald[1] wrote "Why Why Functional Programming Matters Matters." That's also a really good read, with many useful insights.
fogus[2] has included WFPM in his list of 10 technical papers every programmer should read twice.[3] There's also a StackOverflow discussion[4], probably several.
There are many, many programmers out there, capable, productive, useful programmers getting stuff done, but knowing nothing (significant) about FP. And that's fine. But personally, I think that if you want to be serious about your craft, you should learn about FP, what it is, what it's good for, how to think in it, when to use it, and when to avoid it. It's not the One True Style(tm), but it is
[0] http://www.cse.chalmers.se/~rjmh/Papers/whyfp.html
[1] https://news.ycombinator.com/submitted?id=raganwald
[2] https://news.ycombinator.com/submitted?id=fogus
[3] http://blog.fogus.me/2011/09/08/10-technical-papers-every-pr...
[4] http://stackoverflow.com/questions/36504/why-functional-lang...
Why should you care about functional programming? Here are a few reasons:
1. It's different, especially from what it seems you're already doing. Therefore it is likely that if you do FP in earnest, you will learn new problem-solving techniques, many of which will prove useful elsewhere.
2. In FP, the abstraction and composition model – functions – scales down farther than do objects, modules, or compilation units. That is, functions remain practical in smaller places where objects or modules would be too expensive to insert, syntactically or semantically. In FP languages, then, you can use abstraction and composition even within the "leaves" of your logic, and that's where the bulk of your code actually lives.
3. Functions have a mathematical basis that you can draw on for intuition. In some FP languages, the mathematical parallels are so reliable that you can even calculate the code you need. Calculation can work in places where main force of intellect and the search for "aha! moments" fail.
There you go, three reasons: it's different, it not only scales up but down, and it comes with useful theory.
1. It's different, especially from what it seems you're already doing. Therefore it is likely that if you do FP in earnest, you will learn new problem-solving techniques, many of which will prove useful elsewhere.
2. In FP, the abstraction and composition model – functions – scales down farther than do objects, modules, or compilation units. That is, functions remain practical in smaller places where objects or modules would be too expensive to insert, syntactically or semantically. In FP languages, then, you can use abstraction and composition even within the "leaves" of your logic, and that's where the bulk of your code actually lives.
3. Functions have a mathematical basis that you can draw on for intuition. In some FP languages, the mathematical parallels are so reliable that you can even calculate the code you need. Calculation can work in places where main force of intellect and the search for "aha! moments" fail.
There you go, three reasons: it's different, it not only scales up but down, and it comes with useful theory.
Functional programming is about composing functions with functions. Just like OOP is about composing objects with objects.
FP and OOP is not in conflict with each other and you can certainly use both. FP is more in conflict with procedural programming. With procedural programming you are teaching the computer how to do things step by step (for i while i < len i++). With FP, you declare what you want to be done, and the computer figure it out on its own on how to get it done. An example would be for each loop on a collection. You are not telling the computer to increment index. You just want to iterate each element. The computer is free to spawn 20 threads to iterate the collection, so long that it accomplish the goal of your expression.
FP and OOP is not in conflict with each other and you can certainly use both. FP is more in conflict with procedural programming. With procedural programming you are teaching the computer how to do things step by step (for i while i < len i++). With FP, you declare what you want to be done, and the computer figure it out on its own on how to get it done. An example would be for each loop on a collection. You are not telling the computer to increment index. You just want to iterate each element. The computer is free to spawn 20 threads to iterate the collection, so long that it accomplish the goal of your expression.
Well, some toughts are easier to represent in abstract algebra. Really.
Functional programming is too generical a label to make it possible to answer your question. It may apply to anything from C-like functions to no side effect languages, but I guess most people around here use the term to mean Lisp.
If that's what you meant, Lisp'll give you a huge amount of reflexivity that you can use to change the language so it better fits your problem. In addition, Lisp has a very simple environment, what makes it the go-to language when you must embbeb an interpreter.
Now, if you were thinking about Haskell, it has a very powerfull type system that helps creating bug-fre software, and makes type-driven programming possible. It also has explicit side effects, what makes it possible for the compiler to paralelize your program for you, and a quite good optimizer (but not as good as you'll get by manually writting low level code).
And there are plenty of other languages people call functional. Each of them have some interesting aspects that may be usefull for solving a problem. Again, it's a very generic label, and a quite old one, so that all current languages already adopt functional features - in consequence it's not a very usefull label.
Functional programming is too generical a label to make it possible to answer your question. It may apply to anything from C-like functions to no side effect languages, but I guess most people around here use the term to mean Lisp.
If that's what you meant, Lisp'll give you a huge amount of reflexivity that you can use to change the language so it better fits your problem. In addition, Lisp has a very simple environment, what makes it the go-to language when you must embbeb an interpreter.
Now, if you were thinking about Haskell, it has a very powerfull type system that helps creating bug-fre software, and makes type-driven programming possible. It also has explicit side effects, what makes it possible for the compiler to paralelize your program for you, and a quite good optimizer (but not as good as you'll get by manually writting low level code).
And there are plenty of other languages people call functional. Each of them have some interesting aspects that may be usefull for solving a problem. Again, it's a very generic label, and a quite old one, so that all current languages already adopt functional features - in consequence it's not a very usefull label.
I strongly agree that functional/expression-based code is usually a nicer way of expressing algorithms. Composing functions like map and filter feels like a better way of processing collections than loops. Higher-order functions in general were a big revelation for me. Once I read about algebraic data types, they started showing up in my pseudo-code, even before I'd programmed with them. In general, declarative relationships, such as FP lets you write, tend to be easier to grok than procedures.
For the OP, let me see if this connects: In C/C++ and others, we have the ternary operator, which lets you write conditional expressions. status = n > 9000 ? "yes" : "no"; This is pretty convenient, but annoying to read afterwards, especially if you start nesting. Functional languages let you have the same convenience without the annoyance. You can do things like this OCaml: status := if n > 9000 then"yes" else "no" Yes, OCaml lets you have mutable state if you need it.
FWIW, the "just functional" language in my head is a lot closer to OCaml than Lisp. Haskell and Lisp both add a lot of "extra" stuff that's cool but (obviously) not needed to be "functional". Not that OCaml doesn't have "extra stuff", but it's not as dominant. YMMV.
For the OP, let me see if this connects: In C/C++ and others, we have the ternary operator, which lets you write conditional expressions. status = n > 9000 ? "yes" : "no"; This is pretty convenient, but annoying to read afterwards, especially if you start nesting. Functional languages let you have the same convenience without the annoyance. You can do things like this OCaml: status := if n > 9000 then"yes" else "no" Yes, OCaml lets you have mutable state if you need it.
FWIW, the "just functional" language in my head is a lot closer to OCaml than Lisp. Haskell and Lisp both add a lot of "extra" stuff that's cool but (obviously) not needed to be "functional". Not that OCaml doesn't have "extra stuff", but it's not as dominant. YMMV.
As someone who only got experience in Haskell I have to ask the same thing. Why should I use X OOP language when I can write almost anything in Haskell with ease? I tried learn programming in C and Python and even the beginner language Scheme, still I did not get programming until I learnt Haskell. Code shouldn't be abstracted as real world objects becuase code is instruction not something concrete you can touch, its information. Abstracting it using a more mathematical approuch very much makes sense and let even a beginner be able to handle quite complex problems.
Higher order functions is what made it click for me. Immutability, purity and the other stuff is nice, but certainly not mandatory to use FP.
The reduction in verbosity due to easier ways to approach things is what sold FP to me. Common patterns that I'd write several times in OO are easy to abstract in FP.
Tim Sweeny, of Epic Games (Unreal Engine) says that in their C++ codebase, 90% of loops are functional folds or maps.
I imagine the biggest issue, especially as an embedded programmer, is the lack of excellent optimizing compilers that let you write functional, but still obtain the same performance.
The reduction in verbosity due to easier ways to approach things is what sold FP to me. Common patterns that I'd write several times in OO are easy to abstract in FP.
Tim Sweeny, of Epic Games (Unreal Engine) says that in their C++ codebase, 90% of loops are functional folds or maps.
I imagine the biggest issue, especially as an embedded programmer, is the lack of excellent optimizing compilers that let you write functional, but still obtain the same performance.
As others have said, immutability and referential transparency enable you to reason about your programs much more cleanly that you are generally able to in imperative code. Functional programming also forces you to think about problems in a recursive manner, which is inherently very valuable, as the idea of solving problems by recursion is extremely powerful. Take a simple problem like this: given k different denominations of coins, how many ways can you make change for n dollars? (This is a problem from SICP, which also seems to come up on HN a lot). The functional code for this is about 10 lines. I don't even know how you would solve this imperatively...
You may look at this example and think it's contrived, that "real world problems" aren't anything like counting ways to make change. But just because you haven't encountered it in your daily work doesn't mean that other people haven't, or that it's not valuable as a tool in itself. And ultimately, having conceptual tools for thinking about problems is always going to be a good thing.
Btw, I find it slightly amusing that you don't want "to write programs like I'm writing on a Turing machine" since imperative programming maps pretty directly onto TMs as a model of computation...but I digress :)
You may look at this example and think it's contrived, that "real world problems" aren't anything like counting ways to make change. But just because you haven't encountered it in your daily work doesn't mean that other people haven't, or that it's not valuable as a tool in itself. And ultimately, having conceptual tools for thinking about problems is always going to be a good thing.
Btw, I find it slightly amusing that you don't want "to write programs like I'm writing on a Turing machine" since imperative programming maps pretty directly onto TMs as a model of computation...but I digress :)
I think a lot of people gloss over a bit too quickly on the actual implications at a hardware level on what it means to have immutable state. If you have 10mb of numbers that you need to add 1 to, immutable state means you now have 20mb of numbers guaranteed. Allowing for mutable state means you can stick to the 10mb and mutate it.
So in general, forced immutable state means that any class of problems which require both high performance and a lot of mutation are going to be very poor candidates for functional programming. This is actually a large share of embedded programming and why the OP probably hasn't found that much use for it - embedded programming often comes down to accepting large amounts of data from sensors and manipulating it.
The problems where functional programming shine are where you have a chunk of data and need to create an answer from it. Given 10mb of data, should I do X or Y? Functional programming is a very good tool here because of aspects such as composability allowing for extra steps to be chained into a calculation, and immutable data ensuring that multiple steps can be run on the same data at the same time.
Hopefully that helps a bit in deciding when functional would be useful or less useful.
So in general, forced immutable state means that any class of problems which require both high performance and a lot of mutation are going to be very poor candidates for functional programming. This is actually a large share of embedded programming and why the OP probably hasn't found that much use for it - embedded programming often comes down to accepting large amounts of data from sensors and manipulating it.
The problems where functional programming shine are where you have a chunk of data and need to create an answer from it. Given 10mb of data, should I do X or Y? Functional programming is a very good tool here because of aspects such as composability allowing for extra steps to be chained into a calculation, and immutable data ensuring that multiple steps can be run on the same data at the same time.
Hopefully that helps a bit in deciding when functional would be useful or less useful.
> If you have 10mb of numbers that you need to add 1 to immutable state means you now have 20mb of numbers guaranteed
You'd only have 20mb of 'numbers' if you needed to keep a reference to the original data structure. Otherwise its' garbage.
>So in general, forced immutable state means that any class of problems which require both high performance and a lot of mutation are going to be very poor candidates for functional programming
FP does not necessitate 'immutable state', even in pure languages like Haskell. What it enforces is purity, or more concrete, referential transparency. Often the easiest way to obtain referential transparency is immutable data (non destructive data structures, etc), but you can have mutable data and maintain referential transparency for high performance (see ST Monad).
You'd only have 20mb of 'numbers' if you needed to keep a reference to the original data structure. Otherwise its' garbage.
>So in general, forced immutable state means that any class of problems which require both high performance and a lot of mutation are going to be very poor candidates for functional programming
FP does not necessitate 'immutable state', even in pure languages like Haskell. What it enforces is purity, or more concrete, referential transparency. Often the easiest way to obtain referential transparency is immutable data (non destructive data structures, etc), but you can have mutable data and maintain referential transparency for high performance (see ST Monad).
Something like "map myints (+ 1)" can be performed in a mutable way, without compromising on functional purity. (So long you're not using the old values afterwards, of course.)
Not all compilers perform such optimizations, but there's no fundamental conflict.
I believe Haskell's GHC compiler has "fusion" which allows it to wrap chains of maps/filters/folds into a single tight loop. That way you keep functional style, but get performance of writing it imperatively.
Not all compilers perform such optimizations, but there's no fundamental conflict.
I believe Haskell's GHC compiler has "fusion" which allows it to wrap chains of maps/filters/folds into a single tight loop. That way you keep functional style, but get performance of writing it imperatively.
You're getting a bit bogged down in details, I meant it more as a general case. There are numerous cases where it is simply more efficient to be operating directly on data - the most used example I can think of are fragment and vertex shaders. It's irrelevant to the issue anyway though: simply having the ability to do something does not make it the best way to do something. You could code in a composable and immutable way using FORTRAN or COBOL if you tried hard enough.
The OP was asking for cases where you'd prefer to be using FP over the type of code he is used to. Manipulating blobs of sensor data are simply unlikely to be a productive use of FP - although it could certainly be done. However, calculating statistics from that data is going to be a place where FP would shine and would be a good choice for it.
Not everything is a nail simply because you can spend hours hitting it with enough transformations to make it look like one.
EDIT: Thinking about it, fragment and vertex shaders actually seem like a very good place to use FP.
The OP was asking for cases where you'd prefer to be using FP over the type of code he is used to. Manipulating blobs of sensor data are simply unlikely to be a productive use of FP - although it could certainly be done. However, calculating statistics from that data is going to be a place where FP would shine and would be a good choice for it.
Not everything is a nail simply because you can spend hours hitting it with enough transformations to make it look like one.
EDIT: Thinking about it, fragment and vertex shaders actually seem like a very good place to use FP.
Even when doing direct mutation of a byte array, FP can come in handy. The key part is being able to pass around and return functions. While I don't have experience "manipulating blobs of sensor data", most code can benefit from passing functions around and using closures. Even in otherwise imperative code, using folds and maps may make things clearer. Although, it might not be practical due to performance reasons.
Immutable state does not necessarily imply 20mb of numbers guaranteed. It depends on the compiler and on the implementation of data structures. Functional data structures can use mutation behind the scenes to produce a facade of immutability. Clojure has an efficient "immutable" array data structure offering constant-time (~log32 n) performance and better space usage. Shallow binding is another general technique that produces space and time performance nearly identical to corresponding mutable data structures by wrapping the mutable data structure with older versions storing a pointer to the incrementally newer version plus the incremental change. Both of these use mutation behind the scenes. It's not necessary to sacrifice performance to program in a functional way.
That's a somewhat naive view of immutability. Garbage collection should take care of the 10mb of numbers you aren't using anymore. True, there are issues for more complex data structures, but that's why persistent data structures were invented. The goal is only to retain what you use.
This isn't that different from an imperative world. If someone somewhere has a reference to some part of that data (or may have a reference), what are you going to do? Presumably you'll make a copy. In an immutable world, adding references to an existing of data is trivial. So either you accrete more data, in which case you're in the same place as the imperative world, or it's not new data, no one is using it, and you can throw it out.
In practice it's probably more complex than that, and depends on the language implementation.
There's also something to be said for something which, implementation-wise, mutates in place when possible but semantically acts as if it is immutable.
I'm not advocating for FP in embedded systems, just trying to flesh out more detail about the trade-off here. I think even if you don't program with a language with immutability, it may be worth asking questions (which may have a simple answer in the affirmative!) about the extent to which you need mutability in a given situation vs not. "Don't optimize yet" and all that.
This isn't that different from an imperative world. If someone somewhere has a reference to some part of that data (or may have a reference), what are you going to do? Presumably you'll make a copy. In an immutable world, adding references to an existing of data is trivial. So either you accrete more data, in which case you're in the same place as the imperative world, or it's not new data, no one is using it, and you can throw it out.
In practice it's probably more complex than that, and depends on the language implementation.
There's also something to be said for something which, implementation-wise, mutates in place when possible but semantically acts as if it is immutable.
I'm not advocating for FP in embedded systems, just trying to flesh out more detail about the trade-off here. I think even if you don't program with a language with immutability, it may be worth asking questions (which may have a simple answer in the affirmative!) about the extent to which you need mutability in a given situation vs not. "Don't optimize yet" and all that.
Totally agree on the hardware implementation.
Mutable state has two uses. First, when what you are modelling really has state the varies with time (e.g. real world stuff, UI stuff). Second, as a kind of optimization.
One good reason for learning FP is that you think more clearly about what mutable state you are using and why.
For example, OOP programmers tend to instinctively write
Mutable state has two uses. First, when what you are modelling really has state the varies with time (e.g. real world stuff, UI stuff). Second, as a kind of optimization.
One good reason for learning FP is that you think more clearly about what mutable state you are using and why.
For example, OOP programmers tend to instinctively write
obj.setX(x)
obj.doG()
y = obj.doF()
y = F(G(x))
State rarely varies with time other than the wall clock, mostly in between time X and Y some events happened.
With FP systems you generally have to model these things explicitly and attach a particular time to a particular state which makes the entire system much easier to reason about.
Cached state really bothers me because invariably even after caching it, the state can't be reused because of the difficulty of reasoning about the whole system.
I think in reality the appeal of FP is that the people who use it are generally more experienced programmers who tend not to make too much mess in non-FP languages anyway. The benefits are more the peer group than anything else.
With FP systems you generally have to model these things explicitly and attach a particular time to a particular state which makes the entire system much easier to reason about.
Cached state really bothers me because invariably even after caching it, the state can't be reused because of the difficulty of reasoning about the whole system.
I think in reality the appeal of FP is that the people who use it are generally more experienced programmers who tend not to make too much mess in non-FP languages anyway. The benefits are more the peer group than anything else.
> I think in reality the appeal of FP is that the people who use it are generally more experienced programmers who tend not to make too much mess in non-FP languages anyway. The benefits are more the peer group than anything else.
Careful, you're arguing here that FP is better because the person using it is a better person. There is nothing wrong with that statement if it is actually an unbiased fact, but simply choosing to use or not use FP would automatically bias you towards declaring that true or false. Avoiding that line of reasoning is a good idea because it's not useful.
Stick to the merits of FP itself.
Careful, you're arguing here that FP is better because the person using it is a better person. There is nothing wrong with that statement if it is actually an unbiased fact, but simply choosing to use or not use FP would automatically bias you towards declaring that true or false. Avoiding that line of reasoning is a good idea because it's not useful.
Stick to the merits of FP itself.
Actually I think the poster is right, and I don't think we should avoid all lines of argument that might be subject to personal bias.
The poster was saying that FP has a certain cachet (no pun intended) that is probably due to the quality of the average programmer.
It's a bit like if someone was asked to explain higher salaries in Silicon Valley. Are we allowed to refer to the quality of programmers? Of course there is a risk of personal bias, but how could one have this discussion without referring to this factor?
The poster was saying that FP has a certain cachet (no pun intended) that is probably due to the quality of the average programmer.
It's a bit like if someone was asked to explain higher salaries in Silicon Valley. Are we allowed to refer to the quality of programmers? Of course there is a risk of personal bias, but how could one have this discussion without referring to this factor?
You'd have to bring in actual results: programmers in SV create great products that are widely used around the world. This could be used as evidence to imply they are better.
FP is a bit tougher here - the vast majority of programs used by the general public are not FP. You could argue that FP is 'newer' and not had time to create these widely used programs yet - but FP is far from new. Basically there is no real world evidence to back up that FP programmers are higher skilled and only personal biases and assumptions.
FP is a bit tougher here - the vast majority of programs used by the general public are not FP. You could argue that FP is 'newer' and not had time to create these widely used programs yet - but FP is far from new. Basically there is no real world evidence to back up that FP programmers are higher skilled and only personal biases and assumptions.
This is kind of the dirty secret of any kind of programming, that it's very hard to make arguments that this or that technology leads to categorically better or different software. I mean, yes, there are specific domains, like trying to write webapps in C vs Ruby, but in general the programming languages well-suited for a task tend to be much, much more alike than different.
We love to hate on PHP but it basically runs a non-trivial proportion of the web, incl Facebook. Python and Ruby look different but they're not so different that we can say, on average, a programmer is X times more productive using language Y instead of Z.
That said I am kinda sorta sympathetic to the idea that people who veer off the beaten path of Java are self-selecting and may be open to more ideas, more adaptable, etc. But I'm leery of extending that to saying they're "better" in any measurable or definitive way.
The one thing I would say re: FP being "newer" is that while the ideas aren't newer, it's "newer" in the sense that it's not as battle-tested as Java or C. Although, then again, when you see Ruby (a friendlier, no parens Lisp-alike), JS, higher-order functions in C++, list comprehensions in Python, and so on, it stops looking quite so much like FP is really all that foreign. People are already using a lot of it. So perhaps I should be more specific and say that while the concepts aren't new, the implementations are new to industry or production systems.
We love to hate on PHP but it basically runs a non-trivial proportion of the web, incl Facebook. Python and Ruby look different but they're not so different that we can say, on average, a programmer is X times more productive using language Y instead of Z.
That said I am kinda sorta sympathetic to the idea that people who veer off the beaten path of Java are self-selecting and may be open to more ideas, more adaptable, etc. But I'm leery of extending that to saying they're "better" in any measurable or definitive way.
The one thing I would say re: FP being "newer" is that while the ideas aren't newer, it's "newer" in the sense that it's not as battle-tested as Java or C. Although, then again, when you see Ruby (a friendlier, no parens Lisp-alike), JS, higher-order functions in C++, list comprehensions in Python, and so on, it stops looking quite so much like FP is really all that foreign. People are already using a lot of it. So perhaps I should be more specific and say that while the concepts aren't new, the implementations are new to industry or production systems.
I find that a bit limiting, I would rather hear people's opinions even if they might be subject to bias. And I don't think that an opinion based on personal experience is "only personal biases and assumptions".
It is also important that your approach be uniformly enforced: you would have to encourage people to refrain from saying that FP programmers have the same skill level as non-FP programmers, since that opinion is also not based on any evidence.
It is also important that your approach be uniformly enforced: you would have to encourage people to refrain from saying that FP programmers have the same skill level as non-FP programmers, since that opinion is also not based on any evidence.
> I think in reality the appeal of FP is that the people who use it are generally more experienced programmers
I would wager that FP is used more in universities than in any other setting, and by underclassmen, not just graduate students. So I don't exactly think this is true.
I would wager that FP is used more in universities than in any other setting, and by underclassmen, not just graduate students. So I don't exactly think this is true.
> If you have 10mb of numbers that you need to add 1 to, immutable state means you now have 20mb of numbers guaranteed.
Immutable state also means that you don't need to add 1 to the numbers you won't use, and that you can discard the old numbers if you only wanted the incremented ones.
Garbage colletors and run-time optimizers are added complexity, and may be too much for an embbebed system (or maybe not, controlers are getting bigger each day). But that's a context dependent conclusion, not a certainty.
Immutable state also means that you don't need to add 1 to the numbers you won't use, and that you can discard the old numbers if you only wanted the incremented ones.
Garbage colletors and run-time optimizers are added complexity, and may be too much for an embbebed system (or maybe not, controlers are getting bigger each day). But that's a context dependent conclusion, not a certainty.
Don't the compilers optimize this? I always believed functional programming to be popular among compiler experts because it's easier to write certain optimizations on a program because of it's many properties like state not changing etc. I don't know if this leads to realistically faster programs in real life.
That is just theory. In practice, it is going to be very hard for the compiler to know, at the time that you change a single value in a huge array, that you are never going to refer to the old array, and so it can just mutate the array.
I am not as experienced as you, far from it, but my -very limited - understanding of functional programming is that it's all about limiting side effects, which means limiting the ways you can screw up something. I could try to explain what I think, but John Carmack did it much better than I would ever do in a blog post, that was once submitted on HN: http://www.altdevblogaday.com/2012/04/26/functional-programm...
Great article by Carmack. Thanks.
[deleted](1)
Can anyone provide advice on which language to pick? I started learning C++ and Java in high school and college because those were forced by the curriculum. My first two jobs out of school forced me also to use C# which was close enough to Java. When I wanted to make the leap to script languages, my choice between Python and Ruby basically boiled down to the strength of the local Ruby / Python communities, and Ruby clearly won in that realm.
But with functional programming, I want to learn, but I have far less direction. Scala is perhaps the biggest, but still small. Haskell seems to be the most "hardcore", which appeals to me, whereas I keep anecdotally seeing people who like Ruby and Javascript (my two main languages) also tend to gravitate towards Clojure, and I can't explain why.
So far I've hated Scala's syntax and formatting (much like I hated Objective-C). Haskell is making my brain melt. Clojure seems quite simple and reminds me of my days writing Scheme in school. Any thoughts?
But with functional programming, I want to learn, but I have far less direction. Scala is perhaps the biggest, but still small. Haskell seems to be the most "hardcore", which appeals to me, whereas I keep anecdotally seeing people who like Ruby and Javascript (my two main languages) also tend to gravitate towards Clojure, and I can't explain why.
So far I've hated Scala's syntax and formatting (much like I hated Objective-C). Haskell is making my brain melt. Clojure seems quite simple and reminds me of my days writing Scheme in school. Any thoughts?
I'm a bit biased, but I'd recommend giving Scala a try. There is a lot of non-sense floating on the internet about the language, but I suggest making up your own mind.
Tooling and IDE support is better than Haskell and Clojure, so you can focus on the language instead of fighting with tools.
If you know Scala a bit, it's easy to move to something else. For instance, if you decide that you don't need types, you can move to Clojure; if you decide you don't need unchecked side-effects you can move to Haskell; if you decide that you don't need functional programming, you can move to a different OOP language (or keep using Scala).
Tooling and IDE support is better than Haskell and Clojure, so you can focus on the language instead of fighting with tools.
If you know Scala a bit, it's easy to move to something else. For instance, if you decide that you don't need types, you can move to Clojure; if you decide you don't need unchecked side-effects you can move to Haskell; if you decide that you don't need functional programming, you can move to a different OOP language (or keep using Scala).
short and sweet: Functional programming as a paradigm is "about" the following things as I understand it:
1) functions as first class citizens (this is a great read that relates to this: http://steve-yegge.blogspot.com/2006/03/execution-in-kingdom...)
2) immutable data: mutable state is the enemy of threading/concurrency/parallelism. Things can change underneath you in unexpected ways causing unforeseen consquences. When the data never changes, this sort of thing becomes much easier.
3) Referential transparency: This is related to #2. Whenever you pass A into function Z, you get B. There's no chance of side effects or other possible outcomes. A goes in, B comes out.
1) functions as first class citizens (this is a great read that relates to this: http://steve-yegge.blogspot.com/2006/03/execution-in-kingdom...)
2) immutable data: mutable state is the enemy of threading/concurrency/parallelism. Things can change underneath you in unexpected ways causing unforeseen consquences. When the data never changes, this sort of thing becomes much easier.
3) Referential transparency: This is related to #2. Whenever you pass A into function Z, you get B. There's no chance of side effects or other possible outcomes. A goes in, B comes out.
tl;dr: State is bad for optimizations, FP encourages you to write most of the code state-free.
One of the perks of FP is that it exposes pure functions, functions that have a fixed output for a given input. Since it's really bad at storing and mutating state(canonically can only be done via monads), it encourages you to separate your code into pure and mutable states. Let's take, for instance, a function that reads a file and xor's that with a random byte stream. A non functional approach would be something like this:
But remember that most "pure" FP languages are bad at keeping state, turning a huge game(for instance) into a challenge both to the programmer and to the compiler.
[1] - http://en.wikipedia.org/wiki/Currying
[2] - http://en.wikipedia.org/wiki/Map_%28higher-order_function%29
EDIT: Formmating, not used to posting to hackernews comments
One of the perks of FP is that it exposes pure functions, functions that have a fixed output for a given input. Since it's really bad at storing and mutating state(canonically can only be done via monads), it encourages you to separate your code into pure and mutable states. Let's take, for instance, a function that reads a file and xor's that with a random byte stream. A non functional approach would be something like this:
void xor_file_stream(file input, stream random_stream){
result = []
file_byte = read_byte(input)
while(file_byte != EOF){
stream_byte = read_byte(random_stream)
res.append(file_byte^stream_byte)
file_byte = read_byte(input)
}return result
} byte xor(byte b1, byte b2){
return byte(byte b2){
return b1^b2
}
}
//Here I assume the language has file and random_stream as iterables
void xor_file_stream(file input,stream random_stream){
return map(map(xor,random_stream),input)
}But remember that most "pure" FP languages are bad at keeping state, turning a huge game(for instance) into a challenge both to the programmer and to the compiler.
[1] - http://en.wikipedia.org/wiki/Currying
[2] - http://en.wikipedia.org/wiki/Map_%28higher-order_function%29
EDIT: Formmating, not used to posting to hackernews comments
I would be interested in your views about OOP; it shares some ideals with FP, namely, encapsulation. OOP puts a lot of emphasis on encapsulating state, and binding the associated behavior to it so that no other part of the program can access it and cause problems, and so that contracts can be expressed by clean, abstract interfaces. It also typically uses procedural workflow and algorithmic and adds to it classes as a code organisation paradigm. OOP is boxes; stuff is in boxes so you don't see it.
FP tries to do that but with behavior, so that any given operation can be cleanly plugged into other behavior via function composition and higher-levels such as maps, folds and their math-ey equivalents functors and monads. FP is pipes. Stuff goes in, other stuff comes out, and we have an elaborate system of pipe ends to make sur one's out fits into the other's in.
OOP is good when a lot of persistent state has to hang around; objects in a game, objects floating around in a GUI window, client, transaction, product objects in a commercial application that must be created, viewed, modified, deleted. Boxes in a warehouse, bills and client data in boxes, running and shooting boxes in your Xbox.
FP is good when data has to come in, be transformed, and get out. Text treatment for example. grep is obviously made out of super-low-level procedural C, but it would be a perfect fit for the FP model: a bunch of text comes in, gets chopped into lines, a search algo is performed on each one, and it all gets spitted out, possibly into another, well, pipe. Compilers are also a nice fit: yet some more text comes in, it's chopped into a flat list of tokens, it's parsed into an abstract syntax tree, then into a type-checker, then a few other pieces of optimisation piping, and finally gets spit out as some other lower-level representation.
Server-side scripting is the most common real-life application of functional-ish langs; a request comes in, a few DB requests are made, the return values are chopped into HTML (or whatever mid-level representation then html), and that stuff is spit on the wire, which as far as it goes is isomorphic to a pipe.
Symbolic computing is a pretty sweet match for FP: some notation for an equation comes in, is turned into an expression tree, then into some standard form, then gets solved or integrated or whatever (recursive tree-walks all over the place), then turned back into a human-friendly form and spit back to the display.
Pandoc is a neat Haskell lib that takes some document format (LaTeX, HTML, markdown, whatever), turns it into a unified representation, then spits it back as another format.
JavaScript is funny because it totally embraces the fact that behavior is data, that pipes are also boxes. You give a callback to a handler, which then puts stuff like it as if were a pipe, and then you have functions that implicitly act on a global, implicit data structure (the DOM), some of them that take a description of where does one want to go in the DOM and returns a node of that recursively-defined tree, such that functions can still be chained and it's relevant to do so even if they're mostly used for their side-effects. JS is the weirdest and most wonderful thing. It even has JSON, which is the funniest example of the code-is-data ethos since LISP and it's exposed AST.
But yeah, it's in the nature of FP to not care about state, so it's normal that keeping state around in FP-styled programs is a pain. Passing a global data structure/container to a function that reaches down into the structure and returns a new, modified instance of that global and re-assigning the new into the old is just... Yuck. If a player gets killed in my game state, I want to just delete it from my player list, not replace my player list with a new one.
It's in the nature of OOP to see behavior as a second-class citizen to state. In patterns parlance, FP means using the iterator and the strategy pattern all over the place as they were themselves objects. Just like OOP lets me put two objects together and have a new one, and perhaps put an array of those inside another object. In FP, I can put two different strategies together, and have another strategy, and put that compound in the iterator pattern/strategy, and have another strategy that I can still compound. Recursion (and the oldest FP performance tweak for it, the tco) is a nice way to express iteration without the use of a dedicated iteration construct, which is totally not a nice piece of pipe; you can do function composition by nesting calls [ much(foo(grunk(one_cookie_yum))); ], but it gets weird [ much(foo(grunk_many_cookies(many_cookies_yum))); but you can't do something like {for one_cookie : many_cookies much(foo(grunk(one_cookie)))}; or you could but it would be a built-in construct which you wouldn't have control over, while in haskell you can just do map (much . foo . grunk) many_cookies ]
So yeah.
If you can picture your program like a pipe, it's probably a good fit for FP. The kind of embedded softare you do probably is not remotely looking like a pipe, so you can keep on not understanding FP. If it can provide you some comfort and perhaps a laugh, the legendary pure-functional no-side-fx-allowed the-ghc-actually-is-the-flagsip-project Haskell has a runtime made out of C.
FP tries to do that but with behavior, so that any given operation can be cleanly plugged into other behavior via function composition and higher-levels such as maps, folds and their math-ey equivalents functors and monads. FP is pipes. Stuff goes in, other stuff comes out, and we have an elaborate system of pipe ends to make sur one's out fits into the other's in.
OOP is good when a lot of persistent state has to hang around; objects in a game, objects floating around in a GUI window, client, transaction, product objects in a commercial application that must be created, viewed, modified, deleted. Boxes in a warehouse, bills and client data in boxes, running and shooting boxes in your Xbox.
FP is good when data has to come in, be transformed, and get out. Text treatment for example. grep is obviously made out of super-low-level procedural C, but it would be a perfect fit for the FP model: a bunch of text comes in, gets chopped into lines, a search algo is performed on each one, and it all gets spitted out, possibly into another, well, pipe. Compilers are also a nice fit: yet some more text comes in, it's chopped into a flat list of tokens, it's parsed into an abstract syntax tree, then into a type-checker, then a few other pieces of optimisation piping, and finally gets spit out as some other lower-level representation.
Server-side scripting is the most common real-life application of functional-ish langs; a request comes in, a few DB requests are made, the return values are chopped into HTML (or whatever mid-level representation then html), and that stuff is spit on the wire, which as far as it goes is isomorphic to a pipe.
Symbolic computing is a pretty sweet match for FP: some notation for an equation comes in, is turned into an expression tree, then into some standard form, then gets solved or integrated or whatever (recursive tree-walks all over the place), then turned back into a human-friendly form and spit back to the display.
Pandoc is a neat Haskell lib that takes some document format (LaTeX, HTML, markdown, whatever), turns it into a unified representation, then spits it back as another format.
JavaScript is funny because it totally embraces the fact that behavior is data, that pipes are also boxes. You give a callback to a handler, which then puts stuff like it as if were a pipe, and then you have functions that implicitly act on a global, implicit data structure (the DOM), some of them that take a description of where does one want to go in the DOM and returns a node of that recursively-defined tree, such that functions can still be chained and it's relevant to do so even if they're mostly used for their side-effects. JS is the weirdest and most wonderful thing. It even has JSON, which is the funniest example of the code-is-data ethos since LISP and it's exposed AST.
But yeah, it's in the nature of FP to not care about state, so it's normal that keeping state around in FP-styled programs is a pain. Passing a global data structure/container to a function that reaches down into the structure and returns a new, modified instance of that global and re-assigning the new into the old is just... Yuck. If a player gets killed in my game state, I want to just delete it from my player list, not replace my player list with a new one.
It's in the nature of OOP to see behavior as a second-class citizen to state. In patterns parlance, FP means using the iterator and the strategy pattern all over the place as they were themselves objects. Just like OOP lets me put two objects together and have a new one, and perhaps put an array of those inside another object. In FP, I can put two different strategies together, and have another strategy, and put that compound in the iterator pattern/strategy, and have another strategy that I can still compound. Recursion (and the oldest FP performance tweak for it, the tco) is a nice way to express iteration without the use of a dedicated iteration construct, which is totally not a nice piece of pipe; you can do function composition by nesting calls [ much(foo(grunk(one_cookie_yum))); ], but it gets weird [ much(foo(grunk_many_cookies(many_cookies_yum))); but you can't do something like {for one_cookie : many_cookies much(foo(grunk(one_cookie)))}; or you could but it would be a built-in construct which you wouldn't have control over, while in haskell you can just do map (much . foo . grunk) many_cookies ]
So yeah.
If you can picture your program like a pipe, it's probably a good fit for FP. The kind of embedded softare you do probably is not remotely looking like a pipe, so you can keep on not understanding FP. If it can provide you some comfort and perhaps a laugh, the legendary pure-functional no-side-fx-allowed the-ghc-actually-is-the-flagsip-project Haskell has a runtime made out of C.
I asked, "When is FP the right tool for the job?"
"If you can picture your program like a pipe..." may well be the answer I was looking for. Thanks!
"If you can picture your program like a pipe..." may well be the answer I was looking for. Thanks!
This is the best explanation of functional programming I've ever read.
Congrats.
Congrats.
This was really excellent
This past week I bought three books about Erlang. I'm going to be studying and going through all of those books in the next month or two.
Why? Because I have the same questions you do, only I'm a much worse programmer/developer.
Insert Nike slogan here.
What's the worst thing that can happen by learning something?
Why? Because I have the same questions you do, only I'm a much worse programmer/developer.
Insert Nike slogan here.
What's the worst thing that can happen by learning something?
> What's the worst thing that can happen by learning something?
If nothing else, opportunity cost. I can't use the same time to learn something else.
If nothing else, opportunity cost. I can't use the same time to learn something else.
But you can't really know the real value (or lack of) time learning X, until you have done X. (Thou somethings will always appear to have more value before you start)
> Why functional programming?
At least for the sake of thinking in another paradigm, having another «environment» to think about same issues.
I have scarce experience in functional language but AFAIK there are not only good for in->process->out like it's explained in other post.
At least for the sake of thinking in another paradigm, having another «environment» to think about same issues.
I have scarce experience in functional language but AFAIK there are not only good for in->process->out like it's explained in other post.
Because functional code is on average half the size of imperative code. The shorter the code, the better. If that argument doesn't convince you, then you need to first understand why writing short code is so important.
I don't have an answer for you, but for the last year I have been learning Clojure, and I have found it fascinating. I do not think this will satisfy you, but I would like to share some of the things that I find interesting about Clojure.
Lately I've been studying the source code of Aleph, a web server written by Zach Tellman. Zach is a very smart guy, and reading his source code is an education (I've actually found this true of the majority of Clojure projects: the quality of the code is very good, and Clojure is the only language I've worked with so far where I have been able to learn so much reading the source code of the 3rd party libraries that I rely on).
Let's start with an interesting comment about Java, on StackOverflow:
Why doesn't more Java code use PipedInputStream / PipedOutputStream?
http://stackoverflow.com/questions/484119/why-doesnt-more-ja...
In response to that question, matt_b suggests this is part of the reason why people don't use pipedinputstream or pipedoutputstream:
"Typically, data is read from a PipedInputStream object by one thread and data is written to the corresponding PipedOutputStream by some other thread. Attempting to use both objects from a single thread is not recommended, as it may deadlock the thread."
So now, look at how PipedInputStream and PipedOutputStream are used in Aleph:
https://github.com/ztellman/aleph/blob/perf/src/aleph/format...
In particular, look at this function:
(defn channel->input-stream "Consumes messages from a channel that emits bytes, and feeds them into an InputStream."
On a different subject, I have learned a new style of programming by using closures in a functional language. Maybe I am a bit stupid, but it took me a long time before I began to realize how useful closures are. My only other exposure to closures was with Ruby and Javascript, but I never did concurrent programming with Ruby or Javascript, and it seems to me the greatest use of closures is when you are dealing with a lot of threads. When I first started writing Clojure code, I did not use closures at all. I wrote some clumsy code. For instance, I wrote a function that got some data from the database, and I wrote another function that stored that data in a hashmap in memory (using the memory as a cache) and I wrote another function that would check the hashmap for the data, before making the call to the database, and then I wrote another function that ran every 15 minutes and erased items in the hashmap, thus invalidating the cache. But then I realized I could combine several of these operations by creating a closure at the moment that I store data in the hashmap: the closure could call Thread.sleep(900000) and then delete the data from the hashmap. I found it convenient to create the closure while I was storing the data in the hashmap as, at that moment, I have all the information I need, including the key with which that data is being stored. Then I push the closure onto a queue and it gets processed in 15 minutes. This simplifies things quite a bit.
Again, Zach Tellman's code has been educating me about how powerful closures can be in the context of concurrent programming. Look here:
https://github.com/ztellman/aleph/blob/perf/src/aleph/http.c...
And look especially at this function:
(defn wrap-ring-handler "Takes a normal Ring handler, and turns it into a handler that can be consumed by Aleph's start-http-server. If the Ring handler returns an async-promise, this will be handled properly and sent along as a long-poll response whenever it is realized. This should be the outermost middleware around your function. To use an Aleph handler at a particular endpoint within the scope of this middleware, use wrap-aleph-handler."
You are asking why functional programming is a big deal. I'm sure different people will have different answers. I think the difficulty in answering is that there is no one thing that by itself makes an answer, rather, it is a whole series of things that combine in powerful ways. For me, the immutable data structures of Clojure are an education. I have learned a lot. I think using closures in the context of concurrent programming could be an absolute nightmare, if it was not for the safety offered by immutable data structures. And yet, I don't think immutable data structures are a big deal by themselves, nor do I think that closures are a big deal by themselves, nor do I think concurrency is a big deal by itself. Only when I look at the combination of these 3 do I find myself enlightened, like I've just discovered a whole new level of programming, a level of power I have never felt in any other language.
A final note: Adam Bard has a blog post "Effortless async by design" in which he says "I dare you to find a thread-safer language. Well, maybe Erlang." And then he also uses some code from Zach Tellman to demonstrate what he means. Bard's post is worth a read:
http://adambard.com/blog/why-clojure-part-2-async-magic/
I could list a lot of things that seem to me very powerful about the coding style I've learned this last year: composability, currying, closures, immutable data structures, thread-safe, etc. But I think descriptions like that tend to be a bunch of meaningless buzz words until you've dived in deep enough to see how they all interact with each other. Because its the interaction of these features that is really powerful, not any of them alone.
Lately I've been studying the source code of Aleph, a web server written by Zach Tellman. Zach is a very smart guy, and reading his source code is an education (I've actually found this true of the majority of Clojure projects: the quality of the code is very good, and Clojure is the only language I've worked with so far where I have been able to learn so much reading the source code of the 3rd party libraries that I rely on).
Let's start with an interesting comment about Java, on StackOverflow:
Why doesn't more Java code use PipedInputStream / PipedOutputStream?
http://stackoverflow.com/questions/484119/why-doesnt-more-ja...
In response to that question, matt_b suggests this is part of the reason why people don't use pipedinputstream or pipedoutputstream:
"Typically, data is read from a PipedInputStream object by one thread and data is written to the corresponding PipedOutputStream by some other thread. Attempting to use both objects from a single thread is not recommended, as it may deadlock the thread."
So now, look at how PipedInputStream and PipedOutputStream are used in Aleph:
https://github.com/ztellman/aleph/blob/perf/src/aleph/format...
In particular, look at this function:
(defn channel->input-stream "Consumes messages from a channel that emits bytes, and feeds them into an InputStream."
([ch]
(channel->input-stream ch "utf-8"))
([ch charset]
(let [out (PipedOutputStream.)
in (PipedInputStream. out 16384)
bytes (map* #(bytes->byte-array % charset) ch)]
(future
(try
(loop []
(when-let [msg @(read-channel* bytes :on-drained nil)]
(.write out ^bytes msg)
(recur)))
(finally
(.close out))))
in)))
On a different subject, I have learned a new style of programming by using closures in a functional language. Maybe I am a bit stupid, but it took me a long time before I began to realize how useful closures are. My only other exposure to closures was with Ruby and Javascript, but I never did concurrent programming with Ruby or Javascript, and it seems to me the greatest use of closures is when you are dealing with a lot of threads. When I first started writing Clojure code, I did not use closures at all. I wrote some clumsy code. For instance, I wrote a function that got some data from the database, and I wrote another function that stored that data in a hashmap in memory (using the memory as a cache) and I wrote another function that would check the hashmap for the data, before making the call to the database, and then I wrote another function that ran every 15 minutes and erased items in the hashmap, thus invalidating the cache. But then I realized I could combine several of these operations by creating a closure at the moment that I store data in the hashmap: the closure could call Thread.sleep(900000) and then delete the data from the hashmap. I found it convenient to create the closure while I was storing the data in the hashmap as, at that moment, I have all the information I need, including the key with which that data is being stored. Then I push the closure onto a queue and it gets processed in 15 minutes. This simplifies things quite a bit.
Again, Zach Tellman's code has been educating me about how powerful closures can be in the context of concurrent programming. Look here:
https://github.com/ztellman/aleph/blob/perf/src/aleph/http.c...
And look especially at this function:
(defn wrap-ring-handler "Takes a normal Ring handler, and turns it into a handler that can be consumed by Aleph's start-http-server. If the Ring handler returns an async-promise, this will be handled properly and sent along as a long-poll response whenever it is realized. This should be the outermost middleware around your function. To use an Aleph handler at a particular endpoint within the scope of this middleware, use wrap-aleph-handler."
[f]
(fn [ch request]
(run-pipeline request
{:error-handler (fn [ex] (error ch ex))}
;; call into handler
(fn [{:keys [body content-type character-encoding] :as request}]
(if (channel? body)
(if (options/channel-ring-requests? request)
;; leave channels as is
(f (assoc request ::channel ch))
;; move onto another thread, since there will be blocking reads
(task "input-stream-reader"
(f (assoc request
::channel ch
:body (formats/channel->input-stream body character-encoding)))))
(f (assoc request
::channel ch
:body (formats/bytes->input-stream body character-encoding)))))
;; send response
(fn [response]
(when-not (::ignore response)
(enqueue ch response))))))
You are asking why functional programming is a big deal. I'm sure different people will have different answers. I think the difficulty in answering is that there is no one thing that by itself makes an answer, rather, it is a whole series of things that combine in powerful ways. For me, the immutable data structures of Clojure are an education. I have learned a lot. I think using closures in the context of concurrent programming could be an absolute nightmare, if it was not for the safety offered by immutable data structures. And yet, I don't think immutable data structures are a big deal by themselves, nor do I think that closures are a big deal by themselves, nor do I think concurrency is a big deal by itself. Only when I look at the combination of these 3 do I find myself enlightened, like I've just discovered a whole new level of programming, a level of power I have never felt in any other language.
A final note: Adam Bard has a blog post "Effortless async by design" in which he says "I dare you to find a thread-safer language. Well, maybe Erlang." And then he also uses some code from Zach Tellman to demonstrate what he means. Bard's post is worth a read:
http://adambard.com/blog/why-clojure-part-2-async-magic/
I could list a lot of things that seem to me very powerful about the coding style I've learned this last year: composability, currying, closures, immutable data structures, thread-safe, etc. But I think descriptions like that tend to be a bunch of meaningless buzz words until you've dived in deep enough to see how they all interact with each other. Because its the interaction of these features that is really powerful, not any of them alone.
Uh, hey, I'm Zach. Thanks for the shout out. I'm mildly uncomfortable with you using Aleph as an example of good, idiomatic Clojure, but I suppose I can live with it.
If you're looking through my code in the future and have questions, please feel free to get in touch.
If you're looking through my code in the future and have questions, please feel free to get in touch.
Much of Clojure itself is written in Clojure. I would recommend going right to the source for idiomatic examples. https://github.com/clojure/clojure
I used to agree with the common objection that Clojure has poor documentation coverage, but then I realized that, usually, reading the source code was an excellent way to get a handle on things, given the habit of functional purity that Clojure developers seem to have. This is even more true with tools like Cider and Fireplace, which give you instant access.
People mostly benefit from immutable state, which is very useful for avoiding extra bugs. All other "expressiveness" garbage is for those who really dig it.
[deleted]
relevant wikipedia pages:
- http://en.wikipedia.org/wiki/Immutable_object - http://en.wikipedia.org/wiki/Persistent_data_structure
- http://en.wikipedia.org/wiki/Immutable_object - http://en.wikipedia.org/wiki/Persistent_data_structure
I can appreciate your position, because I mostly write programs that model the physical world. The state is what you care about, and the state changes over time, and you have to solve for the state iteratively-- which means that the transition from state [i] to state [i+1] includes a bunch of intermediate guesses at the state. Furthermore, those state updates take place element by element-- you rarely update an entire vector at once, but rather update one or two elements at a time, based on calculations related to some item in the physical world.
In short, you have a vector of reals that you update over and over, one element at a time. This is simply not an operation that functional programming excels at (there is an idea in the functional programming community that the state of a system, because it is a snapshot in time, never changes, and that therefore it is appropriate to model this as a bunch of immutable vectors that stream along in time-- but in practice I don't find that a very powerful, or useful, abstraction).
So my reaction to functional programming, after reading a couple of articles and books, was "Huh, it's cool, but not that applicable to my work."
I can imagine how this also would be the case in embedded systems programming. My brother designs and builds robotic manufacturing systems, and from my limited experience looking over his shoulder, it sure looks like those systems are all about managing state transitions.
Nevertheless, there are a lot of powerful ideas alive in the functional programming world, and over time these have influenced the way I do things. At the most basic, when I design programs, I try to represent the lowest-level descriptions of physical processes as pure functions-- no globals, all state passed in as immutable arguments, and all results returned as numbers. I do find this style of programming easier to test, and to reason about, and that's one of the primary benefits of FP you hear about.
There are other things, too, like my approach to input and output, and how I think of certain operations as "maps" now, but I think you get the point. In the big picture, I'm scavenging for ideas, rather than adopting an approach wholesale. It's like reading about Extreme Programming, and shifting to test-driven development but not pair programming.
My advice would be, look to adopt FP ideas piecewise, rather than all at once. Look for the small ideas and approaches that will make your programming more clear, or easier to test, or easier to re-apply from one system to another. If you've put in an honest effort to understand FP, and you just don't see how any of it applies to your work, then don't worry about it. There's no need for you to "be convinced" of anything. It's about building up your toolbox of tricks that work for your application domain.
In short, you have a vector of reals that you update over and over, one element at a time. This is simply not an operation that functional programming excels at (there is an idea in the functional programming community that the state of a system, because it is a snapshot in time, never changes, and that therefore it is appropriate to model this as a bunch of immutable vectors that stream along in time-- but in practice I don't find that a very powerful, or useful, abstraction).
So my reaction to functional programming, after reading a couple of articles and books, was "Huh, it's cool, but not that applicable to my work."
I can imagine how this also would be the case in embedded systems programming. My brother designs and builds robotic manufacturing systems, and from my limited experience looking over his shoulder, it sure looks like those systems are all about managing state transitions.
Nevertheless, there are a lot of powerful ideas alive in the functional programming world, and over time these have influenced the way I do things. At the most basic, when I design programs, I try to represent the lowest-level descriptions of physical processes as pure functions-- no globals, all state passed in as immutable arguments, and all results returned as numbers. I do find this style of programming easier to test, and to reason about, and that's one of the primary benefits of FP you hear about.
There are other things, too, like my approach to input and output, and how I think of certain operations as "maps" now, but I think you get the point. In the big picture, I'm scavenging for ideas, rather than adopting an approach wholesale. It's like reading about Extreme Programming, and shifting to test-driven development but not pair programming.
My advice would be, look to adopt FP ideas piecewise, rather than all at once. Look for the small ideas and approaches that will make your programming more clear, or easier to test, or easier to re-apply from one system to another. If you've put in an honest effort to understand FP, and you just don't see how any of it applies to your work, then don't worry about it. There's no need for you to "be convinced" of anything. It's about building up your toolbox of tricks that work for your application domain.
I appreciate your comment, but I think it's inaccurate to say that functional programming is suboptimal whenever you're dealing with stateful systems.
For me, the primary characteristic of functional programming is that it hems pretty closely to mathematics. And the fundamental language of mathematics is "pure." The calculus that underlies engineering is expressed in a pure language where time is simply a function parameter -- and this in itself doesn't mean that calculus is ill-suited for dynamic problems.
If you look at the Haskell world, for example, there is a ton of fascinating work being done with stateful systems, reactive systems, and so on. Functional programming doesn't exclude state and dynamicity. But it insists on modelling these things mathematically, that is, with a foundation of immutability, equational reasoning, etc.
See what I mean? I'm not saying that current implementations of functional languages are perfect in every scenario -- that's clearly false. But basically, or theoretically, functional programming is eminently capable of modelling state, and could even be argued to model it more coherently and intelligently than other paradigms.
For me, the primary characteristic of functional programming is that it hems pretty closely to mathematics. And the fundamental language of mathematics is "pure." The calculus that underlies engineering is expressed in a pure language where time is simply a function parameter -- and this in itself doesn't mean that calculus is ill-suited for dynamic problems.
If you look at the Haskell world, for example, there is a ton of fascinating work being done with stateful systems, reactive systems, and so on. Functional programming doesn't exclude state and dynamicity. But it insists on modelling these things mathematically, that is, with a foundation of immutability, equational reasoning, etc.
See what I mean? I'm not saying that current implementations of functional languages are perfect in every scenario -- that's clearly false. But basically, or theoretically, functional programming is eminently capable of modelling state, and could even be argued to model it more coherently and intelligently than other paradigms.
I'm not trying to be a troll. I'm trying to understand. And I don't.
I've been a professional programmer for 28 years. I don't care about "the one right way" or "ideological purity" or anything like that. I care about efficiently building programs that work. (Answers that say that functional programming is in fact the one right way will be dismissed as ideologues or trolls.)
I don't think that functional programming can be the one right way, because I have spent most of my career in embedded systems. Those programs tend to be full of states that changed many function calls ago, but are still relevant. If I understand correctly, that doesn't play well with functional programming.
But once you don't buy that functional programming is the one right way, then you have to ask, "When is it the right tool for the job? And when is it not?"
(And by the way, once you don't' think that functional programming is the one right way, then things like tail recursion and the Y combinator stop looking like elegant solutions to important problems. They start looking like crocks that you were forced into because you're using the wrong tool for the job.)
Look, I get that all of computer programming is mathematics. It's all Turing machines, too, but that doesn't mean that I want to write programs like I'm writing on a Turing machine. I don't want to write them like I'm writing an abstract algebra paper, either - unless there's a real benefit.
Again, I recognize that this sounds like a troll, but I'm really not. I can be convinced. Somebody say something convincing (hopefully in terms that a professional programmer with very little abstract algebra can understand).