Functional Python Made Easy(hackflow.com)
hackflow.com
Functional Python Made Easy
http://hackflow.com/blog/2013/10/13/functional-python-made-easy/
8 comments
Forgive me if this is a silly question -- I'm an F# developer by trade, and I only know a bit of Python -- but why would you rather add a bunch of features common to strongly+statically-typed functional programming languages to Python instead of simply writing your code in a language which already has support for such features?
For example, you could write code you want to use GADTs and pattern-matching for in Haskell, then create a Python wrapper for scripting purposes. I understand that having your solution written in two languages generally requires more effort to keep things working, and it's more difficult to find qualified developers to hire. To me, it just seems like it would make more sense to use each language's strengths where you need them instead of trying to graft all the features you could ever want into a single language (which, IMO, is basically what C++ does, to it's detriment).
For example, you could write code you want to use GADTs and pattern-matching for in Haskell, then create a Python wrapper for scripting purposes. I understand that having your solution written in two languages generally requires more effort to keep things working, and it's more difficult to find qualified developers to hire. To me, it just seems like it would make more sense to use each language's strengths where you need them instead of trying to graft all the features you could ever want into a single language (which, IMO, is basically what C++ does, to it's detriment).
My main reason is that I don't write Scala or Haskell professionally. However, I do maintain Python at work. So, while dropping Python and switching to a functional language is not an option, importing a simple library is.
Programming "functionally" in python was one of the best changes I made to my coding practices (it certainly makes dealing with multiprocessing a lot easier!).
Programming "functionally" in python was one of the best changes I made to my coding practices (it certainly makes dealing with multiprocessing a lot easier!).
> F# developer by trade
I thought F# was quite a niche language and there weren't any "F# developers", just .NET/C# that also know some F#. How did you end up on this career path?
About grafting features onto Python: most Python guys come from the Linux land and have quite a problem with using a .NET/Mono language, and Haskell is/seems too intimidating and complex. Most also seem to dislike/fear the JVM. So there's only OCaml left and this is even more of a "niche language" than F#. Only viable option is extending Python, unfortunately Python is such a pain in the ass to properly extend so people just make libraries to "ease the pain".
I thought F# was quite a niche language and there weren't any "F# developers", just .NET/C# that also know some F#. How did you end up on this career path?
About grafting features onto Python: most Python guys come from the Linux land and have quite a problem with using a .NET/Mono language, and Haskell is/seems too intimidating and complex. Most also seem to dislike/fear the JVM. So there's only OCaml left and this is even more of a "niche language" than F#. Only viable option is extending Python, unfortunately Python is such a pain in the ass to properly extend so people just make libraries to "ease the pain".
That was true some years back before F# was released as a "first-class" language (F# 2.0 in VS2010), and the F# community is still (and probably always will be) dwarfed by the sheer number of C# programmers in the world. On the upside, the F# community has grown quite a lot in the last year; there's even an official F# Foundation now: http://fsharp.org
I ended up working with F# through my previous job; I founded a startup whose product as a .NET -> GPU JIT compiler. The idea was similar to what Xamarin is doing now with their C# (and now F#!) tools for building iOS and Android apps, in that we aimed to make GPU programming accessible to the everyday programmer.
I wrote the first version of our product in C#; it was around 30KLoC, still missing quite a lot of functionality, and buggier than I was really happy with. I'd heard that F# was based on ML, and that ML was designed for work like building compilers and theorem provers, so I decided to take the plunge and learn it. It was a little difficult to wrap my head around at first, but I stuck to it, and I ported+simplified our original C# codebase to F# within a couple of months, ending up with ~5KLoC and a new version of our product which was much faster, had fewer bugs, and overall easier to maintain than before. It sounds a little cheesy, but making the jump to F# and putting in the effort to learn it really well was one of the best career decisions I've ever made, and I haven't regretted it for a second. It's made me a much stronger developer overall.
I think you're spot on about extending Python with libraries to "ease the pain". As I've said before, Python is quite a good, useful language; however, it seems like a good chunk of the core "Python" libraries are actually C libraries designed for use with a Python wrapper.
And sure, Haskell can be a bit obtuse at first, but it's really not too bad if you actually take the time to learn it. I think the real problem is that "senior" developers get comfortable in whatever language they use day-to-day for a long time, then they try Haskell -- which is probably much different than other languages they've used before -- and since they don't immediately get it, they assume it's the language's fault and give up.
I ended up working with F# through my previous job; I founded a startup whose product as a .NET -> GPU JIT compiler. The idea was similar to what Xamarin is doing now with their C# (and now F#!) tools for building iOS and Android apps, in that we aimed to make GPU programming accessible to the everyday programmer.
I wrote the first version of our product in C#; it was around 30KLoC, still missing quite a lot of functionality, and buggier than I was really happy with. I'd heard that F# was based on ML, and that ML was designed for work like building compilers and theorem provers, so I decided to take the plunge and learn it. It was a little difficult to wrap my head around at first, but I stuck to it, and I ported+simplified our original C# codebase to F# within a couple of months, ending up with ~5KLoC and a new version of our product which was much faster, had fewer bugs, and overall easier to maintain than before. It sounds a little cheesy, but making the jump to F# and putting in the effort to learn it really well was one of the best career decisions I've ever made, and I haven't regretted it for a second. It's made me a much stronger developer overall.
I think you're spot on about extending Python with libraries to "ease the pain". As I've said before, Python is quite a good, useful language; however, it seems like a good chunk of the core "Python" libraries are actually C libraries designed for use with a Python wrapper.
And sure, Haskell can be a bit obtuse at first, but it's really not too bad if you actually take the time to learn it. I think the real problem is that "senior" developers get comfortable in whatever language they use day-to-day for a long time, then they try Haskell -- which is probably much different than other languages they've used before -- and since they don't immediately get it, they assume it's the language's fault and give up.
I thought F# was quite a niche language and there weren't any "F# developers"
I know of a couple full time F# developers, both working in finance. The finance sector seems to be a big advocate of functional languages with F#, Haskell, various ML's and APL derivatives being much more common than they are in other sectors
I know of a couple full time F# developers, both working in finance. The finance sector seems to be a big advocate of functional languages with F#, Haskell, various ML's and APL derivatives being much more common than they are in other sectors
Because for many of us F# and Haskell are not necessarily realistic options. Python runs in a lot more environments and it's still lacking a few (most importantly browsers).
Python is comfortable. I'd very much like a nice statically typed functional language, but that's neither F# nor Haskell. Clojure(script) + Typed Clojure is about the closest practical thing.
Python is comfortable. I'd very much like a nice statically typed functional language, but that's neither F# nor Haskell. Clojure(script) + Typed Clojure is about the closest practical thing.
I'm curious -- what do you like about Clojure that you don't like about F# and Haskell? Do you find Clojure's syntax easier to learn than F#'s or Haskell's, or is there another feature that particularly draws you to Clojure?
Clojure is closer to Python, thus more comfortable. The macros, persistent collections and core.async are a nice bonus. Also, in extreme cases I could relatively easily monkey-patch things, just as I can in Python.
F# (and particularly Haskell) are one extra degree of separation away. I love useful type systems, but I haven't been able to practically use one so far.
Typed Clojure looks like I could start using it piecemeal, but its type inference is very limited (much like Scala), so I'm not sure of its long-term usability.
F# (and particularly Haskell) are one extra degree of separation away. I love useful type systems, but I haven't been able to practically use one so far.
Typed Clojure looks like I could start using it piecemeal, but its type inference is very limited (much like Scala), so I'm not sure of its long-term usability.
Note that most functional languages, and definitely Haskell have persistent data structure libraries as that is the natural way to write data structures in those languages.
Sure, but Python doesn't, which was my point.
I have a similar opinion as the commenter above: I like Clojure more than F# and Haskell.
F# is no-go because of the .NET/Mono dependency. Haskell is no-go because of the community attitude. ML (the non-.NET F#) is an option; as well as Miranda (precursor to Haskell with less attitude).
Sometimes these discussions forget to take the human side of things on board, which is really important for many applications and long-term projects.
F# is no-go because of the .NET/Mono dependency. Haskell is no-go because of the community attitude. ML (the non-.NET F#) is an option; as well as Miranda (precursor to Haskell with less attitude).
Sometimes these discussions forget to take the human side of things on board, which is really important for many applications and long-term projects.
F# depending on Mono is no worse than Clojure depending on the JVM, really.
The Haskell community is very friendly, actually. I've always received useful advice.
The Haskell community is very friendly, actually. I've always received useful advice.
In addition, Mono has improved tremendously over the past 12-18 months since Xamarin has taken over the project. It's not yet in the category of some of the very highly-tuned JVM implementations (or the .NET CLR on Windows), but performance and support are plenty good enough for most applications.
What don't you like about the Haskell "community attitude"? My brief encounter with the Haskell community was very positive, so I'm curious.
Macros
Because it's just a library.
I don't think it's as easy or prudent to hop between languages as you make it out to be. To turn it around, if you can extend your language with features, why would you introduce a whole new language dependency into your project if you don't have to?
I don't think it's as easy or prudent to hop between languages as you make it out to be. To turn it around, if you can extend your language with features, why would you introduce a whole new language dependency into your project if you don't have to?
I think it's better to introduce a language dependency instead of extending an existing language with new features if those new features won't be a natural fit for your existing language.
For example, GADTs are natural in strongly+statically-typed languages (e.g., Haskell) because they're typically used to write code which you want the compiler to check for correctness (as much as possible); if your existing language is a dynamically-typed language (in this case, Python) the interpreter may still be able to enforce type-safety at run-time, but you'll lose the more practical benefit of knowing at compile-time that the code won't break. Adding a feature like this to Python would add additional complexity to the language while also taking a significant amount of time before it was supported well-enough across the entire range of Python environments to really catch on; and for all that hassle -- which you're going through just to stay within Python, remember -- you're receiving minimal gain, and it's likely to cause more headaches in the long term than it solves.
FWIW, I'm actually quite a big fan of Python, even though I don't use it much; in particular, the Python open-source community is excellent. I just think that languages should focus on solving whatever problem they solve -- it's easy to get carried away with adding "more power" and end up ruining something which already worked well for a particular problem.
For example, GADTs are natural in strongly+statically-typed languages (e.g., Haskell) because they're typically used to write code which you want the compiler to check for correctness (as much as possible); if your existing language is a dynamically-typed language (in this case, Python) the interpreter may still be able to enforce type-safety at run-time, but you'll lose the more practical benefit of knowing at compile-time that the code won't break. Adding a feature like this to Python would add additional complexity to the language while also taking a significant amount of time before it was supported well-enough across the entire range of Python environments to really catch on; and for all that hassle -- which you're going through just to stay within Python, remember -- you're receiving minimal gain, and it's likely to cause more headaches in the long term than it solves.
FWIW, I'm actually quite a big fan of Python, even though I don't use it much; in particular, the Python open-source community is excellent. I just think that languages should focus on solving whatever problem they solve -- it's easy to get carried away with adding "more power" and end up ruining something which already worked well for a particular problem.
Maybe because you want to keep your code base consistent, maybe because there lots of libraries and wrappers in python, maybe because you can run python on nearly all platforms including the jvm, etc.
I can see the argument for pattern matching, but algebraic data types seem entirely out of place. The primary benefit would be type checking and exhaustivity checks: neither of which fit with the dynamic nature of Python.
Functional != Strongly Typed
Functional != Strongly Typed
Argument matching has been discussed, there've been PEPs on that, and they couldn't come up with a satisfactory syntax. And that whatever you try, 'if/elif' statements are just more readable and streamlined than any syntax they've try to come up with.
As to strongly typed. Has anyone considered a strict attribute in python, I wonder? Because I'd love [1] to have compile time type checks for types/variables that I'm worried about.
[1] certainly not at a cost of having to contend with type hierarchies of a strongly typed language.
As to strongly typed. Has anyone considered a strict attribute in python, I wonder? Because I'd love [1] to have compile time type checks for types/variables that I'm worried about.
[1] certainly not at a cost of having to contend with type hierarchies of a strongly typed language.
The regularity of ADTs is nice, too, though tagged unions feel weird in dynamic languages.
[deleted]
Guido brought up Python functional programming during the 2012 PyCon keynote:
http://www.youtube.com/watch?v=EBRMq2Ioxsc#t=44m25s
This is also helpful - http://docs.python.org/2.7/howto/functional.html
I really like how re_find gives you the actual regex match right off the bat, instead of having to go through match_result.groups()!
Using groups lets you pull out parts of a match more easily. Perhaps it's a bit of a corner case, but for parsing logs, I find it invaluable.
re_find returns tuple of all captures if there are more than one. See parse ini example
That's definitely a bonus. But in that case, what about the "entire line" match, that you would get from match.groups(0)?
And if the captures are named? Is there a performance penalty for always using search instead of match? Can I pass re.* flags to alter the behavior of the engine? can I pre-compile regexes that I use frequently?
Please don't get me wrong; I appreciate the effort that went into this, but there appears to be a lot of flexibility (and performance) lost in the re_find function.
This is cool,
And if the captures are named? Is there a performance penalty for always using search instead of match? Can I pass re.* flags to alter the behavior of the engine? can I pre-compile regexes that I use frequently?
Please don't get me wrong; I appreciate the effort that went into this, but there appears to be a lot of flexibility (and performance) lost in the re_find function.
This is cool,
dict(imap(re_finder('(\w+)=(\w+)'), ini.splitlines()))
dict((line.split('=') for line in ini if '=' in line))You can pass flags and compiled regular expression. Pattern with named captures causes re_find to return dict.
You won't get entire match if you have several captures, but you can add a pair of parentheses around your regexp to circumvent this limitation.
As you can see, plenty of flexibility here. Also, you can fallback to naked re once or twice a year, not that much trouble.
You won't get entire match if you have several captures, but you can add a pair of parentheses around your regexp to circumvent this limitation.
As you can see, plenty of flexibility here. Also, you can fallback to naked re once or twice a year, not that much trouble.
Very neat, and great examples. I do a lot of data manipulation myself, and could see this being very helpful/useful.
neat, i do a lot of FP idioms in my code.
another option i frequently borrow from is google's "goopy": http://goog-goopy.sourceforge.net/goopy.functional.html
another option i frequently borrow from is google's "goopy": http://goog-goopy.sourceforge.net/goopy.functional.html
btw, Python 3.3 introduced ChainMap for "merging" dicts.
http://docs.python.org/dev/library/collections#collections.C...
http://docs.python.org/dev/library/collections#collections.C...
[deleted]
This is the closest thing i found for that: https://github.com/lihaoyi/macropy