Python Idioms in Rust(benjamincongdon.me)

182 pointsby cicerovor 8 Jahren78 comments

http://benjamincongdon.me/blog/2018/03/23/Python-Idioms-in-Rust/

12 comments

marshrayvor 8 Jahren

Very strange to see the core primitives of functional programming being sold as 'Pythonic'.

I suppose we all look at the world from the perspective of our own experience.

apendletonvor 8 Jahren

Honestly I suspect that this stems from a perception particularly among people from a non-traditional CS background that "functional programming" is intimidating, whereas Python is seen as approachable in a get-shit-done kind of way for programming novices. Python definitely did co-opt a lot of traditionally functional stuff, but did it in a way that obviates needing to know what a monad is, concern oneself with functional purity, etc., so there's a community of people now who don't care about any of that but are still familiar with some of these idioms, and this post is meant to make Rust look more accessible to them in a way that "Rust supports functional programming idioms" would not.

coldteavor 8 Jahren

>Honestly I suspect that this stems from a perception particularly among people from a non-traditional CS background that "functional programming" is intimidating, whereas Python is seen as approachable in a get-shit-done kind of way for programming novices.

Just people from a "non-traditional CS background"? Most CS graduates today will say the same thing with regards to e.g. Python vs Scheme or CL.

>Python definitely did co-opt a lot of traditionally functional stuff, but did it in a way that obviates needing to know what a monad is, concern oneself with functional purity, etc.

If you hanged around HN pre 2010, you'd seen that FP was all about Lisp (and things like map, reduce, macros, first class functions, etc), and few gave a rat's arse about "monads" and "purity" -- it was only starting to emerge in general consciousness, not dominant as it is today in FP circles.

ansiblevor 8 Jahren

>: If you hanged around HN pre 2010, you'd seen that FP was all about Lisp (and things like map, reduce, macros, first class functions, etc), and few gave a rat's arse about "monads" and "purity" -- it was only starting to emerge in general consciousness, not dominant as it is today in FP circles.

My recollection was a bit different. I got interested in Haskell around 2007 or so. And pre-2010, I recall encountering several articles about that on HN, though obviously there were often articles about the Lisp family.

pjmlpvor 8 Jahren

As someone that learned FP when Haskell did not exist and Miranda papers were still fresh, this mentality that only Haskell matters as FP notion really feels odd.

OtterCodervor 8 Jahren

I personally feel like Haskell makes a good reference implementation for functional programming. Whenever someone is explaining how to do some interesting FP thing in, say, JavaScript, I usually just ignore them and go read about how the same thing is done in Haskell, which is usually clearer and less burdened with the complexity of getting a mutable, proceedural, eager language to behave like an immutable, lazy, functional one.

jholmanvor 8 Jahren

If you think that immutability and laziness are inherently part of functional, then you're part of the new-school-is-the-only-school problem.

I mean, I'm not saying that immutability and laziness aren't interesting. But FP has a long long history, most of it's not lazy, and immutability is nowhere near universal.

willtimvor 8 Jahren

Completely agree. Anyone interested in functional programming should learn Haskell for this reason alone. Skip the blog post that falsely claims a "functor is a box" or a "monad is a burrito" and go read the original papers.

supergarfieldvor 8 Jahren

This is tangential to your main point, but:

> did it in a way that obviates needing to know what a monad is, concern oneself with functional purity,

Caring about both of these things in the context of functional programming essentially dates from the early 1990s. According to Wikipedia (I have no better source, but it matches what I know of pre-1990s functional languages),

> Eugenio Moggi first described the general use of monads to structure programs in 1991.

Functional programming existed before then. Scheme, an early functional language, is from the 1970s. It has basically all the FP idioms that you have in Python, is not pure, is dynamically typed, and the doc never once references monads.

I'd actually say the definition of FP has shifted towards pure, category-theory oriented languages, but "traditionally functional stuff" definitely doesn't include monads.

coldteavor 8 Jahren

>Functional programming existed before then [1991]. Scheme, an early functional language, is from the 1970s.

Lisp is even older, as was touted as the de facto functional language since forever. And earlier MLs didn't have monads either.

And of course, just because Moggi described monads in 1991, it doesn't mean it didn't take over a decade for them to make any kind of significant dent.

(Assuming that they have made much of a dent today, which outside the world of Haskell, their explicit (as opposed to implicit) use is close to 0, anyway).

grumpyprolevor 8 Jahren

The problem with "traditionally functional stuff" was that one could only use functional programming for a subset of problems. Monads allow Haskell programmers to use FP to solve problems involving IO, mutation and effects. This is not redefining FP, it is extending it to a wider class of problems.

papitovor 8 Jahren

You are saying that Python is like the Bill Nye of functional programming: not really a scientist, but does make it accessible to the masses.

vvandersvor 8 Jahren

Saying someone who was a mechanical engineer was "not really a scientist" kinda does him a bit of a disservice.

What made Bill Nye awesome was not only his passions for the subject matter but a solid background that let him communicate complex topics in a simple and engaging way.

kbensonvor 8 Jahren

Not to mention I don't recall him really calling himself a "scientist" in any formal sense, but my exposure to him has always been tangential. He was the "Bill Bye the Science Guy" in his original show, and afterwards I've always seem him explained as a science advocate.

I mean, what's a scientist anyway? If it's someone in constant pursuit of knowledge, does a constant reading of new material to learn more count?

icebrainingvor 8 Jahren

I'd say a scientist is someone who advances science, not just someone who reads results by others (unless they publish a study based on those results, like a meta-analysis or similar).

kbensonvor 8 Jahren

Does that mean the scientist that tests hypothesis that never works out, and never bothers to publish, isn't a scientist?

What about the person that actually does advance knowledge, for their self, but doesn't share it? Are you not a scientist if you don't let everyone know what you found? You would conceivably still be performing science.

Maybe the easiest way to define it as someone that intends to discover something new (or at least add new data to existing data, for corroboration or refutation)? I don't consider myself to be a scientist, but if I wanted to explore some topic in depth, and and started reading about it, and maybe performing my own small scale experiments to confirm what I had read, I think I would be "doing science" and I would then be at a minimum am amateur scientist.

[deleted]vor 8 Jahren

ceronmanvor 8 Jahren

Python had the primitives of functional programming since its conception. Those are not exclusive of pure FP languages.

Some particular names used in the examples were introduced by Python. For example the `enumerate` function or `zip`. I haven't seen those names used in programming languages older than Python, I know I might wrong. The point is that it's not crazy to believe that Rust design does borrow some things from Python (among other languages of course).

The article is comparing some particular Python idioms with Rust. The fact that those are functional I think is not particularly relevant. It's like someone seeing a comparison of Java and Go and saying that those are just the principles of imperative programming, or object oriented programming.

agumonkeyvor 8 Jahren

Python FP primitives came from an early friend or codev of Guido. He had experience in FP or Lisp and felt he needed them. Guido never really wanted that and just left them there.

All these names and ideas are clearly rooted in FP circles

fjhvor 8 Jahren

> Some particular names used in the examples were introduced by Python. For example the `enumerate` function or `zip`. I haven't seen those names used in programming languages older than Python, I know I might wrong.

All the ML dialects that I know have a zip function and ML pre-dates Python by about 20 years. This is particularly relevant since Rust is clearly strongly influenced by ML.

> The fact that those are functional I think is not particularly relevant. It's like someone seeing a comparison of Java and Go and saying that those are just the principles of imperative programming, or object oriented programming.

The comparison feels right, but I think it would certainly be odd if somebody explained loops in Go as "a Java idiom".

c3534lvor 8 Jahren

I suppose it makes sense in weird way, since both Python and Rust have these features implemented in similar ways because it stems from more traditional functional programming. You could also compare Java to C++ on account of them both being inspired by C.

ubernostrumvor 8 Jahren

It's possible for these constructs to be present in many languages and predate Python's adoption of them, and for it to be simultaneously true that a lot of idiomatic Python relies on these constructs.

slaydemonsvor 8 Jahren

Except the article contained non-functional concepts in it also. But hey at least you got to announce the fact you use/like functional programming languages.

tzaholavor 8 Jahren

I think it's nice that more and more languages take inspiration from the Python community.

For example, I've recently found a way to use Python-like type hints in C++. Crazy, huh? The trick is to replace "auto" with the concrete type name, e.g.:

auto x = 5;

becomes:

int x = 5;

tathougiesvor 8 Jahren

This is a joke right?

tzaholavor 8 Jahren

Joke? No, you can actually do this in C++!

tathougiesvor 8 Jahren

Of course you can do this in c++. The second form of your declaration is the standard way of doing things. How did you learn c++, that this was a surprise to you?

Secondly, that’s not a ‘type hint’ and c++ did not get this from python. Prior to c++17, there was no auto and all declarations were like this. If anything, Python got the names from c++

kbensonvor 8 Jahren

> I suppose we all look at the world from the perspective of our own experience.

Yes, but in addition to that, I think you can likely sell an idea to people much easier when it's couched in something they are already familiar with. There's been lots said about the functional programming capabilities of Rust, but if people don't already equate what they do as following some functional paradigm, they might find themselves uninterested in the discussion.

I think Python especially, with its "one way to do it" and (my perception) how people adhere to "the way to do it in Python", there's a good chance some people don't really know when they are using common functional patterns.

spyspyvor 8 Jahren

> Python especially, with its "one way to do it"

Despite being a core tenet of the language, Python is _really, really_ bad at following that rule. I agree with everything else you're saying, though. Those are still Python idioms even if they originated elsewhere.

davidcuddebackvor 8 Jahren

I think people take the "one way to do it" tenet a little too seriously. It comes from PEP20 [1], which to me seems to have a more jestful than dogmatic tone:

> There should be one-- and preferably only one --obvious way to do it.

> Although that way may not be obvious at first unless you're Dutch.

If I remember my history correctly, this was poking fun as Perl's "TIMTOWTDI" [2].

It also says one obvious way to do it. It doesn't preclude there being more than one way.

[1] https://www.python.org/dev/peps/pep-0020/

[2] https://en.wikipedia.org/wiki/There%27s_more_than_one_way_to...

jerfvor 8 Jahren

In 2004, "There should be only one way to do it" was a more realistic goal. 2004 puts us at Python 2.4 or so, but I think that's just when it got formalized into a PEP, it was around for a while before then. If we step back two years to 2002, we get to Python 2.1. We are now pre-generators, pre-with statement, pre-itertools, pre-decorators, and a lot of other things (useful itertools, etc.). There more often was only one sensible way to do a thing. Python evolved, but PEP20 didn't.

davidcuddebackvor 8 Jahren

> 2004 puts us at Python 2.4 or so, but I think that's just when it got formalized into a PEP

I didn't notice the date on the PEP, but I think you're right. I remember reading it when I learned Python circa 2002.

Found this (from 1999): https://mail.python.org/pipermail/python-list/1999-June/0019....

xapatavor 8 Jahren

The "obvious way" changes over time, as the language improves.

codetrottervor 8 Jahren

How so? Comprehensions vs loops vs recursive functions vs in some cases map/reduce/filter is the only example I can think of.

strkekvor 8 Jahren

There's also strings.

    'foo ' + bar
    'foo {}'.format(bar)
    'foo %s' % bar
    f'foo {bar}'

BerislavLopacvor 8 Jahren

As well as the string templates almost nobody knows about (https://docs.python.org/3.6/library/string.html#template-str...):

    Template('$foo bar').substitute(foo='foo')

tomsthumbvor 8 Jahren

Python has good facilities for a solid chunk of functional programming “in the small”. List comprehensions, map, filter, and reduce are all there with clear, easy to understand syntax. It’s no Haskell, but these are definitely niceties.

That said, I don’t see where the author put forward the idea that functional programming is Pythonic so much as “these are functional things that are great in python, here are some analogs in rust”.

ubernostrumvor 8 Jahren

I feel like list comprehensions have always been emphasized as an important part of writing idiomatic Python. And as Python added more constructs in that vein (generator expressions, dict and set comprehensions) the tutorials people used didn't really keep up.

In the same way, map and filter were often covered, but not always with an explanation of the style of programming they supported, or why that style could be useful. I do remember the original Dive Into Python did a good job of this, and of encouraging people to explore a more functional style, but I don't recall many other resources, at least from the (2004-ish) era when I was first learning Python, doing that.

As a result I have to be a bit careful these days; I still use Python's more functional constructs regularly, but I've learned to go easy on them when collaborating with others, since I know a lot of people who learn Python now are using materials that mostly only touch on list comprehensions.

repsilatvor 8 Jahren

The "real functional" map/reduce/filter functions tend to be discouraged by programmers and linters too. It is becoming (has become?) an established part of the culture -- "Lambda in a `map`? Use a list comprehension instead."

I guess it's a "sheep as a lamb" thing -- "The 'functional way' is either points-free or pointless."

Maybe it's better in Rust because the method-style `x.filter().map().reduce()` lets you read left-to-right, and the "global function" `reduce(map(filter(x)))` has to be read inside-out. If there is a good reason to do Python's global function way, maybe D's UFCS can bring them the best of both worlds.

ubernostrumvor 8 Jahren

It is becoming (has become?) an established part of the culture -- "Lambda in a `map`? Use a list comprehension instead."

Which is annoying, because there are times when using map or filter is cleaner; for example, if you're doing pipeline-y stuff, using the function-based approach is much easier to read and reason about than doing a bunch of nested comprehensions.

I also tend to think people don't use itertools enough. But it catches people by surprise enough that several of my messing-with-interviewers canned solutions to common problems make heavy use of itertools (including a FizzBuzz which contains no 'for' or 'if' statements).

yxhuvudvor 8 Jahren

> Personally, I like Rust’s extensive iterator functions even better than Python’s

Funny, when the rust example that gave rise to the comment looks more or less like Ruby[ * ] but with other syntax for blocks. The writer should get around more and try more other languages.

[ * ] Or any of a zillion other object oriented languages that have decent support for functional looking code.

apendletonvor 8 Jahren

My understanding, though, is that idiomatic Ruby inverts the control for iteration (array.each(|thing| do stuff)), which leads to slightly different ergonomics, whereas idiomatic Python is like what Rust is now; see http://journal.stuffwithstuff.com/2013/01/13/iteration-insid... for a comparison. Interestingly, Rust used to have iteration styled after Ruby's, and explicitly switched in rust 0.7: https://www.reddit.com/r/programming/comments/1hl2qr/rust_07...

I think the comparison here is also to Python because the specifics of the API, rather than just the patterns, are pretty obviously inspired specifically by Python (names of functions like zip, etc.), as opposed to other external-iterator languages like C#

sampovor 8 Jahren

Not sure if there is any point in implementing map_on_vec using map, but this code

    fn map_on_vec(vec: &Vec<i32>, func: fn(i32) -> i32) -> Vec<i32>
    {
        let mut new_vec = Vec::new();
        for i in 0..vec.len() {
            new_vec.push(func(vec[i]));
        }
        new_vec
    }

doesn't need to be based on mut, push and for. Something like this works, too:

    fn map_on_vec(vec: &Vec<i32>, func: fn(i32) -> i32) -> Vec<i32>
    {
        let new_vec: Vec<i32> = vec
            .iter()
            .map(|x| func(*x))
            .collect();
        new_vec
    }

At least it runs a tiny bit faster, without all the pushing.

shepmastervor 8 Jahren

And getting into idiomatic Rust + a bit of code golf + a bit of showing-off:

    fn map_on_vec(vec: &[i32], func: F)
    where
        F: Fn(i32) -> i32,
    {
        vec.iter().map(|x| func(*x)).collect()
    }

- There's no need for the temporary variable, can just return directly

- The `vec` argument should be a `&[T]` not a `&Vec<T>` [discouraged]

- The `func` argument shouldn't be a function pointer but instead take an unboxed, generic closure (no forced pointer indirection, can take a closure). You may even want to accept a `FnMut` to allow the closure to mutate itself.

[discouraged]: https://stackoverflow.com/q/40006219/155423

leshowvor 8 Jahren

shep! you forgot map_on_vec's return type. Thanks for writing the comment though, I was just about to point out Fn

vvandersvor 8 Jahren

I'm also looking forward to the impl trait work where you can return partial iterators as well instead of the massive type signature that they are today.

recursivevor 8 Jahren

> Neither Python nor Rust has ternary operators (which may be for the best).

Then it goes on to show the ternary conditional operator from python.

dom96vor 8 Jahren

If as a Python programmer you're looking for a compiled language that is similar to Python, check out Nim: https://nim-lang.org/.

ubernostrumvor 8 Jahren

Since it just came up again yesterday, it's worth noting that the term "compiled language" is a tricky one to use here:

https://nedbatchelder.com//blog/201803/is_python_interpreted...

So: is Python compiled? Yes. Is Python interpreted? Yes. Sorry, the world is complicated...

leshowvor 8 Jahren

I can't speak for OP but I think they are just differentiating between languages that produce an executable binary versus one that is interpreted/runs in a VM/is compiled to bytecode/whatever else

Can_Notvor 8 Jahren

And Nim produces a small binary with the ability to tell the GC if/when/how long to run, no need for any interpreter or VM. It targets c before final compilation.

leshowvor 8 Jahren

Okay? I wasn't talking about Nim or any language specifically.

alehander42vor 8 Jahren

There is even a library(in a very early stage) that tries to semi-automatically translate Python programs to equivalent Nim idioms: https://github.com/metacraft-labs/py2nim

raymondhvor 8 Jahren

Thanks for the nicely written Python/Rust comparison :-)

Is Rust's flat_map() equivalent to this in Python?

    def flat_map(func, it):
        return map(func, chain.from_iterable(it))

edflsafoiewqvor 8 Jahren

No, it's

    def flat_map(func, it):
        return chain.from_iterable(map(func, it))

[deleted]vor 8 Jahren

monocasavor 8 Jahren

The 'single line if' is even cooler than that in rust, IMO. Everything being an expression allows really neat combinations inside statements.

stochastic_monkvor 8 Jahren

It’s worth noting that C++ has been adopting python’s functional attributes, from range-based loops, <algorithm> , duck-typing, and fold expressions, to libraries supporting Python’s style more directly. Range-v3 [0] and cppitertools [1] stand out to me. The former is in a current proposal for inclusion in the C++20 standard.

[0] https://github.com/ericniebler/range-v3

[1] https://github.com/ryanhaining/cppitertools

ubernostrumvor 8 Jahren

I'm a bit surprised nobody has tried to start up the "what are tuples really for in Python" argument in this thread.

The definition given -- "Python tuples act like immutable lists" -- is correct as a description of behavior, but the decision of when and whether to use a tuple versus a list or some other structure can be a pretty contentious one in some Python circles.

cup-of-teavor 8 Jahren

Lists are for iterating, tuples are for unpacking.

kzrdudevor 8 Jahren

Wouldn't you say that lists are for homogenous collections and tuples for short heterogeneous ones? Of course we use the Python "duck typing" definition for homogenous.

cup-of-teavor 8 Jahren

You could, but I think mine is better. (firstname, lastname) and (x,y,z) are homogeneous collections. But they probably should be tuples. In a tuple the position describes the structure, in a list the position merely describes the order. Hence why tuples are best if you plan to unpack them (ie. firstname, lastname = name).

Trouble is Python doesn't completely separate the concepts like some languages do (e.g. C array/struct, R array/list). You can unpack lists and store heterogeneous items in them. Many libraries do this (e.g. SQL libraries will give a list per row by default). And if you follow the bash read idiom in Python using str.split you'll unpack a list as well. For me the distinction is about documenting my code.

The only real practical advantage I'm aware of is tuples are hashable. I don't think they are implemented anything like structs underneath.

lmmvor 8 Jahren

> (firstname, lastname) and (x,y,z) are homogeneous collections.

Disagree. In a language with a decent type system you would give them different types, because using a first name as a last name or an x as a z is always an error.

shoovor 8 Jahren

Yeah.

Another rule of thumb is that a lot of the time you can make code that uses tuples clearer by rewriting it to use namedtuple-defined struct values. If the language supported it, it'd probably also make sense most of the time to define a type for each tuple field, and have a type-checker look for errors.

Occasionally you see tuples used where they want immutable and hashable lists, but most of the time they're used as shorthand struct/"pod" tours types.

cup-of-teavor 8 Jahren

In what language would you make firstname and lastname or x and y different types?

lmmvor 8 Jahren

Well, I would in Rust for starters.

cup-of-teavor 8 Jahren

Interesting. I should probably try one of these typed languages. When I was thinking of types I was thinking about "physical" types like in C, not abstract types.

dllthomasvor 8 Jahren

Yeah, it's pretty common to think of types as "representational", and that really doesn't get at most of what they are good for.

Even in C you can get some good help from the type checker if you know how to use it. For instance, see https://dlthomas.github.io/using-c-types-talk/slides/slides.... for a technique that can turn a flakey segfault from misuse of an API into a compile time error (later slides do just that for a simple SDL program, with a modified header file).

kzrdudevor 8 Jahren

Even in C, we could use abstract types more; we could use wrapper structs more, to introduce abstraction where we often idiomatically do not.

(Again about Rust and C++ equally, if we'd only care about "physical" type, the string and the vector would be the same type. The distinct string type allows us to form the rules around exactly what kind of values we allow for or optimize for using that type.)

tathougiesvor 8 Jahren

Haskell

weatherlightvor 8 Jahren

looks more like ruby to me than python.

cup-of-teavor 8 Jahren

The Rust version of "zip" seems a bit odd as it's implemented like a method rather than a function. Little things like make me not enjoy a language as much.

coldteavor 8 Jahren

>as it's implemented like a method rather than a function

Why should it be a function? It can be a method on sequence like objects (trait)...

Python itself has tons of those...

cup-of-teavor 8 Jahren

Because the parameters to zip aren't positional, they are just multiple parameters of the same type.

Python does have some that annoy me a bit. I think str.join would be clearer as a function, but at least that does have a positional parameter.

coldteavor 8 Jahren

>Because the parameters to zip aren't positional, they are just multiple parameters of the same type.

How is that a counter-argument? It makes sense then that the method belongs to that type (iteratables).

microtonalvor 8 Jahren

The itertools crate has you covered:

https://docs.rs/itertools/0.7.8/itertools/fn.zip.html

By the way, the Iterator's zip function is no magic. You could trivially implement a standalone function using the zip method yourself:

https://docs.rs/itertools/0.7.8/src/itertools/free.rs.html#7...

Ruskyvor 8 Jahren

There's also UFCS:

    for (a, b) in Iterator::zip(as, bs) {
        // ...
    }

tabtabvor 8 Jahren

Sorry, but language doesn't matter that much if the entire environment/stack is decent. If you are using overly "clever" features of the language, then you may be writing write-only code that has a high probability of tripping up new hires.

I see too many trying to reinvent the database or operating system in application code. Don't.

GolDDranksvor 8 Jahren

What that has to do with anything? The author didn't advocate for reinventing the database or operating system in application code.

Also, Rust is very much an antithesis of write-only code; it's specifically aimed for programming at large.

leshowvor 8 Jahren

We have different definitions of clever. Using iterator combinators is pretty par for the course and sort of a minimum understanding I would expect anyone to have.