Write More Classes(lucumr.pocoo.org)
lucumr.pocoo.org
Write More Classes
http://lucumr.pocoo.org/2013/2/13/moar-classes/
146 comments
Make your libraries easy to do things that are easy, but make hard things possible too.
IME Ronacher follows this dictum very well. I'd suggest that anyone who wants to see this try his Flask and Werkzeug packages.
As for storing extra state, about which many here have complained, I've found it really helpful that I can set werkzeug.http._accept_re to my own RE when I want to do something weird with media types. That is state that the vast number of users won't need to touch, yet the fact that it exists makes life better for someone who does need it. I'm sure there are numerous other examples I haven't had to bother with yet. Would we really be better off if this RE had to be passed in every time we handled a request? (Although I would understand if you argued this value should be stored in an object not in a module. I haven't needed that yet.)
On the other hand, routing to and handling resources with these packages is typically done with functions and decorators only, although the route decorators are methods of the application object. So Ronacher is not any sort of hardass about classes; he just does what works.
IME Ronacher follows this dictum very well. I'd suggest that anyone who wants to see this try his Flask and Werkzeug packages.
As for storing extra state, about which many here have complained, I've found it really helpful that I can set werkzeug.http._accept_re to my own RE when I want to do something weird with media types. That is state that the vast number of users won't need to touch, yet the fact that it exists makes life better for someone who does need it. I'm sure there are numerous other examples I haven't had to bother with yet. Would we really be better off if this RE had to be passed in every time we handled a request? (Although I would understand if you argued this value should be stored in an object not in a module. I haven't needed that yet.)
On the other hand, routing to and handling resources with these packages is typically done with functions and decorators only, although the route decorators are methods of the application object. So Ronacher is not any sort of hardass about classes; he just does what works.
I think there is an aspect to this article that won't be understood unless you're really part of the Python community. For a while now, one of Python's selling points to users from other languages has been "you don't need to write all that code" (most likely directed at Java), more specifically you can simply use a function rather than having to define a class just to define static methods etc. Over time, this has grown into the mantra of "You don't need a class just use a function/module". A lot of people seem to follow this viewpoint blindly, so much so that they consider it to be "Pythonic"; the way you "should" write python.
I think Armin's post is somewhat in line with the following post from the google testing blog. http://googletesting.blogspot.com/2008/12/static-methods-are...
I think Armin's post is somewhat in line with the following post from the google testing blog. http://googletesting.blogspot.com/2008/12/static-methods-are...
Is there anything to this debate that doesn't boil down to style? Python has a pretty decent object system. It also has first-class, nested functions with closures. You could write good OO code or good functional code according to your taste, and python libraries don't seem to strongly prefer one style over the other. I'm not terribly interested debating OO vs. functional in the general case, but I am interested in the pros and cons of each style in specific contexts.
For example, Python's lack of variable declarations sometimes leads to bugs involving scoping. (I've run into this a couple of times myself). Does this quirk become more of an issue in heavily functional code? Are there other language quirks that become troublesome in heavily OO code? In what circumstances might one style be preferred over another?
For example, Python's lack of variable declarations sometimes leads to bugs involving scoping. (I've run into this a couple of times myself). Does this quirk become more of an issue in heavily functional code? Are there other language quirks that become troublesome in heavily OO code? In what circumstances might one style be preferred over another?
In Python, the distinction is less strong than it is in some other languages. Since a class can implement __call__, allowing instances to be directly called like a function, even if an API specifies a function, you can pass a class instance in with a suitable __call__. So using functions doesn't have to tie to to it being a function forever and ever in the future (or breaking reverse compatibility, the way it does in most other languages. This isn't a perfect answer to the objections, but the objection is at its most weak in Python.
I must have missed something in my reading of the article - what is the relevance of "Condoms for Onions" to the essay?
I think the original video this is a response to, was really just saying 'A class should not be one method, that's a function'
That was only one of the things the video said. Overall, it was pointing out how overuse of classes was resulting in lots of extra complexity and additional lines of code, making programs harder to understand and maintain
[deleted]
The Java would have been prettier if he had used the new try-with-resources from Java 7.
You can do the same with python's msgpack as the C# version
Try this:
And while I agree with most of the article, I don't think writing more classes is a one-size-fits-all solution. Classes IMO, only makes sense from a heavily OOP point of view.
Try this:
packer = msgpack.Packer()
serialized = packer.pack('stuff you wanna pack')
unpacker = msgpack.Unpacker()
unpacker.feed(serialized)
print unpacker.unpack()
And while I agree with most of the article, I don't think writing more classes is a one-size-fits-all solution. Classes IMO, only makes sense from a heavily OOP point of view.
I dont think the OP was pointing to OOP when talking about classes. You can have classes without a meaningful class hierarchy that still conforms to the OP's desired programming style using protocols like the iterator protocol.
A better way to phrase the OP's salient point while sidestepping the polemic of OOP is "Standardize and use protocols".
A better way to phrase the OP's salient point while sidestepping the polemic of OOP is "Standardize and use protocols".
> You can do the same with python's msgpack as the C# version
You can't. The unpacker in msgpack for Python only reads full objects. The C# version lets me go into and out of streaming at will and even skip past objects without reading them into memory. The only thing the Python version of msgpack can do with the unpacker is buffering up bytes internally until an object is ready. That object however could still expand into a 10GB blob of data which I would then have to postprocess.
You can't. The unpacker in msgpack for Python only reads full objects. The C# version lets me go into and out of streaming at will and even skip past objects without reading them into memory. The only thing the Python version of msgpack can do with the unpacker is buffering up bytes internally until an object is ready. That object however could still expand into a 10GB blob of data which I would then have to postprocess.
Personally, I find it more natural to use closures when parsing in Python, not classes. BTW, my day job frequently involves writing parser in C.
if classes didn't have such a terribly verbose syntax in basically every language that has them, i'd be less opposed to using them.
in java, c++, or c#, to add a variable to a class, you have to repeat its name 4 times. once to declare, once in the constructor parameters, and once on each side of the assignment. why am i writing the same thing 4 times for what should be a core part of the language?
in haskell, you write it once (i'm not saying haskell's records are nice, but they got that part right). same with rust.
and with a function, you write it once.
in java, c++, or c#, to add a variable to a class, you have to repeat its name 4 times. once to declare, once in the constructor parameters, and once on each side of the assignment. why am i writing the same thing 4 times for what should be a core part of the language?
in haskell, you write it once (i'm not saying haskell's records are nice, but they got that part right). same with rust.
and with a function, you write it once.
FWIW, with C#'s object initialisation syntax you can cut that down to once too. eg
public class Foo
{
public string Bar { get; set;}
}
var foo = new Foo { Bar = "Hey there" };I've come to love Scala case classes:
case class Foo(bar: String, foo: String = "hello")
new Foo(bar = "Hey there")
new Foo(foo = "Hey there", bar = "Hello")
new Foo("Hey there")I get what you're saying, but good OO for me usually suggests nullary constructors, letting a class know when something happens as opposed to setting internal state directly, and the class saying let me do something for you using my state instead of giving you my state directly.
Providing a getter at least is sometimes practically unavoidable, so there's 1 repetition, but with a good IDE that's just a quick key combination over the variable name.
Providing a getter at least is sometimes practically unavoidable, so there's 1 repetition, but with a good IDE that's just a quick key combination over the variable name.
> good OO for me usually suggests nullary constructors
I'm curious, could you please elaborate on this point? If a class has any dependency, i usually find it better to require that dependency to be passed in the constructor, so the constructed instance can always be in a valid initialized state. Why do you find nullary constructors to be good OO?
I'm curious, could you please elaborate on this point? If a class has any dependency, i usually find it better to require that dependency to be passed in the constructor, so the constructed instance can always be in a valid initialized state. Why do you find nullary constructors to be good OO?
I came off too heavy-handed there. My intention was more to just say that variables shouldn't be put in constructors if they don't have to be. So, just railing more against the pattern that Java seems to have popularized where every private variable automatically has a getter/setter and can be set through the constructor. Certainly if an object requires some initial state that can never change, passing the value in through the constructor is usually most appropriate. Even then, I sometimes like to have a nullary constructor with an init() method. Just makes certain design patterns utilizing e.g. reflection or pooling a bit easier to manage in some languages.
Ok, i didn't get your point then. Well, i definitely agree on that the pattern/trend of auto-generating getters, setters and constructor parameters for every attribute in a class without considering if those attributes should be visible (or, worse, mutable) from the outside is pretty hideous.
About the nullary constructor + init() method for deserializing purposes, i don't have any strong opinion really, as long as consistency is kept throughout the code base/module that relies on that. Using Java's reflection API though, you can extract all necessary type information from non-nullary constructors in order to call them with the required dependencies, which is not essentially more complicated than instantiating the class and then setting its properties thought setter methods.
About the nullary constructor + init() method for deserializing purposes, i don't have any strong opinion really, as long as consistency is kept throughout the code base/module that relies on that. Using Java's reflection API though, you can extract all necessary type information from non-nullary constructors in order to call them with the required dependencies, which is not essentially more complicated than instantiating the class and then setting its properties thought setter methods.
I don't think that nullary constructors are so good. If you have dependencies then after constructing your object it'll be in an invalid state because the dependencies aren't set up correctly.
http://news.ycombinator.com/item?id=5207262
Is there a preferred way on HN to address a "duplicate" reply?
Is there a preferred way on HN to address a "duplicate" reply?
I would argue that this repetition is not essential. It happens when you assign a constructor argument to an attribute directly without any modification. But there are case where it doesn't appear:
How often such circumstances occur is another issue. The form of initialization that you mentioned is probably the most common so languages can provide shortcuts in this case.
- the attribute has a different name
- an attribute is set whose value is a function of more than one argument
- an attribute is set to a constant value independent of constructor arguments
How often such circumstances occur is another issue. The form of initialization that you mentioned is probably the most common so languages can provide shortcuts in this case.
> in java, c++, or c#, to add a variable to a class, you have to repeat its name 4 times. once to declare, once in the constructor parameters, and once on each side of the assignment. why am i writing the same thing 4 times for what should be a core part of the language?
Because Java, C++, and C# suck at this. I'm trying to see the relevance to the OP, which is about Python, which rather decidedly lacks this problem. To add a new instance variable in a class, you need to mention it once, or twice if your using a __slots__-based class for compact memory footprint (because then you have to add it to __slots__ and then actually assign it a value somewhere.)
Because Java, C++, and C# suck at this. I'm trying to see the relevance to the OP, which is about Python, which rather decidedly lacks this problem. To add a new instance variable in a class, you need to mention it once, or twice if your using a __slots__-based class for compact memory footprint (because then you have to add it to __slots__ and then actually assign it a value somewhere.)
Scala and TypeScript do it right, though.
Seems like he's asking for modularization so we can make special-case adjustments to libraries without completely monkey patching or rewriting them. Seems like a reasonable ask of a mature library.
On the streaming verus resident working set argument... Most of what most programmers deal with doesn't have to scale to deal with huge streaming datasets, so it doesn't get the attention.
On the streaming verus resident working set argument... Most of what most programmers deal with doesn't have to scale to deal with huge streaming datasets, so it doesn't get the attention.
Can anyone explain the statement about simplejson for me please?
Some libraries manage to skip the token part. (I'm looking at you
simplejson, a library that even with the best intentions in mind
is impossible to teach stream processing)When you are trying to parse extremely large documents or you only care about a small subset of a document, needing to load the entire parsed file into memory is problematic. Simplejson doesn't provide an API that allows for any other option.
This is in contrast to something like an XML sax parser, that allows you to register for events like "a foobar element was loaded". You get the foobar element while all the other tokens are thrown out the window as soon as they are parsed.
The complaint is that, somewhere, under the hood, simplejson is doing that token parsing, but because of their API, a user can't plug into it.
This is in contrast to something like an XML sax parser, that allows you to register for events like "a foobar element was loaded". You get the foobar element while all the other tokens are thrown out the window as soon as they are parsed.
The complaint is that, somewhere, under the hood, simplejson is doing that token parsing, but because of their API, a user can't plug into it.
> The complaint is that, somewhere, under the hood, simplejson is doing that token parsing, but because of their API, a user can't plug into it.
I read it exactly the other way around: simplejson skips an internal tokenization step, so even if you fork the library it is pretty much impossible to make it streaming, because there's no token stream to handle stream state.
I read it exactly the other way around: simplejson skips an internal tokenization step, so even if you fork the library it is pretty much impossible to make it streaming, because there's no token stream to handle stream state.
That's more or less right, masklinn. To the above writers, look at the Python source (I can't vouch for how the C implementation does things). It's not beautiful, but it's also not hard to understand.
The effective work is all done inside a big function. To customize it, you'd have to split that function into pieces and then glue it back together using a class or your own function coordinating the parts.
The effective work is all done inside a big function. To customize it, you'd have to split that function into pieces and then glue it back together using a class or your own function coordinating the parts.
IMO, the Flask example is pushing it - considering how large a job rendering a template is, the number of customization points is probably appropriate, but at some point you're going to get lost in a rabbit hole of wrapper functions that call wrapper functions and end up with three equally appropriate levels you might hook into because the code was written to keep you from having to duplicate a single line of code from the library in your alternative implementation-- never mind how confusing that makes things to the casual debugger (who wants to get to the actual meaty code to see what's wrong with it). Flexibility is useful, but it must justify the loss of simplicity.
But yes, in the JSON cases, having more flexibility than a single 'loads' is clearly justified.
But yes, in the JSON cases, having more flexibility than a single 'loads' is clearly justified.
Some note: When it comes to debugging I've rarely seen a better stacktrace output than the one Flask presents you in debug mode (provided by Werkzeug). It actually lets you open the damn sourcefile right there. Right now I'm (mis?)using Flask together with CouchDB to build a db application where you can dynamically define your entities, fields, code hooks and so on at runtime and using Flask for this has been great so far. Ronacher's attitude clearly shows through in how easy I could get Flask to behave in ways it was possibly never meant to (dynamically add view definitions, document mappings and so on).
So Armin, if you read this, thank you, it was a joy spending time with your framework (and source code) over the weekend ;)!
So Armin, if you read this, thank you, it was a joy spending time with your framework (and source code) over the weekend ;)!
> IMO, the Flask example is pushing it - considering how large a job rendering a template is, the number of customization points is probably appropriate, but at some point you're going to get lost in a rabbit hole of wrapper functions that call wrapper functions
The actual implementations do a bit more. I have pretty much overriden every single part of that at one point, if for no other reason than debugging. Some of those hooks were added later because people requested them.
The actual implementations do a bit more. I have pretty much overriden every single part of that at one point, if for no other reason than debugging. Some of those hooks were added later because people requested them.
I find that I like it when classes store configuration that's known at load time, rather than state. That way it's a lot like a Common Lisp program, except multiple programs can run in the same process, have different configurations, and communicate with each other.
I think the real point the author is trying to make is: use modular, layered designs of composable components, instead of monolithic APIs that only have a single entry point. The single-entry-point model imposes costs and decisions onto the application that are hard to work around.
I think this is a good point. I think that it's hard to get from there to "more classes are always better" though. More classes don't always make a design more flexible. You have to consciously design for flexibility and reusability, and even then it takes a lot of deep thought and hard work to get your interfaces right, such that they are truly reusable in practice.
I think this is a good point. I think that it's hard to get from there to "more classes are always better" though. More classes don't always make a design more flexible. You have to consciously design for flexibility and reusability, and even then it takes a lot of deep thought and hard work to get your interfaces right, such that they are truly reusable in practice.
Quoting from the end of the article: "And our solution to monolithic code in Python are classes."
So, I guess the question I ask as someone who doesn't write a ton of python code is: Is that true? And if so, why? Is it not possible to compose a layered API that doesn't rely on inheritance / classes in Python?
So, I guess the question I ask as someone who doesn't write a ton of python code is: Is that true? And if so, why? Is it not possible to compose a layered API that doesn't rely on inheritance / classes in Python?
> So, I guess the question I ask as someone who doesn't write a ton of python code is: Is that true?
I don't think it is. His objections to the json library API can be addressed without introducing a single class.
He seems to want access to the internal implementation of converting bytes into a single Python object. That is probably just an internal function that can just be exposed. If he wants to override it from the top level, the library could permit that by just adding the function as a default parameter.
He wants it to deal with streaming. Python generators can help with this. The library could be refactored into implementing a json.load_stream function instead, which takes a string generator and returns some kind of object stream generator. The original json.loads function could be replaced with a wrapper around it for backwards compatibility and for API users who don't want the additional complexity.
None of this involves writing classes. A generator could be seen as a type of class instance, and one can implement a generator with classes, but generators have a well known and simple API which eliminates the need to define a new API, as the traditional writing of classes would do.
It seems to me that the writer has little experience of not using classes, so wants to solve every problem he has with a class. The API problems he want to address are genuine, but he's blind to any solution that doesn't use classes.
I don't think it is. His objections to the json library API can be addressed without introducing a single class.
He seems to want access to the internal implementation of converting bytes into a single Python object. That is probably just an internal function that can just be exposed. If he wants to override it from the top level, the library could permit that by just adding the function as a default parameter.
He wants it to deal with streaming. Python generators can help with this. The library could be refactored into implementing a json.load_stream function instead, which takes a string generator and returns some kind of object stream generator. The original json.loads function could be replaced with a wrapper around it for backwards compatibility and for API users who don't want the additional complexity.
None of this involves writing classes. A generator could be seen as a type of class instance, and one can implement a generator with classes, but generators have a well known and simple API which eliminates the need to define a new API, as the traditional writing of classes would do.
It seems to me that the writer has little experience of not using classes, so wants to solve every problem he has with a class. The API problems he want to address are genuine, but he's blind to any solution that doesn't use classes.
> It seems to me that the writer has little experience of not using classes, so wants to solve every problem he has with a class.
That's a bold statement.
Where is the trend to forcefully avoid classes here coming from? Classes are a perfectly valid tool in Python to solve problems. Someone sent me a mail this morning proposing a dictionary with callback functions in addition to a function as if that would solve anything.
That's a bold statement.
Where is the trend to forcefully avoid classes here coming from? Classes are a perfectly valid tool in Python to solve problems. Someone sent me a mail this morning proposing a dictionary with callback functions in addition to a function as if that would solve anything.
> Where is the trend to forcefully avoid classes here coming from?
See the "Stop Writing Classes" video.
Classes hold state. More state adds more complexity, and typical OO design add lots of layers of indirection which also add complexity. Most of the time, you don't need the flexibility that the extra indirection gives you. So you get complexity for little benefit.
If you don't need to hold state, then a collection of functions will usually suffice. In Python, you can use modules for this.
By all means use classes if they are the best way to solve a problem. But for those who know nothing else, how do they even know this?
My problem with this article is that it says "I have a problem; I can solve it with classes; therefore everyone should use classes more". It fails to even consider anything else.
See the "Stop Writing Classes" video.
Classes hold state. More state adds more complexity, and typical OO design add lots of layers of indirection which also add complexity. Most of the time, you don't need the flexibility that the extra indirection gives you. So you get complexity for little benefit.
If you don't need to hold state, then a collection of functions will usually suffice. In Python, you can use modules for this.
By all means use classes if they are the best way to solve a problem. But for those who know nothing else, how do they even know this?
My problem with this article is that it says "I have a problem; I can solve it with classes; therefore everyone should use classes more". It fails to even consider anything else.
> Classes hold state. More state adds more complexity, and typical OO design add lots of layers of indirection which also add complexity. Most of the time, you don't need the flexibility that the extra indirection gives you. So you get complexity for little benefit.
Functions hold state as well, they just encapsulate it. You can still break up your algorithm into a class with multiple template methods and then encapsulate it in a function if you're afraid of leaking state elsewhere.
What I see instead is that people can't get rid of state and put it as global variables into Python modules. Case in point: pickle. It uses sys.modules and pickle.dispatch_table to store some of its state.
> If you don't need to hold state, then a collection of functions will usually suffice. In Python, you can use modules for this.
Classes have another advantage: virtual method calls. If one function in a module calls into another function in the module I can only do two things: a) copy/paste the library and change one of the functions or b) a monkeypatch which modifies a shared resource someone else might want to use with the old semantics.
A class I can subclass and change the behavior for each method separately.
Functions hold state as well, they just encapsulate it. You can still break up your algorithm into a class with multiple template methods and then encapsulate it in a function if you're afraid of leaking state elsewhere.
What I see instead is that people can't get rid of state and put it as global variables into Python modules. Case in point: pickle. It uses sys.modules and pickle.dispatch_table to store some of its state.
> If you don't need to hold state, then a collection of functions will usually suffice. In Python, you can use modules for this.
Classes have another advantage: virtual method calls. If one function in a module calls into another function in the module I can only do two things: a) copy/paste the library and change one of the functions or b) a monkeypatch which modifies a shared resource someone else might want to use with the old semantics.
A class I can subclass and change the behavior for each method separately.
> You can still break up your algorithm into a class with multiple template methods
Template methods are an abomination. They are only widely used because many OO languages don't have first-class functions. A template method means that the method is being parameterized by one or more functions. Parameters should be expressed as parameters! You wouldn't design a Circle class so that you need to subclass it to specify the radius of the circle. The radius is a parameter. Likewise for the callback functions used by a template method. Template methods are intrinsically difficult to understand and spaghetti-like, unless well-documented, because it is not immediately clear what the parameters to it are. And when overriding a callback method, it's not immediately clear what template method the overridden method is paramaterizing. Or that it is even parameterizing anything. (Caveat: Sometimes template methods are warranted, but they are greatly overused.)
> Classes have another advantage: virtual method calls.
You don't need virtual method calls in order to parameterize functionality. See previous paragraph. Furthermore, it is not safe to override a method without knowing if it has been designed for overriding, and what the class expects of the overriding method. See Effective Java, "Item 17: Design and document for inheritance or else prohibit it". Also see "Item 16: Favor composition over inheritance", and a similar discussion in the Gang of Four book.
> If one function in a module calls into another function in the module I can only do two things: a) copy/paste the library and change one of the functions or b) a monkeypatch which modifies a shared resource someone else might want to use with the old semantics.
If a method is not designed to be overridden, you shouldn't override it anyway. (See previous paragraph.) When using a functional approach (in a language with first-class functions), you can usually design for flexibility just as easily or more easily than when using an inheritance-based OO approach for flexibility.
Template methods are an abomination. They are only widely used because many OO languages don't have first-class functions. A template method means that the method is being parameterized by one or more functions. Parameters should be expressed as parameters! You wouldn't design a Circle class so that you need to subclass it to specify the radius of the circle. The radius is a parameter. Likewise for the callback functions used by a template method. Template methods are intrinsically difficult to understand and spaghetti-like, unless well-documented, because it is not immediately clear what the parameters to it are. And when overriding a callback method, it's not immediately clear what template method the overridden method is paramaterizing. Or that it is even parameterizing anything. (Caveat: Sometimes template methods are warranted, but they are greatly overused.)
> Classes have another advantage: virtual method calls.
You don't need virtual method calls in order to parameterize functionality. See previous paragraph. Furthermore, it is not safe to override a method without knowing if it has been designed for overriding, and what the class expects of the overriding method. See Effective Java, "Item 17: Design and document for inheritance or else prohibit it". Also see "Item 16: Favor composition over inheritance", and a similar discussion in the Gang of Four book.
> If one function in a module calls into another function in the module I can only do two things: a) copy/paste the library and change one of the functions or b) a monkeypatch which modifies a shared resource someone else might want to use with the old semantics.
If a method is not designed to be overridden, you shouldn't override it anyway. (See previous paragraph.) When using a functional approach (in a language with first-class functions), you can usually design for flexibility just as easily or more easily than when using an inheritance-based OO approach for flexibility.
> Furthermore, it is not safe to override a method without knowing if it has been designed for overriding, and what the class expects of the overriding method.
This is true, but likewise, its not safe to parameterize functionality by passing callbacks unless you know what the function you are passing the callback to expects of callback functions (in terms of arguments, return values, and side effects.)
Both of these, really, are the same principle: its not safe to call code without knowing what the code expects of you when calling it.
This is true, but likewise, its not safe to parameterize functionality by passing callbacks unless you know what the function you are passing the callback to expects of callback functions (in terms of arguments, return values, and side effects.)
Both of these, really, are the same principle: its not safe to call code without knowing what the code expects of you when calling it.
> This is true, but likewise, its not safe to parameterize functionality by passing callbacks unless you know what the function you are passing the callback to expects of callback functions
Of course; this goes without saying. But when programming in an OO style, programmers often override methods thinking that they are only changing the behavior of that one method, when in fact they could be inadvertently changing the behavior of other methods in the class.
When passing in functions as arguments, on the other hand, this makes it much more explicit what is supported and what isn't.
Of course; this goes without saying. But when programming in an OO style, programmers often override methods thinking that they are only changing the behavior of that one method, when in fact they could be inadvertently changing the behavior of other methods in the class.
When passing in functions as arguments, on the other hand, this makes it much more explicit what is supported and what isn't.
> Of course; this goes without saying. But when programming in an OO style, programmers often override methods thinking that they are only changing the behavior of that one method
This is not a problem with using classes or overriding methods, its a problem either with either failure to document behavior on the part of the library author or failure to read documentatio on the part of the library consumer.
Neither of those is any less a danger in the case here the design of the library involves parameterizing functionality by passing functions around as parameters.
This is not a problem with using classes or overriding methods, its a problem either with either failure to document behavior on the part of the library author or failure to read documentatio on the part of the library consumer.
Neither of those is any less a danger in the case here the design of the library involves parameterizing functionality by passing functions around as parameters.
> This is not a problem with using classes or overriding methods, its a problem either with either failure to document behavior on the part of the library author or failure to read documentation on the part of the library consumer.
Missing or poor documentation is a sad reality of programming in the real world. And is the normal state when working on a active project with other developers. This would be mitigated if programmers were very careful about declaring methods to be final if it's not perfectly safe to override them, but most are not that careful, or even fully aware of the issues. And in many programming languages there is no way to declare a method to be final.
> Neither of those is any less a danger in the case here the design of the library involves parameterizing functionality by passing functions around as parameters.
When programming in a functional style, functionality that is not intended to be modified by the client will not provide a parameter for doing so, so this significant source of confusion is eliminated.
Missing or poor documentation is a sad reality of programming in the real world. And is the normal state when working on a active project with other developers. This would be mitigated if programmers were very careful about declaring methods to be final if it's not perfectly safe to override them, but most are not that careful, or even fully aware of the issues. And in many programming languages there is no way to declare a method to be final.
> Neither of those is any less a danger in the case here the design of the library involves parameterizing functionality by passing functions around as parameters.
When programming in a functional style, functionality that is not intended to be modified by the client will not provide a parameter for doing so, so this significant source of confusion is eliminated.
> This would be mitigated if programmers were very careful about declaring methods to be final if it's not perfectly safe to override them
Its always perfectly safe to override a method if you maintain its required features. Its rarely, if ever, perfectly safe to do so if you don't. The issue is correctly documenting the required features.
Its always perfectly safe to override a method if you maintain its required features. Its rarely, if ever, perfectly safe to do so if you don't. The issue is correctly documenting the required features.
> What I see instead is that people can't get rid of state and put it as global variables into Python modules.
What I see is people writing classes that hold state in instances for far longer than is needed. Other code written around these classes then need to take this into account. This tends to the same place where global variables are. A tangled mess.
> If one function in a module calls into another function in the module I can only do two things...
3) Adjust the function to include the behaviour that you need. In Python, default parameters often mean that you can do this without breaking backwards compatibility. If a function is deficient in some way, why not fix the function rather than working around it?
What I see is people writing classes that hold state in instances for far longer than is needed. Other code written around these classes then need to take this into account. This tends to the same place where global variables are. A tangled mess.
> If one function in a module calls into another function in the module I can only do two things...
3) Adjust the function to include the behaviour that you need. In Python, default parameters often mean that you can do this without breaking backwards compatibility. If a function is deficient in some way, why not fix the function rather than working around it?
I'd be so bold as to make a categorical statement: If your classes are invariably stateful, or if they can reasonably be described with the phrase "lots of layers", then you're doing it wrong. You're not doing object-oriented programming so much as the mongrel "class-oriented programming" style that rose to prominence among C++ programmers in the 1990s.
Good well-modularized object-oriented code should not be an alternative to a collection of functions. It should be a technique for making your collections of functions more effective. It allows you to write a smaller, more concise collection of functions that can operate in a consistent manner on a whole plethora of datatypes. It accomplishes this because you can write your functions against a common set of mixins that your datatypes inherit rather than having to worry about specializing them for every datatype.
Good well-modularized object-oriented code should not be an alternative to a collection of functions. It should be a technique for making your collections of functions more effective. It allows you to write a smaller, more concise collection of functions that can operate in a consistent manner on a whole plethora of datatypes. It accomplishes this because you can write your functions against a common set of mixins that your datatypes inherit rather than having to worry about specializing them for every datatype.
The question is, what classes do that module systems don't? The answer is generally inheritance, and through it, class polymorphism.
With first class function, you hardly need class polymorphism. Just pass the function as argument already, don't bother with writing a whole new class just to override one method.
The other use for inheritance is plain code reuse. This is bad most of the time because it promotes thick interfaces, and function calls are great at code reuse anyway.
With first class function, you hardly need class polymorphism. Just pass the function as argument already, don't bother with writing a whole new class just to override one method.
The other use for inheritance is plain code reuse. This is bad most of the time because it promotes thick interfaces, and function calls are great at code reuse anyway.
> With first class function, you hardly need class polymorphism. Just pass the function as argument already, don't bother with writing a whole new class just to override one method.
Here, here! After returning to functional programming after a 30 year hiatus, I am beginning to realize just how detrimental the OO orthodoxy has been, and just how impoverished the world has been without first class functions. If only Scheme had caught on in the mainstream when it was invented, the programming world would be a much happier and safer place.
Here, here! After returning to functional programming after a 30 year hiatus, I am beginning to realize just how detrimental the OO orthodoxy has been, and just how impoverished the world has been without first class functions. If only Scheme had caught on in the mainstream when it was invented, the programming world would be a much happier and safer place.
> Here, here!
Where, where? ITYM "Hear! Hear!"
http://en.wikipedia.org/wiki/Hear,_hear
"Hear, hear is an expression used as a short, repeated form of hear him, hear him. It represents a listener's agreement with the point being made by a speaker."
Where, where? ITYM "Hear! Hear!"
http://en.wikipedia.org/wiki/Hear,_hear
"Hear, hear is an expression used as a short, repeated form of hear him, hear him. It represents a listener's agreement with the point being made by a speaker."
The question is, what classes do that module systems don't?
Modules (at least in Python) are singletons.
Modules (at least in Python) are singletons.
Doesn't count. When you define a single data structure in a module, (along with functions to manipulate it) you can instantiate it just like you would a class.
Ugh. That's like saying that high-level programming languages "don't count" because you can do that stuff in assembler.
I'm not arguing Turing completeness here. I'm arguing trivial one to one mapping. Syntax sugar. Compare:
struct foo {
int bar;
float baz;
};
int get_bar(foo);
float get_baz(foo);
class foo {
public:
int get_bar();
float get_baz();
private:
int bar;
float baz;
};
You can do pretty much anything with default arguments, rather than overridable class methods. First class functions make this very flexible.
Sometimes this is a little inconvenient. I think the sign you really need a class is when you are sending in something called "object_type", and a dict of "object_type -> function".
Sometimes this is a little inconvenient. I think the sign you really need a class is when you are sending in something called "object_type", and a dict of "object_type -> function".
...indeed, the one thing I "hate" about Python is that somehow it pushes your mind away from functional solutions. You end up writing either OO or procedural code, although in a language with first class functions and immutable strings you's expect to slip more to the functional side.
Anyone else had the feeling that although Python allows functional style, it somehow pushes your mind away from it?
Anyone else had the feeling that although Python allows functional style, it somehow pushes your mind away from it?
Personally I find myself avoiding writing in a functional style in Python because of it's lack of good anonymous function literals and poor handling of closures in nested functions.
Edit: when I say "poor handling of closures in nested functions" I'm referring to having to use nonlocal in order to modify variables in the parent function's scope.
Edit: when I say "poor handling of closures in nested functions" I'm referring to having to use nonlocal in order to modify variables in the parent function's scope.
yeah, I think lack of real anonymous functions is a small annoyance that can get quite "big" from the yucky feeling you get when defining functions just to pass them on. But... what do you mean by "poor handling of closures in nested functions"?
Edit: ok, I get what you mean now :) ...though I consider this "poor handling" a feature of Python - mutating outer scope variables is a side-effect and what it encourages is what I like to call "callback hell functions with data Javascript-style nightmare" :) (true, some people know to keep their sanity and write readable code even in this style, and if you enjoy it you're probably one of them few, but most don't...). By "functional style", I and some people refer to "functional as in mathematical functions", so only using side-effects when actually needed (yeah, some "extremist" languages like Haskell try to never do them, but...) and you get this by having immutable values and passing around anonymous functions (that don't modify their outer scope).
...and this is another part of the paradox: this "feature" of Python should encourage what I call functional style, but why oh why it doesn't? ...maybe Rich Hickey is on to something with Clojure and it's actually more about the data-structures than the language syntax or semantics?
Edit: ok, I get what you mean now :) ...though I consider this "poor handling" a feature of Python - mutating outer scope variables is a side-effect and what it encourages is what I like to call "callback hell functions with data Javascript-style nightmare" :) (true, some people know to keep their sanity and write readable code even in this style, and if you enjoy it you're probably one of them few, but most don't...). By "functional style", I and some people refer to "functional as in mathematical functions", so only using side-effects when actually needed (yeah, some "extremist" languages like Haskell try to never do them, but...) and you get this by having immutable values and passing around anonymous functions (that don't modify their outer scope).
...and this is another part of the paradox: this "feature" of Python should encourage what I call functional style, but why oh why it doesn't? ...maybe Rich Hickey is on to something with Clojure and it's actually more about the data-structures than the language syntax or semantics?
Well it does not really help that Python lacks tail call elimination (last I checked), which is fairly important to (practical) functional programming.
Meh, you'd use fold, map or filter 99% of the time in a functional language like scala or haskell. TCO is great, but you can still write a lot of idiomatic functional stuff even without it.
Right! TCO is not essential to practical functional programming. TCO is essential to practical use of the subset of functional programming that insists on recursion instead of iteration, including fold/map/filter.
TCO is an under-the-hood behaviour that the programmer would count on for practical performance. There's no reason that under-the-hood magic can't be in the implementation of fold/map/filter as primitive control ops.
It's just an accident of history that the [mp]atriarch functional programming language is Lisp, a language that was created in the name of proving a point about recursion. Recursion as the solution-to-all-problems is not essential to functional thinking. The essential ideas are first-class functions, higher-order functions, and referential transparency.
I think this mistake is comparable to thinking that classes are essential to object-orientation. I think many people are confused by university professors who love Scheme (for reasonable reasons), and who like to pretend that Lisps are the only alternatives to Fortran (arguably true in the '60s).
TCO is an under-the-hood behaviour that the programmer would count on for practical performance. There's no reason that under-the-hood magic can't be in the implementation of fold/map/filter as primitive control ops.
It's just an accident of history that the [mp]atriarch functional programming language is Lisp, a language that was created in the name of proving a point about recursion. Recursion as the solution-to-all-problems is not essential to functional thinking. The essential ideas are first-class functions, higher-order functions, and referential transparency.
I think this mistake is comparable to thinking that classes are essential to object-orientation. I think many people are confused by university professors who love Scheme (for reasonable reasons), and who like to pretend that Lisps are the only alternatives to Fortran (arguably true in the '60s).
One can write Python in a procedural manner, and even in a functional way, so one can use the same techniques as are used in those paradigms (OO is probably more natural for Python though).
You can also easily write component-based architectures in Python, for example using zope.interface/zope.component (ie: the modules only, not the full Zope CMS). For example the Trac bug tracker/wiki works like this, and for its purpose I think it's a perfect fit. It allows re-usable and pluggable components with declared, adaptable interfaces that are not restricted by some hierarchical taxonomy the framework designer came up with.
At the end of the day there will always be different ways to build layered architectures using re-usable components, and classes are just one of them. I'm pretty sure the author of the article didn't intend it like that, but 'write more classes' doesn't sound like great advice for novice/inexperienced programmers.
At the end of the day there will always be different ways to build layered architectures using re-usable components, and classes are just one of them. I'm pretty sure the author of the article didn't intend it like that, but 'write more classes' doesn't sound like great advice for novice/inexperienced programmers.
It is perfectly acceptable to design a library with a simplified, restricted interface that covers the majority of use cases, but not all. e.g. requests vs urllib2 For me, that's a win for common sense. I have never even considered streaming JSON.
The lack of streaming support should be a clue that the OP needs to look at whether what he's attempting is even sensible or could be better served by a different transport protocol with built in streaming support.
JSON supports streaming. You not having considered it is not an argument.
Firstly:
http://www.ietf.org/rfc/rfc4627.txt
"An implementation may set limits on the size of texts that it accepts. " - How does that "support" streaming?
My argument is that most people don't need it most of the time (I can't prove this). It's also NOT a protocol designed for streaming. As regards API design, it's a matter of taste but my preference and that of many others who use python, as we can see with the popularity of requests, Flask, is that an API should be designed for simplicity for the common use cases rather than for absolute completeness. I'm not saying don't expose a low-level API full of BufferedReaders, JSONParserCursors, but don't force it on me to parse a tiny JSON document of known length.
http://www.ietf.org/rfc/rfc4627.txt
"An implementation may set limits on the size of texts that it accepts. " - How does that "support" streaming?
My argument is that most people don't need it most of the time (I can't prove this). It's also NOT a protocol designed for streaming. As regards API design, it's a matter of taste but my preference and that of many others who use python, as we can see with the popularity of requests, Flask, is that an API should be designed for simplicity for the common use cases rather than for absolute completeness. I'm not saying don't expose a low-level API full of BufferedReaders, JSONParserCursors, but don't force it on me to parse a tiny JSON document of known length.
The protocol can support streaming very well. How easy it is or if it's possible at all is an implementation issue.
With the Go standard library's json implementation, for example, you can decode JSON streams very easily: http://golang.org/pkg/encoding/json/#example_Decoder
[deleted]
Write no classes!
Joe Armstrong: "I think the lack of reusability comes in object-oriented languages, not in functional languages. Because the problem with object-oriented languages is they've got all this implicit environment that they carry around with them. You wanted a banana but what you got was a gorilla holding the banana and the entire jungle. If you have referentially transparent code, if you have pure functions-all the data comes in its input arguments and everything goes out and leaves no state behind-it's incredibly reusable. You can just reuse it here, there, and everywhere. When you want to use it in a different project, you just cut and paste this code into your new project. Programmers have been conned into using all these different programming languages and they've been conned into not using easy ways to connect programs together. The Unix pipe mechanism-A pipe B pipe C-is trivially easy to connect things together. Is that how programmers connect things together? No. They use APIs and they link them into the same memory space, which is appallingly difficult and isn't cross-language. If the language is in the same family it's OK-if they're imperative languages, that's fine. But suppose one is Prolog and the other is C. They have a completely different view of the world, how you handle memory. So you can't just link them together like that. You can't reuse things. There must be big commercial interests for whom it is very desirable that stuff won't work together."
- Peter Seibel, Coders at Work: Reflections on the Craft of Programming
Joe Armstrong: "I think the lack of reusability comes in object-oriented languages, not in functional languages. Because the problem with object-oriented languages is they've got all this implicit environment that they carry around with them. You wanted a banana but what you got was a gorilla holding the banana and the entire jungle. If you have referentially transparent code, if you have pure functions-all the data comes in its input arguments and everything goes out and leaves no state behind-it's incredibly reusable. You can just reuse it here, there, and everywhere. When you want to use it in a different project, you just cut and paste this code into your new project. Programmers have been conned into using all these different programming languages and they've been conned into not using easy ways to connect programs together. The Unix pipe mechanism-A pipe B pipe C-is trivially easy to connect things together. Is that how programmers connect things together? No. They use APIs and they link them into the same memory space, which is appallingly difficult and isn't cross-language. If the language is in the same family it's OK-if they're imperative languages, that's fine. But suppose one is Prolog and the other is C. They have a completely different view of the world, how you handle memory. So you can't just link them together like that. You can't reuse things. There must be big commercial interests for whom it is very desirable that stuff won't work together."
- Peter Seibel, Coders at Work: Reflections on the Craft of Programming
Nice quote, but did you read the article at all?
Something else I want to mention: what's written above will most likely result in some sort of warmed up discussion in regards to object oriented programming versus something else. Or inheritance versus strategies. Or virtual methods versus method passing. Or whatever else hackernews finds worthy of a discussion this time around.
All of that is entirely irrelevant to the point I'm making which is that monolithic pieces of code are a bad idea. And our solution to monolithic code in Python are classes. If your hammer of choice is Haskell then use whatever the equivalent in Haskell looks like. Just don't force me to fork your library because you decided a layered API is not something you want to expose to your user.
Something else I want to mention: what's written above will most likely result in some sort of warmed up discussion in regards to object oriented programming versus something else. Or inheritance versus strategies. Or virtual methods versus method passing. Or whatever else hackernews finds worthy of a discussion this time around.
All of that is entirely irrelevant to the point I'm making which is that monolithic pieces of code are a bad idea. And our solution to monolithic code in Python are classes. If your hammer of choice is Haskell then use whatever the equivalent in Haskell looks like. Just don't force me to fork your library because you decided a layered API is not something you want to expose to your user.
Aha! Good point. TL;DR! Only skimmed, as others have commented, it was far too long. Perhaps the title should have been Python rant: Write More Classes!. The point is, the themes touched upon in the article are greater. The author's wish not to raise them is .. hereby noted, but not understood, really, when appealing to a larger audience. You can't have it both ways.
classes are exactly the pattern that tends to evolve into monolithic piece of code. Worse, they are harder to read and debug
Two points:
1. When OO was on the brink of becoming mainstream, let's say early 90's, there was an awful lot of the most idiotic rubbish talked about reusability. For example I recall one magazine article that, with a completely straight face, told readers that you would be able to buy an "Aeroplane" class that you could slot into pretty much any application that dealt with aeroplanes. So Joe has a point here.
2. This is not an argument for not using classes within a single codebase. And indeed when I see Erlang functions pushing all the relevant state around as parameters in a big mess of tuples and lists like a crazy baglady with all her possessions in a shopping cart, the benefits of encapsulating it all become clear.
1. When OO was on the brink of becoming mainstream, let's say early 90's, there was an awful lot of the most idiotic rubbish talked about reusability. For example I recall one magazine article that, with a completely straight face, told readers that you would be able to buy an "Aeroplane" class that you could slot into pretty much any application that dealt with aeroplanes. So Joe has a point here.
2. This is not an argument for not using classes within a single codebase. And indeed when I see Erlang functions pushing all the relevant state around as parameters in a big mess of tuples and lists like a crazy baglady with all her possessions in a shopping cart, the benefits of encapsulating it all become clear.
Write no classes: http://selflanguage.org :)
Love it when people write about OO without OO experience.
Are you saying Joe Armstrong doesn't know what he's talking about? He's a very googleable guy.
I'm not aware of Joe Armstrong having extensive OO experience - either as a programmer or a language designer. Could you fill me in? Perhaps the picture I have is completely wrong. Erlang is from 1986. Did he gather extensive OO experience before? In Smalltalk? Or on the side while developing Erlang?
According to a message he posted[0] on the erlang-questions mailing list, since 1967 he has written in Fortran, Lisp, Prolog, Smalltalk, Erlang, C, C++, Java, Python, Ruby, Lua, Javascript, Haskell, ML and OCaml.
I italicised all the object-oriented ones. Saying that Joe Armstrong doesn't have "extensive OO experience" is pretty comical!
[0] http://erlang.org/pipermail/erlang-questions/2013-January/07...
I italicised all the object-oriented ones. Saying that Joe Armstrong doesn't have "extensive OO experience" is pretty comical!
[0] http://erlang.org/pipermail/erlang-questions/2013-January/07...
Extensive experience?
About Java: "I tried Java"
About C++: "I saw C++ coming and read the book - or at least tried to read the book - there's a dent in the wall behind my piano, where the book hit the wall"
About Smalltalk: "I learnt (with various degrees of proficiency) [...] smalltalk [...] and became proficient in Prolog (aggghhh - the beauty ....)"
About the others: " I also (later) tried Python (ok), Ruby (ok)".
We might vary in our perception, but I would not call this extensive experience, "trying" some languages and "trying" to read a book.
I would call "I've written some systems in Java for 10 years" extensive Java experience.
About Java: "I tried Java"
About C++: "I saw C++ coming and read the book - or at least tried to read the book - there's a dent in the wall behind my piano, where the book hit the wall"
About Smalltalk: "I learnt (with various degrees of proficiency) [...] smalltalk [...] and became proficient in Prolog (aggghhh - the beauty ....)"
About the others: " I also (later) tried Python (ok), Ruby (ok)".
We might vary in our perception, but I would not call this extensive experience, "trying" some languages and "trying" to read a book.
I would call "I've written some systems in Java for 10 years" extensive Java experience.
He's humble.
If you read what he wrote (not just this quote), it is clear that he has a better grasp of OO than most OO practitioners and language designers.
And especially since OO is not well defined (see pg's quote of Jon Rees' characterization), most OO people can, and often do, claim everyone who uses a different OO definition than they do has no experience.
If you read what he wrote (not just this quote), it is clear that he has a better grasp of OO than most OO practitioners and language designers.
And especially since OO is not well defined (see pg's quote of Jon Rees' characterization), most OO people can, and often do, claim everyone who uses a different OO definition than they do has no experience.
The quote posted in the initial comment really has no relevance to Armin's article so it's fair enough to call it out in this context.
To attack the quote itself by simply assuming the author is inexperienced in OO just silly. Particularly so, given that Joe Armstrong has a grasp of computer languages that most of us could only ever dream of.
To attack the quote itself by simply assuming the author is inexperienced in OO just silly. Particularly so, given that Joe Armstrong has a grasp of computer languages that most of us could only ever dream of.
To attack the quote itself by simply assuming the author is inexperienced in OO just silly.
No one is assuming anything about his OO skills, they are reading the words written and assuming the author isn't lying.
No one is assuming anything about his OO skills, they are reading the words written and assuming the author isn't lying.
Erm, really?
Love it when people write about OO without OO experience.
Love it when people write about OO without OO experience.
I stand to this quote, Joe Armstrong has no documented OO experience (OO development for some years, development of an OO language etc.) and is quoted as an expert on the topic of classes. None of the people who have voted down all my comments have shown any experience in OO/classes development of Joe Armstrong. Joe Armstrong admitted to bash OO in a blog post ("silly" see quote above) to stir up people.
[Edit] It's the same when people quote Linus on GUIs. He might be an excellent kernel hacker, with a great understanding of large scale open source development, but his opinion on GUI is as good as everyone else. Or how people quote Wozniak on everything. This is not science but a guru cult. I'm proposing that we move our industry to facts and experiments and end the pop culture and guru cults. I'm not with Raganwald that we should embrace this pop culture. This is the reason that I vehemently disagree when people cite Joe Armstrong as an expert on classes or when Joe Armstrong tries to stir up people my bashing OO.
[Edit] It's the same when people quote Linus on GUIs. He might be an excellent kernel hacker, with a great understanding of large scale open source development, but his opinion on GUI is as good as everyone else. Or how people quote Wozniak on everything. This is not science but a guru cult. I'm proposing that we move our industry to facts and experiments and end the pop culture and guru cults. I'm not with Raganwald that we should embrace this pop culture. This is the reason that I vehemently disagree when people cite Joe Armstrong as an expert on classes or when Joe Armstrong tries to stir up people my bashing OO.
Designing Erlang is an impressive accomplishment as computing goes. I don't know why you insist Armstrong is a dilettante without any basis.
(as a side note, your argument does not make logical sense. Designing A does not make me an expert in B.)
I do not "insist" that he is a "dilettante", I question people on this thread quoting him as an expert on OO as there without showing that he is an expert on OO. Beside I give circumstantial evidence why he isn't ("Tried Java ...", "Threw C++ book at the wall ..."). I think it's important to differentiate strong opinion from expert.
And there are more indications: His understanding of "isolation" in the quoted text above is very influenced by him being the Erlang creator and the way Erlang thinks of isolation (processes). None of the (C++/Java/Ruby) OO programmers I know would use "isolation" in an OO context that sense (one error crashing other programs in the same VM).
I do not "insist" that he is a "dilettante", I question people on this thread quoting him as an expert on OO as there without showing that he is an expert on OO. Beside I give circumstantial evidence why he isn't ("Tried Java ...", "Threw C++ book at the wall ..."). I think it's important to differentiate strong opinion from expert.
And there are more indications: His understanding of "isolation" in the quoted text above is very influenced by him being the Erlang creator and the way Erlang thinks of isolation (processes). None of the (C++/Java/Ruby) OO programmers I know would use "isolation" in an OO context that sense (one error crashing other programs in the same VM).
So you are saying that you are a better language expert, and know better than him?
Why can't he say that?
Several people in HN should know more about OO than Armstrong.
There are academics that have taught and worked for decades on OO systems and concepts among the crowd, unlike Joe.
In general, popular language creators are hardly the more knowledgable of the various PL concepts and issues. Simon Peyton Jones probably knows hundred times more about such stuff than Matz or Guido do, for example.
Several people in HN should know more about OO than Armstrong.
There are academics that have taught and worked for decades on OO systems and concepts among the crowd, unlike Joe.
In general, popular language creators are hardly the more knowledgable of the various PL concepts and issues. Simon Peyton Jones probably knows hundred times more about such stuff than Matz or Guido do, for example.
It's worth pointing out that the original comment claimed Armstrong had no OO experience... not simply that Armstrong wasn't the most knowledgable OO programmer on HN.
[deleted]
OCaml, as the name ("Objective Caml") implies, also supports object-oriented programming. It actually has a pretty clever implementation of it as well. That said, many people tend to stick to the functional bits of OCaml and don't venture out into the OOP bits very often.
I thought about italicising OCaml as well, but I'd hesitate to call it an object-oriented language. It's more like a functional language that happens to have an object system.
Likewise Common Lisp has an object system (CLOS) but I wouldn't call it an object-oriented language.
Likewise Common Lisp has an object system (CLOS) but I wouldn't call it an object-oriented language.
From http://www.infoq.com/interviews/johnson-armstrong-oop
" Is Erlang object oriented? Joe Armstrong: Smalltalk got a lot of the things right. So if your question is about what I think about object oriented programming, I sort of changed my mind over that. I wrote a an article, a blog thing, years ago - Why object oriented programming is silly. I mainly wanted to provoke people with it. They had a quite interesting response to that and I managed to annoy a lot of people, which was part of the intention actually. I started wondering about what object oriented programming was and I thought Erlang wasn't object oriented, it was a functional programming language.
Then, my thesis supervisor said "But you're wrong, Erlang is extremely object oriented". He said object oriented languages aren't object oriented. I might think, though I'm not quite sure if I believe this or not, but Erlang might be the only object oriented language because the 3 tenets of object oriented programming are that it's based on message passing, that you have isolation between objects and have polymorphism.
Alan Kay himself wrote this famous thing and said "The notion of object oriented programming is completely misunderstood. It's not about objects and classes, it's all about messages". He wrote that and he said that the initial reaction to object oriented programming was to overemphasize the classes and methods and under emphasize the messages and if we talk much more about messages then it would be a lot nicer. The original Smalltalk was always talking about objects and you sent messages to them and they responded by sending messages back.
But you don't really do that and you don't really have isolation which is one of the problems. Dan Ingalls said yesterday (I thought it was very nice) about messaging that once you got messaging, you don't have to care where the message came from. You don't really have to care, the runtime system has to organize the delivery of the message, we don't have to care about how it's processed. It sort of decouples the sender and the receiver in this kind of mutual way. That's why I love messaging.
The 3 things that object oriented programming has it's messaging, which is possibly the most important thing. The next thing is isolation and that's what I talked about earlier, that my program shouldn't crash your program, if the 2 things are isolated, then any mistakes I make in my program will not crash your program. This is certainly not true with Java. You cannot take 2 Java applications, bung them in the JVM and one of them still halts the machine and the other one will halt as well. You can crash somebody else's application, so they are not isolated.
The third thing you want is polymorphism. Polymorphism is especially regarding messaging, that's just there for the programmer's convenience. It's very nice to have for all objects or all processes or whatever you call them, to have a printMe method - "Go print yourself" and then they print themselves. That's because the programmers, if they all got different names, the programmer is never going to remember this, so it's a polymorphism. It just means "OK, all objects have a printMe method. All objects have a what's your size method or introspection method."
Erlang has got all these things. It's got isolation, it's got polymorphism and it's got pure messaging. From that point of view, we might say it's the only object oriented language and perhaps I was a bit premature in saying that object oriented languages are about. You can try it and see it for yourself."
It depends a lot on how one defines OO, but in this regard he has -- or at least he has given some thought for it.
" Is Erlang object oriented? Joe Armstrong: Smalltalk got a lot of the things right. So if your question is about what I think about object oriented programming, I sort of changed my mind over that. I wrote a an article, a blog thing, years ago - Why object oriented programming is silly. I mainly wanted to provoke people with it. They had a quite interesting response to that and I managed to annoy a lot of people, which was part of the intention actually. I started wondering about what object oriented programming was and I thought Erlang wasn't object oriented, it was a functional programming language.
Then, my thesis supervisor said "But you're wrong, Erlang is extremely object oriented". He said object oriented languages aren't object oriented. I might think, though I'm not quite sure if I believe this or not, but Erlang might be the only object oriented language because the 3 tenets of object oriented programming are that it's based on message passing, that you have isolation between objects and have polymorphism.
Alan Kay himself wrote this famous thing and said "The notion of object oriented programming is completely misunderstood. It's not about objects and classes, it's all about messages". He wrote that and he said that the initial reaction to object oriented programming was to overemphasize the classes and methods and under emphasize the messages and if we talk much more about messages then it would be a lot nicer. The original Smalltalk was always talking about objects and you sent messages to them and they responded by sending messages back.
But you don't really do that and you don't really have isolation which is one of the problems. Dan Ingalls said yesterday (I thought it was very nice) about messaging that once you got messaging, you don't have to care where the message came from. You don't really have to care, the runtime system has to organize the delivery of the message, we don't have to care about how it's processed. It sort of decouples the sender and the receiver in this kind of mutual way. That's why I love messaging.
The 3 things that object oriented programming has it's messaging, which is possibly the most important thing. The next thing is isolation and that's what I talked about earlier, that my program shouldn't crash your program, if the 2 things are isolated, then any mistakes I make in my program will not crash your program. This is certainly not true with Java. You cannot take 2 Java applications, bung them in the JVM and one of them still halts the machine and the other one will halt as well. You can crash somebody else's application, so they are not isolated.
The third thing you want is polymorphism. Polymorphism is especially regarding messaging, that's just there for the programmer's convenience. It's very nice to have for all objects or all processes or whatever you call them, to have a printMe method - "Go print yourself" and then they print themselves. That's because the programmers, if they all got different names, the programmer is never going to remember this, so it's a polymorphism. It just means "OK, all objects have a printMe method. All objects have a what's your size method or introspection method."
Erlang has got all these things. It's got isolation, it's got polymorphism and it's got pure messaging. From that point of view, we might say it's the only object oriented language and perhaps I was a bit premature in saying that object oriented languages are about. You can try it and see it for yourself."
It depends a lot on how one defines OO, but in this regard he has -- or at least he has given some thought for it.
I always asked myself why Alan Kay didn't name it "message oriented" then.
The answer to this question is supposed to be in "The Early History of Smalkalk", though I haven't re-read it.
Anyway, I just asked on the Fonc mailing list, and got an incredibly detailed answer from Alan Kay himself. To sum it up, someone asked him what he was doing, and he didn't quite have a name for that yet. Heck, he was at the stage of narrowing it down mathematically. So he just answered "object oriented programming", because it fitted with the vocabulary of the time, and went back to work.
He did later regretted this label.
Now, it's not even sure a better terminology would have helped much. Alan Kay reckons that "it might have helped to have better terminology", but it would probably have been hijacked too:
> The success of the ideas made what we were doing popular, and people wanted to be a part of it. This led to using the term "object oriented" as a designer jeans label for pretty much anything (there was even an "object-oriented" COBOL!). This appropriation of labels without content is a typical pop culture "fantasy football" syndrome.
Anyway, I just asked on the Fonc mailing list, and got an incredibly detailed answer from Alan Kay himself. To sum it up, someone asked him what he was doing, and he didn't quite have a name for that yet. Heck, he was at the stage of narrowing it down mathematically. So he just answered "object oriented programming", because it fitted with the vocabulary of the time, and went back to work.
He did later regretted this label.
Now, it's not even sure a better terminology would have helped much. Alan Kay reckons that "it might have helped to have better terminology", but it would probably have been hijacked too:
> The success of the ideas made what we were doing popular, and people wanted to be a part of it. This led to using the term "object oriented" as a designer jeans label for pretty much anything (there was even an "object-oriented" COBOL!). This appropriation of labels without content is a typical pop culture "fantasy football" syndrome.
Because "communicating with messages" is one of the important features of objects, but objects are the focus.
The confusion, I think, mostly came about because in the late 1980s early 1990s, an explosion of "Object-oriented" programming happend by gluing classes into existing popular structured programming languages -- producing ObjectPascal and C++ which became the popular "OO" languages which became what "OO" meant to lots of people (Objective-C was a contemporary, and despite being C-based like C++ was more like Kay's idea of OO than C++, but didn't become popular until later.)
So what became popularly conflated with "object oriented" programming was statically-typed, class-oriented programming typified by C++ and similar languages (e.g., Java.)
There are certainly ways (often not the same ways between languages) in which newer languages -- Erlang, Python/Ruby, etc. -- are more "object oriented" than Java/C++-style languages.
The confusion, I think, mostly came about because in the late 1980s early 1990s, an explosion of "Object-oriented" programming happend by gluing classes into existing popular structured programming languages -- producing ObjectPascal and C++ which became the popular "OO" languages which became what "OO" meant to lots of people (Objective-C was a contemporary, and despite being C-based like C++ was more like Kay's idea of OO than C++, but didn't become popular until later.)
So what became popularly conflated with "object oriented" programming was statically-typed, class-oriented programming typified by C++ and similar languages (e.g., Java.)
There are certainly ways (often not the same ways between languages) in which newer languages -- Erlang, Python/Ruby, etc. -- are more "object oriented" than Java/C++-style languages.
Joe Armstrong is very opinionated. I would want to hear what he has to say but I wouldn't take him as an unbiased authority.
being googleable and knowing what he is talking about doesnt mean that his this quote is relavant in this context. What he said is a good piece of advice, but only when looked through larger window of programming. When we are talking about only python and choices of coding style in python, his quote looks like santa on valentines day!
Lets stick to the argument made rather than the person making it.
I could not understand how classes can help with JSON-example.
It looks the same as XML SAX vs DOM: you feed NN-Mb to DOM-parser (SQLServer xml-datatype for example) and you have problems. No matter: classes or functions.
It looks the same as XML SAX vs DOM: you feed NN-Mb to DOM-parser (SQLServer xml-datatype for example) and you have problems. No matter: classes or functions.
By default, I go with the inclination to put code in functions when I tackle a project, because most of my projects are just products and applications without a public API and testing or debugging the flow in a functional paradigm is much simpler. I can take a function and plug it in the interactive interpreter (in Python) and run it without needing to instantiate other state that's needed for the test.
However, in certain cases, like where I need to write a public API, I have found that having classes as wrappers to the functionality helps it a bit. So really, the "stop-writing-classes-unless-you- absolutely-need-to" guideline still holds true for me.
However, in certain cases, like where I need to write a public API, I have found that having classes as wrappers to the functionality helps it a bit. So really, the "stop-writing-classes-unless-you- absolutely-need-to" guideline still holds true for me.
As Java was mentioned. The Java API is very nice, but there is a layer missing on top. The Java API was written with a early 90s mindset and got most users after 2000. This goes for most Java APIs. IO, Swing, ... The idea is to have LEGO building blocks (BufferedReader) you can plug together. But you need to plug them together all the time. The missing layer e.g. is IO.readIntoString(file). Apache IOUtils, StringUtils etc. fill in this layer for many Java APIs.
The one API that is not powerful enough is Collection. There you have the top layer without the LEGO building blocks. Compare this to the Scala collection API which has the top layers and the building blocks.
For a good API you need both, building blocks to tailer to your specific need (20% of the time) and an easy top layer (80%) to prevent writing the same stuff all the time.
The one API that is not powerful enough is Collection. There you have the top layer without the LEGO building blocks. Compare this to the Scala collection API which has the top layers and the building blocks.
For a good API you need both, building blocks to tailer to your specific need (20% of the time) and an easy top layer (80%) to prevent writing the same stuff all the time.
To be fair, they have mostly fixed the IO API with Java 7 (NIO2). This covers various Files# helpers as well as a proper Path abstraction.
15 (?) years late ;-)
Why does that matter? If version 1 is bad, we should never use any newer versions because version 1 was bad? The iPhone didn't support 3g in the first version, so we shouldn't use current iPhones?
In some ways, yes. Plenty of people moved to other languages because of those handicaps, and they have no good reason to come back.
You make it sound like it's some kind of popularity contest, where people not choosing the language means it 'loses'. I disagree pretty heavily there - your hammer doesn't lose if you use your saw. The language's user loses if he is such a fanboy of hammers he refuses to use the saw. If Java improves to become the best tool (not saying it has/will), then choosing not use use Java because you 'moved on', is a failure on your part, not the tool.
The thing with analogies is that you have to know when they break down.
Sure the language loses. An hammer is self-sufficient; a language needs libraries, community support, job/project candidates that can replace an absent developer, etc. Do you think having NumPy isn't a win for Python?
Of course, if the language is sufficiently better than the others, maybe you're willing to write your own e.g. Websockets library to use it. But there's still a large handicap there.
For example, Plan9 is in many ways a better OS than GNU/Linux. But I still use the latter, because I rather have the support and thousands of binary packages that a larger community brings.
Sure the language loses. An hammer is self-sufficient; a language needs libraries, community support, job/project candidates that can replace an absent developer, etc. Do you think having NumPy isn't a win for Python?
Of course, if the language is sufficiently better than the others, maybe you're willing to write your own e.g. Websockets library to use it. But there's still a large handicap there.
For example, Plan9 is in many ways a better OS than GNU/Linux. But I still use the latter, because I rather have the support and thousands of binary packages that a larger community brings.
I must be way out of touch because I had no idea there was an anti-class movement for which the OP has to argue for more classes?
He mentions stop writing classes talk in the very beginning.
Which he doesn't really address, nor do the two opinions necessarily clash (at least not totally). Diederich's talk is mostly about avoiding to write superfluous classes. Quite often a simple module with one or two function suffices. His examples look quite different from those in the blog post.
(Although I have to admit that I skimmed most of them, as I'm in serious eyeroll territory whenever I see the old Java IO API used as a bad example again.)
(Although I have to admit that I skimmed most of them, as I'm in serious eyeroll territory whenever I see the old Java IO API used as a bad example again.)
Good point. I think author doesn't actually care as much about classes per se as he does about interfaces and extension points. Complexity might get out of hand incredibly quickly, though.
This is a bad article.
First, write nothing. If you can solve your problem without writing code (or even better, by deleting code), that is the best solution.
Next, write code which fits your architecture. Sometimes functional composition is the best system for representing your computational structure. Tree operations, for example, are especially amenable to recursive function-based computation.
Sometimes, when you are writing a state machine, for example, an object is the best possible representation. Objects are entirely focused around hidden-implementation finite state machines, and thus mirror your computation almost exactly.
Funny how most people who have only practical training in a handful of languages and no programming language theory at all tend to advocate for the one style of programming that they know well. When all you have is a hammer...
Note: The small paragraph at the end of this article seems to hedge by agreeing with me and essentially calling the reader to disregard what he previously wrote. If he had followed my step one he could have avoided writing the entire article. Think of the complexity saved!
First, write nothing. If you can solve your problem without writing code (or even better, by deleting code), that is the best solution.
Next, write code which fits your architecture. Sometimes functional composition is the best system for representing your computational structure. Tree operations, for example, are especially amenable to recursive function-based computation.
Sometimes, when you are writing a state machine, for example, an object is the best possible representation. Objects are entirely focused around hidden-implementation finite state machines, and thus mirror your computation almost exactly.
Funny how most people who have only practical training in a handful of languages and no programming language theory at all tend to advocate for the one style of programming that they know well. When all you have is a hammer...
Note: The small paragraph at the end of this article seems to hedge by agreeing with me and essentially calling the reader to disregard what he previously wrote. If he had followed my step one he could have avoided writing the entire article. Think of the complexity saved!
From what you wrote, I'm not sure you read the article. It's not about objects vs functions; it's about hiding a lot of functionality behind an overly-simple interface, with particular attention paid to an example of a parser that takes a whole string rather than a streaming interface.
What annoyed me most was his constant apparent conflation between bytes and characters. But I didn't really see much evidence of treating all the world like a nail.
What annoyed me most was his constant apparent conflation between bytes and characters. But I didn't really see much evidence of treating all the world like a nail.
I know what he was trying to say, but I don't think that's actually what he said.
What he should have said is that you should pick an interface that suits your computation. The problem was not that the interface was too simple, but that it didn't fit the computation. One can easily imagine an object-oriented interface for the same wrapper computation that also hides the true nature of the computation.
What he should have said is that you should pick an interface that suits your computation. The problem was not that the interface was too simple, but that it didn't fit the computation. One can easily imagine an object-oriented interface for the same wrapper computation that also hides the true nature of the computation.
I saw no conflation. Everything there is a byte.
Quote:
For instance while I have not yet reached the end of the string I would feed my characters into a buffer until I found the end of the string
The point however is that instead of reading characters step by step some lexers will instead have a look at a buffer of a certain size and move a pointer around in that buffer and modify some bytes in place (in-situ).
He talks about this buffer, at one moment in terms of bytes, another moment in terms of characters.
It's a pedantic quibble, but an easy source of errors, especially if you're using a language that doesn't distinguish strongly between the two concepts, and e.g. you encode your characters as UTF8 with a weak alias to unsigned char just like bytes.
For instance while I have not yet reached the end of the string I would feed my characters into a buffer until I found the end of the string
The point however is that instead of reading characters step by step some lexers will instead have a look at a buffer of a certain size and move a pointer around in that buffer and modify some bytes in place (in-situ).
He talks about this buffer, at one moment in terms of bytes, another moment in terms of characters.
It's a pedantic quibble, but an easy source of errors, especially if you're using a language that doesn't distinguish strongly between the two concepts, and e.g. you encode your characters as UTF8 with a weak alias to unsigned char just like bytes.
So in Java, a parameter named 'filename' is automatically aliased to 'path'? Doh...
static String readFirstLine(String filename) { try { BufferedReader br = new BufferedReader(new FileReader(path)); ....
So people writes this everyday, yet still fail to do it correctly...
Also, this article was written 1 day in the future. The future looks bleak to me...
static String readFirstLine(String filename) { try { BufferedReader br = new BufferedReader(new FileReader(path)); ....
So people writes this everyday, yet still fail to do it correctly...
Also, this article was written 1 day in the future. The future looks bleak to me...
No. Don't write classes. Write useful abstractions. It doesn't matter what the abstraction is (a class, a function, or something else you programming language supports)as long as it fits your current architecture and can be easily modified for possible future extensions.(I will agree with the author that one giant monolithic function is probably a bad abstraction).
+1, one of the best points in the discussion so far. :)
The programming industry walks around in circles and there is no beacon in this darkness of ignorance.
There is no professional culture and new generations of developers successfully forget all the experience previous generations have accumulated.
Also, some piece of advice from a seasoned programmer to the web-programming-children: if you are not really a serious developer, if you write your freaking websitee on Django, or whatever a framework there is, you don't really need a methodology, because you are doing an easy task, you can write in whatever language/style/paradigm you like, even on Brainfuck. But please don't extrapolate your humble experience to the entire industry and don't tell people working on large complicated (real) projects how they must write code, because they have some experience you don't have.
There is no professional culture and new generations of developers successfully forget all the experience previous generations have accumulated.
Also, some piece of advice from a seasoned programmer to the web-programming-children: if you are not really a serious developer, if you write your freaking websitee on Django, or whatever a framework there is, you don't really need a methodology, because you are doing an easy task, you can write in whatever language/style/paradigm you like, even on Brainfuck. But please don't extrapolate your humble experience to the entire industry and don't tell people working on large complicated (real) projects how they must write code, because they have some experience you don't have.
Because programmers who happen to implement complex, huge, real systems that happen to interface users via a browser are of course less of a "real" programmers then the ones who happen to implement rather straightforward, but native, CRUD interfaces to some legacy DB.
Wait, what?
"Domain pissing match"? Or just inexperience in and misconceptions about writing web apps on your part?
Anyway, that's irrelevant - the advice here is not specific to web programming at all and is a sound one, no matter how experienced you are or what you work on at the moment. Writing modular, reusable and easily customisable code is just a good idea.
Heh, what am I doing. It's still early for me and I'm responding to an obvious trolling attempt from frustrated Java programmer who uses the word "humble" without a shred of humility in his own writing. I should just stop and try porting this "freaking website on Django" that my team has been working on for the last three years to Brainfuck - I'd be more productive that way and I guess it would be more fun, too.
Wait, what?
"Domain pissing match"? Or just inexperience in and misconceptions about writing web apps on your part?
Anyway, that's irrelevant - the advice here is not specific to web programming at all and is a sound one, no matter how experienced you are or what you work on at the moment. Writing modular, reusable and easily customisable code is just a good idea.
Heh, what am I doing. It's still early for me and I'm responding to an obvious trolling attempt from frustrated Java programmer who uses the word "humble" without a shred of humility in his own writing. I should just stop and try porting this "freaking website on Django" that my team has been working on for the last three years to Brainfuck - I'd be more productive that way and I guess it would be more fun, too.
> But please don't extrapolate your humble experience to the entire industry and don't tell people working on large complicated (real) projects how they must write code, because they have some experience you don't have.
Don't take this the wrong way, I mostly agree with your points regarding the missing "professional culture" among us programmers, just yesterday I sent an email out to our internal list urging my coworkers not to use one-letter variables anymore and explaining to them why that is bad, but when it comes to web projects somehow not being "real" programming I'm afraid you're wrong.
It is true, some momentum was lost when the "open-data" mantra that was flying in the air around 2004-2005 gave way to today's walled gardens and one-page AJAXy apps, but there are still interesting things happening on the web.
Don't take this the wrong way, I mostly agree with your points regarding the missing "professional culture" among us programmers, just yesterday I sent an email out to our internal list urging my coworkers not to use one-letter variables anymore and explaining to them why that is bad, but when it comes to web projects somehow not being "real" programming I'm afraid you're wrong.
It is true, some momentum was lost when the "open-data" mantra that was flying in the air around 2004-2005 gave way to today's walled gardens and one-page AJAXy apps, but there are still interesting things happening on the web.
Don't you think writing about it is a good to minimize the amount forgotten generation to generation? Even if they write something that is flat out wrong, someone will probably tell them and everyone might learn. Also I wouldn't be so quick to dismiss building websites on Django (or any other framework) as easy. I imagine many things are easier, but I bet many things are harder as well, I'd think you'd be surprised at how quickly "good" designs go to shit when business requirements change every two weeks and how this effects development.
There's a fairly effective rule of thumb: classes--or more accurately objects--are a way to cleanly encapsulate state. If you rely on a mutating state, you probably want an object/class. If not, a function is often better.
Now you can bundle related functions together in a structure, but this structure is morally a module, not an object, let alone a class. Some languages will force you to encode those modules as classes / prototypes / singletons, but that's just a design pattern to circumvent a limitation of the language.
Now you can bundle related functions together in a structure, but this structure is morally a module, not an object, let alone a class. Some languages will force you to encode those modules as classes / prototypes / singletons, but that's just a design pattern to circumvent a limitation of the language.
A good example of classes in Python is the dnspython library.
It's tasked with returning records from parsed zones and every single record is a class. I like it and I don't understand the first sentence of this blog post.
Try not to put too much weight on what others tell you, make up your own mind. It's a classic human issue.
It's tasked with returning records from parsed zones and every single record is a class. I like it and I don't understand the first sentence of this blog post.
Try not to put too much weight on what others tell you, make up your own mind. It's a classic human issue.
The discussion on the thread has rotten to much extent, the article doesnt address different languages or paradigms, he just mentions about classes being better than block codes in python. Something more rudimentary, a valuable piece for quick-fix programmers, for hobby programmers. Classes are one way of abstractions.
So he asks for using abstractions instead on no abstraction (block code) but if you have other ways of abstracting code, please go ahead. THis article is good for people with few / no tools, people who are still learning
So he asks for using abstractions instead on no abstraction (block code) but if you have other ways of abstracting code, please go ahead. THis article is good for people with few / no tools, people who are still learning
I really don't like the way this was presented, even if I may agree with the idea underneath. It's not about classes at all. It's about better design, but even the examples are strange.
Just from the JSON example:
- Why do I need a class for streaming JSON - Python's got a perfectly good `yield` for returning tokens in such situations.
- Why would I ever design the JSON library to be extendable at the tokenizer level? If you need serialiser / deserialiser, why not just provide a map of types to callbacks / callback as a parameter? Do you really want to extend JSON format itself?
- The 2GB JSON example is just weird. If you care about such use cases, you a) most likely have a limit on data size at webserver level, b) use proper formats for handling that size of data (I really doubt there's no better data representation once you get to GB sizes).
I see his point of view, but he's arguing for one single "hammer" solution, rather than arguing against the monolithic design. His story seems to present some weird back-story really: "I needed to make my data easier to handle, so I started automatically serialising objects into JSON, then they became huge so I have to start streaming them otherwise just parsing of them takes way too long".
Just from the JSON example:
- Why do I need a class for streaming JSON - Python's got a perfectly good `yield` for returning tokens in such situations.
- Why would I ever design the JSON library to be extendable at the tokenizer level? If you need serialiser / deserialiser, why not just provide a map of types to callbacks / callback as a parameter? Do you really want to extend JSON format itself?
- The 2GB JSON example is just weird. If you care about such use cases, you a) most likely have a limit on data size at webserver level, b) use proper formats for handling that size of data (I really doubt there's no better data representation once you get to GB sizes).
I see his point of view, but he's arguing for one single "hammer" solution, rather than arguing against the monolithic design. His story seems to present some weird back-story really: "I needed to make my data easier to handle, so I started automatically serialising objects into JSON, then they became huge so I have to start streaming them otherwise just parsing of them takes way too long".
> - Why do I need a class for streaming JSON - Python's got a perfectly good `yield` for returning tokens in such situations.
See the msgpack-cli example at the bottom. Say you have a function that returns a generator for tokens in Python. You would need another function that builds objects out of them. How do you customize how objects are being built? A class makes that simpler because each of the methods are extension points you can override.
But yeah, a token stream would be much appreciated.
> - Why would I ever design the JSON library to be extendable at the tokenizer level?
For instance if you want to skip past objects. That's what we're doing for instance for unwanted input data. That implicitly also prevents hash collision DOS attacks because we never build hash tables for things that we don't want. It also gets rid of the suspension of execution when a garbage collector runs and cleans up a nested structure. I can make you tons of examples where the "build a tree, reject a tree" approach destroys an evented Python server.
> - The 2GB JSON example is just weird. If you care about such use cases, you a) most likely have a limit on data size at webserver level, b) use proper formats for handling that size of data (I really doubt there's no better data representation once you get to GB sizes).
Try using a streaming JSON API like Twitter's firehose. Most people just give up and use XML because there is SAX or if they go with JSON they newline delimit it because most libraries are horrible or useless for parsing streamed JSON.
See the msgpack-cli example at the bottom. Say you have a function that returns a generator for tokens in Python. You would need another function that builds objects out of them. How do you customize how objects are being built? A class makes that simpler because each of the methods are extension points you can override.
But yeah, a token stream would be much appreciated.
> - Why would I ever design the JSON library to be extendable at the tokenizer level?
For instance if you want to skip past objects. That's what we're doing for instance for unwanted input data. That implicitly also prevents hash collision DOS attacks because we never build hash tables for things that we don't want. It also gets rid of the suspension of execution when a garbage collector runs and cleans up a nested structure. I can make you tons of examples where the "build a tree, reject a tree" approach destroys an evented Python server.
> - The 2GB JSON example is just weird. If you care about such use cases, you a) most likely have a limit on data size at webserver level, b) use proper formats for handling that size of data (I really doubt there's no better data representation once you get to GB sizes).
Try using a streaming JSON API like Twitter's firehose. Most people just give up and use XML because there is SAX or if they go with JSON they newline delimit it because most libraries are horrible or useless for parsing streamed JSON.
> You would need another function that builds objects out of them.
I.e. you don't bundle your serialiser with your json parser. I think that's a good idea. How do you customise? most likely a callback that builds an object or returns the json fragment unmodified if it can't. It can be streamed too in order to accumulate / rebuild fragments of the tree.
Whether object is simpler here (builder / factory style), or another generator, that's a matter of taste mostly.
> For instance if you want to skip past objects.
You can't skip past objects at a tokenizer level, unless you implement logic of skipping whole structure. But that's what the parser does. Why don't you just skip the objects based on the streaming API? You don't have to construct them first - it's up to your deserialiser implementation if you want to ignore parts of the structure.
> Try using a streaming JSON API like Twitter's firehose.
From the documentation (unless I'm reading the wrong format description):
"""The body of a streaming API response consists of a series of newline-delimited messages, where "newline" is considered to be \r\n (in hex, 0x0D 0x0A) and "message" is a JSON encoded data structure or a blank line."""
That's not a huge JSON document. They use the newline delimiter as you described later. I don't think that's because "most libraries are horrible or useless for parsing streamed JSON". Why would you ever want to stream JSON which is an infinite and never complete object? What's wrong about splitting by newline? By making newlines the message separators, you'll never have to worry that your stream becomes broken due to some parsing error: bad message? ignore it skip to newline, continue with next one.
I.e. you don't bundle your serialiser with your json parser. I think that's a good idea. How do you customise? most likely a callback that builds an object or returns the json fragment unmodified if it can't. It can be streamed too in order to accumulate / rebuild fragments of the tree.
Whether object is simpler here (builder / factory style), or another generator, that's a matter of taste mostly.
> For instance if you want to skip past objects.
You can't skip past objects at a tokenizer level, unless you implement logic of skipping whole structure. But that's what the parser does. Why don't you just skip the objects based on the streaming API? You don't have to construct them first - it's up to your deserialiser implementation if you want to ignore parts of the structure.
> Try using a streaming JSON API like Twitter's firehose.
From the documentation (unless I'm reading the wrong format description):
"""The body of a streaming API response consists of a series of newline-delimited messages, where "newline" is considered to be \r\n (in hex, 0x0D 0x0A) and "message" is a JSON encoded data structure or a blank line."""
That's not a huge JSON document. They use the newline delimiter as you described later. I don't think that's because "most libraries are horrible or useless for parsing streamed JSON". Why would you ever want to stream JSON which is an infinite and never complete object? What's wrong about splitting by newline? By making newlines the message separators, you'll never have to worry that your stream becomes broken due to some parsing error: bad message? ignore it skip to newline, continue with next one.
I think
> A class makes that simpler because each of the methods are extension points you can override.
is a strong argument in favour of classes. They're more extensible even if the author doesn't consider it. However - as soon as your class has an implementation like
I don't know the story in other languages - I assume Java has it as you see a great emphasis on dependency injection. If dynamic lookup of constants was present I think classes would be more unintentionally extensible however they were written - as it is, you have to be as careful writing classes for extension as for functional code where you have to manually provide extension points.
> A class makes that simpler because each of the methods are extension points you can override.
is a strong argument in favour of classes. They're more extensible even if the author doesn't consider it. However - as soon as your class has an implementation like
def to_json(str)
JSONParser.parse(str) # JSONParser is not streamed
end
I don't know the story in other languages - I assume Java has it as you see a great emphasis on dependency injection. If dynamic lookup of constants was present I think classes would be more unintentionally extensible however they were written - as it is, you have to be as careful writing classes for extension as for functional code where you have to manually provide extension points.
In Perl, classes/packages are dynamically scoped. So you can safely localise any monkey-patching. For eg:
use 5.016;
use warnings;
package JSONParser {
sub parse {
my ($self, $str) = @_;
"JSONParser::parse $str";
}
}
package Foo {
sub new { my $class = shift; bless {}, $class }
sub to_json {
my ($self, $str) = @_;
JSONParser->parse($str);
}
}
my $foo = Foo->new;
say $foo->to_json("foo");
{
# OK... I want to amend that JSONParser->parser behaviour
# but just in this scope!
no warnings 'redefine';
local *JSONParser::parse = sub {
my ($self, $str) = @_;
"No Longer JSONParser::parser! $str";
};
say $foo->to_json("bar");
}
say $foo->to_json("baz");
JSONParser::parse foo
No Longer JSONParser::parser! bar
JSONParser::parse bazIn Java, you often see a class JsonClass with a toJson method like you presented and then a StreamingJsonClass with an overriden toJson that streams - this can cause problems with types (because the most obvious return type for toJson when it's written is JsonObject, but you really want an Iterator<JsonObject>), but Ruby/Python have the same problem (only hidden).
It's quite idiomatic Java, though, to remove all static calls from methods, even if you have to use anonymous classes (i.e. extensible method dependencies) to do so, for exactly the reasons you outline.
It's quite idiomatic Java, though, to remove all static calls from methods, even if you have to use anonymous classes (i.e. extensible method dependencies) to do so, for exactly the reasons you outline.
> - Why do I need a class for streaming JSON - Python's got a perfectly good `yield` for returning tokens in such situations.
I use msgpack in Python (twisted, tornado) precisely because it can consume byte buffers which are not token-aligned.
I use msgpack in Python (twisted, tornado) precisely because it can consume byte buffers which are not token-aligned.
Could not agree more with the idea that people should write more well designed, OO code. This exact same issue is prevalent in JS land and it leads to people saying JS can't be used for real development, when in fact the problem is that the code they write is a mess and that's what's holding them back.
Expanding on Armin's dichotomy, top-down designs like Python's open() or jquery plugins start with giving 70-80% of users APIs that are as simple as possible for their most frequent use cases while shielding them from the sausages underneath.
Bottom-up designs like Java's standard library or POSIX start with LEGO building blocks that solve the most fundamental pieces of largely academic computer science problems and just give them out to their end users and expect them to be able to assemble Tower Defense by solving this LEGO puzzle first.
The problem with sticking entirely to their 2 approaches is that you end up either ignoring power users or making healthy adults with normal IQs feel stupid. There is no reason you can't serve 100% of your user base by incrementally evolving your API approaches and provide 2 sets of APIs within the same library, with the top-down easy one shielding the bottom-up sausage factory that takes care of the meat grinding for you. Most API designers don't realize this and won't ever go there. Extremists and egoists with lots of opinions will spending hundreds of man years to promote their One True Way of doing things. They'll say things like "no leaky abstractions!" or "these enterprise people are just making things too complicated to create jobs!", when the simple truth is probably just that they don't understand how people think.
Make your libraries easy to do things that are easy, but make hard things possible too.