Making Python's __init__ method magical(blog.lerner.co.il)
blog.lerner.co.il
Making Python's __init__ method magical
http://blog.lerner.co.il/making-init-methods-magical-with-autoinit/
14 comments
Very nice; it does seems far more reasonable to implement this as a decorator than as an explicit function call __init__.
One thing I like a lot better about it in particular is that it doesn't reach into the stack frame, which strikes me as a very messy grab at internals people should never ever touch. The arity and name information from the compiled function object, however, has legitimate use in tooling and I suspect a lot of decorators do this sort of thing.
It's an interesting balance between one being more magical in its use of the system (inspecting live stack frames) and the other being more magical in appearance (having a function with just pass in it actually do something).
It's an interesting balance between one being more magical in its use of the system (inspecting live stack frames) and the other being more magical in appearance (having a function with just pass in it actually do something).
nice but it doesn't work if not all kwargs are passed..
Here is another version implemented as a decorator.
Actually I’d also like not to use inspect, but didn’t find any nice way to get the kwargs default values without it, any suggestion?
import inspect
def autoinit(decorated_init):
def _wrap(*args, **kwargs):
obj, nargs= args[0], args[1:]
names = decorated_init.func_code.co_varnames[1:len(nargs)+1]
nargs = {k: nargs[names.index(k)] for k in names}
kwa_keys = decorated_init.func_code.co_varnames[len(nargs)+1:]
kwa_defaults = inspect.getargspec(decorated_init).defaults
for k in kwa_keys:
nargs[k] = kwargs[k] if (k in kwargs) else kwa_defaults[kwa_keys.index(k)]
for k, v in nargs.iteritems():
setattr(obj, k, v)
return decorated_init(*args, **kwargs)
return _wrap
if __name__ == '__main__':
class Foo():
@autoinit
def __init__(self, a, b, x=0, y=0):
self.p = 'Hello, I am set inside the __init__'
self.s = self.a + self.b + self.x + self.y
f = Foo(1, 2, x=120, y=5)
for param in 'abxyps':
print param, '=>', getattr(f, param)
f = Foo(31, 33, x=64)
for param in 'abxyps':
print param, '=>', getattr(f, param)
f = Foo(7, 121)
for param in 'abxyps':
print param, '=>', getattr(f, param)[deleted]
Good call. It's amazing how many things you have to be careful of when writing decorators.
Really really minor nitpick, but when talking about languages and their traits these little details can become important for the reader to understand, lest it confuse them about other languages:
> While many people think of __init__ as a constructor, that’s not really the case. Rather, __init__ is invoked on our new object after it has been created
I'm not aware of a language (though I don't claim to know them all!) in which this is not exactly the definition of a constructor. In C++, `operator new` does the allocation, the constructor is called after. In java it's similar. In ruby it's Class#alloc (which is called by Class#new as roughly `ClassName.alloc.tap{|o| o.initialize(*args); o}`)
So the reader is correct to interpret it as a constructor. Constructors initialize the state of an uninitialized object.
> While many people think of __init__ as a constructor, that’s not really the case. Rather, __init__ is invoked on our new object after it has been created
I'm not aware of a language (though I don't claim to know them all!) in which this is not exactly the definition of a constructor. In C++, `operator new` does the allocation, the constructor is called after. In java it's similar. In ruby it's Class#alloc (which is called by Class#new as roughly `ClassName.alloc.tap{|o| o.initialize(*args); o}`)
So the reader is correct to interpret it as a constructor. Constructors initialize the state of an uninitialized object.
So, we've got three roles and three names... just let's call the allocator "allocator", the initializer "initializer", and the thing which does both the "constructor", maybe? :)
In Python you have a "real" constructor you can use: __new__. So I think it's good not to call __init__ a constructor. Unless of course you differ in my opinion that __new__ is a constructor?
I do. __new__ is effectively an allocator, or an interface to it. It allocates an unconstructed object. Then the constructor, __init__ is called on that.
Edited to add: Reading around, it appears that smalltalk has no separation between allocation and construction, having only the equivalent to `operator new`/`Class#new`/`obj.__new`, with initialization also happening in overloads of that. So some confusion may stem from that. I do think the distinction is meaningful and important in languages that separate these two things out, though.
Edited to add: Reading around, it appears that smalltalk has no separation between allocation and construction, having only the equivalent to `operator new`/`Class#new`/`obj.__new`, with initialization also happening in overloads of that. So some confusion may stem from that. I do think the distinction is meaningful and important in languages that separate these two things out, though.
I'll have to check my terminology to understand this better. But I can assure you that most of the programmers who take my Python classes (who tend to come from Java, C#, and C++) are rather surprised that the constructor modifies an existing object, and doesn't actually create the object.
That said, I'm totally willing to believe that neither they nor I understood the difference between an "allocator" and a "constructor." I'll check into this further, both for myself and for the sake of future writing and lectures!
That said, I'm totally willing to believe that neither they nor I understood the difference between an "allocator" and a "constructor." I'll check into this further, both for myself and for the sake of future writing and lectures!
Well, they have a poor understanding[1] of the sequence of events in C++ if they think that the constructor allocates the object there (I won't speak too strongly to Java or C#, since I don't work in them if I don't have to, but I believe that they both work in a very similar way). This would in fact be impossible, since many constructors are usually called for any given object (and all constructors in the object's graph are always called, unlike the more traditional OO languages where the upcall is usually explicit). Also because where an object is allocated is a decision of the caller. It can be allocated with new on the heap, on the stack with a simple declaration, or at an arbitrary memory location with placement new.
In C++ the heap object initialization goes like this:
`operator new` is an allocator and the constructor initializes the raw bytes returned from it.
You can actually do the process explicitly in its entirety like so using placement new, which is a special form that calls the constructors on an arbitrary location:
[1] Note: I don't really mean this as a judgement, understanding of C++ internals are pretty poor in a lot of programmers and for pretty good reasons a lot of the time.
In C++ the heap object initialization goes like this:
Obj *obj = new Obj;
`operator new` is an allocator and the constructor initializes the raw bytes returned from it.
You can actually do the process explicitly in its entirety like so using placement new, which is a special form that calls the constructors on an arbitrary location:
Obj *obj = ::operator new(sizeof(Obj);
new(obj) Obj;
[1] Note: I don't really mean this as a judgement, understanding of C++ internals are pretty poor in a lot of programmers and for pretty good reasons a lot of the time.
Hi, I'm a C# developer who has done some Java and C++ in the past. You may consider me in the box of people who thinks the constructor creates the object, and the object not existing until the constructor finishes. The allocation of memory is not what I would count as creation of an object, as a model of memory is basically unrelated to the idea of objects and language semantics as I use them. If the object cannot be used, I would consider it not yet created.
I am not trying to argue (on the contrary, I would defer to your expertise), but rather I mean to add a small data point of what perspective developers with my background may have.
I am not trying to argue (on the contrary, I would defer to your expertise), but rather I mean to add a small data point of what perspective developers with my background may have.
In the discussion of C++ constructors, someone usually (IME) says something like, "now remember, it's not a real Foo until the constructor finishes!" because the invariants haven't been set up yet. That could be where the misunderstanding you mentioned comes from.
In Smalltalk, 'new' is a class method which allocates a new instance, but which then has no special access to the internals of that newly created instance. An instance method, 'initialize' is then almost always used to set the initial state (by convention - this is not an actual language feature). Most commonly, initialize is called by new, but this again is just a convention, and a fairly recent one.
Interesting. Yes, it is very similar in both ruby and python, except that the convention is stronger by virtue of being the default behaviour of #new or .__new__ respectively. The placement of the methods (new on the class object and init[ialize] on the instance) is identical.
I really should do something in smalltalk. I only know details about it from reading about it, really, I've never had an opportunity to use it directly, though I go out of my way to understand its influences on more contemporary languages.
I really should do something in smalltalk. I only know details about it from reading about it, really, I've never had an opportunity to use it directly, though I go out of my way to understand its influences on more contemporary languages.
My very limited experience working with the Smalltalk language and environment was quite negative, although I hear people sing its praises, so perhaps I just didn't know what I was doing.
That said, it seems to me that many Ruby developers (and to a more limited degree, Python developers) are learning Smalltalk nowadays, for the same reason as English speakers learn Latin: to understand the origins of the language, and thus get a deeper understanding of how it works.
That said, it seems to me that many Ruby developers (and to a more limited degree, Python developers) are learning Smalltalk nowadays, for the same reason as English speakers learn Latin: to understand the origins of the language, and thus get a deeper understanding of how it works.
Yes, and while I'm generally of the view that Smalltalk has lessons to teach, of course they aren't all good ones - in particular I sometimes feel that the way construction and initialization is done in Smalltalk has tended to incentivise having getters and setters for every internal variable, and has perhaps contributed to the rather brittle style of program construction that is decried as part of "everything that's wrong with OOP".
Good point. Python's __init__ is closer to Common Lisp's 'initialize-instance :after' except for the fact that Common Lisp provides :initarg to allow make-instance to fully construct an object in most cases.
Here's another magical version that uses a class decorator:
class Autoinit(object):
def __call__(self, cls):
orig_init = cls.__init__
arg_count = orig_init.func_code.co_argcount
arg_vars = orig_init.func_code.co_varnames[1:1 + arg_count]
class Wrapped(cls):
def __init__(self, *args, **kwargs):
for pos, var in enumerate(arg_vars[:len(args)]):
setattr(self, var, args[pos])
for var in kwargs:
setattr(self, var, kwargs[var])
return orig_init(self, *args, **kwargs)
return Wrapped
if __name__ == '__main__':
@Autoinit()
class Test(object):
def __init__(self, a, b):
pass
t = Test(1, b=2)
print t.a, t.b
I don't see much value autoassignining args. I do see being useful at times to auto-assign kwargs.
class Base(object):
def __init__(self, *args, **kws):
self.__dict__.update(kws)
class Foo(Base):
def__init__("foo", age=30):
self.name = "foo"
super(Foo, self).__init__("foo", age=30)
def update(init_meth):
def wrapped(self, *args, **kws):
init_meth(self, *args, **kws)
self.__dict__.update(kws)
return wrapped
class Foo(object):
@update
def__init__("foo", age=30):
self.name = "foo"Oh boy, so I started using Python first. I took "reflection everywhere" for granted so hard without even knowing it. I learned Clojure next, and while it had its rough points I never thought, "Man this is really hard!"
Then, I learned some Scala & Java. That's when I realized how for granted I took this thing called "reflection". Especially when it came to ORMs. Holy mackerel. I got over it but it is definitely a huge thing -- for better and worse -- that is so easy in Python that really spoils beginners.
Then, I learned some Scala & Java. That's when I realized how for granted I took this thing called "reflection". Especially when it came to ORMs. Holy mackerel. I got over it but it is definitely a huge thing -- for better and worse -- that is so easy in Python that really spoils beginners.
The JVM does have powerful reflection capabilities, even though most programmers don't use it directly.
You do see it used for mocking, ORMs, dependency injection, etc.
You do see it used for mocking, ORMs, dependency injection, etc.
CoffeeScript has a nice syntactic sugar for setting properties by prefixing the argument name with "@":
class Foo
constructor: (@x, @y) ->
@z = @x + @y
setFoo: (@foo) ->
Interesting you mentioned CoffeeScript and syntactic sugar, because adding an equivalent "self." in python inits' parameter names doesn't work the same way in CoffeeScript.
def __init__(self, self.x, self.y): # fail
pass
No, I just thought the way CoffeeScript handles it is pretty slick. I looked around, and it seems like proposals to add something like this to Python did come up [1] [2], but weren't very welcomed.
[1] https://mail.python.org/pipermail/python-dev/2005-July/05456...
[2] http://bytes.com/topic/python/answers/101594-__autoinit__-pr...
[1] https://mail.python.org/pipermail/python-dev/2005-July/05456...
[2] http://bytes.com/topic/python/answers/101594-__autoinit__-pr...
I prefer the simpler:
class A(object):
def __init__(self, x, y):
self.__dict__.update((k,v) for k,v in locals().items() if k != 'self')I don't like code to casually access the inner dict. Now you have to maintain your `k != 'self'` (ugly in its own right) for each argument you want to exclude from this rule. The explicit version is superior, this is needlessly slow and complicated.
I had a different take on this - it's somewhat inspired by collections.namedtuple but mutable and has some other convenient features for data classes: https://gist.github.com/tomstrummer/8240282
(see the examples at the bottom of the file)
This could also be tweaked to use __dict__ instead of __slots__ so it can act as an expando class which I believe is more similar to what the OP achieves.
(see the examples at the bottom of the file)
This could also be tweaked to use __dict__ instead of __slots__ so it can act as an expando class which I believe is more similar to what the OP achieves.
Please don't.
def __init__(self, x, y):
self.x = x
self.y = y
This is pretty cool (reminds me of the scala approach to constructors). But please listen to the author's caveats, and never do something like this in a production system; there are few things worse than magic when you're debugging.
For me it's rather common (more than 75%) for the __init__ parameters be other than simple assignments to attributes.
This kind of complexity and the esoterica involved is exactly what terrifies me about Python.
The thing with Python is that you don't need to know this stuff to use the language effectively. Yes, after a while, you'll want to dive into the innards -- but they're optional for most people during their initial time learning Python.
Yes, but the depth of the pool is far from clear when you are considering diving in. :-)
I remember seeing something like this in Matellis' Python Cookbook.
[note: I'm aware this does not do even remotely a complete job as a decorator. Use at your own risk, it is a demonstration only.]