Tips for a new Django developer(zeroandone.posterous.com)
zeroandone.posterous.com
Tips for a new Django developer
http://zeroandone.posterous.com/top-10-tips-to-a-new-django-developer
8 comments
This is some really useful stuff. I learned a lot of this over time, and wished I had known it sooner.
The point about schema evolution is important.
I've taken to using a pickled python object in a text field in every model. That flexibility to store arbitrary data in a hash is amazing. I've used it to, for example, represent twitter accounts that are oauthed, with password, following the business account, or just nothing special. 4 kinds of accounts, with logic in the model to deal with it properly, all without evolving the schema. I only started with the first 2, and added the latter two without a problem.
I'm pushing this a bit further now, and transitioning away from models altogether, and experimenting with a non-relational database like couchdb. I'm debating moving to google app engine too, but found some details (like getting on your own domain) to be pretty annoying.
The point about schema evolution is important.
I've taken to using a pickled python object in a text field in every model. That flexibility to store arbitrary data in a hash is amazing. I've used it to, for example, represent twitter accounts that are oauthed, with password, following the business account, or just nothing special. 4 kinds of accounts, with logic in the model to deal with it properly, all without evolving the schema. I only started with the first 2, and added the latter two without a problem.
I'm pushing this a bit further now, and transitioning away from models altogether, and experimenting with a non-relational database like couchdb. I'm debating moving to google app engine too, but found some details (like getting on your own domain) to be pretty annoying.
Really don't like the sound of that, pickling allows you to serialize any python code, and consequently deserialize it. Presumably you are placing user submitted data into this pickle and therefore without careful sanity checking wouldn't this potentially allow the user to inject code which may be run on deserialization?
Someone else may be able to shed more light on this and correct me (please do!) but thats my concern and would love to hear if it was a valid one or not.
Someone else may be able to shed more light on this and correct me (please do!) but thats my concern and would love to hear if it was a valid one or not.
Whoa! I was assuming this worked like doing a JSON grab-bag column in Rails or J2EE, which I am a fan of. Executable code would make me much, muuuuuuuuuch cooler on the idea.
Can you just do the same thing with JSON, Yaml, or your favorite serialization method of choice that does not have catastrophic consequences if parsing gets subverted?
Can you just do the same thing with JSON, Yaml, or your favorite serialization method of choice that does not have catastrophic consequences if parsing gets subverted?
> Can you just do the same thing with JSON, Yaml, or your favorite serialization method of choice that does not have catastrophic consequences if parsing gets subverted?
Sure. You can even extend the `json` module to serialize custom data types not defined in the JSON spec.
Sure. You can even extend the `json` module to serialize custom data types not defined in the JSON spec.
No, I'm just serializing dicts. Any user input is already in a string, so sanitized. I also usually make helper functions to get/set things in the dict, so I don't operate on the field outside the model often.
This snippet is really helpful: http://www.djangosnippets.org/snippets/513/
This snippet is really helpful: http://www.djangosnippets.org/snippets/513/
It is possible to build a poisonous pickle which, on depickling, calls arbitrary standard library functions, so accepting pickles from untrusted source is a No.
One may think that if only the internal model of the user input is pickled, and the user only gets to update this internal model through web requests in defined ways, thus never getting to upload pickles or construct arbitrary complicated python objects, it should be relatively safe.
However, what remains is that if someone manages to blindly inject SQL, he can inject executable code into your application, instead of just injecting data.
One may think that if only the internal model of the user input is pickled, and the user only gets to update this internal model through web requests in defined ways, thus never getting to upload pickles or construct arbitrary complicated python objects, it should be relatively safe.
However, what remains is that if someone manages to blindly inject SQL, he can inject executable code into your application, instead of just injecting data.
Come to think of it, I think Hacker News works like this.
When you upvote and need to sign in, the code to execute the upvote is stored to be executed once you sign in.
When you upvote and need to sign in, the code to execute the upvote is stored to be executed once you sign in.
Pickling python objects and storing is an very old practice in Python world. Zope has been persisting objects in ZODB (a giant pickle jar with some transactional features) since 1998.
New to me :)
I love it
I love it
> I've taken to using a pickled python object in a text field in every model.
That's basically what App Engine does right there. It stores by an id, then the value is your object serialized as a protocol buffer.
The App Engine datastore allows you to look up by other properties than id because it indexes every field of your object in the background without you having to deal with it. You can query by pair-wise or more-wise indexes too, but you have to create those manually.
That's basically what App Engine does right there. It stores by an id, then the value is your object serialized as a protocol buffer.
The App Engine datastore allows you to look up by other properties than id because it indexes every field of your object in the background without you having to deal with it. You can query by pair-wise or more-wise indexes too, but you have to create those manually.
Storing pickled python objects is fine as long as you don't care about relational data. But if you aren't careful, data integrity will probably suffer over time as the complexity of your application increases. And that is not good. Anyway I'm sure a lot of people here know this stuff already. Just do your research and make sure the route you go makes sense for your application and your scaling requirements.
Would be interested to hear your take on Django+CouchDB - are you using a library like Couchdbkit, or rolling your own? Are you connecting it to the built-in admin?
Still exploring. I'll post something when it actually works.
Man, a lot of those sound familiar. A lot of hard-coded path problems are especially annoying since the official Django documentation often tells you to do it that way.
On a semi-related topic - I have an old 0.96 Django app that was using an old database migration framework that isn't supported in the newer versions of Django. For that framework I wrote the Django migrations in manually generated SQL. Anyone know how I could update to django-evolution or South with this hand coded SQL and still preserve my existing data?
The documentation for django-evolution says:
> Django Evolution imposes one important restriction on your development process: If you intend to make a change to your database, you must do it through Django Evolution. If you modify the database outside of Django and the Django Evolution framework, you're on your own.
Which makes me think I can't use that.
On a semi-related topic - I have an old 0.96 Django app that was using an old database migration framework that isn't supported in the newer versions of Django. For that framework I wrote the Django migrations in manually generated SQL. Anyone know how I could update to django-evolution or South with this hand coded SQL and still preserve my existing data?
The documentation for django-evolution says:
> Django Evolution imposes one important restriction on your development process: If you intend to make a change to your database, you must do it through Django Evolution. If you modify the database outside of Django and the Django Evolution framework, you're on your own.
Which makes me think I can't use that.
I love django-evolution, I'm using it on AppRabbit.com to regenerate dynamic models and it's been a lifesaver. The only hitches I've found (granted the relationships between my models are fairly restrictive) are related to adding null=False fields and such. But really, it handles most things seamlessly.
Great tips. When I was starting, some of these didn't occur to me and the others I knew I should be doing but didn't know how.
Thank you for this. I am starting and hopefully this'll give me a solid foundation.
thanks for this, i just started django development couple days ago, and this is very useful.
another thing worth mention is that use generic view, it will save lots typing:)
another thing worth mention is that use generic view, it will save lots typing:)
These are pretty good. I really liked #6, the tip for template_tags. I hadn't thought of that before and will definitely do that. I will also look into extensions of the template system as suggested at the bottom. Those were really helpful, even for someone who has been using Django for a few months.
I disagree with #4, the suggestion to not include business logic in views. What Django calls 'views' are what other frameworks call 'controllers' (MTV=MVC), and the business logic should reside there. I'm totally for separating the code into helper functions in other files, but putting business logic into models definitely violates the idea of MVC.
For #5, I would recommend the second approach of having a separate settings.py for production vs development. Or at least code it where DEBUG=False is only overridden if socket confirms you are on a development machine - not the other way around. Debug contains too much info to risk exposing in production, but maybe my fear of socket screwing up is unfounded.
I disagree with #4, the suggestion to not include business logic in views. What Django calls 'views' are what other frameworks call 'controllers' (MTV=MVC), and the business logic should reside there. I'm totally for separating the code into helper functions in other files, but putting business logic into models definitely violates the idea of MVC.
For #5, I would recommend the second approach of having a separate settings.py for production vs development. Or at least code it where DEBUG=False is only overridden if socket confirms you are on a development machine - not the other way around. Debug contains too much info to risk exposing in production, but maybe my fear of socket screwing up is unfounded.
Not to get into a pedantic argument with you, friend, but the generally accepted definition of "model" in the MVC pattern is "the place that holds your business logic." See http://en.wikipedia.org/wiki/Model-view-controller, for example (the article uses the term "domain logic," but it's the same thing - http://en.wikipedia.org/wiki/Domain_logic).
Re: #4 My 2 cents: I come from a Java web background, so I may be a bit biased.
I like to think of the view functions as controlling state in an app and deciding what do show on a page (handing input or sticking stuff in the request). Something in between the presentation logic (in templates) or "business logic".
I think the django docs suggest that business (or domain) logic should go in the model (in an extra method) if it pertains to a particular instance of the model - or row in the db. Class level logic should go in a Manager for the model - like doing special queries, for example.
As an aside, I think that this approach makes my code pretty easy to test, I can whip up a test case that uses data in fixtures without having to worry about the view. Using something like the command_extensions makes it pretty easy to prototype model/manager code in the python shell against real data in my dev env. This is probably my favorite thing about working with python/django over java.
I like to think of the view functions as controlling state in an app and deciding what do show on a page (handing input or sticking stuff in the request). Something in between the presentation logic (in templates) or "business logic".
I think the django docs suggest that business (or domain) logic should go in the model (in an extra method) if it pertains to a particular instance of the model - or row in the db. Class level logic should go in a Manager for the model - like doing special queries, for example.
As an aside, I think that this approach makes my code pretty easy to test, I can whip up a test case that uses data in fixtures without having to worry about the view. Using something like the command_extensions makes it pretty easy to prototype model/manager code in the python shell against real data in my dev env. This is probably my favorite thing about working with python/django over java.
For template tags, is doing it that way better than importing them in a base template which is included by the children templates?
Templatetags don't inherit (at least not in my experience) so if you don't want to have to load the tags on every template, add them to your global templatetags: http://www.djangosnippets.org/snippets/160/
Of-course, be careful while doing so though 'cause the stuff that isn't documented can change at anytime, and may break your code.