Fragments of Code

Django: Use your context

May 26, 2009 · 2 Comments

Sometimes your application needs certain common values in its templates, such as a base URL to external media files (not served under MEDIA_URL).  Initially I thought a custom template tag was the way to go, and it does work, but there are some disadvantages.  You have to load your template tag library into each template explicitly, and, especially for simple “static value” uses, it makes your code somewhat more complex and obscure.  OTOH, a template context processor isn’t really appropriate for an application because it applies at site or project level and so affects requests to all applications.  Now, we can pass values from views to templates, but being mindful of the DRY principle, we want some way to factor out common code such as this.  My most recent answer is a factory function in my views module that returns a django.template.RequestContext instance with some common variables:

from django.template import RequestContext

def _context(request, extra={}):
    """Returns a RequestContext instance with some custom variables."""
    extra['foo'] = 'bar'
    return RequestContext(request, extra)

Then each view can add this custom context to its return value:

return render_to_response(my_template, context_instance=_context(request))

→ 2 CommentsCategories: Django · code
Tagged: , , ,

Tweaking Django auth admin

May 26, 2009 · Leave a Comment

So, I wanted one of those horizontal multi-selects with “available” and “chosen” boxes to associate users with groups in the Django auth admin UI.  Turns out it all I had to do was import the UserAdmin class from django.contrib.auth.admin and override the filter_horizontal attribute:

from django.contrib.auth.admin import UserAdmin
UserAdmin.filter_horizontal = (’user_permissions’, ‘groups’)
from django.contrib.auth.admin import UserAdmin

UserAdmin.filter_horizontal = ('user_permissions', 'groups')

Since importing the UserAdmin class runs the admin module from django.contrib.auth, the UserAdmin and GroupAdmin classes are registered for the admin site.  All I have to do then is import my custom admin module in my URLconf instead of the one from django.contrib.auth to make sure my customizations are applied in my admin site.

→ Leave a CommentCategories: Django · code
Tagged: ,

Vista x64 challenges

March 14, 2009 · Leave a Comment

On balance I’m satisfied with Vista x64, but I’ve run into a few challenges and at least one casualty.

The casualty was my Palm Tungsten E2, which can’t connect to Vista64 with the USB cable.  According to Palm, you can connect with Bluetooth, but I don’t have Bluetooth on the computer.  Fortunately I don’t use the Palm much anymore and I synced mainly to back it up.

As for the challenges, it seems that some applications, notably OpenOffice.org 3, have to be installed using XP-SP2 compatibility mode.  Also, some installation software doesn’t trigger privilege elevation which may be required to write to certain registry keys or system folders, so the “Run as Administrator” and “CMD prompt here as administrator” Elevation PowerToys are practically necessary.  Sun’s Java RE (which I probably wouldn’t install except that it’s a dependency of OOo) has an annoying bug in it’s control panel such that you can’t change the settings, so you can’t disable automatic updating.  The workaround is to run bin/javacpl.exe as administrator using the Elevation PowerToy.  Using 32-bit Windows help files (.hlp) requires a special download (this affects all Vista versions) as Microsoft is no longer updating the older help program and so doesn’t distribute it with the OS.  Only two of the four buttons on my Kensington Expert Trackball Mouse work, and Vista thinks it’s a regular mouse — but it wasn’t working fully under XP-SP3 either.  Don’t bother to install the MouseWorks software, although some folks claim they’ve gotten it work (or at least the OS to use the driver).  I would guess at this point Kensington isn’t going to release a Vista-compatible driver for the older devices.

I can live with two buttons.

→ Leave a CommentCategories: Uncategorized
Tagged: , , ,

lxml makes XML (almost) fun

March 13, 2009 · Leave a Comment

Working with XML isn’t my favorite task. Sure, I get why it’s useful for data transfer, but I’d rather deal with an interface that hides the gory details … meaning the raw XML.  Of course, sometimes that isn’t possible.  The Python standard library modules as of version 2.4 only offers the DOM and SAX APIs, which, while useful, just don’t offer enough power and flexibility to do ad hoc XML processing without significant pain, at least for the casual user. Python 2.5 adds the ElementTree API (xml.etree.ElementTree), but we’re still missing full XPath and XSLT support.

In steps lxml, a library that really makes everything else obsolete. Prior to lxml there was 4Suite, but its latest release was posted in December 2006 and the project appears to be dead. Fortunately, lxml runs on Python 2.3 or later (I’m stuck on 2.4 for the time being).  The only hiccup, depending on your environment, can be getting the C dependencies installed – libxml and libxslt. But if you’re going to do some serious work with XML, it’s worth the effort to get lxml installed.  You won’t want to go back.

DOM

>>> xmlsrc = '<root><element>text</element></root>'
>>> from xml.dom.minidom import parseString
>>> parseString(xmlsrc).getElementsByTagName('element')[0].firstChild.nodeValue
u'text'

Yikes!

lxml

>>> xml = '<root><element>text</element></root>'
>>> from lxml import etree
>>> etree.fromstring(xml).findtext('//element')
'text'

Ahh … better.

→ Leave a CommentCategories: Python · XML · code
Tagged: ,

Django gotcha: related objects deleted by default

March 6, 2009 · 2 Comments

I discovered “accidentally” recently that the Django model delete() method not only deletes the model instance, but all its related objects — at least those which are related to the original object via a ForeignKey field.  (The source code is so labyrinthine that I gave up trying to determine exactly what it does.)  This is not necessarily a bad thing; many times it’s exactly what you want to preserve the “referential integrity” of your data.  For instance, if you have some kind of “collection” object and a number of “item” objects which are related to it in a one-to-many relationship, it’s reasonable to expect that when the collection is deleted, the related items are also deleted.

Now, in the Django admin app, when you click the button to delete an item you get a helpful warning informing you of all the other objects you will delete if you proceed.  But what if you’re using the API directly?  No warning there, it just whacks the whole lot.  At first I thought maybe this was an ORM thing, but it’s not.  In fact it has nothing to do with the backend database: Django manually fetches all the related objects and deletes them.  For example, in my case the backend is MySQL 5.0 and MyISAM tables.  In MySQL 5.0 only InnoDB tables support foreign key constraints; MyISAM tables will parse the syntax but do nothing with it.  In any case, the constraints that Django generates do not include an ON DELETE clause, so MySQL 5.0 would use “RESTRICT” as the default value, meaning that the database will not allow the deletion of a row from the “parent” table if the primary key value exists in the referenced foreign key of a row in the related table.  I suppose that’s good as far as it goes, but you shouldn’t be messing with the database directly, right?  Anyway, that’s not relevant because we’re not talking about raw SQL commands, but the Django model API methods.

So, there’s an outstanding proposal to add keyword arguments to the ForeignKey model field to control how DELETE and UPDATE on the parent model affect the related model.  The status of the proposal is unclear and there hasn’t been a lot of serious discussion on the mailing lists AFAICT.  I would guess that since you can override the delete() method on a per-model basis, and presumably other issues are more pressing, that the core developers don’t want to worry about this right now.  They may be right, but I sure had an unpleasant surprise when I discovered this behavior the wrong way.

I have an organizational directory database with Persons and OrgUnits.  Persons are automatically added and removed based on information retrieved an another data source (LDAP).  An OrgUnit (e.g., a department) can have a “head”, which is a person:

head = models.ForeignKey(Person, blank=True, null=True, related_name='head_of')

Now, of course, if a Person who happened to be the head of a department left the organization I wouldn’t want the department to be deleted, right?  In this case, as it turns out, it was worse than that.  Since the OrgUnit structure is hierarchical, the OrgUnit model has a one-to-many relationship with itself:

parent = models.ForeignKey('self', null=True, blank=True, related_name='children')

Now what happens if the head of a top-level OrgUnit is deleted?  The OrgUnit of which he/she was head is deleted, and every “descendant” OrgUnit under that one!  So, I had to override the delete() methods on both the Person and OrgUnit models.

Person:

def delete(self):
"""
Override default model method so an OrgUnit is not deleted
when its head is deleted.
"""
self.head_of.clear()
super(Person, self).delete()

OrgUnit:

def delete(self):
"""
Override default model method so that OrgUnit children are
not deleted when the parent OrgUnit is deleted.
"""
self.children.clear()
super(OrgUnit, self).delete()

On balance, Django’s default behavior is probably the right thing to do — as long as you’re aware of it!

→ 2 CommentsCategories: Django · MySQL · code
Tagged: ,

Doctests make me love unit testing

March 6, 2009 · Leave a Comment

I’ve long had the nagging feeling that I’m not writing enough tests for my code. It’s kind of like doing the dishes, not exactly a task I look forward to.  The fun of programming is solving the problems, not writing code that tests whether your solutions actually work! 

What I love about Python’s doctest is that it fits so naturally into the flow of coding: you write the function signature, then write the docstring saying what it’s supposed to do, right?  So then, right there, you write one or more simple tests to demonstrate the basic functionality you expect — before you’ve written a line of code in the body of the function. You write the function, save the file and then — boom! — run the tests.  Immediate feedback.  And I start to feel like my approach to programming is changing, because I can see that writing tests is not just about due diligence or whatever, but it’s a technique that can actually help me write code.

And — you can paste in commands from the Python interactive interpreter unchanged.  Yet another reason why Python feels like it’s designed with the needs of programmers, human beings, in mind.  It’s trying to make your life easier, simpler, and more productive.  While Python isn’t perfect (no language is), those things alone make it rate pretty high in my book.

→ Leave a CommentCategories: Python
Tagged: ,

Get a fully-qualified URL for the current Django site

February 24, 2009 · 2 Comments

You need to generate a fully-qualified URL to a Django page, in particular outside of a web request context (in which you would have access to server variables), such as an automated process that generates e-mail with links.  You may be able to generate a root-relative URL from a reverse lookup; there’s also get_absolute_url() of course, but it’s provided on a per-model basis, and in any case shouldn’t be coupled with URL elements such as protocol and host name.  You can get the domain part of the host name from the current site object, but Django currently (as of version 1.0.2) provides no means for reliably generating a fully-qualified URL (including protocol and port) outside of a web request context.  In the function current_site_url(), below, I have used two custom settings, MY_SITE_PROTOCOL and MY_SITE_PORT.  (My current practice is to prefix custom settings with MY_,  place them in a parallel module in the project called my_settings.py, and import the custom settings into the project settings module.)

def current_site_url():
    """Returns fully qualified URL (no trailing slash) for the current site."""
    from django.contrib.sites.models import Site
    current_site = Site.objects.get_current()
    protocol = getattr(settings, 'MY_SITE_PROTOCOL', 'http')
    port     = getattr(settings, 'MY_SITE_PORT', '')
    url = '%s://%s' % (protocol, current_site.domain)
    if port:
        url += ':%s' % port
    return url

Now, I still don’t really have enough information to construct a fully-qualified URL for the most general case, because in taking advantage of the django.root setting, my code no longer “knows” what Django’s root path is.  That was good for decoupling the URLconf from the web server conf, but again, I need to generate fully-qualified URLs outside of a web request context, so I don’t have access the django.root setting.  My solution has been to add another custom setting, MY_DJANGO_URL_PATH, which corresponds to the django.root setting (a comment Django’s mod_python handler module indicates that the handler must be called before importing any settings in order for os.environ to be set up correctly with respect to settings).  With that, I can get my Django root URL with this function:

def django_root_url(fq=False):
    """Returns base URL (no trailing slash) for the current project.

    Setting fq parameter to a true value will prepend the base URL
    of the current site to create a fully qualified URL.

    The name django_root_url is used in favor of alternatives
    (such as project_url) because it corresponds to the mod_python
    PythonOption django.root setting used in Apache.
    """
    url = getattr(settings, 'MY_DJANGO_URL_PATH', '')
    if fq:
        url = current_site_url() + url
    return url

With these functions and Django’s reverse URL lookup, I can construct fully-qualified URLs.

→ 2 CommentsCategories: Apache · Django · code
Tagged: , ,

GenericSetup and separation of concerns

February 17, 2009 · Leave a Comment

As I continue my investigation of and attempt to implement a GenericSetup approach to customizations for my Plone site, I am seeing more clearly the need for this transition, even if the reasoning behind some of the design choices escapes me.  

In TTW customization of Zope 2 there was no natural way to cleanly separate customizations from default functionality.  Perhaps more significantly, the system did not really provide a canonical way of separating presentation from business logic.  Zope Page Templates (and the older DTML methods) were, and are, too powerful for their own good.  On the other side “Script (Python)” scripts are too limited in power (due to security restrictions) and scope to be able to handle general purpose business logic.  This state of affairs leads almost unavoidably to “spaghetti code” strung across a mixture of page templates, DTML methods, “Script (Python)” scripts and “External Methods”. 

The addition of “Controller Page Template”, “Controller Python Script”, and “Controller Validator” objects appears to have been an earlier attempt to address the problem of a lack of a clear MVC-style “controller”.  While these objects provided some conveniences, at least for form processing, they didn’t eliminate the deeper problem.  I’m still not sure that Plone 3/Zope 2/Five has really resolved the fundamental issue, but definite progress has been made.  

I do regret the profusion in GenericSetup of XML configuration files (a la Java web apps) which IMO humans should not have to read, much less write. The use of XML-like “ZCML” for the main configuration files seems particularly strange. There must have been some initial reason for not using XML, but now it just seems arbitrary and bewildering.  Personally, I would much prefer a pure Python implementation, but what do I know?

→ Leave a CommentCategories: Plone · Python · Zope
Tagged: , , , , , ,

Plone: Transitioning from TTW customization to GenericSetup

February 13, 2009 · Leave a Comment

So, I buy into the idea of GenericSetup, but I’m finding the transition to using it for my site customizations rather painful.  In part, this is due to the fact that most of the relevant documentation on plone.org is a bit old and thin.  It seems that the best bet for a quick start is to use the generator script that ships with DIYPloneStyle.  There is still a certain amount of guessing or trial and error, but at least the configuration file comments are helpful.  You can also find examples in the CMFPlone package.

My Plone site goes back to Plone 2.0.5, so I went though the painful transition to Archetypes-based content types. That change was mandatory to get to Plone 2.1+, and now I’m seeing the writing on the wall in Plone 3 with GenericSetup and Zope 3 technologies.  While I worried in a previous post that this change was adding complexity that would make customizing Plone more difficult for the casual developer, I also recognize that the new approach provides a level of modularity across the site that was not possible with TTW customization alone.  But since I was fairly heavily invested in the TTW approach, I am now having to untangle all the little things that were added or changed in various places, from site properties to actions and workflow.

→ Leave a CommentCategories: Plone · Zope
Tagged: , , ,

Python script for Apache RewriteMap

February 4, 2009 · Leave a Comment

It turned out that it didn’t need this after all, but thought I’d post it here anyway …

The use case was to base64-encode a URL so that it could be passed as a query parameter to a login page.  The login routes through a third page and returns to the logn page, which redirects the client back to the original URL which was base64-encoded.  Without the encoding, the third page could mangle the original URL.

#!/usr/bin/python

import binascii
import sys

while sys.stdin:
    print binascii.b2a_base64(sys.stdin.readline().rstrip()) ,
    sys.stdout.flush()

→ Leave a CommentCategories: Apache · Python
Tagged: , , ,