Getting the name of a class

I’m posting this here because the answer wasn’t easy to find — in fact, I got it by guessing.

The problem is: How to get the name of a class as a string — not the class name of an instance, but the name of class object itself.  The answer turns out to be simply the “magic” attribute __name__.  Unfortunately, you can’t see this attribute with dir(), and the Python tutorial section on classes (as of 2.6.4) makes no mention of it.  To be sure, this is not a common use case, but I happened to have one in hand.

>>> from mymodule import User
>>> u = User
>>> u.__class__.__name__
'type'
>>> u.__name__
'User'
>>> user = u('id')
>>> user.__class__.__name__
'User'

xlwt convenience methods

As I started using xlwt, I found myself wanting some more convenient methods for dumping tabular data into a worksheet, especially when all the data can be treated as strings.  Tabular data in this context is an iterable of iterables, such as a list of tuples.

Here’s what I’ve got so far:

"""Excel utilities.

'Tabular data' in this context is an iterable of iterables.
"""

import xlwt

def to_workbook(tabular_data, workbook=None, sheetname=None):
    """
    Returns the Excel workbook (creating a new workbook
    if necessary) with the tabular data written to a worksheet
    with the name passed in the 'sheetname' parameter (or a
    default value if sheetname is None or empty).
    """
    wb = workbook or xlwt.Workbook()
    ws = wb.add_sheet(sheetname or 'Data')
    to_worksheet(tabular_data, ws)
    return wb

def to_worksheet(tabular_data, worksheet):
    """
    Writes the tabular data to the worksheet (returns None).
    Thanks to John Machin for the tip on using enumerate().
    """
    for row_index, row_data in enumerate(tabular_data):
        worksheet_row = worksheet.row(row_index)
        for col_index, col_data in enumerate(row_data):
            worksheet_row.write(col_index, col_data)

In a Django context, then, you have a very straightforward way turning a query into an Excel file using the values_list() QuerySet method, e.g.:

wb = to_workbook(MyModel.objects.values_list())

Since values_list() outputs the attribute values for each object in the same order in which they’re defined in the model class, you can insert a row of headings to your table:

table = MyModel.objects.values_list()
headings = [f.name for f in MyModel._meta.fields]
table.insert(0, headings)
wb = to_workbook(table)

A Generic context processor for Django settings

I found myself creating a number of simple custom context processors which simply return custom settings that I have added to my Django settings module (I actually keep these custom settings in my_settings.py and import then into settings.py, just to keep them separate).  I decided it was a good idea to add exception handling to these functions so that I would get a useful error message if I tried to use a particular context processor without implementing its required setting(s).  So, after some refactoring, I came up with this:

from django.conf import settings
from django.core.exceptions import ImproperlyConfigured

class SettingsContextProcessor(object):
    """
    Class for creating simple context processors that
    return one or more Django settings.
    """

    def __init__(self, *setting_names):
        self.setting_names = setting_names

    def __call__(self, request):
        extra_context = {}
        for sn in self.setting_names:
            try:
                extra_context[sn] = getattr(settings, sn)
            except AttributeError, e:
                raise ImproperlyConfigured('Missing required setting: %s' % sn)
        return extra_context

Now I can create custom settings-based context processors like this:

google_analytics = SettingsContextProcessor('GOOGLE_ANALYTICS_PROFILE_ID')
jquery = SettingsContextProcessor('JQUERY_VERSION')
jqueryui = SettingsContextProcessor('JQUERYUI_VERSION')
static_media = SettingsContextProcessor('STATIC_MEDIA_URL')
yui = SettingsContextProcessor('YUI_VERSION')

Tweaking Django auth admin

So, I wanted one of those horizontal multi-selects with “available” and “chosen” boxes to associate users with groups in the Django auth admin UI.  Turns out it all I had to do was import the UserAdmin class from django.contrib.auth.admin and override the filter_horizontal attribute:

from django.contrib.auth.admin import UserAdmin

UserAdmin.filter_horizontal = (‘user_permissions’, ‘groups’)

from django.contrib.auth.admin import UserAdmin

UserAdmin.filter_horizontal = ('user_permissions', 'groups')

Since importing the UserAdmin class runs the admin module from django.contrib.auth, the UserAdmin and GroupAdmin classes are registered for the admin site.  All I have to do then is import my custom admin module in my URLconf instead of the one from django.contrib.auth to make sure my customizations are applied in my admin site.

lxml makes XML (almost) fun

Working with XML isn’t my favorite task. Sure, I get why it’s useful for data transfer, but I’d rather deal with an interface that hides the gory details … meaning the raw XML.  Of course, sometimes that isn’t possible.  The Python standard library modules as of version 2.4 only offers the DOM and SAX APIs, which, while useful, just don’t offer enough power and flexibility to do ad hoc XML processing without significant pain, at least for the casual user. Python 2.5 adds the ElementTree API (xml.etree.ElementTree), but we’re still missing full XPath and XSLT support.

In steps lxml, a library that really makes everything else obsolete. Prior to lxml there was 4Suite, but its latest release was posted in December 2006 and the project appears to be dead. Fortunately, lxml runs on Python 2.3 or later (I’m stuck on 2.4 for the time being).  The only hiccup, depending on your environment, can be getting the C dependencies installed – libxml and libxslt. But if you’re going to do some serious work with XML, it’s worth the effort to get lxml installed.  You won’t want to go back.

DOM

>>> xmlsrc = '<root><element>text</element></root>'
>>> from xml.dom.minidom import parseString
>>> parseString(xmlsrc).getElementsByTagName('element')[0].firstChild.nodeValue
u'text'

Yikes!

lxml

>>> xml = '<root><element>text</element></root>'
>>> from lxml import etree
>>> etree.fromstring(xml).findtext('//element')
'text'

Ahh … better.

Django gotcha: related objects deleted by default

I discovered “accidentally” recently that the Django model delete() method not only deletes the model instance, but all its related objects — at least those which are related to the original object via a ForeignKey field.  (The source code is so labyrinthine that I gave up trying to determine exactly what it does.)  This is not necessarily a bad thing; many times it’s exactly what you want to preserve the “referential integrity” of your data.  For instance, if you have some kind of “collection” object and a number of “item” objects which are related to it in a one-to-many relationship, it’s reasonable to expect that when the collection is deleted, the related items are also deleted.

Now, in the Django admin app, when you click the button to delete an item you get a helpful warning informing you of all the other objects you will delete if you proceed.  But what if you’re using the API directly?  No warning there, it just whacks the whole lot.  At first I thought maybe this was an ORM thing, but it’s not.  In fact it has nothing to do with the backend database: Django manually fetches all the related objects and deletes them.  For example, in my case the backend is MySQL 5.0 and MyISAM tables.  In MySQL 5.0 only InnoDB tables support foreign key constraints; MyISAM tables will parse the syntax but do nothing with it.  In any case, the constraints that Django generates do not include an ON DELETE clause, so MySQL 5.0 would use “RESTRICT” as the default value, meaning that the database will not allow the deletion of a row from the “parent” table if the primary key value exists in the referenced foreign key of a row in the related table.  I suppose that’s good as far as it goes, but you shouldn’t be messing with the database directly, right?  Anyway, that’s not relevant because we’re not talking about raw SQL commands, but the Django model API methods.

So, there’s an outstanding proposal to add keyword arguments to the ForeignKey model field to control how DELETE and UPDATE on the parent model affect the related model.  The status of the proposal is unclear and there hasn’t been a lot of serious discussion on the mailing lists AFAICT.  I would guess that since you can override the delete() method on a per-model basis, and presumably other issues are more pressing, that the core developers don’t want to worry about this right now.  They may be right, but I sure had an unpleasant surprise when I discovered this behavior the wrong way.

I have an organizational directory database with Persons and OrgUnits.  Persons are automatically added and removed based on information retrieved an another data source (LDAP).  An OrgUnit (e.g., a department) can have a “head”, which is a person:

head = models.ForeignKey(Person, blank=True, null=True, related_name='head_of')

Now, of course, if a Person who happened to be the head of a department left the organization I wouldn’t want the department to be deleted, right?  In this case, as it turns out, it was worse than that.  Since the OrgUnit structure is hierarchical, the OrgUnit model has a one-to-many relationship with itself:

parent = models.ForeignKey('self', null=True, blank=True, related_name='children')

Now what happens if the head of a top-level OrgUnit is deleted?  The OrgUnit of which he/she was head is deleted, and every “descendant” OrgUnit under that one!  So, I had to override the delete() methods on both the Person and OrgUnit models.

Person:

def delete(self):
"""
Override default model method so an OrgUnit is not deleted
when its head is deleted.
"""
self.head_of.clear()
super(Person, self).delete()

OrgUnit:

def delete(self):
"""
Override default model method so that OrgUnit children are
not deleted when the parent OrgUnit is deleted.
"""
self.children.clear()
super(OrgUnit, self).delete()

On balance, Django’s default behavior is probably the right thing to do — as long as you’re aware of it!

Get a fully-qualified URL for the current Django site

You need to generate a fully-qualified URL to a Django page, in particular outside of a web request context (in which you would have access to server variables), such as an automated process that generates e-mail with links.  You may be able to generate a root-relative URL from a reverse lookup; there’s also get_absolute_url() of course, but it’s provided on a per-model basis, and in any case shouldn’t be coupled with URL elements such as protocol and host name.  You can get the domain part of the host name from the current site object, but Django currently (as of version 1.0.2) provides no means for reliably generating a fully-qualified URL (including protocol and port) outside of a web request context.  In the function current_site_url(), below, I have used two custom settings, MY_SITE_PROTOCOL and MY_SITE_PORT.  (My current practice is to prefix custom settings with MY_,  place them in a parallel module in the project called my_settings.py, and import the custom settings into the project settings module.)

def current_site_url():
    """Returns fully qualified URL (no trailing slash) for the current site."""
    from django.contrib.sites.models import Site
    current_site = Site.objects.get_current()
    protocol = getattr(settings, 'MY_SITE_PROTOCOL', 'http')
    port     = getattr(settings, 'MY_SITE_PORT', '')
    url = '%s://%s' % (protocol, current_site.domain)
    if port:
        url += ':%s' % port
    return url

Now, I still don’t really have enough information to construct a fully-qualified URL for the most general case, because in taking advantage of the django.root setting, my code no longer “knows” what Django’s root path is.  That was good for decoupling the URLconf from the web server conf, but again, I need to generate fully-qualified URLs outside of a web request context, so I don’t have access the django.root setting.  My solution has been to add another custom setting, MY_DJANGO_URL_PATH, which corresponds to the django.root setting (a comment Django’s mod_python handler module indicates that the handler must be called before importing any settings in order for os.environ to be set up correctly with respect to settings).  With that, I can get my Django root URL with this function:

def django_root_url(fq=False):
    """Returns base URL (no trailing slash) for the current project.

    Setting fq parameter to a true value will prepend the base URL
    of the current site to create a fully qualified URL.

    The name django_root_url is used in favor of alternatives
    (such as project_url) because it corresponds to the mod_python
    PythonOption django.root setting used in Apache.
    """
    url = getattr(settings, 'MY_DJANGO_URL_PATH', '')
    if fq:
        url = current_site_url() + url
    return url

With these functions and Django’s reverse URL lookup, I can construct fully-qualified URLs.

My life with Plone, part 3

A continuation of parts 1 and 2.

Workflow

Plone’s workflow functionality, which is the portal_workflow tool of the underlying Zope CMF, can do just about anything you want it to do, provided that you can figure out how to do it. Plone itself still doesn’t have a UI for creating and configuring workflow, so you have to do it in the ZMI, which AFAICT has changed little in the time I have used Zope and the CMF.  Plone has fairly recently integrated placeful workflow, adding the ability to override sitewide workflows at the folder level, which is an important consideration in a large and complex site.  Hopefully, they’ve fixed the bugs in the UI (both Plone and the ZMI) which I encountered when I first tried using it.

There are a couple things about CMF/Plone workflow that are not exactly intuitive, or at least are different from how other CMS systems implement workflow functionality:

1. A content item is always under workflow, if its content type is configured to use workflow at all. In other words, a content item doesn’t go “through” workflow; rather, the item is bound to its workflow.
2. Workflow has no security per se.  In other words, access to objects in various workflow states is determined by the security settings on the objects in the ZODB. Workflow transitions implement changes in ZODB security settings.

An access control gotcha

Zope security settings are, so to speak, atomic.  Inheritance of permissions can be toggled on/off at any level of the object hierarchy and explicitly overridden on any object.  So far, so good.  However, unlike file systems, access to objects is not strictly dependent on access to their containers.  For example, a “private” folder which is not publicly accessible can contain “published” or “public draft” items which are.  The security problem is that content authors may assume, not without reason, that any content they put in a private folder will also be private.  This is a deep issue because CMF workflow explicitly sets object permissions and Zope security only considers the permissions on an object, not its container (factoring in inherited permissions, of course).  A workaround I have implemented is a modification of the default workflow to force setting the initial state of an object to “private” if its parent object is in the private state.  At least this provides a modicum of protection to unwitting content authors; they have to explicity “make visible” or “publish” these items to make them more widely accessible.

1. Wrote a script hideInPrivateParent:

## Script (Python) "hideInPrivateParent"
##bind container=container
##bind context=context
##bind namespace=
##bind script=script
##bind subpath=traverse_subpath
##parameters=state_change
##title=Transition state_change.object to private state if parent object is private
##
wftool = context.portal_workflow
PRIVATE_TRANSITION = 'hide'
PRIVATE_STATE = 'private'
obj = state_change.object
parent = obj.aq_inner.getParentNode()
if parent is not None:
    try:
        if wftool.getInfoFor(parent, 'review_state') == PRIVATE_STATE:
            wftool.doActionFor(obj, PRIVATE_TRANSITION)
    except:
        pass

2. Created a new workflow state called “initialized” and set it as the default workflow state.

3. Created a new workflow transition call “initialize”.  Set the “trigger type” to “automatic”.  Set the “destination state” to “visible”. Set “script (after)” to hideInPrivate Parent.

The result is that when a new content item is created with this workflow, its state will be private if its parent object (typically a folder) is private, otherwise it will be “visible”.

Continued in Part 4.

Adding object tools to Django admin pages

Sometimes I want to add an item to the list of “object tools” on the change form for a model. The base admin/change_form.html template, however, does not (as of Django 1.0.2) include a block inside the object tools list:

<ul class="object-tools"><li><a href="history/" class="historylink">{% trans "History" %}</a></li>
{% if has_absolute_url %}<li><a href="../../../r/{{ content_type_id }}/{{ object_id }}/" class="viewsitelink">{% trans "View on site" %}</a></li>{% endif%}
</ul>

I don’t want to reproduce this code to add something to the list, so instead I write a JavaScript function with jQuery:

function appendObjectTools(tools) {
    /* Appends items to the object tool list.
     *
     * 'tools' param is an Array of Objects in which each Object
     * has 'url' and 'title' properties corresponding to the
     * URL and href for each link.
     */
    html = '';
    for (var i=0; i < tools.length; i++) {
	html += '<li><a href="' + tools[i].url + '">' + tools[i].title + '</a></li>';
    }
    $('#content-main .object-tools').append(html);
}

I can then override the change form for a model:

{% extends "admin/change_form.html" %}
{% block extrahead %}
    {{ block.super }}
    <script type="text/javascript">
$(function() {
    tools = [
        {
            url: "{% url collections id=original.id %}",
            title: "Collections"
        }
    ];
    appendObjectTools(tools);
});
    </script>
{% endblock %}

jQuery and the custom script function can be added using ModelAdmin media definitions.

Now on my change form I get a pretty oval button.

Subclassing Django’s ModelAdmin

After writing about sweetening URLFields on Django admin forms, I got to thinking, why not subclass ModelAdmin instead? It works!

from django.contrib import admin

class LocalModelAdmin(admin.ModelAdmin):
    class Media:
        js = (
            'js/tinymce/tiny_mce.js',
            'js/init/tinymce_advanced.js',
            'js/init/url_features.js',
            )

admin.py:

from path.to.module import LocalModelAdmin

class MyModelAdmin(LocalModelAdmin):
    ...

MyModel list and add/change forms now get the js files from LocalModelAdmin.