Use the framework

I don’t know how many times I’ve developed or implemented some clever enhancement of an application or framework (e.g., Plone, Django, others) only to regret it later.  Of course it always seems like a reasonable idea at the time, but the more repetitions of this pattern I experience, the more skeptical I am of fancy solutions (included many of my own published on this site) and add-ons that promise short-term feature additions but too often incur long-term maintenance costs that simply aren’t worth the benefits.  Here’s how I’m thinking about it now:

  • Don’t reverse engineer the system.  By all means, read the code, just don’t try to outsmart it.
  • If you can help it, don’t use methods or attributes marked as private (yes, I have a beef with Django’s Model._meta).
  • Don’t use a third-party product for something you can do relatively easily without it.  Stop and ask yourself if the coding you may save is worth the headache of maintaining the product through framework upgrades, etc.
  • Keep an eye on framework developments.  Watch the roadmap.  Read the release notes!  I’ve been pleasantly surprised a number of times to find out about new (or existing) features by re-reading documentation.
  • Participate in the core community.  Everybody benefits from improvements to the core framework.  This is not to say that every good product should ultimately aim for inclusion in the framework, but with a strong group of core developers new features are subjected to critical analysis, rigorous testing and benefit from wide usage and long term support.
Advertisements

,

Leave a comment

Django management command barfs on sites framework

As always, the story is a bit convoluted …

I had recently made changes to the get_absolute_url() methods on a couple of application models (Django 1.3/Python 2.4).  For various reasons, the admin UI for the app is on an intranet site, which is a different host from the public site (represented by a different Django Site object).  The changes involved calling a new function that uses the current site object to determine the appropriate domain for constructing a fully-qualified URL for get_absolute_url().  As originally implemented, I set a variable at the module level in models.py:

BASE_PUBLIC_URL = 'http://%s/path/to/app' % get_public_domain()

where get_public_domain() imports the Site model class from django.contrib.sites, calls Site.objects.get_current() and returns the appropriate public domain.  The get_absolute_url() methods then used BASE_PUBLIC_URL in constructing the final URL.

This worked fine in the normal application contexts, i.e., the admin site and the public site.  However, a custom management command which updates app data from an external source raised an ImportError on the relevant model.  The significant part of the traceback was as follows:

  File "/path/to/pyenv/lib/python2.4/site-packages/django/contrib/sites/models.py", line 25, in get_current
    current_site = self.get(pk=sid)

Ad hoc testing showed the ImportError to be a red herring — and, in any case, it didn’t square with the fact that the rest of the app (minus the admin command) was functional.

Now, the management command doesn’t actually call get_absolute_url(), so I figured that maybe the solution was to wrap the base public URL in a memoized function, so that the current site object is accessed lazily:

@memoize
def get_base_public_url():
    from path.to.mymodule import get_public_domain
    return 'http://%s/path/to/app' % get_public_domain()

That did the trick.  I’m still not sure exactly why the sites framework barfed, and it doesn’t seem worth digging for …

, ,

Leave a comment

Django 1.3 First Impressions

On the whole I am very pleased with Django 1.3.  The developers did a good job of taking care of some of the outstanding warts in Django, particularly the lack of “static file” management.  While I have not yet used the django.contrib.staticfiles app, it appears to solve the problem in a reasonable way.  Now I can retire the workarounds (such as this) that I had developed to deal with the problem.  The addition of built-in logging support is certainly welcome.  Improving and fixing inconsistencies in certain template tags all seems good.  I previously praised the render() shortcut, which will eliminate the repetitive nonsense of render_to_response(context_instance=RequestContext(request)).  The ORM got an important patch allowing configuration of on-delete behavior for ForeignKey (and OneToOneField) fields, about which I had also posted.  One interesting, small, but nice improvement came unannounced: help_text on a form field is now wrapped by a <span></span> when rendered as HTML by the as_*() methods.  I actually filed a ticket reporting this omission from the announcement.  For some reason, it seems that getting this change into the code and documentation has been a challenge.

I suspect there may be some moaning around the deprecation of function-based generic views in favor of class-based views.   Class-based views make sense, but it looks like there will be a little pain in the transition, partly because the keyword arguments for Django’s built-in generic view functions don’t map exactly to the generic view class keyword arguments.  It would have been nice to provide a little smoother transition there.  Also, the class-based view documentation is rather dense because you have to refer to the mixin classes that compose the actual generic view class you want to use.  I’m sure it will get easier with time, but it does feel like a jump in complexity that could make generic views more difficult for new users.    For example, where I did this before in a URLconf module:

from django.conf.urls.defaults import *
from django.views.generic.simple import direct_to_template

urlpatterns = patterns('',
    (r'^status/$', direct_to_template,
     {'template': 'sitetest/status.txt', 'mimetype': 'text/plain'}),
)

I now have to do something like this:

from django.conf.urls.defaults import *
from django.views.generic.base import TemplateView

class PlainTextTemplateView(TemplateView):
    """A plain text generic template view."""
    def render_to_response(self, context, **kwargs):
        return super(PlainTextTemplateView, self).render_to_response(
            context, content_type='text/plain', **kwargs
            )

urlpatterns = patterns('',
    (r'^status/$', PlainTextTemplateView.as_view(template_name='sitetest/status.txt')),
)

While class-based views may have been a step in the right direction for the framework, I wonder how it will play out.

1 Comment

Apache LDAP authentication and Active Directory

I needed to authenticate users in Apache against Active Directory using mod_authnz_ldap.  Normally I would have set the URL and base DN like this:

ldaps://example.com
ou=CompanyPeople,dc=example,dc=com

In this case, however, the users spanned two different top-level containers or “domains”:

ou=CompanyPeople,dc=example,dc=com
ou=OtherPeople,dc=example,dc=com

So, I tried setting the base DN to the top level:

dc=example,dc=com

but authentication failed with this ugly error in the log:

[ldap_search_ext_s() for user failed][Operations error]

It took some hunting, but I finally found that if you want to query the Active Directory “Global Catalog” (GC) via LDAP, you have to use port 3268 or 3269 (LDAPS) instead of the usual default port 389 or 636. So, the working URL and base DN are:

ldaps://example.com:3269
dc=example,dc=com

,

2 Comments

My favorite Python packages

Python generally lives up to its motto, “Batteries included.”  Here I want to give credit to folks who have provided some of my extra “batteries” — freely available Python tools that make my work easier and better.

Django — The de facto standard for Python web application development.  I’ve learned a lot from studying its code.  Includes a library of useful utilities (django.utils) that can used outside of web application contexts (e.g., check out django.utils.datastructures.SortedDict).

sphinx — Also a de facto standard in the Python universe.  It’s made me appreciate reStructured Text and improve my code documentation practices.  Ironically I find its own documentation rather hard to use.

virtualenv — How did we manage without it?

pip — Better package management than easy_install.

ipython — Worth it for the command history alone.

decorator — Almost essential for writing decorators, especially if you’re on Python < 2.5.

Fabric — A great addition to the developer’s or sysadmin’s toolkit.

lxml — For XML processing, I almost never use Python’s builtin XML libraries.

pycurl — Brings the power of libcurl into Python, filling gaps left by urllib/urllib2 and httplib (e.g., multiple asynchronous requests, multipart form data).

xlrd, xlwt — Good API for MS Excel processing.  Unfortunately, no support (yet) for Excel 2007 XML format.

simplejson — The standard JSON library for Python < 2.6.

py.test — Anything that makes writing and running unit tests easier is very good.

unittest2 — Makes available to Python 2.4-2.6 the significant enhancements made to the standard unittest module in Python 2.7.

I also want to thank Christof Gohlke for his “Unofficial Windows Binaries” site, since up-to-date versions of lxml and pycurl would be difficult to use on Windows without his builds.

And finally, there are essential libraries that I depend on without normally using directly: MySQL-Python, pysqlite (stuck on CentOS 5/Python 2.4), python-ldap, docutils, setuptools.

Leave a comment

Back to Windows

I’m a little wistful about it … but I’ve returned to Windows for my primary office working environment, after a lengthy experiment with Ubuntu. I really like Ubuntu, but the challenges of using it in a Windows-centric environment (Active Directory, Exchange, etc.) just made the extra effort to adapt hard to justify.  CrossOver with Office 2007 is OK, but just annoying enough; same with Evolution and Exchange.  Dealing with Active directory shares when your OS is “not supported” is frustrating.  And Windows generally has better GUI tools, although it’s hard to say whether those benefits outweigh the vast superiority of the Linux command line.

In any case, now I have to deal again with the problems that made me want to switch to Linux in the first place — principally that it’s a much better environment for open source software development.   I may give Cygwin another try, or run Ubuntu in a VM, we’ll see.  I’m pretty attached to Emacs, and have used it in Windows, so that’s good.

It’s rainy and cold outside today …

Before wrapping this up I’m going to post how I replaced GRUB with the Windows 7 boot loader, since I found a number of variations on how to do this.  I booted to a Win7 DVD, took the repair option, deselected Window 7 from the Windows OS auto-find list (one article I read stressed this deselection; I’m not sure it really matters), clicked Next, then selected the Command Prompt.  The two commands that worked to restore the Win7 boot loader were, in order:

bootrec.exe /fixboot
bootrec.exe /fixmbr

Exit the command prompt and restart from the HD.

,

3 Comments

Be explicit when setting Apache host access controls

I recently discovered that I had made an incorrect assumption regarding the use of the host-based authorization directives in Apache: I thought that if, for example, a directory defined Order, Deny, and Allow directives, that the use of an Allow directive in a subdirectory was simply “additive”, i.e., extending the existing rules as if the rules from the parent directory were “inherited” and the extra Allow was added to the list of Allows from the parent.

This is most definitely NOT the case, at least in Apache 2.2, and the documentation does not address this specific issue.  Worse, I think it is reasonable to believe based on the mod_authz_host docs (the module which provides the OrderDeny, and Allow directives) and the “How Sections Are Merged” section of the Apache Configuration Sections doc that in fact the configuration would behave in the way I have expected.  (Of course, one should always test.)

Here’s the problem:

<Directory /abc>
    Order Deny,Allow
    Deny from all
    Allow from example1.com
</Directory>

<Directory /abc/def>
    Allow from example2.com
</Directory>

You might think that access to /abc/def is restricted to hosts from example1.com and example2.com domains, or perhaps just example2.com, but in fact, it’s open to the world!  In other words, the /abc/def block is not equivalent to:

<Directory /abc/def>
    Order Deny,Allow
    Deny from all
    Allow from example1.com
    Allow from example2.com
</Directory>

as I thought it would be, or even to:

<Directory /abc/def>
    Order Deny,Allow
    Deny from all
    Allow from example2.com
</Directory>

The result is the same even if the Order directive in /abc is set to Allow,Deny. It seems as though mod_authz_host resets all its directives whenever one is set in a “directory” context.  The reset state is to allow all by default because neither Allow nor Deny have default values, and the default value of Order is Deny,Allow.

,

Leave a comment