So, I’ve been slaving away at a web application that connects to its data store via RESTful web services. Because I have to use HTTP PUT and DELETE methods in addition to the usual GET and POST methods, I chose to use Python’s standard library module httplib (urllib and urllib2 only support GET and POST). Everything’s going along great until I get around to a couple of methods I’d been putting off that involve file uploading.
Now, I’m reasonably knowledgeable about HTTP, not an expert by any means, and I’m generally happy to stay out of the gory details as much as possible by using standard library modules. So I was a bit surprised–and dismayed–to discover that if I wanted to submit a multipart form (truly multipart, having both string params and a file) I had to construct the request body by hand.
Of course, constructing a multipart form isn’t terribly difficult, but that’s not the point. Constructing HTTP request headers isn’t difficult, but how many people do it by hand? A file upload form isn’t exactly an edge case, so it’s curious to me that it would not be handled by httplib at this late date (there is a feature request for Python 2.7).
In any case, I’ve decided to use PycURL for the multipart forms. I would be perfectly happy with that if only I didn’t have to translate C API docs into Python (the PycURL docs give you just enough hints that you can get by with some trial and error).
Once upon a time there was a village. And the people of the village all worked very hard creating “content”. And the content was good and useful.
In the course of commerce the people saw that other villages also produced content which was good and useful. So one day elders from all the surrounding villages held a meeting to see about how they could work together and “share” their content more freely and easily. After talking briefly the elders agreed that this would be a good thing, and they gave it a name: “content management”. And the elders returned to their villages to tell the people how wonderful it was going to be that all the content which they created in their separate villages would be available to everyone in all the villages. And the people rejoiced.
When the village elders reconvened to start working toward achieving this dream they immediately saw that it wasn’t going to be so easy. It seemed that folks had many different ideas about how the content sharing should work. Some people said, “I just want to share content with the people in my neighborhood. That’s all I care about.” Others said, “I want to be able to pick and choose what gets shared with other villages.” And on and on. The elders tried their best to come up with a way that all the needs and desires of their constituents could be met, but alas they did not succeed. Eventually, they realized that content sharing could happen in many different ways without having “one system” that governed all. The villagers indeed over time worked out many ways to share their content, which, while far from being the promised land of “content management”, served their needs pretty well. And they were reasonably happy.
Now there was another village in another land. And the people of this village worked very hard creating and organizing “resources”. And these resources were highly sought after by people from all over the land, well beyond the boundaries of the village itself. And the people of the village stored their resources in giant silos in different places throughout the village. And each silo was accessible by different means: one by train, another by boat, another by cart and horse. And they thought to themselves, “Wouldn’t it be wonderful if the resources from all these silos were accessible by the same means, whether by train, or boat, or horse-cart? Then people could come to one place for all the resources they desire.” And so the village elders met to discuss this, and agreeing it was a good thing, they gave it a name: “resource discovery”. And the elders told the people how wonderful it was going to be …
I’m posting this here because the answer wasn’t easy to find — in fact, I got it by guessing.
The problem is: How to get the name of a class as a string — not the class name of an instance, but the name of class object itself. The answer turns out to be simply the “magic” attribute __name__. Unfortunately, you can’t see this attribute with dir(), and the Python tutorial section on classes (as of 2.6.4) makes no mention of it. To be sure, this is not a common use case, but I happened to have one in hand.
>>> from mymodule import User
>>> u = User
>>> u.__class__.__name__
'type'
>>> u.__name__
'User'
>>> user = u('id')
>>> user.__class__.__name__
'User'
As I started using xlwt, I found myself wanting some more convenient methods for dumping tabular data into a worksheet, especially when all the data can be treated as strings. Tabular data in this context is an iterable of iterables, such as a list of tuples.
Here’s what I’ve got so far:
"""Excel utilities.
'Tabular data' in this context is an iterable of iterables.
"""
import xlwt
def to_workbook(tabular_data, workbook=None, sheetname=None):
"""
Returns the Excel workbook (creating a new workbook
if necessary) with the tabular data written to a worksheet
with the name passed in the 'sheetname' parameter (or a
default value if sheetname is None or empty).
"""
wb = workbook or xlwt.Workbook()
ws = wb.add_sheet(sheetname or 'Data')
to_worksheet(tabular_data, ws)
return wb
def to_worksheet(tabular_data, worksheet):
"""
Writes the tabular data to the worksheet (returns None).
Thanks to John Machin for the tip on using enumerate().
"""
for row_index, row_data in enumerate(tabular_data):
worksheet_row = worksheet.row(row_index)
for col_index, col_data in enumerate(row_data):
worksheet_row.write(col_index, col_data)
In a Django context, then, you have a very straightforward way turning a query into an Excel file using the values_list() QuerySet method, e.g.:
wb = to_workbook(MyModel.objects.values_list())
Since values_list() outputs the attribute values for each object in the same order in which they’re defined in the model class, you can insert a row of headings to your table:
table = MyModel.objects.values_list()
headings = [f.name for f in MyModel._meta.fields]
table.insert(0, headings)
wb = to_workbook(table)
I found myself creating a number of simple custom context processors which simply return custom settings that I have added to my Django settings module (I actually keep these custom settings in my_settings.py and import then into settings.py, just to keep them separate). I decided it was a good idea to add exception handling to these functions so that I would get a useful error message if I tried to use a particular context processor without implementing its required setting(s). So, after some refactoring, I came up with this:
from django.conf import settings
from django.core.exceptions import ImproperlyConfigured
class SettingsContextProcessor(object):
"""
Class for creating simple context processors that
return one or more Django settings.
"""
def __init__(self, *setting_names):
self.setting_names = setting_names
def __call__(self, request):
extra_context = {}
for sn in self.setting_names:
try:
extra_context[sn] = getattr(settings, sn)
except AttributeError, e:
raise ImproperlyConfigured('Missing required setting: %s' % sn)
return extra_context
Now I can create custom settings-based context processors like this:
google_analytics = SettingsContextProcessor('GOOGLE_ANALYTICS_PROFILE_ID')
jquery = SettingsContextProcessor('JQUERY_VERSION')
jqueryui = SettingsContextProcessor('JQUERYUI_VERSION')
static_media = SettingsContextProcessor('STATIC_MEDIA_URL')
yui = SettingsContextProcessor('YUI_VERSION')
OK, this doesn’t really disable it, but it does make the “browser uploader” the default …
<Directory /path/to/wp/wp-admin>
<Files media-new.php>
# Force browser uploader instead of Flash
RewriteCond %{QUERY_STRING} !=flash=0
RewriteRule ^/(.*) /$1?flash=0
</Files>
</Directory>
(WordPress 2.8.4)
It should clear from this blog that I’m a big fan of Django. I use it for as much of my work as possible. Recently a couple of other developers in my shop have entered the Django arena, which, as a general proposition, is a good thing — in that we’re using the framework more widely. But as a result, one of Django’s few weaknesses has been made more painfully obvious, namely, the fragility of Django projects.
The problem is that one import error in any installed app’s models, any referenced URLconf, view module, or other module imported by one of those breaks the entire project immediately and horribly. This makes the installation of new apps and updates of installed apps inherently risky to the entire site, which IMO is a *bad thing*. Now, I haven’t delved into the guts of Django’s initialization process to see what, if anything, could be done about this, but on a conceptual level it seems that the project as a whole should have some way to recover from a bad app or module, unless it’s related to the core functionality of the project (like a middleware or context processor module).
Is that unreasonable?
Or, developing Django data applications without a database.
In recent months I have been working intensively on a user interface to a data store that lives behind a web services API. While this might not be considered a natural fit for Django, the smart de-coupling of URLs, views, and templates from data models means that the former retain their value even without the latter. And hey, it’s all just Python, right? Django models are just one way to handle data in your application, albeit a very convenient and powerful one when dealing with an RDBMS.
Before working on this project, I had already developed two other Django apps based on data sources accessed via HTTP, both of which were read-only, which of course made things quite a bit simpler. The first of these was an XML-RPC interface (built with django_xmlrpc, Python’s standard xmlrpclib module and python-ldap) to an LDAP directory. I manage user and group information for a number of staff applications, including Django itself, for which it is very useful to draw upon a central source of personnel data. The Django-based service provides convenient methods for common operations while hiding the complexities of LDAP connections and search queries.
The second app dealt with requests to web services of a library catalog which return content in a custom XML format. Since only HTTP GET requests were required, I used Python’s standard urllib and urllib2 modules for the request/response handling, and lxml.etree for the XML parsing and XSLT application (see also my previous post on the virtues of lxml).
The current project, as opposed to the previous two, involves both read and write actions on the backend data store. Also, because the web services API implements a REST architecture, the “client” code I was tasked with has to support a wider range of HTTP request methods (not just GET and POST) and responses. Finally, the client code stack includes a full-blown user interface (the ultimate purpose of the app) for managing the backend data. In my next post, I’ll talk about how I broke down the problem.
So, I wanted one of those horizontal multi-selects with “available” and “chosen” boxes to associate users with groups in the Django auth admin UI. Turns out it all I had to do was import the UserAdmin class from django.contrib.auth.admin and override the filter_horizontal attribute:
from django.contrib.auth.admin import UserAdmin
UserAdmin.filter_horizontal = (‘user_permissions’, ‘groups’)
from django.contrib.auth.admin import UserAdmin
UserAdmin.filter_horizontal = ('user_permissions', 'groups')
Since importing the UserAdmin class runs the admin module from django.contrib.auth, the UserAdmin and GroupAdmin classes are registered for the admin site. All I have to do then is import my custom admin module in my URLconf instead of the one from django.contrib.auth to make sure my customizations are applied in my admin site.
On balance I’m satisfied with Vista x64, but I’ve run into a few challenges and at least one casualty.
The casualty was my Palm Tungsten E2, which can’t connect to Vista64 with the USB cable. According to Palm, you can connect with Bluetooth, but I don’t have Bluetooth on the computer. Fortunately I don’t use the Palm much anymore and I synced mainly to back it up.
As for the challenges, it seems that some applications, notably OpenOffice.org 3, have to be installed using XP-SP2 compatibility mode. Also, some installation software doesn’t trigger privilege elevation which may be required to write to certain registry keys or system folders, so the “Run as Administrator” and “CMD prompt here as administrator” Elevation PowerToys are practically necessary. Sun’s Java RE (which I probably wouldn’t install except that it’s a dependency of OOo) has an annoying bug in its control panel such that you can’t change the settings, so you can’t disable automatic updating. The workaround is to run bin/javacpl.exe as administrator using the Elevation PowerToy. Using 32-bit Windows help files (.hlp) requires a special download (this affects all Vista versions) as Microsoft is no longer updating the older help program and so doesn’t distribute it with the OS. Only two of the four buttons on my Kensington Expert Trackball Mouse work, and Vista thinks it’s a regular mouse — but it wasn’t working fully under XP-SP3 either. Don’t bother to install the MouseWorks software, although some folks claim they’ve gotten it work (or at least the OS to use the driver). I would guess at this point Kensington isn’t going to release a Vista-compatible driver for the older devices.
I can live with two buttons.
