Following on from yesterday's post, I decided to try implement proper content negotiation. After a fair amount of time spent getting to grips with [Lua][], I got a [script][] which works very nicely. It implements [server driven][] content negotiation for [media types][mime].

The basic idea of content negotiation is that a resource (e.g., this [graph][]) exists in multiple formats (in this case, [SVG][graph-svg], [PNG][graph-png] and [GIF][graph-gif]). When a user agent requests the resource, it indicates which formats it understands by listing them in the Accept header. The server compares these to the available formats and sends the best one. So a browser which can display [SVG][] will receive the diagram in SVG format, while a browser which can't will receive it in [PNG][] (or [GIF][]) format.

(The following description assumes knowledge of the Accept header format.)

The script works by searching the directory for files with the requested name but with an additional extension (each of which is a variant). The [media type][mime] is inferred from the extension using /etc/mime.types, and the quality of the type is set by a hardcoded table in the script. Each variant is checked against the acceptable types sent by the user agent, and the overall quality calculated by multiplying the quality with the q parameter in the Accept header. The variant with the highest overall quality is then chosen.

Some browsers include wildcard entries such as image/* and */* in the Accept header without specifying a q parameter. This parameter defaults to 1 (the highest value), which means that no preference is actually indicated. The script implements the same [hack][] that [Apache][] does in order to compensate for this. It also handles directory index files by defaulting to files named "index".

To install the [script][], download and save it somewhere (such as /etc/lighttpd/). Then add the following to the site definition.

magnet.attract-physical-path-to = ("/etc/lighttpd/negotiate.lua")

URLs shouldn't really contain file extensions (like .html, .png) since they are supposed to identify a resource and not a particular representation/format thereof. The format is indicated by the Content-Type header sent in the response. Modern CMSs do this already (for example, the URL of this page doesn't include .html).

Doing the same for static files (i.e. files served directly by the webserver) isn't straightforward because most webservers use the file extension to determine the MIME type to send in the Content-Type header. This means that simply removing the file extension from the filename (or even creating a symlink without a file extension) will cause the webserver to send the wrong Content-Type header.

I decided to try find a solution to this for my webserver of choice, Lighttpd. Lighttpd has a module which embeds a [Lua][] interpreter and allows you to write scripts which modify (or even handle) requests. So I wrote a [script][] which searches the directory for files with the same name as requested but with an extension. This means that any file can be accessed with the file extension removed from the URL while still having the correct Content-Type.

The script currently chooses the first matching file, which means that having multiple files with the same name but different extensions doesn't do anything useful. The proper method however is to actually do [content negotiation][], which chooses the format based on the preferences indicated by the HTTP client in the Accept header.

To use this script, download it and save it somewhere (I use /etc/lighttpd/). Enable mod_magnet, and add the following line to the site definition.

magnet.attract-physical-path-to = ("/etc/lighttpd/extension.lua")

Seeing that I've spent countless hours setting up my Drupal installation, I thought that I would share this with others and document it for future reference. Drupal is an extremely powerful CMS which can be used to create a wide variety of sites. The disadvantage of this is that it requires a fair amount of work to setup a straightforward blog, which involves installing and configuring numerous modules.

Installation

Since there is no Ubuntu package for Drupal 6, I created my own package based on the drupal5 one. I set it up as a virtual host in Lighttpd by simply symlinking the domain name to /usr/share/drupal6. I created a MySQL database for the site and went through the Drupal install process. Since I'm using a multi-site installation, I also needed to alias the /files directory for each site.

$HTTP["host"] == "michael.gorven.za.net" {
    alias.url = ( "/files/" => "/home/mgorven/public_html/" )
}

Clean URLs

Clean URLs allows one to have URLs like /about instead of /index.php?q=about. This however requires that the web server rewrites URLs from the former to the latter. Drupal includes an htaccess file containing settings for Apache, but not for Lighttpd. Lighttpd does have a rewrite module, but it doesn't support the conditions that Drupal needs (such as checking if a file exists).

Lighttpd does however have a module which allows one to add scripts to the request process written in [Lua][]. A script has already been [developed][drupal.lua-devel] which implements the required rewriting for Drupal. The following lines in lighttpd.conf enable this for a specified site (after enabling [mod_magnet][] and downloading the [script][drupal.lua]).

$HTTP["host"] == "michael.gorven.za.net" {
    index-file.names = ( "index.php" )
    magnet.attract-physical-path-to = ( "/etc/lighttpd/drupal.lua" )
}

Blog

When Drupal is first installed, there is no mention of blogging as such. The first step is to enable the Blog core module¹. This creates a blog content type and enables a blog for each user. (The module is designed for a multi-user blog, but can be used for a single user as well.) However, this doesn't give you all the functionality you expect from a blog engine.

Tagging is handled by the Taxonomy core module. You first need to create a vocabulary though, and enable it for blog posts. (This took me ages to discover.) In order to get nice URLs (including the date and post title, for example) you need to install the [Pathauto][] module and configure a pattern for blog posts. You may also want to define a pattern for tags.

There is also no archive functionality. The best way that I can find is the [Views][] module. It includes a pre-defined "archive" view which can display posts from a specific month, and links to the monthly pages. Even after much investigation I couldn't get the archive to behave like typical blog archives (i.e. /blog/2008/07/07, /blog/2008/07 and /blog/2008 for daily, monthly and yearly archives respectively).

Other Blog Features

The [Trackback][] and [Pingback][] modules implement automatic linking with other blogs. (I haven't actually tested these yet.) The Blog API core module allows the blog to be managed with external clients. The [Markdown][markdown-mod] module allows you to write posts using [Markdown][] syntax instead of HTML.

Comments

Drupal enables comments for blog posts by default. The [Akismet][akismet-mod] module implements spam filtering using the [Akismet][] service. The [CAPTCHA][captcha-mod] and [reCAPTCHA][recaptcha-mod] modules allows you to require users to answer a [reCAPTCHA][] when submitting comments. (I haven't actually enabled [CAPTCHAs][captcha] since I haven't gotten any comment spam yet. Or real comments for that matter...)

Posting by email

The [Mailhandler][] module allows you to submit posts via email. The configuration is fairly straightforward, except for the available commands which can be found [here][commands]. These can be specified at the beginning of emails and in the configuration as defaults. I use the following commands.

type: blog
taxonomy: [mail]
promote: 1
status: 1
comment: 2

This creates blog posts and tags them with the "mail" tag. Posts are published and promoted to the front page, and comments are enabled.

The one thing it doesn't handle is attachments (such as images). There are a couple of modules² which support this, but they aren't available for Drupal 6 yet. ([Vhata][] has also hacked together a [photo blogging][phoblog] system, but this isn't implemented as a Drupal module.) I don't really need this feature, so I'm going to wait until these modules are updated.

OpenID

The [OpenID][openid-mod] module allows you to log into your site using [OpenID][]. The [OpenID URL][openidurl] module allows you to delegate your Drupal site as an OpenID by specifying your provider and/or your [Yadis][] document.

Yadis Advertisement

Yadis documents are advertised with a meta header in the HTML document, but this isn't the ideal method of doing so since the relaying party needs to download the entire HTML file. The [preferred methods][intertwingly] are to insert an X-XRDS-Location in the HTTP headers, or to automatically serve the Yadis document if the user agent specifies application/xrds+xml in the Accept header.

The former method can be accomplished with the setenv module for Lighttpd. The second is essentially a conditional rewrite, and so requires some Lua scripting again. The following script will do the job.

if lighty.request["Accept"] == "application/xrds+xml" then
    lighty.env["uri.path"] = "/files/yadis.xrdf"
end

The following lines in lighttpd.conf will announce the Yadis document for the root URL.

$HTTP["url"] == "/" {
    magnet.attract-raw-url-to = ( "/etc/lighttpd/yadis.lua" )
    setenv.add-response-header = ( "X-XRDS-Location" => "http://michael.gorven.za.net/files/yadis.xrdf" )
}

Random Stuff

The tag block is generated by the [Tagadelic][] module. The "Recent Tracks" block is generated from my [LastFM][] feed by the Aggregator core module, and the list of networks is simply a custom block. The [Atom][] feed is generated by the [Atom][atom-mod] module. The contact form, search and file upload are all core modules.

Missing Stuff

The one thing I haven't sorted out is image handling. There are a couple ways to [handle images][drupal-images] in Drupal, but none of these appeal to me (they're too complicated). I will probably just upload images as attachments and insert them manually in the body.