Photo blogging via email with Drupal

The one thing missing from my posting by email setup was support for images. The Mailsave module has finally been updated for Drupal 6, and so I can now submit attachments with email posts. The one shortcoming is that files are simply added to posts as normal attachments, and so images aren't automatically displayed. I therefore have to manually insert images in the body of the post, but I actually prefer this since it's a simpler system and gives me more control.

I also needed a way of resizing images on my phone since they are too big. I found the Nokia Image Editor1 which seems to work fairly well, although it only allows resizing to specific resolutions.

  1. It does work on my phone even though it's supposedly only for the Nokia 3250. 

Cisco un-Clean Access

The [CHPC][] installed a new network this past weekend as part of the [SANReN][] project. The new network consists of [Cisco][] equipment, including their [NAC][] (or "Clean Access") system. This requires all clients to authenticate before they are allowed access to the network, and can also enforce a configured security policy (such as requiring operating system updates and anti-virus).

The system works as follows. By default, the ports on the switch are in an "unauthenticated" [VLAN][]. When a client is connected, it is provided with an IP address (via [DHCP][]) in an "unauthenticated" subnet. The system then presents a captive portal which requires the user to authenticate with a username and password using their browser. If the authentication is successful, the port is moved to a different VLAN (depending on the user's access level), and the switch briefly disconnects the link which causes the client to negotiate a new IP address (in a different subnet).

Before the portal presents the login page it requires that a [Java applet][] be run on the client. The applet gathers various bits of information about the client (including the operating system) and submits this information to the portal. (I assume that the portal uses this information to determine what policies must be enforced. In our setup, Windows machines must have the Clean Access Client installed, while Linux and Mac OS X machines are simply allowed access.) The portal then presents the login page.

Being a geek, I wasn't very happy to go through this rigmarole everytime I connected to the network. (I also couldn't use my [normal browser][konq] since the applet didn't work in it.) So I set out to automate the process. Initially I tried to script everything (including the Java applet) but then I noticed that the output of the applet wasn't sent with the login form submission. The only other information the form contained was a session key and random string, both of which were present on the [HTML][] page which contained the applet. A manual test confirmed that the login page could be submitted successfully as long as the session key and random string were correct — the applet could be bypassed.

I quickly scripted the login process using a

[] script and [wget][]. I then installed it in <code>/etc/network/if-up.d</code> after adding some logic to only execute if the current IP address was on the unauthenticated network. The result is that I can plug in the cable, and my machine automatically authenticates to the system.
While searching for information about the Clean Access system, I came across this [Slashdot article][] about a guy who was suspended from university for bypassing the Clean Access checks. I only realised last night that this is exactly what my script does![^1] I haven't tested it on Windows yet, but the only possible change I can think of is to change the [user agent][]. Seriously Cisco, the fact that I managed to bypass the applet simply by submitting the login form programmatically is ridiculous.
I have attached my script to this post. The way in which I have parsed the HTML page is rather ugly and likely to only work on this specific version of Clean Access. I plan to rewrite it in [Python][] sometime.
<strong>Update:</strong> I have rewritten the script in Python, which should be a bit more solid since it parses the HTML using a [DOM][]. The script requires [libxml2dom][] and [ipy][]. After configuring the parameters it can be dropped in <code>/etc/network/if-up.d</code>[^2] where it should run automatically.
[^1]: Note that it doesn't bypass the authentication: you still need a valid account in order to gain access.
[^2]: Make sure not to use a dot in the filename though.
<em>[CHPC]: Centre for High Performance Computing
</em>[SANReN]: South African National Research Network
<em>[NAC]: Network Admission Control
</em>[VLAN]: Virtual LAN
<em>[DHCP]: Dynamic Host Configuration Protocol
</em>[HTML]: HyperText Markup Language
*[DOM]: Document Object Model
[chpc]: http://www.chpc.ac.za/
[sanren]: http://www.meraka.org.za/sanren.htm
[cisco]: http://en.wikipedia.org/wiki/Cisco_Systems
[vlan]: http://en.wikipedia.org/wiki/Virtual_LAN
[dhcp]: http://en.wikipedia.org/wiki/Dynamic_Host_Configuration_Protocol
[nac]: http://en.wikipedia.org/wiki/Cisco_NAC_Appliance
[java applet]: http://en.wikipedia.org/wiki/Java_applet
[konq]: http://en.wikipedia.org/wiki/Konqueror
[Slashdot article]: http://it.slashdot.org/article.pl?sid=07/04/27/203232
[user agent]: http://en.wikipedia.org/wiki/User_agent
[python]: http://en.wikipedia.org/wiki/Python_(programming_language)
[html]: http://en.wikipedia.org/wiki/HTML
[wget]: http://en.wikipedia.org/wiki/Wget
[bash]: http://en.wikipedia.org/wiki/Bash
[dom]: http://en.wikipedia.org/wiki/Document_Object_Model
[libxml2dom]: http://www.boddie.org.uk/python/libxml2dom.html
[ipy]: http://russell.rucus.net/2008/ipy/

July GeekDinner

I attended my very first GeekDinner last night, for which I managed to get a place at the very last minute (although not everyone made it and I could have gatecrashed anyway ;-) ). It was hosted by Da Capo in Greenmarket Square, which was an awesome venue. (I had the Tomato and Basil Soup for starters, and the Polo Piccata, both of which were delicious.)

Andy spoke about his experiences in Nigeria, which was quite an eye opener. We often complain about Eskom and Telkom here in South Africa, but we actually aren't that bad off. Donna gave a talk about her various activities aimed at encouraging and educating women about IT, and Kerry-Anne spoke about running a daily photo blog.

Mandy then presented a slideshow on personal finance (prepared by Tim) which she had never seen before. The result was hilarious due to the dubious advice in the slideshow, and more humour was provided by the questions afterwards (involving emails from Nigerian princes and the best way to launder money).

All in all it was a fantastic evening which I enjoyed thoroughly. Thanks to Perdeberg for sponsoring the wine (even though I don't drink, it did make things more interesting ;-) ), and thanks to everyone involved with organising the evening. I look forward to the next one.

Slipstreaming Windows XP SP3 in Linux

Unfortunately Windows is still a necessary evil sometimes: I keep a Windows virtual machine for times when it's absolutely necessary, and I still give my friends Windows tech support. I still like to do things properly, and so I wanted to create a Windows XP install CD with Service Pack 3 slipstreamed in1. I had two CDs to do, and slipstreamed the first one using a Windows VM, but then got curious and wondered if I could do it without Windows.

The answer is that it is possible using Wine to run the service pack installer. I followed this blog post (which was interesting since it's in French), but I then found another blog post which explains it in English. The steps are as follows:

  1. Copy contents of original CD to harddrive.
  2. Extract the service pack using cabextract.
  3. Use Wine to run the service pack installer.

    wine ~/sp3/i386/update/update.exe /integrate:~/xp/
  4. Use geteltorito to extract the bootloader from the original CD

  5. Make sure that all the filenames are upper case.

    convmv -r --upper --notest ~/xp/*
  6. Create the new CD image. I did this in K3b with the following settings.

    • Boot emulation: none
    • Boot load segment: 0x7c0
    • Boot load size: 0x4
    • Generate Joilet extensions
    • Omit version numbers in ISO9660 filenames (nothing else enabled under "ISO9660 Settings"
    • ISO Level 1
  7. Test in a virtual machine

It seems to be quite particular about the ISO9660 settings and the upper case filenames, so if it doesn't boot check the settings.

  1. This integrates the service pack into the install CD so that a fresh installation is already updated. 

Mobile interface to Vodacom4me and MyMTN

Vodacom4me and MyMTN allow you to send free SMSs from a computer. Unfortunately those sites are not accessible from a cellphone. I came across a site which provides a mobile interface for Vodacom4me and MyMTN1. This means that you can send SMSs from your cellphone for the cost of the GPRS/UMTS data required to access the site. I having been using this for quite a while, and it works fairly well.

However, there are a few aspects of the site which I don't like, and so I wrote my own version which performs the same function with the following extra features:

  1. Uses cookies to store login data instead of a URL with parameters which needs to be bookmarked (although it will fall back to this method if the user agent doesn't support cookies).
  2. Submits forms using POST instead of GET (but will fall back to GET if the user agent doesn't support POST).
  3. Allows multiple recipients (although only Vodacom4me supports this).
  4. Specifies the maximum message length in the textarea so that phones which support it can show how many characters are left.2
  5. Automatically logs into Vodacom4me/MyMTN again if session has expired.3
  6. Cleaner, less cluttered interface (mainly optimised for my phone ;-) ).
  7. Accessible over HTTPS for extra security.

The site is available at http://m.mene.za.net/ (or with HTTPS). Obviously the restrictions enforced by Vodacom4me and MyMTN still apply. Vodacom4me allows 20 SMSs per day to Vodacom numbers for Vodacom subscribers only. MyMTN allows 5 SMSs per day to MTN numbers for anyone. The source code is available for anyone who is interested (and brave enough).

  1. There is also an interface for CellC's site, but mine does not implement this. 

  2. This is technically not allowed by the HTML specification, but it works on my phone. 

  3. This allows the message composition page to be saved on phones which support this (like my Nokia E70) instead of reloading it every time a message is composed. 

Publishing SSH and GPG keys using DNS

I was looking through a list of DNS record types, and noticed the SSHFP and CERT records. I then proceded to implement these in my domain... just because I can ;-)

SSH Host Keys

The SSHFP record is used to publish the fingerprint of a host's SSH key. When an SSH client connects to a server for the first time, it can verify the host's key by checking for this DNS record. The format of the record is specified in RFC 4255, but there is also a tool which will generate the records for you.

$ sshfp -s mammon.gorven.za.net
mammon.gorven.za.net IN SSHFP 1 1 5e6772b6962f3328a0d73f7765097b7622f21447
mammon.gorven.za.net IN SSHFP 2 1 00e59b1843421f13d75e21abb06bf032a6e60b8b

The SSH client needs to be configured to check these records. Specifying "VerifyHostKeyDNS ask" in ~/.ssh/config will make the client look for SSHFP records, but will still prompt you to accept the key. (It will output a messaging saying that it found a matching key.) Specifying "VerifyHostKeyDNS yes" will skip the prompt if the record exists and matches the key presented by the server.

GPG Keys

The CERT record is used to publish public keys or fingerprints. It can be used for PGP, X.509 or SPKI keys. It is specified in RFC 4398, and there is very little mention of it other than this blog post I found. To generate records you need the make-dns-cert tool which is part of GnuPG. It isn't distributed in the Ubuntu package however, and so I had to compile GnuPG from source.

To determine the name of the record to use, convert your email address into a domain name by replacing the ampersand with a dot1. To publish your entire public key, run the tool as follows.

$ make-dns-cert -k ~/pubkey -n michael

The first parameter specifies the file containing your public key in binary format, and the second parameter specifies the domain name to use. To publish a reference to your public key, run the tool as follows.

$ make-dns-cert -f BF6FD06EA9DAABB6649F60743BD496BD6612FE85 -u http://michael.gorven.za.net/files/mgorven.asc -n michael

The first parameter specifies the fingerprint of your key, and the second parameter the URL at which the public key can be found. It is also possible to only publish the fingerprint or only the URL. Simply add the record which the tool outputs to your zone file2.

There is also another method to publish GPG keys called PKA. The only documentation I can find is a specification in German linked from the blog post mentioned above. I still managed to set it up though. This method uses a TXT record (similar to SPF). Here is my record.

michael._pka.gorven.za.net. TXT

This specifies the fingerprint and URL, just as with the second CERT method above. In order to get gpg to check DNS for keys, you need to specify "--auto-key-locate cert,pka" on the command line or in the configuration file.

  1. So john@example.com becomes john.example.com

  2. It should be possible to clean the record up by using mnemonics, but I couldn't get nsd to accept it and so just left it as is. 

OpenVPN through an HTTP proxy server

I discovered that OpenVPN supports connections through an HTTP proxy server. This makes it possible to establish a VPN from a completely firewalled network where the only external access is through a proxy server1. It takes advantage of the fact that SSL connections are simply tunnelled through the server and aren't interfered with like unencrypted connections.

The server setup is almost identical to a normal configuration, except that the tunnel must use TCP instead of UDP (since the proxy server will establish a TCP connection). Since most proxy servers only allow SSL connections to certain ports, you will also need to change the port number that the server listens on. The best is 443 since that is used for HTTPS, but if the server is also running a web server on port 443, then 563 is probably the next best choice. This port is assigned to NNTPS, and is allowed by the default Squid configuration. The following two lines enable TCP connections and change the port number.

proto tcp-server
port 563

The client configuration is also very similar. It simply needs to enable TCP connections, set the correct port number, and specify the proxy server.

remote vpn.example.com 563
http-proxy cache.saix.net 8080
proto tcp-client

OpenVPN can also authenticate to the proxy server using either Basic or NTLM authentication. To enable this add "stdin basic" or "stdin ntlm" to the http-proxy line. This will prompt for the username and password when the VPN is started. For more details see the OpenVPN documentation.

  1. I am not commenting on the ethics of this. If you need to resort to this method, you probably shouldn't be doing it. 

Python Decorators

For my Masters project I need a method by which the user can specify which functions should be run on an SPE1. This method should be simple, clear and easy to turn on and off. I stumbled upon a blog post a little while ago (I think it was this one) which explained decorators in Python, which is the perfect tool for the job. Decorators are used to transform functions, but without changing the function itself or the calls to it.

def spe(func, *args):
    def run(*args):
        return compile(func, *args)
    return run

def sub(a, b):
    return a - b

print sub(2, 4)

The spe function is the actual decorator. The @spe line applies the decorator to the sub function. Implicitly, the following declaration is made:

sub = spe(sub)

The sub function is being wrapped by the spe function, and so all calls to sub (such as the print line) will use the wrapped function instead. The decorator creates and returns a new function called run which will (eventually) cause the original function to be compiled and executed on an SPE. This means that running a function on an SPE will be as simple as adding @spe before the function declaration2, without having to change the way in which the function is called. Turning it off is as simple as commenting out this line, and it is fairly clear as to what is happening.

  1. Trying to make this decision automatically would be a massive project in itself and would probably be worse than a human decision. 

  2. There will probably be some restrictions on what the function may contain, but that's a different matter. 

Masters project overview

Since I might be posting entries regarding my Masters project, I thought that I would provide a brief overview of the project to put it in perspective. I am doing my MSc in Electrical Engineering at UCT as part of the ACE group headed by Prof. Inggs. The group is based at the CHPC, which is part of the Meraka Institute, which in turn is part of the CSIR. The group's research is focused on developing new platforms and methods for HPC.

My project is to investigate the suitability of the Cell processor for HPC. The Cell processor is found in the PlayStation 3 and in BladeCenters, and is a very powerful processor. It achieves this by using two different types of cores. The one type (PPU) is a general purpose core capable of running an operating system, while the other type (SPU) is designed specifically to crunch numbers.

The disadvantage of this architecture is that it is very difficult to program for. When using the IBM Cell SDK, the user needs to write separate programs for each type of core, and needs to manage the SPEs manually as well as take care of all memory transfers. This requires a good knowledge of the architecture, and results in a lengthy development process and unportable code.

For the Cell processor to be a successfull platform in HPC the development process must be made easier while still making efficient use of the Cell's capabilities. There are a number of commercial and non-commercial tools which aim to do this using a variety of methods. I have looked into these tools and have not found one which is both effective and open.

I therefore aim to create my own platform with which to program the Cell processor. The idea is to use [Python][] as the end user language, and to make a backend which transparently runs certain functions on the SPEs. This will involve converting the functions into C or C++, adding code to manage execution on an SPE and do the required memory transfers, compile it with the [GCC][] compiler and then execute it.

It is quite an ambitious plan, and there are a lot of potential pitfalls. If it succeeds however, I think that it will be a very easy way to develop for the Cell processor while still having portable code.

Disappointed with CSI

I don't watch very many TV series at all. I have watched all the House episodes, which I think are absolutely fantastic. It is a show I enjoy immensely and which is simply awesome. (If you haven't seen House I can strongly recommend it.) I then watched the first season of Heroes, but lost interest partway through the second season. (I found it too disjointed and drawn out.)

The only other series I have enjoyed is CSI. I have watched entire seasons of all three sub-series (Las Vegas, Miami and New York) and have always enjoyed them. The reasons I like it are as follows.

  1. Each episode is independent and doesn't require knowledge of other episodes. This means that I can watch them whenever, and don't need to worry about watching seasons in order.
  2. The content is interesting and well thought out. The plots are intriguing with clever twists, and often provide interesting insights into society.
  3. The emotional content is focused mainly on the victims and people related to the individual cases, as opposed to the main characters which appear in every episode. I tend to form fairly strong attachments to the main characters1, and this means that I don't take the show too seriously since the hectic stuff happens to characters that I don't know well.

However, I have just finished watching the eighth season of CSI Las Vegas, and I am very disappointed with the show. It has continually been breaking points 1 and 3 above, two of the prime reasons that I liked it so much. There have been (significant) recurring stories throughout the season, and the main characters have become personally involved in the cases, which has caused me to take it very seriously.

This has taken the enjoyment out of it for me, and since I watch movies and series to relax and unwind2 it's no longer worth my while to watch CSI. I will continue to watch previous seasons, but I probably won't continue with future Las Vegas seasons.

  1. I probably shouldn't admit this, but hey. 

  2. Yes, it's shallow, and I don't care.