linux

For a current project I needed to include (the most excellent Python XML library) lxml. Getting the right version to run on Mac OS X has been notoriously difficult, though. Easy-installing it was broken (and currently is, for me, as well) and while I have found multiple how-tos, they all dealt with the problem by installing it manually from sources and patching the configure process etc. pp. That would, of course, defeat the whole purpose of using buildout in the first place.

Cheeseshop to the rescue: there is a recipe for installing lxml along with the proper versions of libxml and libxslt, however, its default values didn't work for me. So for the record: the following buildout snippet will get you non-segfaulting lxml on (at least) Mac OS X 10.5.x and Ubuntu linux. YMMV.

parts =
    [...]
    lxml

eggs =
    [...]
    lxml==2.1.2

[lxml]
recipe=plone.recipe.lxml
egg = 
    lxml==2.1.2

libxml2_url = 
    http://xmlsoft.org/sources/libxml2-2.7.1.tar.gz

libxslt_url =
    http://xmlsoft.org/sources/libxslt-1.1.24.tar.gz

p.s. this should be good news for all folks working with Deliverance. Perhaps now I could revisit that...

After receiving my new toy at work three days ago, a SunFire X2200 with 16Gb RAM, today I finally got around to set it up.

Now, while my server OS of choice is definitely FreeBSD this particular machine is going to be used as a development machine for a Plone project which will eventually be hosted at a provider who doesn't support FreeBSD but instead Ubuntu. Also, the machine came without an optical drive. So after a few fruitless attempts to boot from an USB stick, I simply stuck to what I already know and chose a TFTP based route using Mac OS X's built-in tftp server.

Basically, I just followed my previous how-to on how to install FreeBSD on a (headless, keyboardless) soekris machine. Only, this time, I installed the dhcp server from mac ports, instead of compiling it manually:

sudo port install dhcp

Then I downloaded the Ubuntu Server ISO image and mounted it.

In my dhcpd.conf I modified the root-path and filename options thus:

option root-path "/Volumes/Ubuntu-Server 8./install/netboot/";
filename "pxelinux.0";

Next modified the (existing) /System/Library/LaunchDaemons/tftp.plist so that it looked like this:

<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" 
    "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
<plist version="1.0">
<dict>
    <key>InitGroups</key>
    <true/>
    <key>Label</key>
    <string>com.apple.tftpd</string>
    <key>ProgramArguments</key>
    <array>
        <string>/usr/libexec/tftpd</string>
        <string>-i</string>
        <string>/Volumes/Ubuntu-Server 8./install/netboot</string>
        <string>-u</string>
        <string>root</string>
        <string>-s</string>
        <string>/Volumes/Ubuntu-Server 8./install/netboot</string>
        <string>-l</string>
    </array>
    <key>Sockets</key>
    <dict>
        <key>Listeners</key>
        <dict>
            <key>SockServiceName</key>
            <string>tftp</string>
            <key>SockType</key>
            <string>dgram</string>
        </dict>
    </dict>
    <key>inetdCompatibility</key>
    <dict>
        <key>Wait</key>
        <true/>
    </dict>
</dict>
</plist>

Now all I needed to do was to start both daemons...

sudo /opt/local/sbin/dhcpd 
sudo service tftp start

...finally fire up the machine and press F12 at the boot prompt and Bob was my proverbial uncle.

Currently I'm having lots of fun experimenting with WSGI, repoze and Deliverance. But while it's nice to know that it works in a development setup (i.e. deployed with paster) I needed to be sure it would work well in a production environment. And while there are already instructions floating around on how to deploy it with Apache and mod_wsgi, I wanted to know whether I could deploy WSGI-based sites using my trusted workhorse nginx.

Since nginx is much more monolithic than Apache (which is one reason why it can be so noticably more efficient than Apache in certain situations) you can't just drop in a plugin or module. Instead, you must compile nginx from sources and add the module at compile time.

The projects I'm working on will be deployed on Ubuntu and FreeBSD, but of course I will want to be able to test the same setup on my OS X development machine. So I've begun my tests with Ubuntu and OS X. Since nginx is available in all three systems' packaging system, my strategy is to install nginx via its respective package (which will integrate it nicely with start- and shutdown scripts) and simply replace the nginx binary with a self-compiled version that includes mod_wsgi.

So, here it goes: download and expand the sources for nginx (currently 0.5.35) and mod_wsgi (currently version 0.0.6):

wget http://sysoev.ru/nginx/nginx-0.5.35.tar.gz
wget http://hg.mperillo.ath.cx/nginx/mod_wsgi/archive/0.0.6.tar.gz
tar xzf nginx-0.5.35.tar.gz
tar xzf 0.0.6.tar.gz

*By the way, it seems to be a feature of mercurial unknown to the author of mod_wsgi Manlio Perillo to provide .tgz archives not only for the tip but also for each tag. Currently the tip of mod_wsgi doesn't compile on Mac OS X so I'm sticking with version 0.0.6 which has proven to be stable and contains the config fixes for Mac OS X.*

For both Ubuntu and Mac OS X we will need to explicitly tell the mod_wsgi plugin to use Python 2.4 rather than the default 2.5 version that comes with both systems, since I'm intending to run Zope based applications:

$EDITOR mod_wsgi-0.0.6/config

Change the second line of the file to:

PYTHON='python2.4'

Ubuntu

On Ubuntu you will need to install the following packages:

sudo apt-get install gcc
sudo apt-get install python2.4-dev
sudo apt-get install libxslt-dev
sudo apt-get install libssl-dev
sudo apt-get install libpcre3-dev

To take advantage of the start- and stop mechanisms provided by the official nginx package, let's first install that:

sudo apt-get install nginx

Now we can change into the nginx source directory and configure the build process to replace the packaged version of nginx with one that includes mod_wsgi like so:

cd nginx-0.5.35
./configure --add-module=../mod_wsgi-0.0.6/ --prefix=/usr/local --sbin-path=/usr/sbin \
    --conf-path=/etc/nginx/nginx.conf --with-http_ssl_module

You should receive a summary that looks like this:

Configuration summary
  + threads are not used
  + using system PCRE library
  + using system OpenSSL library
  + md5 library is not used
  + sha1 library is not used
  + using system zlib library

  nginx path prefix: "/usr/local"
  nginx binary file: "/usr/sbin"
  nginx configuration file: "/etc/nginx/nginx.conf"
  nginx pid file: "/usr/local/logs/nginx.pid"
  nginx error log file: "/usr/local/logs/error.log"
  nginx http access log file: "/usr/local/logs/access.log"
  nginx http client request body temporary files: "/usr/local/client_body_temp"
  nginx http proxy temporary files: "/usr/local/proxy_temp"
  nginx http fastcgi temporary files: "/usr/local/fastcgi_temp"

Make sure, the packaged instance of nginx is not running (we won't be able to replace it, otherwise):

sudo /etc/init.d/nginx stop

Now you can do the usual make ; sudo make install dance.

Before starting up the instance, we still need to run setup.py from the mod_wsgi folder:

cd ../mod_wsgi-0.0.6/
sudo python2.4 setup.py --prefix=/usr/local/ --sbin-path=/usr/sbin/ --conf-path=/etc/nginx/

Now you can start up your instance:

sudo /etc/init.d/nginx start

Mac OS X

On Mac OS X you will need to have the Developer Tools and MacPorts installed and the install the following in addition:

sudo port install python2.4
sudo port install libxslt # has libxml2 as auto-dependency
sudo port install py-libxml2
sudo port install nginx

For the configure process to find the 2.4 python libraries I found I needed to copy them to /opt/local/lib, as otherwise nginx would load the libraries of the system's 2.5 version at startup time which would throw mod_wsgi off track.

cp /opt/local/Library/Frameworks/Python.framework/Versions/2.4/lib/libpython2.4.dylib /opt/local/lib/

Now we can configure it to match the nginx version from the ports collection like so:

./configure --add-module=../mod_wsgi-0.0.6/ --prefix=/opt/local --conf-path=etc/nginx/nginx.conf --sbin-path=sbin/ --with-http_ssl_module

Again, here's the summary output you should expect:

Configuration summary
  + threads are not used
  + using system PCRE library
  + using system OpenSSL library
  + md5 library is not used
  + sha1 library is not used
  + using system zlib library

  nginx path prefix: "/opt/local"
  nginx binary file: "/opt/local/sbin/"
  nginx configuration file: "/opt/local/etc/nginx/nginx.conf"
  nginx pid file: "/opt/local/logs/nginx.pid"
  nginx error log file: "/opt/local/logs/error.log"
  nginx http access log file: "/opt/local/logs/access.log"
  nginx http client request body temporary files: "/opt/local/client_body_temp"
  nginx http proxy temporary files: "/opt/local/proxy_temp"
  nginx http fastcgi temporary files: "/opt/local/fastcgi_temp"

Now you can do the usual make ; sudo make install dance.

Before starting up the instance, we still need to run setup.py from the mod_wsgi folder:

cd ../mod_wsgi-0.0.6/
sudo python2.4 setup.py --prefix=/opt/local/ --sbin-path=/opt/local/sbin/ --conf-path=/opt/local/etc/nginx/

Now we're finally ready to fire up our new instance. While testing and developing I can't be bothered to use launchctl so I chose a more pedestrian approach:

sudo killall nginx ; sudo /opt/local/sbin/nginx

Now you can take a look at the sample nginx.conf file provided in the examples directory of mod_wsgi to take the provided WSGI demos for a spin and, of course, to serve as a starting point to get your own apps running. Next I'll be looking at getting repoze.plone and repoze.grok running behind nginx+mod_wsgi, so stay tuned.

As part of a current gig I today had to set up two spanking new Ubuntu based servers and deploy a Plone application that I've developed. One task was to have the Plone application start up automatically at boot time. Being a FreeBSD person I had to adapt a bit. In the end I just mashed this init.d how-to with my own homegrown FreeBSD rc.d script and created the following shell script in /etc/init.d/zope.sh:

#!/bin/bash
INSTANCESDIR="/opt/zope/instances"
SUDO=`which sudo`
for instance in `ls $INSTANCESDIR/` ; do
  $SUDO -u www-data $INSTANCESDIR/$instance/bin/zeoserver "$@"
  sleep 5
  $SUDO -u www-data $INSTANCESDIR/$instance/bin/deployment "$@"
done

and issued the following two commands (as root):

update-rc.d zope.sh defaults
chmod a+x /etc/init.d/zope.sh

This ensures that all zope instances in $INSTANCESDIR get started at boot time (and properly shut down at shutdown time!) As you can see, I run a ZEO based setup with a deployment buildout, so, as usual, your mileage may vary etc.

Who'd have thunk it? I was hired to help a client install Gentoo on his notebook today... While it always makes me a little bit nervous to do work outside my core competencies I also welcome the challenge and installing Gentoo is certainly miles more enjoyable than bookkeeping ;-)

Taking TextMate's snippets and macros for a testride (using ZopeEditManager) I popped the notes and links in to the geek section for those of you who might be interested.

This is one of the topics I've been meaning to catch up on. Over the years I have tried several flavours of Linux distributions (SuSE, Redhat, Debian, YellowDogLinux and most recently Gentoo). Despite the obious improvements over the years in the area of user experience I'm still left underwhelmed.

The main reason for that is, that I'm looking at all of these distros purely from the viewpoint of an unexperienced user. I'm looking at Linux as a possible 'Windows cure' for people such as clients, my father-in-law or friends who are a) stuck with the PC-platform for one reason or another (usually no budget for a Mac or just recently having bought some shiny, new PC-box). Personally, I'm quite happy with my 'Mac for desktop' and 'FreeBSD for server' setup, thankyouverymuch.

While all of those distros looked good (except, of course, the horrible default theme that SuSE has slapped onto KDE), but in practice - for a multitude of reasons that I'm plain too lazy to get into right now - they just proved way too cumbersome for the people trying to use the machine.

The good news is, that this might have changed. Enter ubuntu - a new, debian-based Linux distro with two main goals a) useability and b) regular, tested, well-defined (if oddly named) updates. Check out the scoop here, here, especially here and - of course - here.

I played with it on an old 500MHz P2 system for two afternoons before passing on the hardware-cum-system to a good friend who will embark on an IT-education starting next year after being laid off as electrician. For the first time, I had the feeling that this Linux distro might feel accessible to an average user. It's very sensibly pre-configured in my opinion and I'll be following up with great interest on how my friend (a complete beginner to computers, with not even Windows experience, lucky soul!) will fare.