repoze

Currently I'm having lots of fun experimenting with WSGI, repoze and Deliverance. But while it's nice to know that it works in a development setup (i.e. deployed with paster) I needed to be sure it would work well in a production environment. And while there are already instructions floating around on how to deploy it with Apache and mod_wsgi, I wanted to know whether I could deploy WSGI-based sites using my trusted workhorse nginx.

Since nginx is much more monolithic than Apache (which is one reason why it can be so noticably more efficient than Apache in certain situations) you can't just drop in a plugin or module. Instead, you must compile nginx from sources and add the module at compile time.

The projects I'm working on will be deployed on Ubuntu and FreeBSD, but of course I will want to be able to test the same setup on my OS X development machine. So I've begun my tests with Ubuntu and OS X. Since nginx is available in all three systems' packaging system, my strategy is to install nginx via its respective package (which will integrate it nicely with start- and shutdown scripts) and simply replace the nginx binary with a self-compiled version that includes mod_wsgi.

So, here it goes: download and expand the sources for nginx (currently 0.5.35) and mod_wsgi (currently version 0.0.6):

wget http://sysoev.ru/nginx/nginx-0.5.35.tar.gz
wget http://hg.mperillo.ath.cx/nginx/mod_wsgi/archive/0.0.6.tar.gz
tar xzf nginx-0.5.35.tar.gz
tar xzf 0.0.6.tar.gz

*By the way, it seems to be a feature of mercurial unknown to the author of mod_wsgi Manlio Perillo to provide .tgz archives not only for the tip but also for each tag. Currently the tip of mod_wsgi doesn't compile on Mac OS X so I'm sticking with version 0.0.6 which has proven to be stable and contains the config fixes for Mac OS X.*

For both Ubuntu and Mac OS X we will need to explicitly tell the mod_wsgi plugin to use Python 2.4 rather than the default 2.5 version that comes with both systems, since I'm intending to run Zope based applications:

$EDITOR mod_wsgi-0.0.6/config

Change the second line of the file to:

PYTHON='python2.4'

Ubuntu

On Ubuntu you will need to install the following packages:

sudo apt-get install gcc
sudo apt-get install python2.4-dev
sudo apt-get install libxslt-dev
sudo apt-get install libssl-dev
sudo apt-get install libpcre3-dev

To take advantage of the start- and stop mechanisms provided by the official nginx package, let's first install that:

sudo apt-get install nginx

Now we can change into the nginx source directory and configure the build process to replace the packaged version of nginx with one that includes mod_wsgi like so:

cd nginx-0.5.35
./configure --add-module=../mod_wsgi-0.0.6/ --prefix=/usr/local --sbin-path=/usr/sbin \
    --conf-path=/etc/nginx/nginx.conf --with-http_ssl_module

You should receive a summary that looks like this:

Configuration summary
  + threads are not used
  + using system PCRE library
  + using system OpenSSL library
  + md5 library is not used
  + sha1 library is not used
  + using system zlib library

  nginx path prefix: "/usr/local"
  nginx binary file: "/usr/sbin"
  nginx configuration file: "/etc/nginx/nginx.conf"
  nginx pid file: "/usr/local/logs/nginx.pid"
  nginx error log file: "/usr/local/logs/error.log"
  nginx http access log file: "/usr/local/logs/access.log"
  nginx http client request body temporary files: "/usr/local/client_body_temp"
  nginx http proxy temporary files: "/usr/local/proxy_temp"
  nginx http fastcgi temporary files: "/usr/local/fastcgi_temp"

Make sure, the packaged instance of nginx is not running (we won't be able to replace it, otherwise):

sudo /etc/init.d/nginx stop

Now you can do the usual make ; sudo make install dance.

Before starting up the instance, we still need to run setup.py from the mod_wsgi folder:

cd ../mod_wsgi-0.0.6/
sudo python2.4 setup.py --prefix=/usr/local/ --sbin-path=/usr/sbin/ --conf-path=/etc/nginx/

Now you can start up your instance:

sudo /etc/init.d/nginx start

Mac OS X

On Mac OS X you will need to have the Developer Tools and MacPorts installed and the install the following in addition:

sudo port install python2.4
sudo port install libxslt # has libxml2 as auto-dependency
sudo port install py-libxml2
sudo port install nginx

For the configure process to find the 2.4 python libraries I found I needed to copy them to /opt/local/lib, as otherwise nginx would load the libraries of the system's 2.5 version at startup time which would throw mod_wsgi off track.

cp /opt/local/Library/Frameworks/Python.framework/Versions/2.4/lib/libpython2.4.dylib /opt/local/lib/

Now we can configure it to match the nginx version from the ports collection like so:

./configure --add-module=../mod_wsgi-0.0.6/ --prefix=/opt/local --conf-path=etc/nginx/nginx.conf --sbin-path=sbin/ --with-http_ssl_module

Again, here's the summary output you should expect:

Configuration summary
  + threads are not used
  + using system PCRE library
  + using system OpenSSL library
  + md5 library is not used
  + sha1 library is not used
  + using system zlib library

  nginx path prefix: "/opt/local"
  nginx binary file: "/opt/local/sbin/"
  nginx configuration file: "/opt/local/etc/nginx/nginx.conf"
  nginx pid file: "/opt/local/logs/nginx.pid"
  nginx error log file: "/opt/local/logs/error.log"
  nginx http access log file: "/opt/local/logs/access.log"
  nginx http client request body temporary files: "/opt/local/client_body_temp"
  nginx http proxy temporary files: "/opt/local/proxy_temp"
  nginx http fastcgi temporary files: "/opt/local/fastcgi_temp"

Now you can do the usual make ; sudo make install dance.

Before starting up the instance, we still need to run setup.py from the mod_wsgi folder:

cd ../mod_wsgi-0.0.6/
sudo python2.4 setup.py --prefix=/opt/local/ --sbin-path=/opt/local/sbin/ --conf-path=/opt/local/etc/nginx/

Now we're finally ready to fire up our new instance. While testing and developing I can't be bothered to use launchctl so I chose a more pedestrian approach:

sudo killall nginx ; sudo /opt/local/sbin/nginx

Now you can take a look at the sample nginx.conf file provided in the examples directory of mod_wsgi to take the provided WSGI demos for a spin and, of course, to serve as a starting point to get your own apps running. Next I'll be looking at getting repoze.plone and repoze.grok running behind nginx+mod_wsgi, so stay tuned.

Deliverance is the New Kid on the Block[tm] in the Plone world. I find the whole concept very intriguing and have decided to give it a try on both of my current new projects. (No risk, no fun, eh!?)

One of the two projects is smaller in scope where I am both the Zope/Plone developer and the HTML/CSS handyman and the other a larger affair together with another Zopista and an external web agency with up to five more people, none of which is experienced with - or much interested in - Plone, so I am curious, how Deliverance will hold up in these two quite different scenarios.

Now, if you just want to dip your toes into the proverbial water, I suggest you check out Martin Aspeli's Deliverance recipe (does that man ever sleep?) However, Martin's recipe, along with any other tutorial that I found floating about, uses deliverance in its so-called proxy mode where a second daemon is fired up that mediates between the web- and application server. This is not optimal, of course, as it pretty much defeats the whole point of the WSGI concept, so I set out to create a setup, where Plone would be themed by Deliverance configured as a WSGI middleware. Forthwith my notes:

I conducted my test on Mac OS X 10.5.2 with Python 2.4.5 installed via Macports. I didn't use virtualenv (although I should have) but had to make sure throughout the whole process, that python2.4 was used (as opposed to Leopard's default 2.5.1). To do that, I installed easy_install and setuptools explicitly for 2.4:

wget http://peak.telecommunity.com/dist/ez_setup.py
sudo python2.4 ez_setup.py
sudo python ez_setup.py -U setuptools

Before dealing with Deliverance I set up a WSGI enabled Plone instance. The buildout provided by repoze.plone proved to be the best candidate to do so (believe me!)

svn co http://svn.repoze.org/buildouts/repoze.plone/trunk repoze.plone
cd repoze.plone
python2.4 bootstrap.py
./bin/buildout

I then fired up the new instance as per instructions:

./bin/supervisord
./bin/addzope2user admin admin

I then could add a Plone instance foo at localhost:8080 just as always. I then stopped the zope instance so:

./bin/supervisorctl stop zope

and then restarted it using paster:

./bin/paster serve etc/zope2.ini

and sure enough, localhost:8080/foo was still there!

Now the only thing missing was Deliverance itself. To make it available for the instance, I simply easy_installed it into my system (this is the part, where next time I'll use virtualenv!):

sudo easy_install-2.4 Deliverance

This enabled me to add the following snippet to etc/zope2.ini:

[filter:deliverance]
paste.filter_app_factory = deliverance.wsgimiddleware:make_filter
theme_uri = file:///%(here)s/layout.html
rule_uri = file:///%(here)s/rules.xml

To use the deliverance as a filter in the WSGI stack I just had to add it somewhere. I chose to add it just after zope2 itself, so that my pipeline section looked like so:

[pipeline:main]
pipeline = egg:Paste#cgitb
           egg:Paste#httpexceptions
           egg:repoze.retry#retry
           egg:repoze.tm#tm
           egg:repoze.vhm#vhm_xheaders 
           errorlog
           deliverance
           zope2

Before restarting zope (or rather 'reserving it with paster') I had to add the rulefiles mentioned in the filter declaration. Eager as I was at this point, I simply borrowed the trivial default rule and theme from Martin's recipe, spiced it up a little with one of repoze.org's own rule files and created a etc/rules.xml file that looked like this:

<?xml version="1.0" encoding="UTF-8"?>
<rules xmlns:xi="http://www.w3.org/2001/XInclude" 
    xmlns="http://www.plone.org/deliverance">
    <prepend theme="//head" content="//head/link" nocontent="ignore" /> 
    <prepend theme="//head" content="//head/style" nocontent="ignore" /> 
    <append theme="//head" content="//head/script" nocontent="ignore" />    
    <append theme="//head" content="//head/meta" nocontent="ignore" />
    <append-or-replace 
        theme="//head" 
        content="//head/title"
        nocontent="ignore" />
    <append-or-replace theme="//div[@id='content']"
        content="//div[@id='region-content']"
        nocontent="ignore" />
</rules>

and a etc/layout.html file that looked like this:

<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN" "http://www.w3.org/TR/html4/loose.dtd">
<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=iso-8859-15">
<title>The Theme</title>
</head>
<body>
<h1>A theme</h1>
<div id="content">
</div>
<hr>
<address></address>
<!-- hhmts start -->Last modified: Fri Mar 16 09:33:37 CDT 2007 <!-- hhmts end -->
</body> </html>
<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.1//EN"
    "http://www.w3.org/TR/xhtml11/DTD/xhtml11.dtd">

<html xmlns="http://www.w3.org/1999/xhtml" xml:lang="en">
<head>
    <title>layout</title>
</head>
<body>
</body>
</html>

Now I could stop the foreground paster process, fire it up again, hit reload in my browser and voila! Bob was my proverbial Uncle!

Next, I'll give mod_wsgi a try. But, of course, not the one for Apache but... yup, there's a mod_wsgi for nginx, too, yay! Stay tuned...

P.S. I'd like to take this opportunity to thank Chris, Tres and Paul for their efforts to connect Zope and Plone with non-techies and non-Zopistas. I'm very confident that you guys are creating some serious groundwork here that will benefit us immensely during the next eight years! Rock on!