zope

Deliverance is the New Kid on the Block[tm] in the Plone world. I find the whole concept very intriguing and have decided to give it a try on both of my current new projects. (No risk, no fun, eh!?)

One of the two projects is smaller in scope where I am both the Zope/Plone developer and the HTML/CSS handyman and the other a larger affair together with another Zopista and an external web agency with up to five more people, none of which is experienced with - or much interested in - Plone, so I am curious, how Deliverance will hold up in these two quite different scenarios.

Now, if you just want to dip your toes into the proverbial water, I suggest you check out Martin Aspeli's Deliverance recipe (does that man ever sleep?) However, Martin's recipe, along with any other tutorial that I found floating about, uses deliverance in its so-called proxy mode where a second daemon is fired up that mediates between the web- and application server. This is not optimal, of course, as it pretty much defeats the whole point of the WSGI concept, so I set out to create a setup, where Plone would be themed by Deliverance configured as a WSGI middleware. Forthwith my notes:

I conducted my test on Mac OS X 10.5.2 with Python 2.4.5 installed via Macports. I didn't use virtualenv (although I should have) but had to make sure throughout the whole process, that python2.4 was used (as opposed to Leopard's default 2.5.1). To do that, I installed easy_install and setuptools explicitly for 2.4:

wget http://peak.telecommunity.com/dist/ez_setup.py
sudo python2.4 ez_setup.py
sudo python ez_setup.py -U setuptools

Before dealing with Deliverance I set up a WSGI enabled Plone instance. The buildout provided by repoze.plone proved to be the best candidate to do so (believe me!)

svn co http://svn.repoze.org/buildouts/repoze.plone/trunk repoze.plone
cd repoze.plone
python2.4 bootstrap.py
./bin/buildout

I then fired up the new instance as per instructions:

./bin/supervisord
./bin/addzope2user admin admin

I then could add a Plone instance foo at localhost:8080 just as always. I then stopped the zope instance so:

./bin/supervisorctl stop zope

and then restarted it using paster:

./bin/paster serve etc/zope2.ini

and sure enough, localhost:8080/foo was still there!

Now the only thing missing was Deliverance itself. To make it available for the instance, I simply easy_installed it into my system (this is the part, where next time I'll use virtualenv!):

sudo easy_install-2.4 Deliverance

This enabled me to add the following snippet to etc/zope2.ini:

[filter:deliverance]
paste.filter_app_factory = deliverance.wsgimiddleware:make_filter
theme_uri = file:///%(here)s/layout.html
rule_uri = file:///%(here)s/rules.xml

To use the deliverance as a filter in the WSGI stack I just had to add it somewhere. I chose to add it just after zope2 itself, so that my pipeline section looked like so:

[pipeline:main]
pipeline = egg:Paste#cgitb
           egg:Paste#httpexceptions
           egg:repoze.retry#retry
           egg:repoze.tm#tm
           egg:repoze.vhm#vhm_xheaders 
           errorlog
           deliverance
           zope2

Before restarting zope (or rather 'reserving it with paster') I had to add the rulefiles mentioned in the filter declaration. Eager as I was at this point, I simply borrowed the trivial default rule and theme from Martin's recipe, spiced it up a little with one of repoze.org's own rule files and created a etc/rules.xml file that looked like this:

<?xml version="1.0" encoding="UTF-8"?>
<rules xmlns:xi="http://www.w3.org/2001/XInclude" 
    xmlns="http://www.plone.org/deliverance">
    <prepend theme="//head" content="//head/link" nocontent="ignore" /> 
    <prepend theme="//head" content="//head/style" nocontent="ignore" /> 
    <append theme="//head" content="//head/script" nocontent="ignore" />    
    <append theme="//head" content="//head/meta" nocontent="ignore" />
    <append-or-replace 
        theme="//head" 
        content="//head/title"
        nocontent="ignore" />
    <append-or-replace theme="//div[@id='content']"
        content="//div[@id='region-content']"
        nocontent="ignore" />
</rules>

and a etc/layout.html file that looked like this:

<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN" "http://www.w3.org/TR/html4/loose.dtd">
<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=iso-8859-15">
<title>The Theme</title>
</head>
<body>
<h1>A theme</h1>
<div id="content">
</div>
<hr>
<address></address>
<!-- hhmts start -->Last modified: Fri Mar 16 09:33:37 CDT 2007 <!-- hhmts end -->
</body> </html>
<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.1//EN"
    "http://www.w3.org/TR/xhtml11/DTD/xhtml11.dtd">

<html xmlns="http://www.w3.org/1999/xhtml" xml:lang="en">
<head>
    <title>layout</title>
</head>
<body>
</body>
</html>

Now I could stop the foreground paster process, fire it up again, hit reload in my browser and voila! Bob was my proverbial Uncle!

Next, I'll give mod_wsgi a try. But, of course, not the one for Apache but... yup, there's a mod_wsgi for nginx, too, yay! Stay tuned...

P.S. I'd like to take this opportunity to thank Chris, Tres and Paul for their efforts to connect Zope and Plone with non-techies and non-Zopistas. I'm very confident that you guys are creating some serious groundwork here that will benefit us immensely during the next eight years! Rock on!

Any non-trivial deployment of Plone usually consists of at least a web server 'sitting in front of' it, often enough accompanied by some sort of caching service. For my own purposes I have settled on nginx for the former and varnish for the latter[1].

They have proven to be a reasonably good team for me, especially in combination with recent versions of CacheFu, Plone's de facto default caching product. By 'reasonably good' I mean, that varnish and CacheFu in their 'normal' setup (i.e. with the 'official' Plone .vcl config) offer a reasonable trade-off between performance and potentially stale content.

I noticed this behaviour particularly when looking at Zope's access log (i.e. var/log/primary-Z2.log) while accessing my latest Plone installation at wahlcomputer.ccc.de. There was much too much going on, as theoretically there shouldn't be any requests reaching the backend until some content had changed. In essence, the site behaved much slower than it would need to by answering unnecessarily to requests that varnish should be able to satisfy.

However, in order to debug the caching and purging of content in detail I needed a local Mac OS X based setup that mirrored that on the FreeBSD box serving the site as closely, as possible. I habitually took notes while setting it up on my desktop machine and am now retracing my steps as I attempt to duplicate it on my Macbook (Just like a bug isn't fixable until it's reproducible, I never consider my admin work done until I have been able to do it at least twice). I'm confident that somebody out there will find the following notes useful, too (even if it's just myself six months^wweeks down the road)...

Since I'm a happy user of the macports collection already anyway, I let it do the 'heavy lifting' of actually installing nginx and varnish. In addition I provided a launchd startup item for varnish and also added a host entry for wahlcomputer to enable virtual hosting for nginx and varnish. Here it goes:

sudo port install nginx
sudo port install varnish

DNS

Before continuing the setup I first wanted to make sure that I could use the domain name wahlcomputer to access my machine. Instead of doing it 'properly' by adding a new record to OpenLDAP and thus possibly opening up what could amount to yet another can of worms I simply tried to edit ye' good ol' /etc/hosts file and added the following line:

127.0.0.1   wahlcomputer

And sure enough, even in the days of 'OpenLDAP FTW!!' Mac OS X still honours entries made to the hosts file so I could move on.

nginx

Next, I configured the freshly installed nginx. As the port already comes with a launchd start up item, I just needed to load it:

sudo launchctl load /opt/local/etc/LaunchDaemons/org.macports.nginx/org.macports.nginx.plist

It's my practice to leave the default files as untouched as possible and put all my configurations into separate files, that I simply include:

sudo mv /opt/local/etc/nginx/nginx.conf.default  /opt/local/etc/nginx/nginx.conf
sudo mkdir /opt/local/etc/nginx/includes
sudo touch /opt/local/etc/nginx/includes/wahlcomputer.conf
sudo mkdir /opt/local/var/log/nginx/

To make nginx aware of those files I added the following line just before the final closing }:

include etc/nginx/includes/*.conf;

Here's its contents in its full, unabridged glory (and VHM gore):

server {
    server_name wahlcomputer.ccc.de wahlcomputer;
    location / {
        proxy_pass http://127.0.0.1:8071/VirtualHostBase/http/wahlcomputer:80/foo/VirtualHostRoot/;
    }
}

For the changes to take effect, I needed to restart nginx like so (why, oh why, does launchctl with all its bells and whistles not have a simple reload or restart command?!):

sudo launchctl stop org.macports.nginx
sudo launchctl start org.macports.nginx

Now visiting http://wahlcomputer/ showed the front page of the Plone site and I could move on to varnish:

varnish

Unlike nginx, the varnish port (currently) does not come with a startup item, so I made my very own (by blatantly copying from the nginx setup):

sudo mkdir /opt/local/etc/varnish
sudo touch /opt/local/etc/varnish/default.vcl
sudo mkdir /opt/local/etc/LaunchDaemons/org.macports.varnish
sudo touch /opt/local/etc/LaunchDaemons/org.macports.varnish/org.macports.varnishd.plist
sudo chown root:wheel /opt/local/etc/LaunchDaemons/org.macports.varnish/org.macports.varnishd.plist
sudo chmod 644 /opt/local/etc/LaunchDaemons/org.macports.varnish/org.macports.varnishd.plist
sudo ln -s /opt/local/etc/LaunchDaemons/org.macports.varnish/org.macports.varnishd.plist /Library/LaunchDaemons/
sudo mkdir /opt/local/var/varnish
sudo chown -R _www /opt/local/var/varnish

Looking at varnishd's man page - HTTP accelerator daemon - Linux Manual - Digipedia") I saw that there are some options that must be passed in upon start up. I did this by adding them to the ProgramArguments array in the /opt/local/etc/LaunchDaemons/org.macports.varnish/org.macports.varnishd.plist file:

<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
<plist version="1.0">
<dict>
    <key>Disabled</key>
    <false/>
    <key>KeepAlive</key>
    <false/>
    <key>Debug</key>
    <false/>
    <key>Label</key>
    <string>varnishd</string>
    <key>OnDemand</key>
    <false/>
    <key>GroupName</key>
    <string>staff</string>
    <key>UserName</key>
    <string>_www</string>
    <key>ProgramArguments</key>
    <array>
        <string>/opt/local/sbin/varnishd</string>
        <string>-a</string>
        <string>localhost:6081</string>
        <string>-T</string>
        <string>localhost:6082</string>
        <string>-f</string>
        <string>/opt/local/etc/varnish/default.vcl</string>
    </array>
    <key>RunAtLoad</key>
    <false/>
</dict>
</plist>

Now all that was missing was to add some directives to the currently still empty file /opt/local/etc/varnish/default.vcl. It ended up being too long to quote here in its entirety, instead I will simply point you to a nice variation of the 'official zope-plone.vcl that I found that sports some neat enhancements.

Finally I could start up varnish:

sudo launchctl load /opt/local/etc/LaunchDaemons/org.macports.varnish/org.macports.varnishd.plist

In the nginx wahlcomputer.conf file all I needed to change was the port number from 8071 (the Zope instance) to 6081 (varnish).

By keeping a tail on the access logs of nginx and my Zope instance and looking at the response headers with firebug I can now start to tweak my setup to my heart's content. Stay tuned.

[1] A bit of an explanation is perhaps necessary as to why even bother with two types of machinery in front of Plone. The short answer is 'separation of concerns'. While varnish could theoretically take over most of the features that I use nginx for (namely, URL-rewriting, logging and static content delivery) I still would need it for SSL/HTTPS. Also, the logging feature of varnish requires an extra varnishlog process anyway.

Two weeks ago I sat down with Tim Pritlove and he interviewed me about Plone and Zope for his Chaosradio Express podcast.

I thoroughly enjoyed the experience, largely due to the fact that Tim proved to be an excellent host. He gave me enough leeway to ramble while at the same time providing enough structure for the process to let me relax and be myself. Thanks Tim!

In the end we talked over one and a half hours and I can only assume that one has to have a genuine interest in the content matter to listen to it all the way through but afterwards I still felt that I had left much unsaid.

I tried to be honest and upfront about Zope and Plone's weaknesses while still conveying that they both are truly great systems with even greater communities. The interview was conducted in German, though, so I figure the percentage of the audience of this blog that will find it interesting is probably rather small, but nonetheless, here's the link to the episode.

As part of a current gig I today had to set up two spanking new Ubuntu based servers and deploy a Plone application that I've developed. One task was to have the Plone application start up automatically at boot time. Being a FreeBSD person I had to adapt a bit. In the end I just mashed this init.d how-to with my own homegrown FreeBSD rc.d script and created the following shell script in /etc/init.d/zope.sh:

#!/bin/bash
INSTANCESDIR="/opt/zope/instances"
SUDO=`which sudo`
for instance in `ls $INSTANCESDIR/` ; do
  $SUDO -u www-data $INSTANCESDIR/$instance/bin/zeoserver "$@"
  sleep 5
  $SUDO -u www-data $INSTANCESDIR/$instance/bin/deployment "$@"
done

and issued the following two commands (as root):

update-rc.d zope.sh defaults
chmod a+x /etc/init.d/zope.sh

This ensures that all zope instances in $INSTANCESDIR get started at boot time (and properly shut down at shutdown time!) As you can see, I run a ZEO based setup with a deployment buildout, so, as usual, your mileage may vary etc.

Here's a quick snippet for quickly un-expiring content from a ./bin/instance debug session:

plone_site = app.foo
from transaction import commit
import DateTime
now = DateTime.now()
for brain in plone_site.portal_catalog({
    'expires' : {'query' : now, 'range' : 'max'}}):

    obj = brain.getObject()
    obj.setExpirationDate(now + 30)
    obj.reindexObject()

commit()

This sets the expiration date of all currently expired objects 30 days into the future. (You'll need to change app.foo to the id of your plone instance, of course.)