freebsd

Transmission is the BitTorrent client. Period. All other implementations can go take a shower and watch the rest of the game from the bench, if you ask me ;-) (I know you didn't, but it's my blog, afterall...)

In Version 0.8.x transmission introduced a daemon, which listens on a local socket for commands such as adding, starting and stopping torrents. This means, you can simultaneously download multiple torrents from the terminal (you can't start multiple instances of the  'normal' commandline version of transmission, as they all will try to compete for the same port). This comes in handy if you want to download torrents from a remote server (which most likely will have a much fatter pipe to the internet than your desktop machine or notebook).

There is a FreeBSD port for Transmission, but it disables the daemon by default (and the maintainer has told me, that he's not interested in adding or supporting that feature), so I decided to build it manually. This is actually pretty straightforward: just run the usual `./configure ; make install`, bingo! Exce-hept... Transmission uses autoconf to generate its initial configuration. And getting autoconf was the hardest part here for me. In the end it just boils down to installing these packages first, before executing Transmissions `autogen.sh`: libtool15, automake19, autoconf261 (all located in `/devel` in the ports tree).

After installation you can now simply start transmission-daemon (no further parameters necessary, it creates its socket in your home directory and you can - and should! - run it without elevated privileges). You can now user transmission-remote to add torrents and monitor and control their progress. For now, this already makes for a big improvement in regard to my (modest) torrent usage, but I've already written a python client library for transmission-daemon and am now toying with the idea of writing a grok based web fronted for it... anybody interested, perhaps?

Today, the Zope instance hosting this site experienced frequent crashes (as in "every couple of minutes) in the form of coredumps of the zope process. Stumped, bewildered, frustrated and near tears I did what I always do in such cases: pour my little heart out on #plone ;-)

Sure enough, help was at hand. And today's tip of the hat goes to... drumroll... lurker, who alerted me to the fact, that python needs explicit 'huge stack' suppport on FreeBSD. A bit of googling supported this view and so I reinstalled python from the ports, but first ran make config and then enabled HUGE_STACK_SIZE:

After that I ran a few stresstests with ab and the instance easily withheld several thousand requests for the front page. Why this option isn't enabled by default is beyond me. So keep that in mind, kids, when you install python on your BSD boxen: size does matter ;-) (As if the term 'huge stack size' wasn't already obscene enough without such a foot-in-mouth reference...)

To run PostgreSQL in a FreeBSD jail you need to enable security.jail.sysvipc_allowed via sysctl in the jail host. i don’t know how often I have already forgotten this…

Personally, the snow sprint started off with a bit of a shock. Upon arrival at the location I did two things: try to post a blog entry and check my email. First I find out, that I can’t post any blog entries – plone ignores the changes?! At the same time I got email from three clients who report that they can’t edit any content on their plone sites – which, incidentially, are hosted on the very same machine as this blog…

The symptoms (listed to increase the likelihood of somebody experiencing them finding this entry ;-) were that editing or creating any content object in plone that involved a certain amount of data would fail: either the creation would fail with an error message regarding missing data (a.ka. "Title is required") or the modification would just result in an actually unmodifed object and Plone lying out of its teeth claiming, that all changes had been saved.

Luckily I was in the best possible company imaginable to solve such a problem ;-) Matt a.k.a. Hammertoe was kind enough to help me narrow down the problem. Pretty soon we had isolated it to Apache, rather than Zope or Plone itself. Luckily Matt had encountered a very similar problem and immediately remembered, that it was due to a bug specifically in apache version 2.55. Which made sense, because we upgraded our apache just a few days ago…

Fortunately the bug has already been fixed in the trunk and in the 2.0.x-branch. The remaining problem was then to build the new modules specifically on FreeBSD. For future reference (who knows when that will come in handy!) here are the steps:

• cd /tmp
• svn checkout http://svn.apache.org/repos/asf/httpd/httpd/branches/2.0.x
• cd /usr/ports/www/apache20
• make clean
• cp /tmp/2.0.x/modules/proxy/* work/httpd-2.0.55/modules/ don’t replace the folder as that would remove the config information – just copy the files from svn into the proxy directory!
• make deinstall ; make reinstall
• sudo /usr/local/etc/rc.d/apache.sh restart
• write emails to client informing them, that problem is fixed
• post to tomster.org

Thanks again, Matt for the hand-holding ;-)

It’s been a couple of months now since I’ve started to work on a daily basis with jails (and more recently with convenience tool ezjail) and it’s time for a wrap-up: are (FreeBSD-)jails worth the hassle?

Short answer: yes. Long answer: yeeeeeeeeeeeees ;-)

In the beginning I was a bit overwhelmed on a practical level by the “red-tape” involved in setting them up (ezjail has effectively taken care of that, though) and on a conceptual level with constantly having to switch context – mentally, as well as ‘physically’: where you used to be just a cd or su away from your next task you now often need to enter a new jail – which one? go the host system, do a jsl, look for the jail you need, take a note of its id then sudo jexec $ID tcsh... now, where was I?

This is an aspect that still needs improvement. Especially the step of finding the current runtime id of a given jailname is a task best left to a machine.

The top three things that bother me when working inside a jail are:

1. that ping doesn’t work: Papa Scott points out in his comment, that "With sysctl, you can set security.jail.allow_raw_sockets=1 and thus enable ping and traceroute to be run in your jails. ", yay!
2. that localhost and 127.0.0.1 don’t work
3. that a jail can have only one IP-Address

Another thing that still could use a little love in this context is the ports collection. In the current setup you either need to install (and maintain!) one ports collection in each jail or you create a dedicated ‘buidljail’ whose ports collection you keep up-to-date and build packages using the make package-recursive which usually¹ creates the ready-to-install binary packages (including all dependencies) in /usr/ports/packages/All/ which is then mounted read-only via nullfs into each jail.

This sucks twice: first of all it’s inconvenient to create the packages and clean them up again and secondly it requires root access to the build jail which is fine, if you’re just one admin but kind of defeats one of the main purposes of virtualisation when providing other users with their own jails.

This, of course, is fixable and it isn’t really too hard. One just has to set a few environment variables (DISTDIR and WRKDIRPREFIX) in each jail to direct the actual make process of a package away from the default ./work (which is read-only from inside the jail) to somewhere else.

Having said all that it’s now time to report on the goodies. Aside from the security aspects that virtualisation and compartmentalisation bring (think of a huge oil tanker with individual sections vs. one without) there is a huge benefit for you as admin of a jail host – which is, however, somewhat hard to describe. But for example, it’s really practical to, say, tinker with apache to set up a webmail interface without having to worry about breaking your subversion setup. Migrating from bind to tinydns? Just create a new jail, set everything up until it works and then just switch over the IPs of the old bind-jail and the new tinydns jail. Working with jails also means fewer installed packages and less chances for conflicting versions. (Anybody who had to update a dozen or so packages for a new version of gettext just because he/she innocently installed wget will appreciate this…) And also, suddenly every single rc.conf file is just a few lines long and meaningful…

I do wonder, though, how many people looking for information on prisons and life as an inmate will end up at tomster.org since I’ve been blogging about this…

[1] some ports such as djbdns cannot be packaged. Also, portupgrade won’t work in this kind of environment