3. Cyrus IMAP
We're running the cyrus imapd from the ports collection. Current version is 2.1.16, I haven't tested the 2.2 branch.
Since the imapd depends on cyrus-sasl2 and since we want both of them to use Berkeley DB version 4.1, we first install cyrus-sasl with this option and then the imapd.
cd /usr/ports/security/cyrus-sasl2 make WITH_BDB_VER=41 make install cd /usr/ports/mail/cyrus-imapd2 make WITH_BDB_VER=41 make install
Note
If you want to share the sasl authentication with exim you will need to use saslauthd, as well. Since cyrus-sasl2-saslauthd depends on cyrus-sasl2 simple substitute the first with the latter in the above example; if not stick to cyrus-sasl2, as this will avoid running an unnceccessart daemon!
If you're installing saslauthd, you will have to configure it to use sasldb as mechanism. Edit /usr/local/etc/rc.d/saslauthd.sh and change the line with the startup flags to this:
[ -z "$saslauthd_flags" ] && saslauthd_flags="-a sasldb" # Flags to saslauthd program
Next, if you're not using a pre-populated mail directory you must run mkimap
su /usr/local/cyrus/bin/mkimap
Create a server key and certificate
su - cyrus openssl req -new -x509 -nodes -out /var/cert/server.pem -keyout /var/cert/server-key.pem -days 3650
Note
When entering information in the ensuing dialog remember that the field 'Common name' seems to mean FQDN, ie. for Cyrus you should enter the name your clients access the server with i.e. mail.tomster.org!
Make sure the following options exist in /usr/local/etc/imapd.conf:
sasl_pwcheck_method: auxprop # use saslauthd if you want to share with exim tls_key_file: /var/cert/server-key.pem tls_ca_file: /var/cert/server.pem tls_cert_file: /var/cert/server.pem admins: cyrus
Note
If you're using saslauthd change the sasl_pwcheck_method to saslauthd.
In order for timsieved to work you will need to add the following line in /etc/services:
sieve 2000/tcp
Now start up the mailserver and - if necessary - the saslauthd (and don't forget to change the startup script in /usr/local/etc/rc.d/)
/usr/local/etc/rc.d/saslauthd.sh start /usr/local/etc/rc.d/imapd.sh start
Still as user cyrus, create a sasl-user of the same name who will be used as admin account for imap administration:
saslpasswd2 -c cyrus
Note
As of FreeBSD 5.1 resp as of cyrus 2.1.15 you will need to add the following to /etc/rc.conf in order for cyrus to start up via /usr/local/etc/rc.d/imapd.sh:
cyrus_imapd_enable="YES"
Adding IMAP users involves creating an entry in sasldb2 using saslpasswd2 and creating an INBOX using the cm command in cyradm. I.e to add the user jdoe you would do the following as user cyrus:
saslpasswd2 -c jdoe %cyradm localhost Password: <enter password> localhost.tomster.org> cm user.jdoe
Note
saslpasswd uses the current hostname as part of the primary key for looking up passwords. If you should change the primary hostname of your machine later on, all lookups will fail and you will have to re-create all saslusers! See also this note.
Note
If this doesn't work (i.e. you get 'cannot authenticate to server with as cyrus' when using cyradm, make sure that in imapd.conf you've got sasl_pwcheck_method: auxprop.
Additionally you can set quotas using the sq command. However, when adding a lot of users this can become cumbersome. I've found a neat perl script from Amram Bentolila via Venkatesh's Tutorial which I've hacked to accept a tab-separated user/password textfile to create custom sasldb passwords along with the INBOX in one go.
I've created a folder /usr/local/cyrus/tools (to keep the distribution specific /usr/local/cyrus/bin folder clean). The toolset consists of a perlscript for adding a single user and a shellscript for batch adding users.
Heres the perl script:
#!/usr/bin/perl -w
#
# This will create a new mailbox and set a quota on the new user. Just be
# sure that you installed the Cyrus::IMAP perl module. If you did
# 'make all && make install' or installed Cyrus using the FreeBSD ports you
# don't have to do anything at all.
#
# Change the params below to match your mailserver settins, and
# your good to go!
#
# Author: amram@manhattanprojects.com
#
# modified by Tom Lazar tom@tomster.org on 2003-08-26 to use
# a tab separated user - passwd inputfile instead of the standardpassword
use Cyrus::IMAP::Admin;
#
# CONFIGURATION PARAMS
#
my $cyrus_server = "localhost";
my $cyrus_user = "cyrus";
my $cyrus_pass = "CYRUSPASSWORD";
# 100 Megs
my $quota_size = "1000000";
my $mechanism = "login";
#
# EOC
#
if (!$ARGV[0]) {
die "Usage: $0 [user to add] passwd \n";
} else {
$newuser = "$ARGV[0]";
$newpasswd = "$ARGV[1]";
}
sub createMailbox {
my ($user, $subfolder) = @_;
my $cyrus = Cyrus::IMAP::Admin->new($cyrus_server);
$cyrus->authenticate($mechanism,'imap','',$cyrus_user,'0','10000',$cyrus_pass);
if ($subfolder eq "INBOX") {
$mailbox = "user.". $user;
} else {
$mailbox = "user.". $user .".". $subfolder;
}
$cyrus->create($mailbox);
if ($cyrus->error) {
print STDERR "Error: ", $mailbox," ", $cyrus->error, "\n";
} else {
print "Created Mailbox: $mailbox \n";
}
}
sub setQuota {
my ($user) = @_;
my $cyrus = Cyrus::IMAP::Admin->new($cyrus_server);
$cyrus->authenticate($mechanism,'imap','',$cyrus_user,'0','10000',$cyrus_pass);
$mailbox = "user.". $user;
$cyrus->setquota($mailbox,"STORAGE",$quota_size);
if ($cyrus->error) {
print STDERR "Error: ", $mailbox," ", $cyrus->error, "\n";
} else {
print "Setting Quota: $mailbox at $quota_size \n";
}
}
print "Adding User: ", $newuser, "\n";
createMailbox($newuser,'INBOX');
createMailbox($newuser,'Sent');
createMailbox($newuser,'Trash');
createMailbox($newuser,'Drafts');
createMailbox($newuser,'Junk');
setQuota($newuser);
# This portion below will set a password for the user you wanted to
# add.
system "echo ". $newpasswd ." > .saslpass.tmp";
system "saslpasswd2 -p $newuser < .saslpass.tmp";
print "Generated Password: Completed \n";
unlink(".saslpass.tmp");
The script also creates a Drafts, Sent, Trash and Junk folder for the user. This is helpful for setting up IMAP clients to store drafts, sent and deleted mail on the server.
And here the shellscript for batch adding:
#!/bin/sh
#
# Batch add users for cyrus mailserver.
#
## The only dependancy is the cyr_adduser.pl program.
#
# Author: amram@manhattanprojects.com
# modified by Tom Lazar tom@tomster.org on 2003-08-26 to use
# a tab separated user - passwd inputfile instead of the standardpassword
#
CYR_ADDUSER="/usr/local/cyrus/tools/cyr_adduser.pl";
if [ $# -eq 0 ]; then
echo "Usage: "$0 "[users.txt]";
exit 1
else
TXT_FILE=$1
fi
if [ ! -f $CYR_ADDUSER ]; then
echo "Can't open cyr_adduser - "$TXT_FILE
exit 1
fi
if [ ! -f $TXT_FILE ]; then
echo "Can't open file - "$TXT_FILE
exit 1
fi
what="user";
for item in $( cat $TXT_FILE ); do
echo "--------------------------------------\\";
if [ $what = "user" ]; then
user=$item;
what="passwd";
else
passwd=$item;
${CYR_ADDUSER} $user $passwd
what="user";
fi
done After having these in place you simply can call the shellscript with a filename as parameter and watch the mailboxes and passwords being created, voila!
Cyrus doesn't support virtual hosting intrinsically, i.e. the domain name of a user is not even passed onto cyrus, just the local part. Cyrus doesn't 'see' <jdoe@domain.org> but only jdoe. This will become a problem when you've got two domains and each wants an email address such as <info@domain.org> and <info@otherdomain.com>.
That's why it's a good idea to employ a strict scheme when issueing IMAP accounts. We use the following:
localpart_domainname_tld
i.e. <tom@tomster.org> gets the mailbox tom_tomster_org. This way you can always be sure that a) boxnames are unique and b) you can infer the emailaddress from the boxname. Mapping email addresses to mailboxes happens in exim with a virtusertable (See exim docu here).
With cloning a user I mean transfering a cyrus account from one machine to another. This is done by copying the users mailspool directory and then rebuilding the database. I.e. to clone the user tom_tomster_org you would copy /var/spool/imap/user/tom_tomster_org to the new machine (as cyrus, ie. scp -rC /var/spool/imap/user/tom_tomster_org cyrus@remote.host.tld:/var/spool/imap/user/ and then, on the remote host, again as user cyrus run /usr/local/cyrus/bin/reconstruct -rf user.tom_tomster_org.
Use cyradm. To delete a user means to delete his/her IBOX (user.<username>) You first need to give yourself the right to that and then proceed with actually deleting it:
su - cyrus /usr/local/bin/cyradm localhost <enter password of cyrus user> sam user.<username> cyrus d dm user.<username>
Backing up a cyrus installation boils down to making copies of the config directory and the spool directory. In the default setup, they are these:
CONF_PATH=/var/imap SPOOL_PATH=/var/spool/imap
What we want to do here, is to use rsync to keep two such folders in sync (one-way, of course) between two hosts, SOURCE_HOST and TARGET_HOST. We do this by writing a shell script, that performs the necessary actions and then by installing a cronjob on the TARGET_HOST, that will execute this script periodically. For the script to have automated access to the SOURCE_HOST (i.e. without having to enter a password) we will create a key pair for the user cyrus on the TARGET_HOST and add its public key to the SOURCE_HOST.
SOURCE_HOST=mail.tomster.org
On the TARGET_HOST, become user cyrus (first make sure, /usr/local/cyrus belongs to cyrus) and create a key:
sudo chown -R cyrus /usr/local/cyrus sudo su - cyrus ssh-keygen -b 2048 -t dsa
Don't enter a passphrase and wait until ssh-keygen has finished doing its voodoo. (Depending on the CPU of your machine, this could take a while...)
Now we need to add the public to the list of authorized keys on the SOURCE_HOST:
scp /usr/local/cyrus/.ssh/id_dsa.pub username@SOURCE_HOST:/tmp/
On the SOURCE_HOST, become cyrus and add the public key:
sudo su - cyrus cat /tmp/id_dsa.pub > ~/.ssh/authorized_keys
Now, from the TARGET_HOST try to connect to the SOURCE_HOST as user cyrus via ssh. You should get a login prompt without having to enter a password.
Next, on the TARGET_HOST create the target directories:
% mkdir /var/imap % mkdir /var/spool/imap % chown cyrus /var/imap/ % chown cyrus /var/spool/imap/
Now we are ready to write the actual backup script: (save it at /usr/local/cyrus/tools/rsync-imap.sh)
#!/bin/sh SOURCE_HOST=mail.tomster.org CONF_PATH=/var/imap SPOOL_PATH=/var/spool/imap #make sure that cyrus is *not* running: /usr/local/etc/rc.d/imapd.sh stop echo 'Beginning sync at ' `date`'.' echo 'Syncing from '$SOURCE_HOST ' to '$CONF_PATH rsync -rpogze ssh cyrus@$SOURCE_HOST:$CONF_PATH/ $CONF_PATH echo 'Done.' echo 'Syncing from '$SOURCE_HOST ' to '$SPOOL_PATH rsync -rpogze ssh cyrus@$SOURCE_HOST:$SPOOL_PATH/ $SPOOL_PATH echo 'Done.' echo 'Finished sync at ' `date`'.'
Note
Take care with those slashes in the two rsync statements and PATH definitions! Add and leave them exactly as shown.
Note
Depending on the size of your spool directory and the power of your CPUs this operation can put quite a strain on your ressources. Especially on the first run! However, if you haven't got a flat rate for both(!) machines then you really should keep the -z flag in the above rsunc statements to enable compression! Although the most data will stem from attachments (which are typically already compressed i.e. as .zip or .jpg) and not from messages (that compress well) it is worth noticing that the attachments are uuencoded and thus benefit from compression after all.
Run the above script (you might want to add a -v flag in the rsync statements for debugging and monitoring and remove them later again).
With everyhthing in place, we are now ready to make it all happen automatically. As user cyrus on TARGET_HOST create the following crontab entry by issueing crontab -e:
SHELL=/bin/sh MAILTO=admin@tomster.org 0 4 * * * $HOME/tools/rsync-imap.sh >> $HOME/rsync.log 2>&1
This will execute the backupscript every morning at 4 a.m. Adjust for your needs. man 5 crontab is your friend ;-)
Sieve is one of the things that make IMAP great: its a protocol for server-side mailfiltering! Each IMAP user can install a script (i.e. a set of filter-commands) that will be executed with each delivery. The tool to interact with sieve is called sieveshell. It can be used interactively (great for debugging) or in conjunction with pre-written scripts.
To use sieve you must consider the following. Firstly, check that /usr/local/etc/imap.conf has the following:
# Note that duplicate delivery suppression is required for Sieve. # Disabling duplicate delivery suppression will also disable Sieve, # and as such should only be disabled for performance reasons. # duplicatesuppression: yes [...] # If sieveusehomedir is false, this directory is searched for Sieve scripts. # The active Sieve script is s called "default", placed in the users sieve # sieve directory (ie. /var/imap/sieve/u/user). # sievedir: /var/imap/sieve # The pathname of the sendmail executable. Sieve uses sendmail for # sending rejections, redirects and vaca- tion responses. # sendmail: /usr/sbin/sendmail
In /usr/local/etc/cyrus.conf make sure you have an entry for the sieve protocol:
sieve cmd="timsieved" listen="sieve" prefork=0
And in /etc/services this:
sieve 2000/tcp
After this, you will need to restart cyrus (but nothing else).
The neat thing is, that you (a.k.a. user cyrus or any other designated cyrus user with admin privileges) can install scripts for other users:
sieveshell -u cyrus -a <cyrus_user> localhost
connecting to localhost
Please enter your password:<enter cyrus's password!>
> help
Usage:
sieveshell [-u username] [-a authname] [-r realm] <server>
help - this screen
list - list scripts on server
put <filename> [<target name>]
- upload script to server
get <name> [<filename>]
- get script. if no filename display to stdout
delete <name> - delete script.
activate <name> - set a script as the active script
deactivate - deactivate all scripts
quit - quit
To install a script sieve.txt do the following:
put sieve.txt activate sieve.txt quit
The following sample script is taken from the sieve homepage where you can find more for inspiration...
require "fileinto";
if header :is "X-Mailinglist" "suse-linux" {
fileinto "INBOX.Listen.suse-linux";}
elsif header :contains "Mailing-List" "reiserfs" {
fileinto "INBOX.Listen.reiserfs";}
elsif address :contains :all ["to", "cc", "bcc"] "free-clim" {
fileinto "INBOX.Listen.free-clim";}
elsif header :contains "List-Id" "gnupg-users.gnupg.org" {
fileinto "INBOX.Listen.gnupg";}
elsif header :is "X-loop" "isdn4linux" {
fileinto "INBOX.Listen.isdn4linux";}
elsif header :contains "Mailing-list" "qmail-help@list.cr.yp.to"{
fileinto "INBOX.Listen.qmail";}
elsif allof (header :contains "Sender" "owner-info-cyrus@list",
address :contains :localpart ["to", "cc", "bcc"] "info-cyrus"){
fileinto "INBOX.Listen.info-cyrus";}
elsif header :contains "Sender" "ntbugtraq@listserv"{
fileinto "INBOX.Listen.ntbugtraq";}
elsif header :is "list-id" "<ietf-mta-filters.imc.org>"{
fileinto "INBOX.Listen.sieve";}
elsif header :contains "From" "securityportal-l@listserv.securityportal.com"{
fileinto "INBOX.Newsletter.securityportal";}
elsif address :contains :all ["from"] "newsletter@ebay"{
fileinto "INBOX.Newsletter.ebay";}
elsif address :contains :all ["to", "cc", "bcc"] "allegro-cl@cs.berkeley.edu"{
fileinto "INBOX.Listen.allegro-cl";}
elsif address :contains :all ["to", "cc", "bcc"] "plob@lisp.de"{
fileinto "INBOX.Listen.plob";}
else {
fileinto "INBOX";
Here I collect some things I've found helpful in dealing with mailtrouble.
If a user's mailclient has crashed during a pop-session it can happen, that the pop-lockfile won't be deleted. POP doesn't allow for simultaneous sessions. The easiest way to unlock it is simply to send another message to that account. Upon arrival the deserted lock-file will be deleted. The symptom is, that POP clients can't log in and that IMAP clients can log in, but can't delete any mails in the INBOX.
Previous:
2. Exim
