OfflineIMAP

OfflineIMAP is an utility to sync mail from IMAP servers. It does not work with the POP3 protocol or mbox, and is usually paired with a MUA such as Mutt.

As of 06/26/2010, the original author of offlineimap ceased development. A new maintainer has since stepped forward and as of 12/16/2010 developed seemed to be active.

Installing

Install the offlineimap package with pacman:

# pacman -S offlineimap

Configuring

Offlineimap is distributed with two default configuration files, which are both located in /usr/share/offlineimap/. offlineimap.conf contains every setting and is thorougly documented. Alternatively, offlineimap.conf.minimal is is not commented and only contains a small number of settings (see: Minimal).

Copy one of the default configuration files to ~/.offlineimaprc.

Note: Writing a comment after an option/value on the same line is invalid syntax, hence take care that comments are placed on their own separate line.

Minimal

The following file is a commented version of offlineimap.conf.minimal.

~/.offlineimaprc

[general]
# This should contain a comma delimited list of all identifiers of the accounts
# that are to be synced
accounts = main
# If there are two accounts; `main' and `alternative'...
#accounts = main,alternative

[Account main]
# The identifier for the local repository; e.g., the maildir that offlineimap
# will sync with an IMAP server
localrepository = main-local
# The identifier for the remote repository. This is the actual IMAP, which is
# usually foreign to the system
remoterepository = main-remote

[Repository main-local]
# Currently, offlineimap only supports maildir and IMAP for local repositories
type = Maildir
# Where should the mail be placed?
localfolders = ~/Maildir

[Repository main-remote]
# Remote repos can be IMAP or Gmail, the latter being a preconfigured IMAP
type = IMAP
remotehost = host.domain.tld
remoteuser = username

Usage

Before running offlineimap, create any parent directories that were allocated to local repositories:

$ mkdir ~/Maildir

Now, run the program:

$ offlineimap

Mail accounts will now be synced. If anything goes wrong, take a closer look at the error messages. OfflineIMAP is usually very verbose about problems; partly because the developers didn't bother with taking away tracebacks from the final product.

Miscellaneous

Various settings and improvements

Running offlineimap in the background

Most other mail transfer agents assume that the user will be using the tool as a daemon by making the program sync periodically by default. In offlineimap, there are a few settings that control backgrounded tasks.

Confusingly, they are spread thin all-over the configuration file:

~/.offlineimaprc

# In the general section
[general]
# Controls how many accounts may be synced simultaneously
maxsyncaccounts = 1

# In the account identifier
[Account main]
# Minutes between syncs
autorefresh = 5
# Number of quick-syncs between autorefreshes. Quick-syncs do not update if the
# only changes were to IMAP flags
quick = 10

# In the remote repository identifier
[Repository main-remote]
# Instead of closing the connection once a sync is complete, offlineimap will
# send empty data to the server to hold the connection open. A value of 60
# attempts to hold the connection for a minute between syncs (both quick and
# autorefresh)
keepalive = 60

cron job

1. Configure background jobs as shown in #Running offlineimap in the background.

2. Use a log-friendly user interface:

~/.offlineimaprc

[general]
# This will suppress anything but errors
ui = Noninteractive.Quiet

3. Write a wrapper script for daemonizing programs, or use the one shown below:

/usr/local/bin/start_daemon

#!/bin/sh

set -efu

ionice_class=
ionice_priority=
nice=

while getopts c:p:n: f; do
	case $f in
	c) ionice_class=$OPTARG;;
	p) ionice_priority=$OPTARG;;
	n) nice=$OPTARG;;
	*) exit 2;;
	esac
done
shift $((OPTIND - 1))

cmd=$*
io=

if pgrep -u "$(id -u)" -xf -- "$cmd" >/dev/null 2>&1; then
	exit 0
fi

if type ionice >/dev/null 2>&1; then
	[ -n "$ionice_class" ]    && { io=1; cmd="-c $ionice_class $cmd"; }
	[ -n "$ionice_priority" ] && { io=1; cmd="-n $ionice_priority $cmd"; }
	[ -n "$io" ] && cmd="ionice $cmd"
fi

if type nice >/dev/null 2>&1; then
	[ -n "$nice" ] && cmd="nice -n $nice $cmd"
fi

exec $cmd

4. Finally, add a crontab entry:

$ crontab -e

The line should specify the interpreter for correct pgrep -xf results:

*/5 * * * * exec /usr/local/bin/start_daemon -n19 -c2 -p7 python2 /usr/bin/offlineimap 

If you are using offlineimap-git from AUR, you'll want to change python2 to python

The wrapper script is needed because offlineimap has the tendency to sporadically crash, even more so when facing connection problems.

Automatic mutt mailbox generation

Mutt cannot be simply pointed to an IMAP or maildir directory and be expected to guess which subdirectories happen to be the mailboxes, yet offlineimap can generate a muttrc fragment containing the mailboxes that it syncs.

~/.offlineimaprc

[mbnames]
enabled = yes
filename = ~/.mutt/mailboxes
header = "mailboxes "
peritem = "+%(accountname)s/%(foldername)s"
sep = " "
footer = "\n"

Then add source ~/.mutt/mailboxes to ~/.mutt/muttrc.

Gmail configuration

This remote repository is configured specifically for Gmail support, substituting folder names in uppercase for lowercase, among other small additions. Keep in mind that this configuration does not sync the All Mail folder, since it is usually unnecessary and skipping it prevents bandwidth costs:

~/.offlineimaprc

[Repository gmail-remote]
type = Gmail
remoteuser = user@gmail.com
remotepass = password
nametrans = lambda foldername: re.sub ('^\[gmail\]', 'bak',
                               re.sub ('sent_mail', 'sent',
                               re.sub ('starred', 'flagged',
                               re.sub (' ', '_', foldername.lower()))))
folderfilter = lambda foldername: foldername not in '[Gmail]/All Mail'

Note: if you have Gmail set to another language, the folder names may appear translated too, e.g. "verzonden_berichten" instead of "sent_mail".

Note: after version 6.3.5, offlineimap also creates remote folders to match your local ones. Thus you may need a nametrans rule for your local repository too that reverses the effects of this nametrans rule.

Not having to enter the password all the time

.netrc

Add the following lines to your {ic|~/.netrc}:

   machine hostname.tld
   login [your username]
   password [your password]

Don't forget to give the file appropriate rights like 600 or 700:

  chmod 600 ~/.netrc

Gnome Keyring

http://www.clasohm.com/blog/one-entry?entry_id=90957 gives an example of how to use gnome-keyring to store the password.

python-keyring

There's a general solution that should work for any keyring. Install python-keyring from AUR, then change your ~/.offlineimaprc to say something like:

[general]
pythonfile = /home/user/offlineimap.py
…
[Repository RemoteEmail]
remoteuser = username@host.net
remotepasseval = keyring.get_password("offlineimap","username@host.net")
…

and somewhere in ~/offlineimap.py add import keyring. Now all you have to do is set your password, like so:

$ python2 
>>> import keyring
>>> keyring.set_password("offlineimap","username@host.net", "MYPASSWORD")'

and it'll grab the password from your (kwallet/gnome-) keyring instead of having to keep it in plaintext or enter it each time.

Kerberos authentication

Install python2-kerberos from AUR and do not specify remotepass in your .offlineimaprc. OfflineImap figure out the reset all if have a valid Kerberos TGT. If you have 'maxconnections', it will fail for some connection. Comment 'maxconnections' out will solve this problem.

Troubleshooting

Overriding UI and autorefresh settings

For the sake of troubleshooting, it is sometimes convenient to launch offlineimap with a more verbose UI, no background syncs and perhaps even a debug level:

$ offlineimap [ -o ] [ -d <debug_type> ] [ -u <ui> ]

-o

Disable autorefresh, keepalive, etc.

-d <debug_type>

Where <debug_type> is one of imap, maildir or thread. Debugging imap and maildir are, by far, the most useful.

-u <ui>

Where <ui> is one of CURSES.BLINKENLIGHTS, TTY.TTYUI, NONINTERACTIVE.BASIC, NONINTERACTIVE.QUIET or MACHINE.MACHINEUI. TTY.TTYUI is sufficient for debugging purposes.

Note: More recent versions use the following for <ui>: blinkenlights, ttyui, basic, quiet or machineui.

Curses interface (Curses.Blinkenlights) locks terminal

Because of a bug in Python's ncurses package (http://bugs.python.org/issue7567) the Curses interface of OfflineIMAP breaks the terminal on exit. While it appears to irreparably lock the terminal, in reality it only prevents commands from being displayed. The bug has been fixed in Python's SVN for all versions 2.6 up to 3.2 but the current package in the repositories (Python 2.6.5) is still buggy.

In order to solve the issue:

• either append reset to OfflineIMAP's launch command:

$ offlineimap; reset

• or change the ui field in ~/.offlineimaprc to select a fully functional one:

ui = TTY.TTYUI

• or as quick workaround you can just use the following command to skip the reset() function in Curses.py which causes the problem

# sed -i '125i\ \ \ \ \ \ \ \ return' /usr/lib/python2.6/site-packages/offlineimap/ui/Curses.py

netrc authentication

There are some bugs in the current offlineimap which makes it impossible to read the authentication data from ~/.netrc if there are multiple Accounts per remote machine. ( see Mail Archive ) But they are fixed in the GIT package offlineimap-git^AUR (note that is AUR package is flagged as out of date; see the current GitHub external link below). Using the package you can collect all passwords in ~/.netrc. And do not forget to set it's access rights:

chmod 600 ~/.netrc

An example netrc file would be

~/.netrc

machine mail.myhost.de
    login mr_smith
    password secret

socket.ssl deprecation warnings

Depending on the currently installed python version, running offlineimap throws this warning:

DeprecationWarning: socket.ssl() is deprecated.  Use ssl.wrap_socket() instead.

This can be particularly annoying when offlineimap's output is being logged or mailed through cron.

To fix the problem, apply this upstream patch or install offlineimap-git^AUR:

--- offlineimap/imaplibutil.py.orig	2009-08-12 01:24:23.000000000 -0430
+++ offlineimap/imaplibutil.py		2010-06-07 11:17:37.849038683 -0430
@@ -23,9 +23,11 @@
 # Import the symbols we need that aren't exported by default
 from imaplib import IMAP4_PORT, IMAP4_SSL_PORT, InternalDate, Mon2num
 
-# ssl is new in python 2.6
-if (sys.version_info[0] == 2 and sys.version_info[1] >= 6) or sys.version_info[0] >= 3:
+try:
     import ssl
+    ssl_wrap = ssl.wrap_socket
+except ImportError:
+    ssl_wrap = socket.ssl
 
 class IMAP4_Tunnel(IMAP4):
     """IMAP4 client class over a tunnel
@@ -169,7 +171,7 @@
         if last_error != 0:
             # FIXME
             raise socket.error(last_error)
-        self.sslobj = socket.ssl(self.sock, self.keyfile, self.certfile)
+        self.sslobj = ssl_wrap(self.sock, self.keyfile, self.certfile)
         self.sslobj = sslwrapper(self.sslobj)
 
 mustquote = re.compile(r"[^\w!#$%&'+,.:;<=>?^`|~-]")

The diff is relative to the root buildir and it can be applied by using ABS.

External links

• Official OfflineIMAP mailing list
• Gnus, Dovecot, OfflineIMAP, search: a HOWTO
  • This setup worked for me, only difference being I had to add mail_location = maildir:~/Maildir to /etc/dovecot/dovecot.conf. Also, I used the Gmail configuration above. --Unhammer 09:24, 22 October 2010 (EDT)
• Mutt + Gmail + Offlineimap
  • An outline of brisbin's simple gmail/mutt setup using cron to keep offlineimap syncing.
• Current OfflineIMAP maintainer's fork on GitHub
  • Note that a strict build of this on current Arch will fail due to python references unless they are replaced with python2