From ArchWiki
Revision as of 22:30, 11 July 2018 by Progandy (talk | contribs) (Configuring: Link to possible XOAUTH2 login procedure. Method not fully tested and therefore only linked.)
(diff) ← Older revision | Latest revision (diff) | Newer revision → (diff)
Jump to: navigation, search

isync is a command line application to synchronize mailboxes; it supports Maildir and IMAP4 mailboxes. New messages, message deletions and flag changes can be propagated both ways.

Synchronization is based on unique message identifiers (UIDs), so no identification conflicts can occur (as opposed to some other mail synchronizers). Synchronization state is kept in one local text file per mailbox pair; multiple replicas of a mailbox can be maintained.

Note: isync is the name of the project, mbsync is the name of the executable


Install the isync package.


Note: Google appears to block isync from downloading emails by default. If you have 2-step authentication enabled, you need to set up an app password and use that with isync, otherwise you need to go to Google's Security Page and toggle "Allow less secure apps" to "on".
Note: It might be possible to use XOAUTH2 to authenticate with Google. Some hints are in this forum thread
Note: Subfolders setting in MaildirStore now seems to be required to be set: iSync Config SubFolders SubFolders Legacy worked as previous unset - Oct 2017

First create and customize the main configuration file using this example ~/.mbsyncrc:

IMAPAccount gmail
# Address to connect to
Pass ***************
# To store the password in an encrypted file use PassCmd instead of Pass
# PassCmd "gpg2 -q --for-your-eyes-only --no-tty -d ~/.mailpass.gpg"
# Use SSL
# The following line should work. If get certificate errors, uncomment the two following lines and read the "Troubleshooting" section.
CertificateFile /etc/ssl/certs/ca-certificates.crt
#CertificateFile ~/.cert/
#CertificateFile ~/.cert/Equifax_Secure_CA.pem

IMAPStore gmail-remote
Account gmail

MaildirStore gmail-local
Subfolders Verbatim
# The trailing "/" is important
Path ~/.mail/gmail/
Inbox ~/.mail/gmail/Inbox

Channel gmail
Master :gmail-remote:
Slave :gmail-local:
# Exclude everything under the internal [Gmail] folder, except the interesting folders
Patterns * ![Gmail]* "[Gmail]/Sent Mail" "[Gmail]/Starred" "[Gmail]/All Mail"
# Or include everything
#Patterns *
# Automatically create missing mailboxes, both locally and on the server
Create Both
# Save the synchronization state files in the relevant directory
SyncState *

To get rid of the [Gmail]-Stuff (or [Google Mail] as in my case) in each mailbox name, it's possible to use separate Channels for each directory, and later merge them to a group:

Channel sync-googlemail-default
Master :googlemail-remote:
Slave :googlemail-local:
# Select some mailboxes to sync
Patterns "INBOX" "arch"

Channel sync-googlemail-sent
Master :googlemail-remote:"[Google Mail]/Gesendet"
Slave :googlemail-local:sent

Channel sync-googlemail-trash
Master :googlemail-remote:"[Google Mail]/Papierkorb"
Slave :googlemail-local:trash

# Get all the channels together into a group.
Group googlemail
Channel sync-googlemail-default
Channel sync-googlemail-sent
Channel sync-googlemail-trash

As you can see, name-translations are possible this way, as well.


First make any folders that were specified as Maildirs.

$ mkdir -p ~/.mail/gmail

Then to retrieve the mail for a specific channel run:

$ mbsync gmail

or to retrive the mail for all channels:

$ mbsync -a

Tips and tricks

Automatic synchronization

If you want to automatically synchronize your mailboxes, isync can be started automatically with a systemd/User unit. The following service file can start the mbsync command:

Description=Mailbox synchronization service

ExecStart=/usr/bin/mbsync -Va
Note: It's possible this service could trigger without an internet connection. A solution is to add the following into the Unit section: dbus.socket

The following timer configures mbsync to be started 2 minutes after boot, and then every 5 minutes:

Description=Mailbox synchronization timer



Once those two files are created, reload systemd, then enable and start mbsync.timer, adding the --user flag to systemctl.

Tip: The mbsync service now only runs after login. It's also possible to launch the systemd-user instances after boot if you configure Systemd/User#Automatic start-up of systemd user instances.

Integration with notmuch or mu4e

If you want to run notmuch or mu/mu4e after automatically synchronizing your mails, it is preferable to modify the above mbsync.service by adding a post-start hook, like below:

Description=Mailbox synchronization service

ExecStart=/usr/bin/mbsync -Va
ExecStartPost=/usr/bin/notmuch new

You can also index mu by changing the ExecStartPost line to ExecStartPost=/usr/bin/mu index, or to ExecStartPost=/usr/bin/emacsclient -e '(mu4e-update-index)' if you are running emacsclient and would like to index mu4e.

This modification assumes that you have already setup notmuch or mu/mu4e for your user. If the ExecStart command does not execute successfully, the ExecStartPost command will not execute, so be aware of this!


SSL error

If you get certificate related errors like

SSL error connecting ( error:00000012:lib(0):func(0):reason(18) 

you may need to retrieve the server's certificates manually in order for mbsync to correctly verify it.

Step #1: Get the certificates

Tango-inaccurate.pngThe factual accuracy of this article or section is disputed.Tango-inaccurate.png

Reason: This may not always be needed, e.g. for gmail CertificateFile /etc/ssl/certs/ca-certificates.crt in the config file may be suffcient (Discuss in Talk:Isync#Step #1: Get the certificates)

$ mkdir ~/.cert
$ openssl s_client -connect some.imap.server:port -showcerts 2>&1 < /dev/null | sed -ne '/-BEGIN CERTIFICATE-/,/-END CERTIFICATE-/p' | sed -ne '1,/-END CERTIFICATE-/p' > ~/.cert/some.imap.server.pem

This will create a certificate file called ~/.cert/some.imap.server.pem (e.g. ~/.cert/ Alternatively one can download and run it:

$ mkdir ~/.cert
$ wget
$ sh some.imap.server port ~/.cert/

If you wish to do this manually, you may enter:

$ openssl s_client -connect some.imap.server:port -showcerts

and it will display output something like:

depth=1 C = US, O = Google Inc, CN = Google Internet Authority
verify error:num=20:unable to get local issuer certificate
verify return:0
Certificate chain
 0 s:/C=US/ST=California/L=Mountain View/O=Google Inc/
   i:/C=US/O=Google Inc/CN=Google Internet Authority
 1 s:/C=US/O=Google Inc/CN=Google Internet Authority
   i:/C=US/O=Equifax/OU=Equifax Secure Certificate Authority
Server certificate
subject=/C=US/ST=California/L=Mountain View/O=Google Inc/
issuer=/C=US/O=Google Inc/CN=Google Internet Authority
No client certificate CA names sent
SSL handshake has read 2108 bytes and written 350 bytes
New, TLSv1/SSLv3, Cipher is ECDHE-RSA-RC4-SHA
Server public key is 1024 bit
Secure Renegotiation IS supported
Compression: NONE
Expansion: NONE
    Protocol  : TLSv1.1
    Cipher    : ECDHE-RSA-RC4-SHA
    Session-ID: 77136647F42633D82DEDFBB9EB62AB516547A3697D83BD1884726034613C1C09
    Master-Key: 635957FBA0762B10694560488905F73BDD2DB674C41970542ED079446F27234E2CA51CF26938B8CA56DF5BBC71E429A7
    Key-Arg   : None
    PSK identity: None
    PSK identity hint: None
    SRP username: None
    TLS session ticket lifetime hint: 100800 (seconds)
    TLS session ticket:
    0000 - d6 5b a0 a7 10 0e 64 04-72 93 7c 9f 94 fa 07 57   .[....d.r.|....W
    0010 - f1 8b 9d 24 8b 9d 1b f3-a8 b1 4d 2c a9 00 e1 82   ...$......M,....
    0020 - 00 83 1e 3f e5 f2 b2 2c-d2 a8 87 83 16 02 0d 1e   ...?...,........
    0030 - bf b6 c1 d6 75 21 04 e6-63 6b ab 5b ed 94 7a 30   ....u![..z0
    0040 - 1a d0 aa 44 c2 04 9b 10-06 28 b5 7b a0 43 a6 0d   ...D.....(.{.C..
    0050 - 3b 4a 85 1f 2e 07 0a e1-32 9b bd 5d 65 41 4c e2   ;J......2..]eAL.
    0060 - 7c d7 43 ec c4 18 77 53-b5 d4 84 b4 c9 bd 51 d6   |.C...wS......Q.
    0070 - 2d 4f 2e 10 a6 ed 38 c5-8e 9d f8 8b 8a 63 3f 7b   -O....8......c?{
    0080 - ee e6 b8 bf 7a f8 b8 e8-47 92 84 f1 9b 0c 63 30   ....z...G.....c0
    0090 - 76 d8 e1 44                                       v..D

    Start Time: 1352632558
    Timeout   : 300 (sec)
    Verify return code: 20 (unable to get local issuer certificate)
* OK Gimap ready for requests from o67if11168976yhc.67

Simply copy the first block that begins with -----BEGIN CERTIFICATE----- and ends with -----END CERTIFICATE-----, paste into a file, and save with a .pem extension (this is necessary for the next step). Older instructions state that, with Gmail, both certificate blocks must be saved but on testing this was found to be unnecessary.

Now, copy the root issuer certificate to your local certificate folder. In this example (Gmail), the root issuer is Equifax Secure Certificate Authority. This certificate is included in the ca-certificates package.

cp /usr/share/ca-certificates/mozilla/Equifax_Secure_CA.crt ~/.cert/Equifax_Secure_CA.pem

Step #2: Setup mbsync

Configure mbsync to use that certificate:

IMAPAccount gmail
# ...
CertificateFile ~/.cert/

BAD Command with Exchange 2003

When connecting to an MS Exchange 2003 server, there could be problems when using pipelining (i.e. executing multiple imap commands concurrently). Such an issue could look as follows:

sample output of `mbsync -V exchange'

>>> 9 SELECT "arch"^M
* 250 EXISTS
* FLAGS (\Seen \Answered \Flagged \Deleted \Draft $MDNSent)
* OK [PERMANENTFLAGS (\Seen \Answered \Flagged \Deleted \Draft $MDNSent)] Permanent flags
* OK [UNSEEN 241] Is the first unseen message
9 OK [READ-WRITE] SELECT completed.
>>> 10 UID FETCH 1:1000000000 (UID FLAGS)^M
* 1 FETCH (UID 1 FLAGS (\Seen \Answered))
* 2 FETCH (UID 2 FLAGS (\Seen \Answered))
* 249 FETCH (UID 696 FLAGS ())
* 250 FETCH (UID 697 FLAGS (\Seen))
10 OK FETCH completed.
>>> 11 APPEND "arch" (\Seen) {4878+}^M
(1 in progress) >>> 12 UID FETCH 697 (BODY.PEEK[])^M
(2 in progress) >>> 13 UID STORE 696 +FLAGS.SILENT (\Deleted)^M
12 BAD Command is not valid in this state.

So command 9 is to select a new folder, command 10 checks the mail and commands 11, 12 and 13 run in parallel, writing/getting/flagging a mail. In this case, the Exchange server would terminate the connection after the BAD return value and go on to the next channel. (And if all went well in this channel, mbsync would return with 0.) After setting

PipelineDepth 1

in the IMAPStore config part of the Exchange, this problem did not occur any more.

External links