User:Davezerave/Mailman-web

From ArchWiki

Mailman-web is a project based on django, hyperkitty and postorius that serves as a convenient way to create a unified archiver and management interface to a Mailman instance.

Installation

To use mailman-web, a working web server setup is required (e.g. using Apache HTTP Server to forward to the WSGI directly, or using Nginx forwarding requests to an application server such as UWSGI).

Install the mailman-web package.

Warning: Mailman-web should only be accessed over TLS (unless only accessed directly from the machine running it for testing purposes), as it otherwise exposes passwords and user data to the network.

Configuration

The web application is configured in /etc/webapps/mailman-web/settings.py (which is included by the default configurations in base.py and mailman.py).

Note: Mailman-web should store user sensitive data (e.g. sqlite database) in /var/lib/mailman-web/data/, as that directory is only accessible by root and the application itself.

Change the default secret for the application:

/etc/webapps/mailman-web/settings.py
SECRET_KEY = 'something-very-secret'

Make sure to disable debugging when running in production:

/etc/webapps/mailman-web/settings.py
DEBUG = False

Add a valid email configuration (so that the Django application can verify subscribers):

/etc/webapps/mailman-web/settings.py
EMAIL_BACKEND = 'django.core.mail.backends.smtp.EmailBackend'
EMAIL_HOST = 'localhost'
EMAIL_PORT = 25
EMAIL_HOST_USER = username
EMAIL_HOST_PASSWORD = password
Note: The DEFAULT_FROM_MAIL and SERVER_MAIL configuration options can be used to define the From: header of mails sent for internal authentication and error reporting, respectively.

To connect with a running mailman instance's REST API, configuration options have to be added to mailman-web's configuration.

/etc/webapps/mailman-web/settings.py
MAILMAN_REST_API_URL = 'http://localhost:8001'
MAILMAN_REST_API_USER = 'rest_admin'
MAILMAN_REST_API_PASS = 'rest_admin_password'

To configure the archive integration with a mailman instance first setup the integration with hyperkitty on mailman's side and then configure mailman-web to accept those connections:

/etc/webapps/mailman-web/settings.py
MAILMAN_ARCHIVER_KEY = 'SecretArchiverAPIKey'
MAILMAN_ARCHIVER_FROM = ('127.0.0.1', '::1')

The valid hosts or domain names for the application need to be defined:

/etc/webapps/mailman-web/settings.py
ALLOWED_HOSTS = [
    'localhost',
    'lists.example.com'
]

Hosting

This article or section needs expansion.

Reason: Example for a setup using Apache HTTP Server. (Discuss in User talk:Davezerave/Mailman-web)
Note:
  • Mailman-web needs to be run as its own user and group (i.e. mailman-web). It is using /etc/webapps/mailman-web/, /var/lib/mailman-web/ and /run/mailman-web/ for configurations, static caches and (potentially) sockets, respectively.

Nginx and uWSGI

Mailman-web comes with a working uWSGI configuration file in /etc/uwsgi/mailman-web.ini.

Install nginx and uwsgi-plugin-python, create a per-application socket for uWSGI (see UWSGI#Accessibility of uWSGI socket for reference) and activate the uwsgi-secure@mailman-web.socket unit.

For a local test setup, serving mailman-web at http://localhost/mailman-web/ add the following Nginx configuration to your setup:

/etc/nginx/mailman-web.conf
server {
  listen 80;
  server_name localhost;
  charset utf-8;
  client_max_body_size 75M;
  root /usr/share/webapps/mailman-web;
  access_log /var/log/nginx/access.mailman-web.log;
  error_log /var/log/nginx/error.mailman-web.log;

  location /hyperkitty_static {
    alias /var/lib/hyperkitty/static;
  }

  location ~^/(hyperkitty|user-profile)/(.*)$ {
    include /etc/nginx/uwsgi_params;
    uwsgi_pass unix:/run/mailman-web/mailman-web.sock;
  }
}

Setup

Note: Run the following commands as the mailman-web user (e.g. using sudo or su).

After first installation make sure to generate a database:

[mailman-web]$ django-admin migrate --pythonpath /usr/share/webapps/mailman-web/ --settings settings

Afterwards, the static data for the application needs to be collected:

[mailman-web]$ django-admin collectstatic --pythonpath /usr/share/webapps/mailman-web/ --settings settings

To compress the data, run the following:

[mailman-web]$ django-admin compress --pythonpath /usr/share/webapps/mailman-web/ --settings settings

Enable and start the mailman-web-qcluster.service systemd service for required asynchronous operations on the web application.

Populate the database with default data (when setting up for the first time):

[mailman-web]$ django-admin loaddata --pythonpath /usr/share/webapps/mailman-web/ --settings settings first_start

Create a superuser account for the Django application:

[mailman-web]$ django-admin createsuperuser --pythonpath /usr/share/webapps/mailman-web --settings settings

Log in to the admin interface of the Django application at http://localhost/mailman-web/admin to be able to add more Sites besides the default example.com or to add additional Mail domains.

Note: After adding a new site make sure to set the SITE_ID variable in /etc/webapps/mailman-web/settings.py to the respective ID!

Tips and tricks

Importing mailman2 archives

Hyperkitty can import archives from mailman < 3.0.

Note: Run the following commands as the mailman-web user (e.g. using sudo or su).
[mailman-web]$ django-admin hyperkitty_import --pythonpath /usr/share/webapps/mailman-web --settings settings -l ADDRESS mbox_file [mbox_file ...]

Here ADDRESS is the fully-qualified list name (e.g. list@example.com) and the mbox_file argument represents existing archives (in mbox format) to import (usually found in /var/lib/mailman/archives/private/LIST_NAME.mbox/LIST_NAME.mbox).

Afterwards the full-text search index can be updated manually:

[mailman-web]$ django-admin update_index_one_list --pythonpath /usr/share/webapps/hyperkitty --settings settings ADDRESS
Note: The full-text search index should be created by the minutely running cron-job automatically.

Content-Security-Policy Header

It is possible to define a Content-Security-Policy HTTP response header for mailman-web. This allows content and resources only from specified locations.

When using nginx add the following to the server directive in which mailman-web is setup in:

/etc/nginx/mailman-web.conf
[..]
add_header Content-Security-Policy "default-src 'self'; connect-src 'self'; img-src 'self' https://secure.gravatar.com; script-src 'self' 'unsafe-eval' 'unsafe-inline'; style-src 'self' 'unsafe-inline'";
[..]

Disabling Gravatar support

The builtin Gravatar support can be disabled in the configuration:

/etc/webapps/mailman-web/settings.py
HYPERKITTY_ENABLE_GRAVATAR = False

This option was introduced in hyperkitty 1.3.4. In earlier versions, use this instead:

/etc/webapps/mailman-web/settings.py
GRAVATAR_SECURE_URL = ''

Saving mail attachments to disk

By default mailman-web stores mail attachments in its database. However, it can be configured to save the attachments to disk instead:

/etc/webapps/mailman-web/settings.py
HYPERKITTY_ATTACHMENT_FOLDER = /var/lib/mailman-web/data/attachments
Note: The location needs to be accessible and writable by the mailman-web user.

Template customization

Using Django's TEMPLATES-DIRS capabilities, it is possible to override the following templates to change the looks of the application:

  • hyperkitty/headers.html: the content will appear before the </head> tag
  • hyperkitty/top.html: the content will appear before the <body> tag
  • hyperkitty/bottom.html: the content will appear before the </body> tag

Xapian search backend

Mailman-web can make use of a Xapian based search backend. Install the python-xapian-haystack package and configure the backend:

/etc/webapps/mailman-web/settings.py
HAYSTACK_CONNECTIONS = {
    'default': {
        'ENGINE': 'xapian_backend.XapianEngine',
        'PATH': "/var/lib/mailman-web/data/xapian_index",
    },
}

Make sure to create the search index for all lists afterwards. Run the following command as the mailman-web user (e.g. using sudo or su):

[mailman-web]$ django-admin update_index --pythonpath /usr/share/webapps/mailman-web --settings settings

Troubleshooting

SMTP AUTH extension not supported by server

If upon first login with the admin user a SMTNotSupportedError is thrown (logged to /var/log/mailman-web/mailman-web.log), make sure to verify the EMAIL_* settings in /etc/webapps/mailman-web/settings.py.

By default e.g. EMAIL_USE_TLS is set to False which might trigger a failed login via SMTP.

See also