User:Wolegis/Nextcloud

From ArchWiki

From Wikipedia:Nextcloud:

Nextcloud is a suite of client-server software for creating and using file hosting services. It is functionally similar to Dropbox, although Nextcloud is free and open-source, allowing anyone to install and operate it on a private server. In contrast to proprietary services like Dropbox, the open architecture allows adding additional functionality to the server in form of applications.

Nextcloud is a fork of ownCloud. For differences between the two, see Wikipedia:Nextcloud#Differences from ownCloud.

Setup overview

A complete installation of Nextcloud comprises (at least) the following components:

Web server  -->  Application server  -->  Nextcloud (PHP code)  -->  Database

This article will cover MariaDB/MySQL and PostgreSQL as databases and the following combinations of web server and application server:

  • nginx → uWSGI (plus uwsgi-plugin-php)
  • nginx → php-fpm,
  • Apache (using mod_proxy_uwsgi) → uWSGI (plus uwsgi-plugin-php)
  • Apache (using mod_proxy_fcgi) → php-fpm

The Nextcloud package complies with the web application package guidelines. Among other things this mandates that the web application to be run with a dedicated user - in this case nextcloud. This is one of the reasons why the application server comes into play here. For the very same reason it is not possible anymore to execute Nextcloud's PHP code directly in the Apache process by means of php-apache.

Installation

Install the nextcloud package. This will pull in quite a few dependent packages. All required PHP extensions will be taken care of this way. Additionally install the recommended package php-intl for increased translation performance and fixed sorting (preferrably as dependent package with pacman option --asdeps). Other optional dependencies will be covered later depending on your concrete setup (e.g. which database you choose).

Configuration

This guide does not tamper with PHP's central config file /etc/php/php.ini but instead puts Nextcloud specific PHP configuration in places where it does not potentially interfere with settings for other PHP based applications. These places are:

  • A dedicated copy of php.ini in /etc/webapps/nextcloud/php.ini (for the occ command line tool and the background job).
  • Corresponding settings in the configuration of the application server. These will be covered in the section about application servers.

Make a copy of /etc/php/php.ini in /etc/webapps/nextcloud. Although not strictly necessary change ownership of the copy:

chown nextcloud:nextcloud /etc/webapps/nextcloud/php.ini

Most of the prerequisites listed in Nextcloud's installation instructions are already enabled in a bare PHP installation. Additionally enable the following extensions:

/etc/webapps/nextcloud/php.ini
extension=bz2
extension=gd
extension=iconv
# and in case you also installed php-intl (as recommended)
extension=intl

Set date.timezone to your preferred timezone, e.g.:

/etc/webapps/nextcloud/php.ini
date.timezone = Europe/Berlin

Raise PHP's memory limit to at least 512MiB:

/etc/webapps/nextcloud/php.ini
memory_limit = 512M

Optional: For additional security configure open_basedir. This limits the locations where Nextcloud's PHP code can read and write files. Proven settings are

/etc/webapps/nextcloud/php.ini
open_basedir=/var/lib/nextcloud/data:/var/lib/nextcloud/apps:/tmp:/usr/share/webapps/nextcloud:/etc/webapps/nextcloud:/dev/urandom:/usr/lib/php/modules:/var/log/nextcloud:/proc/meminfo

Depending on which additional extensions you configure you may need to extend this list, e.g. /run/redis in case you opt for Redis.

It is not necessary to configure opcache here as this php.ini is only used by the occ command line tool and the background job, i.e. by short running PHP processes.

To make sure the Nextcloud specific php.ini is used by the occ tool set the environment variable NEXTCLOUD_PHP_CONFIG:

export NEXTCLOUD_PHP_CONFIG=/etc/webapps/nextcloud/php.ini

Also add this line to your .bashrc to make this setting permanent.

As a privacy and security precaution create the dedicated directory for session data:

install --owner=nextcloud --group=nextcloud --mode=700 -d /var/lib/nextcloud/sessions

Database

MariaDB/MySQL is the canonical choice for Nextcloud.

The MySQL or MariaDB databases are the recommended database engines.[1]

Most information concerning databases with Nextcloud deals with MariaDB/MySQL. The Nextcloud developers admit to have less detailed expertise with other databases.

PostgreSQL is said to deliver better performance and overall has fewer quirks compared to MariaDB/MySQL. SQLite is mainly supported for test / development installations and not recommended for production. The list of supported databases also contains Oracle Database. This product will not be covered here.

MariaDB / MySQL

Since MariaDB has been the default MySQL implementation in Arch Linux since 2013[2] this text only mentions MariaDB.

In case you want to run your database on the same host as Nextcloud install mariadb (if you haven't done so already). See the corresponding article for details. It is recommended for additional security to configure MariaDB to only listen on a local Unix socket:

/etc/my.cnf.d/server.cnf
[mysqld]
skip_networking
Note: Surprisingly Nextcloud is not compatible with MariaDB version 10.6 or higher (see FS#71549). This is due to MariaDB forcing read-only for compressed InnoDB tables[3] and Nextcloud using these kind of tables:
From MariaDB 10.6.0, tables that are of the COMPRESSED row format are read-only by default. This is the first step towards removing write support and deprecating the feature.

Upstream is aware of this problem but a quick fix seems unlikely.

One easy remedy for this issue is to allow write access to compressed InnoDB tables again by means of MariaDB's system variable innodb_read_only_compressed. Just add the following section to your configuration of MariaDB:

/etc/my.cnf.d/server.cnf
[mariadb-10.6]
innodb_read_only_compressed=OFF

Nextcloud's own documentation recommends to set the transaction isolation level to READ-COMMITTED. This is especially important when you expect high load with many concurrent transactions.

/etc/my.cnf.d/server.cnf
[mysqld]
transaction_isolation=READ-COMMITTED

The other recommendation to set binlog_format=ROW is obsolete. The default MIXED in recent MariaDB versions is at least as good as the recommended ROW. In any case the setting is only relevant when replication is applied.

Start the CLI tool mysql with database user root. (Default password is empty, but hopefully you change it as soon as possible.)

mysql -u root -p

Create the user and database for Nextcloud with

CREATE USER 'nextcloud'@'localhost' IDENTIFIED BY 'xxxxxxxx';
CREATE DATABASE IF NOT EXISTS nextcloud CHARACTER SET utf8mb4 COLLATE utf8mb4_general_ci;
GRANT ALL PRIVILEGES on nextcloud.* to 'nextcloud'@'localhost';
FLUSH privileges;

(XXXXXXXX is a placeholder for the actual password of DB user nextcloud you must choose.) Quit the tool with \q.

Note: MariaDB has a flawed understanding of what UTF8 means resulting in the inability to store any characters with codepoints 0x10000 and above (e.g. emojis). They 'fixed' this issue with version 5.5 by introducing a new encoding called utf8mb4. Bottom line: Never ever use MariaDB's utf8, always use utf8mb4. In case you need to migrate see [4].

So that you have decided to use MariaDB as the database of your Nextcloud installation you have to enable the corresponding PHP extension:

/etc/webapps/nextcloud/php.ini
extension=pdo_mysql

Further configuration (related to MariaDB) is not required (contrary to the information given in Nextcloud's admin manual).

Now setup Nextcloud's database schema with:

occ maintenance:install \
    --database=mysql \
    --database-name=nextcloud \
    --database-host=/run/mysqld/mysqld.sock \
    --database-user=nextcloud \
    --database-pass=xxxxxxxx \
    --admin-pass=zzzzzzzz \
    --admin-email=aaaa@bbbbb \
    --data-dir=/var/lib/nextcloud/data

Mind the placeholders (e.g. xxxxxxxx) and replace them with appropriate values. This command assumes that you run your database on the same host as Nextcloud. Enter occ help maintenance:install and see Nextcloud's documentation for other options.

PostgreSQL

Consult the corresponding article for detailed information about PostgreSQL. In case you want to run your database on the same host as Nextcloud install postgresql (if you haven't done so already). For additional security in this scenario it is recommended to configure PostgreSQL to only listen on a local UNIX socket:

/var/lib/postgres/data/postgresql.conf
listen_addresses = ''

Especially don't forget to initialize your database with initdb. After having done so start PostgreSQL's CLI tool psql

runuser -u postgres -- psql

and create the database user nextcloud and the database of the same name

CREATE USER nextcloud WITH PASSWORD 'xxxxxxxx';
CREATE DATABASE nextcloud TEMPLATE template0 ENCODING 'UNICODE';
ALTER DATABASE nextcloud OWNER TO nextcloud;
GRANT ALL PRIVILEGES ON DATABASE nextcloud TO nextcloud;
\q

(xxxxxxxx is a placeholder for the passwort of database user nextcloud that you have to choose.)

Install the additional package php-pgsql as dependency (pacman option --asdeps) and enable the corresponding PHP extension in /etc/webapps/nextcloud/php.ini:

/etc/webapps/nextcloud/php.ini
extension=pdo_pgsql

Now setup Nextcloud's database schema with:

occ maintenance:install \
    --database=pgsql \
    --database-name=nextcloud \
    --database-host=/run/postgresql \
    --database-user=nextcloud \
    --database-pass=xxxxxxxx \
    --admin-pass=zzzzzzzz \
    --admin-email=aaaa@bbbbb \
    --data-dir=/var/lib/nextcloud/data

Mind the placeholders (e.g. xxxxxxxx) and replace them with appropriate values. This command assumes that you run your database on the same host as Nextcloud. Enter occ help maintenance:install and see Nextcloud's documentation for other options.

Application server

There are two prevalent application servers that can be used to process PHP code: uWSGI or php-fpm. php-fpm as the name suggests is specialized on PHP. The protocol used between the web server and php-fpm is fastcgi. The tool's documentation leaves room for improvement. uWSGI on the other hand can serve code written in a handful of languages by means of language specific plugins. The protocol used is uwsgi (lowercase). The tool is extensively documented - albeit the sheer amount of documentation can become confusing and unwieldy.

Warning: It has to be mentioned that maintenance of uWSGI and especially of it's PHP plugin has been sparse lately[5]. This has already caused issues that could only be solved by patching uWSGI code by the maintainers of Arch Linux packages, i.e. not upstream.

uWSGI

uWSGI has its own article. A lot of useful information can be found there. Install uwsgi and the plugin uwsgi-plugin-php - preferrably as dependencies, i.e. with --asdeps. To run Nextcloud's code with (or in) uWSGI you have to configure one uWSGI specific config file (nextcloud.ini) and define one systemd service.

nextcloud.ini

The Nextcloud package includes a sample config file already in the right place /etc/uwsgi/nextcloud.ini. In almost any case you will have to adapt this file to your requirements and setup. Find a version with lots of commented changes (compared to the packages version). It assumes a no-frills Nextcloud installation for private use (i.e. with moderate load).

See section Background jobs for arguments why not to configure recurring jobs in this file. In general keep the enabled extensions, extension specific settings and open_basedir in sync with /etc/webapps/nextcloud/php.ini (with the exception of opcache).

Tip: The changes to /etc/uwsgi/nextcloud.ini can become extensive. A file named nextcloud.ini.pacnew will be created during package update in case there are changes in the original file provided by the package nextcloud. In order to better track changes in this latter file and apply them to /etc/uwsgi/nextcloud.ini the following approach can be applied:
  • Make a copy of the file as provided by the package (e.g. by extracting from the package) and store it as nextcloud.ini.package.
  • In case an update of package nextcloud produces a nextcloud.ini.pacnew you can identify the changes with diff nextcloud.ini.package nextcloud.ini.pacnew.
  • Selectively apply the changes to your nextcloud.ini depending on whether they make sense with your version or not.
  • Move nextcloud.ini.pacnew over nextcloud.ini.package.

Enable and start

The package uwsgi provides a template unit file (uwsgi@.service). The instance ID (here nextcloud) is used to pick up the right configuration file. That's what we do.

systemctl enable uwsgi@nextcloud.service
systemctl start uwsgi@nextcloud.service

In case you have more than a few (e.g. 2) services started like this and get the impression this is a waste of resource you might consider using emporer mode.

php-fpm

In case you opt to use php-fpm as your application server install php-fpm - preferrably as a dependent package (--asdeps).

Configuration consists of a copy of php.ini relevant for all applications served by php-fpm and a so-called pool file specific for the application (here Nextcloud). For many config parameters it's a matter of taste whether to put them in the first or the second file. We will put most in the pool file. Finally you have to tweak the systemd service file.

php-fpm.ini

As stated earlier this article avoids modifications of PHP's central configuration in /etc/php/php.ini. Instead create a php-fpm specific copy.

cp /etc/php/php.ini /etc/php/php-fpm.ini

Make sure it is owned and only writeable by root. Something along -rw-r--r-- 1 root root ... php-fpm.ini. Enable the op-cache, i.e. uncomment the line

;zend_extension=opcache

and put the following parameters below the existing line [opcache]:

/etc/php/php-fpm.ini
opcache.enable = 1
opcache.interned_strings_buffer = 8
opcache.max_accelerated_files = 10000
opcache.memory_consumption = 128
opcache.save_comments = 1
opcache.revalidate_freq = 1

nextcloud.conf pool file

Next you have to create a so called pool file for php-fpm. It is responsible for spawning dedicated php-fpm processes for the Nextcloud application. Create a file /etc/php/php-fpm.d/nextcloud.conf - you may use this functional version as a starting point.

Again make sure this pool file is owned and only writeable by root (i.e. -rw-r--r-- 1 root root ... nextcloud.conf). You may tweak some settings (especially pm..., php_value[...] and php_flag[...]) to your liking. The settings php_value[...] and php_flag[..] must be consistent with the same settings in /etc/webapps/nextcloud/php.ini (but not /etc/php/php-fpm.ini).

Tip: The package php-fpm comes with its own pool file www.conf that is of little use here. A good approach to get rid of it is to rename it to www.conf.package and create a file www.conf with only comment lines (lines starting with a semicolon). This way www.conf becomes a no-op. It is also not overwritten during installation of a new version of php-fpm. Instead a file www.conf.pacnew is created. You can compare this against www.conf.package to see if anything significant has changed in the pool file that you may have to reproduce in nextcloud.conf. Do not forget to rename www.conf.pacnew to www.conf.package at the end of this procedure.

php-fpm service

php-fpm is (of course) run as a systemd service. You have to modify the service configuration to be able to run Nextcloud. This is best achieved by means of a drop-in file. Use systemctl edit php-fpm.service to create the file and add:

[Service]
ExecStart=
ExecStart=/usr/bin/php-fpm --nodaemonize --fpm-config /etc/php/php-fpm.conf --php-ini /etc/php/php-fpm.ini
ReadWritePaths=/var/lib/nextcloud
ReadWritePaths=/etc/webapps/nextcloud/config
  • It replaces the ExecStart line by a start command that uses the php-fpm.ini covered in the previous section.
  • The directories /var/lib/nextcloud and /etc/webapps/nextcloud/config (and everything below) are made writable. The ProtectSystem=full in the original service definition causes /usr, /boot and /etc to be mounted read-only for the php-fpm processes.

Enable and start

Do not forget to enable and start the php-fpm service.

Keep /etc tidy

The Nextcloud package unconditionally creates the uWSGI configuration file /etc/uwsgi/nextcloud.ini. Of course it is of no use when you run php-fpm instead of uWSGI (and it does no harm whatsoever). In case you nevertheless want to get rid of it just add the following lines to /etc/pacman.conf

/etc/pacman.conf
# uWSGI configuration that comes with Nextcloud is not needed
NoExtract = etc/uwsgi/nextcloud.ini

Web server

There is an abundance of web servers you can choose from. Whatever option you finally pick you have to keep in mind that the Nextcloud application needs to be run with its own system user nextcloud. So you will need to forward your requests to one of the above mentioned application servers.

nginx

Configuration of nginx is way beyond the scope of this article. See the relevant article for further information. Also consult Nextcloud's documentation for an elaborated configuration. Most likely you will have to copy it into a file with an appropriate name below /etc/nginx/sites-available and create the corresponding symbolic link in /etc/nginx/sites-enabled.

The usage of the block upstream php-handler { ... } is not necessary. Just specifiy fastcgi_pass unix:/run/php-fpm/nextcloud.sock; in the location block that deals with forwarding request with PHP URIs to the application server. When using uWSGI instead of php-fpm replace this location block with:

location ~ \.php(?:$|/) {
    include uwsgi_params;
    uwsgi_modifier1 14;
    # Avoid duplicate headers confusing OC checks
    uwsgi_hide_header X-Frame-Options;
    uwsgi_hide_header X-XSS-Protection;
    uwsgi_hide_header X-Content-Type-Options;
    uwsgi_hide_header X-Robots-Tag;
    uwsgi_hide_header X-Download-Options;
    uwsgi_hide_header X-Permitted-Cross-Domain-Policies;
    uwsgi_pass unix:/run/uwsgi/nextcloud.sock;
}

Things you might have to adapt (not exhaustive):

  • Your server name (server_name clauses 2x), i.e. the server part of the URL your Grav installation will be reachable with.
  • The name of the certificate and key you use for SSL / TLS.
  • If and where you want an access log written to.
  • The location where Certbot (or any other ACME client) will put the domain verification challenges. Usage of alias instead of try_files is probably more adequate here.
  • The path used to reach your Nextcloud installation. (The part right to the server name & port section in the URL.)
  • What application server (uWSGI or php-fpm) you are using, i.e. how and where nginx will pass requests that need to trigger some PHP code. (See above.)
  • Configure OCSP stapling.

There is no need to install any additional modules since nginx natively supports both protocols FastCGI and uwsgi.

Apache httpd

Find lots of useful information in the article about the Apache HTTP Server. Nextcloud's documentation has some sample configuration that can also be found in /usr/share/doc/nextcloud/apache.example.conf. Both implicitely rely on mod_php that cannot be used anymore. mod_proxy_fcgi or mod_proxy_uwsgi need to be applied.

Information about how to integrate Apache with php-fpm can be found here in this wiki. uWSGI's documentation has some information about how to integrate Apache with PHP by means of uWSGI and mod_proxy_uwsgi. Mind that the Apache package comes with both modules mod_proxy_fcgi and mod_proxy_uwsgi. They need to be loaded as required.

The following Apache modules are required to run Nextcloud:

/etc/httpd/conf/httpd.conf
# these are already loaded in a standard Apache installation
LoadModule headers_module modules/mod_headers.so
LoadModule env_module modules/mod_env.so
LoadModule dir_module modules/mod_dir.so
LoadModule mime_module modules/mod_mime.so
LoadModule setenvif_module modules/mod_setenvif.so

# these need to be uncommented explicitely
LoadModule rewrite_module modules/mod_rewrite.so
LoadModule ssl_module modules/mod_ssl.so
LoadModule socache_shmcb_module modules/mod_socache_shmcb.so
LoadModule proxy_module modules/mod_proxy.so

# either this one in case you use php-fpm
LoadModule proxy_fcgi_module modules/mod_proxy_fcgi.so
# or this one in case you opt for uWSGI
LoadModule proxy_uwsgi_module modules/mod_proxy_uwsgi.so

Also uncomment the following directive to pull in TLS configuration parameters:

/etc/httpd/conf/httpd.conf
Include conf/extra/httpd-ssl.conf

Consult Mozilla's SSL configurator for details about how to optimize your TLS configuration.

Refer to the following two sample configuration files depending on how you want to access your Nextcloud installation:

  • In case your Nextcloud installation is accessed via a dedicated host name (e.g. https://cloud.example.com/) put this fragment into /etc/httpd/conf/extra/httpd-vhosts.conf.
  • In case your Nextcloud installation is located in a subfolder of your web site (e.g. https://www.example.com/nextcloud/) put this fragment in your /etc/httpd/conf/httpd.conf.

Of course you must adapt these sample configuration files to your concrete setup. Replace the SetHandler directive by SetHandler "proxy:unix:/run/uwsgi/nextcloud.sock|uwsgi://nextcloud/" when you use uWSGI.

The Nextcloud package comes with a .htaccess that already takes care of a lot of rewriting and header stuff. Run oc maintenance:update:htaccess to adapt this file. Parameter htaccess.RewriteBase in /etc/webapps/nextcloud/config/config.php is vital for this.

Background jobs

Nextcloud requires certain tasks to be run on a scheduled basis. See Nextcloud's documentation for some details. The easiest (and most reliable) way to set up these background jobs is to use the systemd service and timer units that are already installed by nextcloud. The service unit needs some tweaking so that the job uses the correct PHP ini-file (and not the global php.ini). Create a drop-in file with systemctl edit nextcloud-cron.service and add:

[Service]
ExecStart=
ExecStart=/usr/bin/php -c /etc/webapps/nextcloud/php.ini -f /usr/share/webapps/nextcloud/cron.php

After that enable and start the timer (not the service).

In-memory caching

Nextcloud's documentation recommends to apply some kind of in-memory object cache to significantly improve performance. This section demonstrates setup of APCu - mainly to pinpoint the details that differ from the instructions given in Nextcloud's documentation. The other options (Redis and memcached) are also sufficiently covered there.

Install php-apcu (as dependency --asdeps). Enable the extension in the relevant configuration files. These are

  • /etc/webapps/nextcloud/php.ini used by the occ command and the background jobs and
  • depending on the application server you use either
    • /etc/uwsgi/nextcloud.ini in case you use uWSGI or
    • /etc/php/php-fpm.d/nextcloud.conf in case you use php-fpm.

The parameter to do so is already there and only needs to be uncommented. Two other configuration parameters related to APCu are also already there.

Restart your application server (not the web server as Nextcloud's documentation claims). Add the following line to your Nextcloud configuration file:

/etc/webapps/nextcloud/config/config.php
'memcache.local' => '\OC\Memcache\APCu',

That's it. Enjoy your performance boost!

Note: Mind that push notify (the Nextcloud service that replaces client polling by notification by the server thus drastically reducing sync latency) is dependend on Redis.