Difference between revisions of "PostgreSQL"

From ArchWiki
Jump to: navigation, search
(Upgrading PostgreSQL: use the same initdb command as in #Installing PostgreSQL)
m (Upgrading PostgreSQL: fix broken section link)
 
(83 intermediate revisions by 22 users not shown)
Line 1: Line 1:
[[Category:Database management systems]]
+
[[Category:Relational DBMSs]]
 
[[it:PostgreSQL]]
 
[[it:PostgreSQL]]
 
[[ja:PostgreSQL]]
 
[[ja:PostgreSQL]]
Line 7: Line 7:
 
{{Related|PhpPgAdmin}}
 
{{Related|PhpPgAdmin}}
 
{{Related articles end}}
 
{{Related articles end}}
[http://www.postgresql.org/ PostgreSQL] is an open source, community driven, standard compliant object-relational database system.
+
[https://www.postgresql.org/ PostgreSQL] is an open source, community driven, standard compliant object-relational database system.
  
This document describes how to set up PostgreSQL. It also describes how to configure PostgreSQL to be accessible from a remote client. Among other applications, PostgreSQL can be substituted for MySQL as part of the [[LAMP]] web stack.
+
== Installation ==
  
== Installing PostgreSQL ==
+
{{Style|Don't duplicate [[sudo]] and [[su]].}}
  
[[Install]] the {{Pkg|postgresql}} package. Then [[Users_and_groups#Other_examples_of_user_management|set a password]] for the newly created ''postgres'' user.
+
[[Install]] the {{Pkg|postgresql}} package. It will also create a system user called ''postgres''.
  
Then, switch to the default PostgreSQL user ''postgres'' by executing the following command:
+
{{Warning|See [[#Upgrading PostgreSQL]] for necessary steps before installing new versions of the PostgreSQL packages.}}
  
* If you have [[sudo]] and your username is in {{ic|sudoers}}:
+
{{Note|Commands that should be run as the ''postgres'' user are prefixed by {{ic|[postgres]$}} in this article.}}
 +
 
 +
You can switch to the PostgreSQL user by executing the following command:
 +
 
 +
* If you have [[sudo]] and are in [[sudoers]]:
  
 
:{{bc|$ sudo -u postgres -i}}
 
:{{bc|$ sudo -u postgres -i}}
  
* Otherwise:
+
* Otherwise using [[su]]:
  
 
:{{bc|<nowiki>
 
:{{bc|<nowiki>
Line 28: Line 32:
 
</nowiki>}}
 
</nowiki>}}
  
See {{man|1|su}} or {{man|8|sudo}} for their usage.
+
See {{man|8|sudo}} or {{man|1|su}} for their usage.
  
{{Note|Commands that should be run as the postgres user are prefixed by {{ic|[postgres]$}} in this article.}}
+
== Initial configuration ==
  
 
Before PostgreSQL can function correctly, the database cluster must be initialized:
 
Before PostgreSQL can function correctly, the database cluster must be initialized:
  
  [postgres]$ initdb --locale $LANG -E UTF8 -D '/var/lib/postgres/data'
+
  [postgres]$ initdb -D '/var/lib/postgres/data'
 +
 
 +
Where {{ic|-D}} is the default location where the database cluster must be stored (see [[#Change default data directory]] if you want to use a different one).
  
Where:
+
Note that by default, the locale and the encoding for the database cluster are derived from your current environment (using [[Locale#LANG: default locale|$LANG]] value). [https://www.postgresql.org/docs/current/static/locale.html]
 +
However, depending on your settings and use cases this might not be what you want, and you can override the defaults using:
 +
* {{ic|--locale ''locale''}}, where ''locale'' is to be chosen amongst the ones defined in the file {{ic|/etc/locale.conf}} (plus {{ic|POSIX}} and {{ic|C}} that are also accepted);
 +
* {{ic|-E ''enconding''}} for the encoding (which must match the chosen locale);
  
* the {{ic|--locale}} is the one defined in the file {{ic|/etc/locale.conf}};
+
Example: {{bc|[postgres]$ initdb --locale en_US.UTF-8 -E UTF8 -D '/var/lib/postgres/data'}}
* the {{ic|-E}} is the default encoding of the database that will be created in the future;
 
* and {{ic|-D}} is the default location where the database cluster must be stored.
 
  
 
Many lines should now appear on the screen with several ending by {{ic|... ok}}:
 
Many lines should now appear on the screen with several ending by {{ic|... ok}}:
Line 47: Line 54:
 
This user must also own the server process.
 
This user must also own the server process.
  
The database cluster will be initialized with locale "en_GB.UTF-8".
+
The database cluster will be initialized with locale "en_US.UTF-8".
 +
The default database encoding has accordingly been set to "UTF8".
 
The default text search configuration will be set to "english".
 
The default text search configuration will be set to "english".
  
Line 58: Line 66:
 
selecting dynamic shared memory implementation ... posix
 
selecting dynamic shared memory implementation ... posix
 
creating configuration files ... ok
 
creating configuration files ... ok
creating template1 database in /var/lib/postgres/data/base/1 ... ok
+
running bootstrap script ... ok
initializing pg_authid ... ok
+
performing post-bootstrap initialization ... ok
[...]
+
syncing data to disk ... ok
 +
 
 +
WARNING: enabling "trust" authentication for local connections
 +
You can change this by editing pg_hba.conf or using the option -A, or
 +
--auth-local and --auth-host, the next time you run initdb.
 +
 
 +
Success. You can now start the database server using:
 +
 
 +
    pg_ctl -D /var/lib/postgres/ -l logfile start
 
}}
 
}}
  
 
If these are the kind of lines you see, then the process succeeded. Return to the regular user using {{ic|exit}}.
 
If these are the kind of lines you see, then the process succeeded. Return to the regular user using {{ic|exit}}.
  
As root, [[start]] and [[enable]] {{ic|postgresql.service}}.
+
{{Note|To read more about this {{ic|WARNING}}, see [[#Restricts access rights to the database superuser by default|local users configuration]].}}
  
 
{{Tip|If you change the root to something other than {{ic|/var/lib/postgres}}, you will have to [[edit]] the service file. If the root is under {{ic|home}}, make sure to set {{ic|ProtectHome}} to false.}}
 
{{Tip|If you change the root to something other than {{ic|/var/lib/postgres}}, you will have to [[edit]] the service file. If the root is under {{ic|home}}, make sure to set {{ic|ProtectHome}} to false.}}
  
{{Warning|If the database resides on a [[Btrfs]] file system, you should consider disabling [[Btrfs#Copy-on-Write (CoW)|Copy-on-Write]] for the directory before creating any database. If the database resides on a [[ZFS]] file system, you should consult [[ZFS#Database]] before creating any database.}}
+
{{Warning|
 +
* If the database resides on a [[Btrfs]] file system, you should consider disabling [[Btrfs#Copy-on-Write (CoW)|Copy-on-Write]] for the directory before creating any database.
 +
* If the database resides on a [[ZFS]] file system, you should consult [[ZFS#Database]] before creating any database.
 +
}}
 +
 
 +
Finally, [[start]] and [[enable]] the {{ic|postgresql.service}}.
  
 
== Create your first database/user ==
 
== Create your first database/user ==
Line 75: Line 96:
 
{{Tip|If you create a PostgreSQL user with the same name as your Linux username, it allows you to access the PostgreSQL database shell without having to specify a user to login (which makes it quite convenient).}}
 
{{Tip|If you create a PostgreSQL user with the same name as your Linux username, it allows you to access the PostgreSQL database shell without having to specify a user to login (which makes it quite convenient).}}
  
Become the postgres user. Add a new database user using the [http://www.postgresql.org/docs/current/static/app-createuser.html createuser] command:
+
Become the postgres user. Add a new database user using the [https://www.postgresql.org/docs/current/static/app-createuser.html createuser] command:
  
 
  [postgres]$ createuser --interactive
 
  [postgres]$ createuser --interactive
  
Create a new database over which the above user has read/write privileges using the [http://www.postgresql.org/docs/current/static/app-createdb.html createdb] command (execute this command from your login shell if the database user has the same name as your Linux user, otherwise add {{ic|-U ''database-username''}} to the following command):
+
Create a new database over which the above user has read/write privileges using the [https://www.postgresql.org/docs/current/static/app-createdb.html createdb] command (execute this command from your login shell if the database user has the same name as your Linux user, otherwise add {{ic|-U ''database-username''}} to the following command):
  
 
  $ createdb myDatabaseName
 
  $ createdb myDatabaseName
Line 87: Line 108:
 
=== Access the database shell ===
 
=== Access the database shell ===
  
Become the postgres user. Start the primary database shell, [http://www.postgresql.org/docs/current/static/app-psql.html psql], where you can do all your creation of databases/tables, deletion, set permissions, and run raw SQL commands. Use the {{ic|-d}} option to connect to the database you created (without specifying a database, {{ic|psql}} will try to access a database that matches your username).
+
Become the postgres user. Start the primary database shell, [https://www.postgresql.org/docs/current/static/app-psql.html psql], where you can do all your creation of databases/tables, deletion, set permissions, and run raw SQL commands. Use the {{ic|-d}} option to connect to the database you created (without specifying a database, {{ic|psql}} will try to access a database that matches your username).
  
 
  [postgres]$ psql -d myDatabaseName
 
  [postgres]$ psql -d myDatabaseName
Line 118: Line 139:
  
 
== Optional configuration ==
 
== Optional configuration ==
 +
 +
The PostgreSQL database server configuration file is {{ic|postgresql.conf}}. This file is located in the data directory of the server, typically {{ic|/var/lib/postgres/data}}. This folder also houses the other main configuration files, including the {{ic|pg_hba.conf}} which defines authentication settings, for both [[#Restricts access rights to the database superuser by default|local users]] and [[#Configure PostgreSQL to be accessible from remote hosts|other hosts ones]].
 +
 +
{{Note|By default, this folder will not be browsable or searchable by a regular user. This is why {{ic|find}} and {{ic|locate}} are not finding the configuration files.}}
 +
 +
=== Restricts access rights to the database superuser by default ===
 +
 +
The defaults {{ic|pg_hba.conf}} '''allow any local user to connect as any database user''', including the database superuser.
 +
This is likely not what you want, so in order to restrict global access to the ''postgress'' user, change the following line:
 +
 +
{{hc|/var/lib/postgres/data/pg_hba.conf|2=
 +
# TYPE  DATABASE        USER            ADDRESS                METHOD
 +
 +
# "local" is for Unix domain socket connections only
 +
local  all            all                                    trust
 +
}}
 +
 +
To:
 +
{{hc|/var/lib/postgres/data/pg_hba.conf|2=
 +
# TYPE  DATABASE        USER            ADDRESS                METHOD
 +
 +
# "local" is for Unix domain socket connections only
 +
local  all            postgres                                peer
 +
}}
 +
 +
You might later add additional lines depending on your needs or software ones.
 +
 +
=== Configure PostgreSQL to be accessible exclusively through UNIX Sockets ===
 +
 +
In the connections and authentications section of your configuration, set:
 +
 +
{{hc|/var/lib/postgres/data/postgresql.conf|2=
 +
listen_addresses = <nowiki>''</nowiki>
 +
}}
 +
 +
This will disable network listening completely.
 +
After this you should [[restart]] {{ic|postgresql.service}} for the changes to take effect.
  
 
=== Configure PostgreSQL to be accessible from remote hosts ===
 
=== Configure PostgreSQL to be accessible from remote hosts ===
  
The PostgreSQL database server configuration file is {{ic|postgresql.conf}}. This file is located in the data directory of the server, typically {{ic|/var/lib/postgres/data}}. This folder also houses the other main configuration files, including the {{ic|pg_hba.conf}}.
+
In the connections and authentications section, set the {{ic|listen_addresses}} line to your needs:
  
{{Note|By default, this folder will not be browsable or searchable by a regular user. This is why {{ic|find}} and {{ic|locate}} are not finding the configuration files.}}
+
{{hc|/var/lib/postgres/data/postgresql.conf|2=
 +
listen_addresses = 'localhost,''my_local_ip_address'''
 +
}}
  
Edit the file {{ic|/var/lib/postgres/data/postgresql.conf}}. In the connections and authentications section, add the {{ic|listen_addresses}} line to your needs:
+
You can use {{ic|'*'}} to listen on all available addresses.
  
listen_addresses = 'localhost,''my_local_ip_address'''
+
{{Note|PostgreSQL uses TCP port {{ic|5432}} by default for remote connections. Make sure this port is open in your [[firewall]] and able to receive incoming connections. You can also change it in the configuration file, right below {{ic|listen_addresses}}}}
#You can use '*' to listen on all local addresses
 
  
Take a careful look at the other lines.
+
Then add a line like the following to the authentication config:
  
Host-based authentication is configured in {{ic|/var/lib/postgres/data/pg_hba.conf}}. This file controls which hosts are allowed to connect. Note that the defaults '''allow any local user to connect as any database user''', including the database superuser. Add a line like the following:
+
{{hc|/var/lib/postgres/data/pg_hba.conf|2=
 +
# TYPE  DATABASE        USER            ADDRESS                METHOD
 +
# IPv4 local connections:
 +
host    all            all            ''ip_address''/32  md5
 +
}}
  
# IPv4 local connections:
+
where {{ic|''ip_address''}} is the IP address of the remote client.
host  all  all  ''my_remote_client_ip_address''/32  md5
 
  
where {{ic|my_remote_client_ip_address}} is the IP address of the client.
+
See the documentation for [https://www.postgresql.org/docs/current/static/auth-pg-hba-conf.html pg_hba.conf].
  
See the documentation for [http://www.postgresql.org/docs/current/static/auth-pg-hba-conf.html pg_hba.conf].
+
{{Note|Neither sending your plain password nor the md5 hash (used in the example above) over the Internet is secure if it is not done over an SSL-secured connection. See [https://www.postgresql.org/docs/current/static/ssl-tcp.html Secure TCP/IP Connections with SSL] for how to configure PostgreSQL with SSL.}}
  
 
After this you should [[restart]] {{ic|postgresql.service}} for the changes to take effect.
 
After this you should [[restart]] {{ic|postgresql.service}} for the changes to take effect.
 
{{Note|PostgreSQL uses port {{ic|5432}} by default for remote connections. Make sure this port is open and able to receive incoming connections.}}
 
  
 
For troubleshooting take a look in the server log file:
 
For troubleshooting take a look in the server log file:
  
  $ journalctl -u postgresql
+
  $ journalctl -u postgresql.service
  
 
=== Configure PostgreSQL authenticate against PAM ===
 
=== Configure PostgreSQL authenticate against PAM ===
Line 154: Line 214:
  
 
For example, the same configuration as above, but with PAM enabled:
 
For example, the same configuration as above, but with PAM enabled:
 
+
{{hc|/var/lib/postgres/data/pg_hba.conf|2=
# IPv4 local connections:
+
# IPv4 local connections:
host  all  all  ''my_remote_client_ip_address''/32  pam
+
host  all  all  ''my_remote_client_ip_address''/32  pam
 +
}}
  
 
The PostgreSQL server is however running without root privileges and will not be able to access {{ic|/etc/shadow}}. We can work around that by allowing the postgres group to access this file:
 
The PostgreSQL server is however running without root privileges and will not be able to access {{ic|/etc/shadow}}. We can work around that by allowing the postgres group to access this file:
  
  setfacl -m g:postgres:r /etc/shadow
+
  # setfacl -m g:postgres:r /etc/shadow
  
 
=== Change default data directory ===
 
=== Change default data directory ===
Line 187: Line 248:
 
=== Change default encoding of new databases to UTF-8 ===
 
=== Change default encoding of new databases to UTF-8 ===
  
{{Note|If you ran {{ic|initdb}} with {{ic|-E UTF8}} these steps are not required.}}
+
{{Note|If you ran {{ic|initdb}} with {{ic|-E UTF8}} or while using an UTF-8 locale, these steps are not required.}}
  
 
When creating a new database (e.g. with {{ic|createdb blog}}) PostgreSQL actually copies a template database. There are two predefined templates: {{ic|template0}} is vanilla, while {{ic|template1}} is meant as an on-site template changeable by the administrator and is used by default. In order to change the encoding of a new database, one of the options is to change on-site {{ic|template1}}. To do this, log into PostgreSQL shell ({{ic|psql}}) and execute the following:
 
When creating a new database (e.g. with {{ic|createdb blog}}) PostgreSQL actually copies a template database. There are two predefined templates: {{ic|template0}} is vanilla, while {{ic|template1}} is meant as an on-site template changeable by the administrator and is used by default. In order to change the encoding of a new database, one of the options is to change on-site {{ic|template1}}. To do this, log into PostgreSQL shell ({{ic|psql}}) and execute the following:
Line 232: Line 293:
 
== Administration tools ==
 
== Administration tools ==
  
 +
* {{App|[[Adminer]]|Web-based database management tool for multiple database systems.|https://www.adminer.org|{{AUR|adminer}}}}
 
* {{App|[[phpPgAdmin]]|Web-based administration tool for PostgreSQL.|http://phppgadmin.sourceforge.net|{{Pkg|phppgadmin}}}}
 
* {{App|[[phpPgAdmin]]|Web-based administration tool for PostgreSQL.|http://phppgadmin.sourceforge.net|{{Pkg|phppgadmin}}}}
* {{App|pgAdmin|GUI-based administration tool for PostgreSQL.|http://www.pgadmin.org/|{{Pkg|pgadmin4}}}}
+
* {{App|pgAdmin|GUI-based administration tool for PostgreSQL.|https://www.pgadmin.org/|{{Pkg|pgadmin4}}}}
 +
* {{App|pgModeler|Graphical schema designer for PostgreSQL.|https://pgmodeler.io/|{{AUR|pgmodeler}}}}
  
== Setup HHVM to work with PostgreSQL ==
+
== Upgrading PostgreSQL ==
  
{{Out of date|hhvm-pgsql fails to compile against HHVM 3.7.0, but upstream has not resolved the problem yet. See https://github.com/PocketRent/hhvm-pgsql/issues/82|section=Setting up HHVM}}
+
{{Style|Don't show basic systemctl commands, etc.}}
  
$ git clone https://github.com/PocketRent/hhvm-pgsql.git
+
Upgrading major PostgreSQL versions requires some extra maintenance.
$ cd hhvm-pgsql
 
  
If you do not use a nightly build, then run this command (verified on HHVM 3.6.1) to avoid compile errors:
+
{{Note|
 +
* Official PostgreSQL [https://www.postgresql.org/docs/current/static/upgrading.html upgrade documentation] should be followed.
 +
* From version {{ic|10.0}} onwards PostgreSQL [https://www.postgresql.org/about/news/1786/ changed its versioning scheme]. Earlier upgrade from version {{ic|9.''x''}} to {{ic|9.''y''}} was considered as major upgrade. Now upgrade from version {{ic|10.''x''}} to {{ic|10.''y''}} is considered as minor upgrade and upgrade from version {{ic|10.''x''}} to {{ic|11.''y''}} is considered as major upgrade.
 +
}}
  
$ git checkout tags/3.6.0
+
{{Warning|The following instructions could cause data loss. Do not run the commands below blindly, without understanding what they do. [https://www.postgresql.org/docs/current/static/backup.html Backup database] first.}}
  
Then build the extension (if you do not need an improved support for Hack language, then remove -DHACK_FRIENDLY=ON):
+
It is recommended to add the following to your {{ic|/etc/pacman.conf}} file:
  
  $ hphpize
+
  IgnorePkg = postgresql*
$ cmake -DHACK_FRIENDLY=ON .
 
$ make
 
  
Then copy the built extension:
+
This will ensure you do not accidentally upgrade the database to an incompatible version. When an upgrade is available, pacman will notify you that it is skipping the upgrade because of the entry in {{ic|pacman.conf}}.  Minor version upgrades are safe to perform. However, if you do an accidental upgrade to a different major version, you might not be able to access any of your data. Always check the [https://www.postgresql.org/ PostgreSQL home page] to be sure of what steps are required for each upgrade. For a bit about why this is the case, see the [https://www.postgresql.org/support/versioning versioning policy].
  
# cp pgsql.so /etc/hhvm/
+
There are two main ways to upgrade your PostgreSQL database. Read the official documentation for details.
  
Add to /etc/hhvm/server.ini:
+
For those wishing to use {{ic|pg_upgrade}}, a {{Pkg|postgresql-old-upgrade}} package is available that will always run one major version behind the real PostgreSQL package. This can be installed side-by-side with the new version of PostgreSQL.  
  
extension_dir = /etc/hhvm
+
Note that the databases cluster directory does not change from version to version, so before running {{ic|pg_upgrade}}, it is necessary to rename your existing data directory and migrate into a new directory. The new databases cluster must be initialized, as described in the [[#Installation]] section.
hhvm.extensions[pgsql] = pgsql.so
 
  
== Upgrading PostgreSQL ==
+
When you are ready, stop the postgresql service, upgrade the following packages: {{Pkg|postgresql}}, {{Pkg|postgresql-libs}}, and {{Pkg|postgresql-old-upgrade}}. Finally upgrade the databases cluster.
  
Upgrading major PostgreSQL versions (i.e. from {{ic|9.''x''}} to {{ic|9.''y''}}) requires some extra maintenance.
+
Stop and make sure PostgreSQL is stopped:
  
{{Note|Official PostgreSQL [http://www.postgresql.org/docs/current/static/upgrading.html upgrade documentation] should be followed.}}
+
# systemctl stop postgresql.service
 +
# systemctl status postgresql.service
  
{{Warning|The following instructions could cause data loss. '''Use at your own risk'''.}}
+
Upgrade the packages:
  
It is recommended to add the following to your {{ic|/etc/pacman.conf}} file:
+
# pacman -S postgresql postgresql-libs postgresql-old-upgrade
  
IgnorePkg = postgresql postgresql-libs
+
Rename the databases cluster directory, and create an empty one:
  
This will ensure you do not accidentally upgrade the database to an incompatible version. When an upgrade is available, pacman will notify you that it is skipping the upgrade because of the entry in {{ic|pacman.conf}}. Minor version upgrades (e.g. 9.0.3 to 9.0.4) are safe to perform. However, if you do an accidental upgrade to a different major version (e.g. 9.0.x to 9.1.x), you might not be able to access any of your data. Always check the [http://www.postgresql.org/ PostgreSQL home page] to be sure of what steps are required for each upgrade. For a bit about why this is the case, see the [http://www.postgresql.org/support/versioning versioning policy].
+
  # mv /var/lib/postgres/data /var/lib/postgres/olddata
 +
# mkdir /var/lib/postgres/data /var/lib/postgres/tmp
 +
# chown postgres:postgres /var/lib/postgres/data /var/lib/postgres/tmp
 +
[postgres]$ initdb -D '/var/lib/postgres/data'
  
There are two main ways to upgrade your PostgreSQL database. Read the official documentation for details.
+
Upgrade the cluster:
  
For those wishing to use {{ic|pg_upgrade}}, a {{Pkg|postgresql-old-upgrade}} package is available that will always run one major version behind the real PostgreSQL package. This can be installed side-by-side with the new version of PostgreSQL.
+
[postgres]$ cd /var/lib/postgres/tmp
 +
[postgres]$ pg_upgrade -b /opt/pgsql-9.6/bin -B /usr/bin -d /var/lib/postgres/olddata -D /var/lib/postgres/data
  
When you are ready, upgrade the following packages: {{Pkg|postgresql}}, {{Pkg|postgresql-libs}}, and {{Pkg|postgresql-old-upgrade}}. Note that the data directory does not change from version to version, so before running {{ic|pg_upgrade}}, it is necessary to rename your existing data directory and migrate into a new directory. The new database must be initialized, as described near the top of this page.
+
{{ic|pg_upgrade}} will perform the upgrade and create some scripts in {{ic|/var/lib/postgres/tmp/}}. Follow the instructions given on screen and act accordingly. You may delete the {{ic|/var/lib/postgres/tmp}} directory once the upgrade is completely over.
 
 
# systemctl stop postgresql.service
 
# mv /var/lib/postgres/data /var/lib/postgres/olddata
 
# mkdir /var/lib/postgres/data
 
# chown postgres:postgres /var/lib/postgres/data
 
[postgres]$ initdb --locale $LANG -E UTF8 -D '/var/lib/postgres/data'
 
  
The upgrade invocation will likely look something like the following. '''Do not run this command blindly without understanding what it does!''' Reference the [http://www.postgresql.org/docs/current/static/pgupgrade.html upstream pg_upgrade documentation] for details.
+
Start the cluster:
  
  [postgres]$ cd /tmp
+
  # systemctl start postgresql.service
[postgres]$ pg_upgrade -b /opt/pgsql-9.6/bin -B /usr/bin -d /var/lib/postgres/olddata -D /var/lib/postgres/data
 
  
 
=== Manual dump and reload ===
 
=== Manual dump and reload ===
Line 296: Line 356:
 
You could also do something like this (after the upgrade and install of {{Pkg|postgresql-old-upgrade}}).
 
You could also do something like this (after the upgrade and install of {{Pkg|postgresql-old-upgrade}}).
  
{{Note|Below are the commands for PostgreSQL 9.6. You can find similar commands in {{ic|/opt/}} for PostgreSQL 9.2.}}
+
{{Note|
 +
* Below are the commands for PostgreSQL 9.6. You can find similar commands in {{ic|/opt/}} for PostgreSQL 9.2.
 +
* If you had customized your {{ic|pg_hba.conf}} file, you may have to temporarily modify it to allow full access to old database cluster from local system. After upgrade is complete set your customization to new database cluster as well and [[restart]] {{ic|postgresql.service}}.
 +
}}
  
 
  # systemctl stop postgresql.service
 
  # systemctl stop postgresql.service
  # /opt/pgsql-9.6/bin/pg_ctl -D /var/lib/postgres/olddata/ start
+
  # mv /var/lib/postgres/data /var/lib/postgres/olddata
  # /opt/pgsql-9.6/bin/pg_dumpall >> old_backup.sql
+
# mkdir /var/lib/postgres/data
  # /opt/pgsql-9.6/bin/pg_ctl -D /var/lib/postgres/olddata/ stop
+
# chown postgres:postgres /var/lib/postgres/data
 +
[postgres]$ initdb -D '/var/lib/postgres/data'
 +
[postgres]$ /opt/pgsql-9.6/bin/pg_ctl -D /var/lib/postgres/olddata/ start
 +
  [postgres]$ pg_dumpall -f /tmp/old_backup.sql
 +
  [postgres]$ /opt/pgsql-9.6/bin/pg_ctl -D /var/lib/postgres/olddata/ stop
 
  # systemctl start postgresql.service
 
  # systemctl start postgresql.service
  # psql -f old_backup.sql postgres
+
  [postgres]$ psql -f /tmp/old_backup.sql postgres
  
 
== Troubleshooting ==
 
== Troubleshooting ==
Line 309: Line 376:
 
=== Improve performance of small transactions ===
 
=== Improve performance of small transactions ===
  
If you are using PostgresSQL on a local machine for development and it seems slow, you could try turning [http://www.postgresql.org/docs/current/static/runtime-config-wal.html#GUC-SYNCHRONOUS-COMMIT synchronous_commit off] in the configuration. Beware of the [http://www.postgresql.org/docs/current/static/runtime-config-wal.html#GUC-SYNCHRONOUS-COMMIT caveats], however.
+
If you are using PostgresSQL on a local machine for development and it seems slow, you could try turning [https://www.postgresql.org/docs/current/static/runtime-config-wal.html#GUC-SYNCHRONOUS-COMMIT synchronous_commit off] in the configuration. Beware of the [https://www.postgresql.org/docs/current/static/runtime-config-wal.html#GUC-SYNCHRONOUS-COMMIT caveats], however.
  
 
{{hc|/var/lib/postgres/data/postgresql.conf|2=
 
{{hc|/var/lib/postgres/data/postgresql.conf|2=
Line 325: Line 392:
 
===  Cannot connect to database through pg_connect()  ===
 
===  Cannot connect to database through pg_connect()  ===
  
Install {{Pkg|php-pgsql}} and edit the {{ic|php.ini}} file uncommenting the lines {{ic|1=extension=pdo_pgsql.so}} and {{ic|1=extension=pgsql.so}}, then restart {{ic|httpd}}.
+
Install {{Pkg|php-pgsql}} and edit the {{ic|php.ini}} file uncommenting the lines {{ic|1=extension=pdo_pgsql}} and {{ic|1=extension=pgsql}}, then restart {{ic|httpd}}.

Latest revision as of 06:26, 13 August 2018

PostgreSQL is an open source, community driven, standard compliant object-relational database system.

Installation

Tango-edit-clear.pngThis article or section needs language, wiki syntax or style improvements. See Help:Style for reference.Tango-edit-clear.png

Reason: Don't duplicate sudo and su. (Discuss in Talk:PostgreSQL#)
Install the postgresql package. It will also create a system user called postgres.
Warning: See #Upgrading PostgreSQL for necessary steps before installing new versions of the PostgreSQL packages.
Note: Commands that should be run as the postgres user are prefixed by [postgres]$ in this article.

You can switch to the PostgreSQL user by executing the following command:

$ sudo -u postgres -i
  • Otherwise using su:
$ su
# su -l postgres

See sudo(8) or su(1) for their usage.

Initial configuration

Before PostgreSQL can function correctly, the database cluster must be initialized:

[postgres]$ initdb -D '/var/lib/postgres/data'

Where -D is the default location where the database cluster must be stored (see #Change default data directory if you want to use a different one).

Note that by default, the locale and the encoding for the database cluster are derived from your current environment (using $LANG value). [1] However, depending on your settings and use cases this might not be what you want, and you can override the defaults using:

  • --locale locale, where locale is to be chosen amongst the ones defined in the file /etc/locale.conf (plus POSIX and C that are also accepted);
  • -E enconding for the encoding (which must match the chosen locale);
Example:
[postgres]$ initdb --locale en_US.UTF-8 -E UTF8 -D '/var/lib/postgres/data'

Many lines should now appear on the screen with several ending by ... ok:

The files belonging to this database system will be owned by user "postgres".
This user must also own the server process.

The database cluster will be initialized with locale "en_US.UTF-8".
The default database encoding has accordingly been set to "UTF8".
The default text search configuration will be set to "english".

Data page checksums are disabled.

fixing permissions on existing directory /var/lib/postgres/data ... ok
creating subdirectories ... ok
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting dynamic shared memory implementation ... posix
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok

WARNING: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.

Success. You can now start the database server using:

    pg_ctl -D /var/lib/postgres/ -l logfile start

If these are the kind of lines you see, then the process succeeded. Return to the regular user using exit.

Note: To read more about this WARNING, see local users configuration.
Tip: If you change the root to something other than /var/lib/postgres, you will have to edit the service file. If the root is under home, make sure to set ProtectHome to false.
Warning:
  • If the database resides on a Btrfs file system, you should consider disabling Copy-on-Write for the directory before creating any database.
  • If the database resides on a ZFS file system, you should consult ZFS#Database before creating any database.

Finally, start and enable the postgresql.service.

Create your first database/user

Tip: If you create a PostgreSQL user with the same name as your Linux username, it allows you to access the PostgreSQL database shell without having to specify a user to login (which makes it quite convenient).

Become the postgres user. Add a new database user using the createuser command:

[postgres]$ createuser --interactive

Create a new database over which the above user has read/write privileges using the createdb command (execute this command from your login shell if the database user has the same name as your Linux user, otherwise add -U database-username to the following command):

$ createdb myDatabaseName

Familiarize with PostgreSQL

Access the database shell

Become the postgres user. Start the primary database shell, psql, where you can do all your creation of databases/tables, deletion, set permissions, and run raw SQL commands. Use the -d option to connect to the database you created (without specifying a database, psql will try to access a database that matches your username).

[postgres]$ psql -d myDatabaseName

Some helpful commands:

Get help:

=> \help

Connect to a particular database:

=> \c <database>

List all users and their permission levels:

=> \du

Show summary information about all tables in the current database:

=> \dt

Exit/quit the psql shell:

=> \q or CTRL+d

There are of course many more meta-commands, but these should help you get started. To see all meta-commands run:

=> \?

Optional configuration

The PostgreSQL database server configuration file is postgresql.conf. This file is located in the data directory of the server, typically /var/lib/postgres/data. This folder also houses the other main configuration files, including the pg_hba.conf which defines authentication settings, for both local users and other hosts ones.

Note: By default, this folder will not be browsable or searchable by a regular user. This is why find and locate are not finding the configuration files.

Restricts access rights to the database superuser by default

The defaults pg_hba.conf allow any local user to connect as any database user, including the database superuser. This is likely not what you want, so in order to restrict global access to the postgress user, change the following line:

/var/lib/postgres/data/pg_hba.conf
# TYPE  DATABASE        USER            ADDRESS                 METHOD

# "local" is for Unix domain socket connections only
local   all             all                                     trust

To:

/var/lib/postgres/data/pg_hba.conf
# TYPE  DATABASE        USER            ADDRESS                 METHOD

# "local" is for Unix domain socket connections only
local   all             postgres                                peer

You might later add additional lines depending on your needs or software ones.

Configure PostgreSQL to be accessible exclusively through UNIX Sockets

In the connections and authentications section of your configuration, set:

/var/lib/postgres/data/postgresql.conf
listen_addresses = ''

This will disable network listening completely. After this you should restart postgresql.service for the changes to take effect.

Configure PostgreSQL to be accessible from remote hosts

In the connections and authentications section, set the listen_addresses line to your needs:

/var/lib/postgres/data/postgresql.conf
listen_addresses = 'localhost,my_local_ip_address'

You can use '*' to listen on all available addresses.

Note: PostgreSQL uses TCP port 5432 by default for remote connections. Make sure this port is open in your firewall and able to receive incoming connections. You can also change it in the configuration file, right below listen_addresses

Then add a line like the following to the authentication config:

/var/lib/postgres/data/pg_hba.conf
# TYPE  DATABASE        USER            ADDRESS                 METHOD
# IPv4 local connections:
host    all             all             ip_address/32   md5

where ip_address is the IP address of the remote client.

See the documentation for pg_hba.conf.

Note: Neither sending your plain password nor the md5 hash (used in the example above) over the Internet is secure if it is not done over an SSL-secured connection. See Secure TCP/IP Connections with SSL for how to configure PostgreSQL with SSL.

After this you should restart postgresql.service for the changes to take effect.

For troubleshooting take a look in the server log file:

$ journalctl -u postgresql.service

Configure PostgreSQL authenticate against PAM

PostgreSQL offers a number of authentication methods. If you would like to allow users to authenticate with their system password, additional steps are necessary. First you need to enable PAM for the connection.

For example, the same configuration as above, but with PAM enabled:

/var/lib/postgres/data/pg_hba.conf
# IPv4 local connections:
host   all   all   my_remote_client_ip_address/32   pam

The PostgreSQL server is however running without root privileges and will not be able to access /etc/shadow. We can work around that by allowing the postgres group to access this file:

# setfacl -m g:postgres:r /etc/shadow

Change default data directory

The default directory where all your newly created databases will be stored is /var/lib/postgres/data. To change this, follow these steps:

Create the new directory and make the postgres user its owner:

# mkdir -p /pathto/pgroot/data
# chown -R postgres:postgres /pathto/pgroot

Become the postgres user, and initialize the new cluster:

[postgres]$ initdb -D /pathto/pgroot/data

Edit postgresql.service to create a drop-in file and override the Environment and PIDFile settings. For example:

[Service]
Environment=PGROOT=/pathto/pgroot
PIDFile=/pathto/pgroot/data/postmaster.pid

If you want to use /home directory for default directory or for tablespaces, add one more line in this file:

ProtectHome=false

Change default encoding of new databases to UTF-8

Note: If you ran initdb with -E UTF8 or while using an UTF-8 locale, these steps are not required.

When creating a new database (e.g. with createdb blog) PostgreSQL actually copies a template database. There are two predefined templates: template0 is vanilla, while template1 is meant as an on-site template changeable by the administrator and is used by default. In order to change the encoding of a new database, one of the options is to change on-site template1. To do this, log into PostgreSQL shell (psql) and execute the following:

First, we need to drop template1. Templates cannot be dropped, so we first modify it so it is an ordinary database:

UPDATE pg_database SET datistemplate = FALSE WHERE datname = 'template1';

Now we can drop it:

DROP DATABASE template1;

The next step is to create a new database from template0, with a new default encoding:

CREATE DATABASE template1 WITH TEMPLATE = template0 ENCODING = 'UNICODE';

Now modify template1 so it is actually a template:

UPDATE pg_database SET datistemplate = TRUE WHERE datname = 'template1';

Optionally, if you do not want anyone connecting to this template, set datallowconn to FALSE:

UPDATE pg_database SET datallowconn = FALSE WHERE datname = 'template1';
Note: This last step can create problems when upgrading via pg_upgrade.

Now you can create a new database:

[postgres]$ createdb blog

If you log back in to psql and check the databases, you should see the proper encoding of your new database:

\l
                              List of databases
  Name    |  Owner   | Encoding  | Collation | Ctype |   Access privileges
-----------+----------+-----------+-----------+-------+----------------------
blog      | postgres | UTF8      | C         | C     |
postgres  | postgres | SQL_ASCII | C         | C     |
template0 | postgres | SQL_ASCII | C         | C     | =c/postgres
                                                     : postgres=CTc/postgres
template1 | postgres | UTF8      | C         | C     |

Administration tools

  • Adminer — Web-based database management tool for multiple database systems.
https://www.adminer.org || adminerAUR
  • phpPgAdmin — Web-based administration tool for PostgreSQL.
http://phppgadmin.sourceforge.net || phppgadmin
  • pgAdmin — GUI-based administration tool for PostgreSQL.
https://www.pgadmin.org/ || pgadmin4
  • pgModeler — Graphical schema designer for PostgreSQL.
https://pgmodeler.io/ || pgmodelerAUR

Upgrading PostgreSQL

Tango-edit-clear.pngThis article or section needs language, wiki syntax or style improvements. See Help:Style for reference.Tango-edit-clear.png

Reason: Don't show basic systemctl commands, etc. (Discuss in Talk:PostgreSQL#)

Upgrading major PostgreSQL versions requires some extra maintenance.

Note:
  • Official PostgreSQL upgrade documentation should be followed.
  • From version 10.0 onwards PostgreSQL changed its versioning scheme. Earlier upgrade from version 9.x to 9.y was considered as major upgrade. Now upgrade from version 10.x to 10.y is considered as minor upgrade and upgrade from version 10.x to 11.y is considered as major upgrade.
Warning: The following instructions could cause data loss. Do not run the commands below blindly, without understanding what they do. Backup database first.

It is recommended to add the following to your /etc/pacman.conf file:

IgnorePkg = postgresql*

This will ensure you do not accidentally upgrade the database to an incompatible version. When an upgrade is available, pacman will notify you that it is skipping the upgrade because of the entry in pacman.conf. Minor version upgrades are safe to perform. However, if you do an accidental upgrade to a different major version, you might not be able to access any of your data. Always check the PostgreSQL home page to be sure of what steps are required for each upgrade. For a bit about why this is the case, see the versioning policy.

There are two main ways to upgrade your PostgreSQL database. Read the official documentation for details.

For those wishing to use pg_upgrade, a postgresql-old-upgrade package is available that will always run one major version behind the real PostgreSQL package. This can be installed side-by-side with the new version of PostgreSQL.

Note that the databases cluster directory does not change from version to version, so before running pg_upgrade, it is necessary to rename your existing data directory and migrate into a new directory. The new databases cluster must be initialized, as described in the #Installation section.

When you are ready, stop the postgresql service, upgrade the following packages: postgresql, postgresql-libs, and postgresql-old-upgrade. Finally upgrade the databases cluster.

Stop and make sure PostgreSQL is stopped:

# systemctl stop postgresql.service
# systemctl status postgresql.service

Upgrade the packages:

# pacman -S postgresql postgresql-libs postgresql-old-upgrade

Rename the databases cluster directory, and create an empty one:

# mv /var/lib/postgres/data /var/lib/postgres/olddata
# mkdir /var/lib/postgres/data /var/lib/postgres/tmp
# chown postgres:postgres /var/lib/postgres/data /var/lib/postgres/tmp
[postgres]$ initdb -D '/var/lib/postgres/data'

Upgrade the cluster:

[postgres]$ cd /var/lib/postgres/tmp
[postgres]$ pg_upgrade -b /opt/pgsql-9.6/bin -B /usr/bin -d /var/lib/postgres/olddata -D /var/lib/postgres/data

pg_upgrade will perform the upgrade and create some scripts in /var/lib/postgres/tmp/. Follow the instructions given on screen and act accordingly. You may delete the /var/lib/postgres/tmp directory once the upgrade is completely over.

Start the cluster:

# systemctl start postgresql.service

Manual dump and reload

You could also do something like this (after the upgrade and install of postgresql-old-upgrade).

Note:
  • Below are the commands for PostgreSQL 9.6. You can find similar commands in /opt/ for PostgreSQL 9.2.
  • If you had customized your pg_hba.conf file, you may have to temporarily modify it to allow full access to old database cluster from local system. After upgrade is complete set your customization to new database cluster as well and restart postgresql.service.
# systemctl stop postgresql.service
# mv /var/lib/postgres/data /var/lib/postgres/olddata
# mkdir /var/lib/postgres/data
# chown postgres:postgres /var/lib/postgres/data
[postgres]$ initdb -D '/var/lib/postgres/data'
[postgres]$ /opt/pgsql-9.6/bin/pg_ctl -D /var/lib/postgres/olddata/ start
[postgres]$ pg_dumpall -f /tmp/old_backup.sql
[postgres]$ /opt/pgsql-9.6/bin/pg_ctl -D /var/lib/postgres/olddata/ stop
# systemctl start postgresql.service
[postgres]$ psql -f /tmp/old_backup.sql postgres

Troubleshooting

Improve performance of small transactions

If you are using PostgresSQL on a local machine for development and it seems slow, you could try turning synchronous_commit off in the configuration. Beware of the caveats, however.

/var/lib/postgres/data/postgresql.conf
synchronous_commit = off

Prevent disk writes when idle

PostgreSQL periodically updates its internal "statistics" file. By default, this file is stored on disk, which prevents disks from spinning down on laptops and causes hard drive seek noise. It is simple and safe to relocate this file to a memory-only file system with the following configuration option:

/var/lib/postgres/data/postgresql.conf
stats_temp_directory = '/run/postgresql'

Cannot connect to database through pg_connect()

Install php-pgsql and edit the php.ini file uncommenting the lines extension=pdo_pgsql and extension=pgsql, then restart httpd.