https://wiki.archlinux.org/api.php?action=feedcontributions&user=Phiresky&feedformat=atomArchWiki - User contributions [en]2024-03-29T14:20:10ZUser contributionsMediaWiki 1.41.0https://wiki.archlinux.org/index.php?title=Cgroups&diff=802280Cgroups2024-03-04T16:58:41Z<p>Phiresky: Add ad-hoc CPU restriction to examples</p>
<hr />
<div>{{Lowercase title}}<br />
[[Category:Kernel]]<br />
[[Category:Virtualization]]<br />
[[ja:Cgroups]]<br />
[[pt:Cgroups]]<br />
{{Related articles start}}<br />
{{Related|Linux Containers}}<br />
{{Related|systemd-nspawn}}<br />
{{Related|Docker}}<br />
{{Related|limits.conf}}<br />
{{Related articles end}}<br />
'''Control groups''' (or '''cgroups''' as they are commonly known) are a feature provided by the Linux kernel to manage, restrict, and audit groups of processes. Compared to other approaches like the {{man|1|nice}} command or {{ic|/etc/security/limits.conf}}, cgroups are more flexible as they can operate on (sub)sets of processes (possibly with different system users).<br />
<br />
Control groups can be accessed with various tools:<br />
<br />
* using directives in [[systemd]] unit files to specify limits for services and slices;<br />
* by accessing the {{ic|cgroup}} filesystem directly;<br />
* via tools like {{ic|cgcreate}}, {{ic|cgexec}} and {{ic|cgclassify}} (part of the {{AUR|libcgroup}} and {{AUR|libcgroup-git}} packages);<br />
* using the "rules engine daemon" to automatically move certain users/groups/commands to groups ({{ic|/etc/cgrules.conf}} and {{ic|cgconfig.service}}) (part of the {{AUR|libcgroup}} and {{AUR|libcgroup-git}} packages); and<br />
* through other software such as [[Linux Containers]] (LXC) virtualization.<br />
<br />
For Arch Linux, systemd is the preferred and easiest method of invoking and configuring cgroups as it is a part of the default installation.<br />
<br />
== Installing ==<br />
<br />
Make sure you have one of these packages [[install]]ed for automated cgroup handling:<br />
<br />
* {{Pkg|systemd}} - for controlling resources of a systemd service.<br />
* {{AUR|libcgroup}}, {{AUR|libcgroup-git}} - set of standalone tools ({{ic|cgcreate}}, {{ic|cgclassify}}, persistence via {{ic|cgconfig.conf}}).<br />
<br />
== With systemd ==<br />
<br />
=== Hierarchy ===<br />
<br />
Current cgroup hierarchy can be seen with {{ic|systemctl status}} or {{ic|systemd-cgls}} command.<br />
<br />
{{hc|$ systemctl status|<nowiki><br />
● myarchlinux<br />
State: running<br />
Jobs: 0 queued<br />
Failed: 0 units<br />
Since: Wed 2019-12-04 22:16:28 UTC; 1 day 4h ago<br />
CGroup: /<br />
├─user.slice <br />
│ └─user-1000.slice <br />
│ ├─user@1000.service <br />
│ │ ├─gnome-shell-wayland.service <br />
│ │ │ ├─ 1129 /usr/bin/gnome-shell<br />
│ │ ├─gnome-terminal-server.service <br />
│ │ │ ├─33519 /usr/lib/gnome-terminal-server<br />
│ │ │ ├─37298 fish<br />
│ │ │ └─39239 systemctl status<br />
│ │ ├─init.scope <br />
│ │ │ ├─1066 /usr/lib/systemd/systemd --user<br />
│ │ │ └─1067 (sd-pam)<br />
│ └─session-2.scope <br />
│ ├─1053 gdm-session-worker [pam/gdm-password]<br />
│ ├─1078 /usr/bin/gnome-keyring-daemon --daemonize --login<br />
│ ├─1082 /usr/lib/gdm-wayland-session /usr/bin/gnome-session<br />
│ ├─1086 /usr/lib/gnome-session-binary<br />
│ └─3514 /usr/bin/ssh-agent -D -a /run/user/1000/keyring/.ssh<br />
├─init.scope <br />
│ └─1 /sbin/init<br />
└─system.slice <br />
├─systemd-udevd.service <br />
│ └─285 /usr/lib/systemd/systemd-udevd<br />
├─systemd-journald.service <br />
│ └─272 /usr/lib/systemd/systemd-journald<br />
├─NetworkManager.service <br />
│ └─656 /usr/bin/NetworkManager --no-daemon<br />
├─gdm.service <br />
│ └─668 /usr/bin/gdm<br />
└─systemd-logind.service <br />
└─654 /usr/lib/systemd/systemd-logind<br />
</nowiki>}}<br />
<br />
=== Find cgroup of a process ===<br />
<br />
The cgroup name of a process can be found in {{ic|/proc/''PID''/cgroup}}.<br />
<br />
For example, the cgroup of the shell:<br />
<br />
{{hc|$ cat /proc/self/cgroup|<br />
0::/user.slice/user-1000.slice/session-3.scope<br />
}}<br />
<br />
=== cgroup resource usage ===<br />
<br />
The {{ic|systemd-cgtop}} command can be used to see the resource usage:<br />
<br />
{{hc|$ systemd-cgtop|<br />
Control Group Tasks %CPU Memory Input/s Output/s<br />
user.slice 540 152,8 3.3G - -<br />
user.slice/user-1000.slice 540 152,8 3.3G - -<br />
user.slice/u…000.slice/session-1.scope 425 149,5 3.1G - -<br />
system.slice 37 - 215.6M - -<br />
}}<br />
<br />
=== Custom cgroups ===<br />
<br />
{{man|5|systemd.slice}} systemd unit files can be used to define a custom cgroup configuration. They must be placed in a systemd directory, such as {{ic|/etc/systemd/system/}}. The resource control options that can be assigned are documented in {{man|5|systemd.resource-control}}.<br />
<br />
This is an example slice unit that only allows 30% of one CPU to be used:<br />
<br />
{{hc|/etc/systemd/system/my.slice|2=<br />
[Slice]<br />
CPUQuota=30%<br />
}}<br />
<br />
Remember to do a [[daemon-reload]] to pick up any new or changed {{ic|.slice}} files.<br />
<br />
=== As service ===<br />
<br />
==== Service unit file ====<br />
<br />
Resources can be directly specified in service definition or as a [[drop-in file]]: <br />
<br />
[Service]<br />
MemoryMax=1G <br />
<br />
This example limits the service to 1 gigabyte.<br />
<br />
==== Grouping unit under a slice ====<br />
<br />
Service can be specified what slice to run in:<br />
<br />
{{bc|1=<br />
[Service]<br />
Slice=my.slice<br />
}}<br />
<br />
=== As root ===<br />
<br />
{{ic|systemd-run}} can be used to run a command in a specific slice. <br />
<br />
# systemd-run --slice=''my.slice'' ''command''<br />
<br />
{{ic|1=--uid=''username''}} option can be used to spawn the command as specific user.<br />
<br />
# systemd-run --uid=''username'' --slice=''my.slice'' ''command''<br />
<br />
The {{ic|--shell}} option can be used to spawn a command shell inside the slice.<br />
<br />
=== As unprivileged user ===<br />
<br />
Unprivileged users can divide the resources provided to them into new cgroups, if some conditions are met.<br />
<br />
Cgroups v2 must be utilized for a non-root user to be allowed managing cgroup resources.<br />
<br />
==== Controller types ====<br />
<br />
Not all resources can be controlled by user. <br />
<br />
{| class="wikitable"<br />
! Controller !! Can be controlled by user !! Options<br />
|-<br />
| cpu || {{Y|Requires delegation}} || CPUAccounting, CPUWeight, CPUQuota, AllowedCPUs, AllowedMemoryNodes<br />
|-<br />
| io || {{Y|Requires delegation}} || IOWeight, IOReadBandwidthMax, IOWriteBandwidthMax, IODeviceLatencyTargetSec<br />
|-<br />
| memory || {{Yes}} || MemoryLow, MemoryHigh, MemoryMax, MemorySwapMax<br />
|-<br />
| pids || {{Yes}} || TasksMax<br />
|-<br />
| rdma || {{No}} || ?<br />
|-<br />
| eBPF || {{No}} || IPAddressDeny, DeviceAllow, DevicePolicy<br />
|}<br />
<br />
{{Note|eBPF is technically not a controller but those systemd options implemented using it and only root is allowed to set them.}}<br />
<br />
==== User delegation ====<br />
<br />
For user to control cpu and io resources, the resources need to be delegated. This can be done with a [[drop-in file]].<br />
<br />
For example if your user id is 1000:<br />
<br />
{{hc|/etc/systemd/system/user@1000.service.d/delegate.conf|2=<br />
[Service]<br />
Delegate=cpu cpuset io<br />
}}<br />
<br />
Reboot and verify that the slice your user session is under has cpu and io controller:<br />
<br />
{{hc|$ cat /sys/fs/cgroup/user.slice/user-1000.slice/cgroup.controllers|<br />
'''cpuset cpu io''' memory pids<br />
}}<br />
<br />
==== User-defined slices ====<br />
<br />
The user slice files can be placed in {{ic|~/.config/systemd/user/}}.<br />
<br />
To run the command under certain slice:<br />
<br />
$ systemd-run --user --slice=''my.slice'' ''command''<br />
<br />
You can also run your login shell inside the slice:<br />
<br />
$ systemd-run --user --slice=''my.slice'' --shell<br />
<br />
=== Run-time adjustment ===<br />
<br />
cgroups resources can be adjusted at run-time using {{ic|systemctl set-property}} command. Option syntax is the same as in {{man|5|systemd.resource-control}}.<br />
<br />
{{Warning| The adjustments will be made '''permanent''' unless {{ic|--runtime}} option is passed. Adjustments are saved at {{ic|/etc/systemd/system.control/}} for system wide options and {{ic|.config/systemd/user.control/}} for user options.}} <br />
<br />
{{Note| Not all resources changes immediately take effect. For example, changing TaskMax will only take effect on spawning new processes.}}<br />
<br />
For example, cutting off internet access for all user sessions:<br />
<br />
$ systemctl set-property user.slice IPAddressDeny=any<br />
<br />
== With libcgroup ==<br />
<br />
You can [[enable]] the {{ic|cgconfig}} service with systemd. This allows you to track any errors in {{ic|cgconfig.conf}} more easily.<br />
<br />
=== Ad-hoc groups ===<br />
<br />
One of the powers of cgroups is that you can create "ad-hoc" groups on the fly. You can even grant the privileges to create custom groups to regular users. {{ic|groupname}} is the cgroup name:<br />
<br />
# cgcreate -a ''user'' -t ''user'' -g memory,cpu:''groupname''<br />
<br />
Now all the tunables in the group {{ic|groupname}} are writable by your user:<br />
<br />
{{hc|$ ls -l /sys/fs/cgroup/memory/''groupname''|<br />
total 0<br />
-rwxrwxr-x 1 user root 0 Sep 25 00:39 cgroup.event_control<br />
-rwxrwxr-x 1 user root 0 Sep 25 00:39 cgroup.procs<br />
-rwxrwxr-x 1 user root 0 Sep 25 00:39 cpu.rt_period_us<br />
-rwxrwxr-x 1 user root 0 Sep 25 00:39 cpu.rt_runtime_us<br />
-rwxrwxr-x 1 user root 0 Sep 25 00:39 cpu.shares<br />
-rwxrwxr-x 1 user root 0 Sep 25 00:39 notify_on_release<br />
-rwxrwxr-x 1 user root 0 Sep 25 00:39 tasks<br />
}}<br />
<br />
Cgroups are hierarchical, so you can create as many subgroups as you like. If a normal user wants to run a {{ic|bash}} shell under a new subgroup called {{ic|foo}}:<br />
<br />
$ cgcreate -g memory,cpu:'''groupname/foo'''<br />
$ '''cgexec''' -g memory,cpu:groupname/foo '''bash'''<br />
<br />
To make sure (only meaningful for legacy (v1) cgroups):<br />
<br />
{{hc|$ cat /proc/self/cgroup|<br />
11:memory:/groupname/foo<br />
6:cpu:/groupname/foo<br />
}}<br />
<br />
A new subdirectory was created for this group. To limit the memory usage of all processes in this group to 10 MB, run the following:<br />
<br />
$ echo 10000000 > /sys/fs/cgroup/memory/groupname/foo/memory.limit_in_bytes<br />
<br />
Note that the memory limit applies to RAM use only -- once tasks hit this limit, they will begin to swap. But it will not affect the performance of other processes significantly.<br />
<br />
Similarly you can change the CPU priority ("shares") of this group. By default all groups have 1024 shares. A group with 100 shares will get a ~10% portion of the CPU time:<br />
<br />
$ echo 100 > /sys/fs/cgroup/cpu/groupname/foo/cpu.shares<br />
<br />
You can find more tunables or statistics by listing the cgroup directory.<br />
<br />
You can also change the cgroup of already running processes. To move all 'bash' commands to this group:<br />
<br />
$ pidof bash<br />
13244 13266<br />
$ '''cgclassify''' -g memory,cpu:groupname/foo `pidof bash`<br />
$ cat /proc/13244/cgroup<br />
11:memory:/groupname/foo<br />
6:cpu:/groupname/foo<br />
<br />
=== Persistent group configuration ===<br />
<br />
{{Note|when using [[Systemd]] ≥ 205 to manage cgroups, you can ignore this file entirely.}}<br />
<br />
If you want your cgroups to be created at boot, you can define them in {{ic|/etc/cgconfig.conf}} instead. For example, the "groupname" has a permission for {{ic|$USER}} and ''users'' of group {{ic|$GROUP}} to manage limits and add tasks. A ''subgroup'' "groupname/foo" group definitions would look like this: <br />
<br />
{{hc|/etc/cgconfig.conf |2=<br />
group '''groupname''' {<br />
perm {<br />
# who can manage limits<br />
admin {<br />
uid = '''$USER''';<br />
gid = '''$GROUP''';<br />
}<br />
# who can add tasks to this group<br />
task {<br />
uid = '''$USER''';<br />
gid = '''$GROUP''';<br />
}<br />
}<br />
# create this group in cpu and memory controllers<br />
cpu { }<br />
memory { }<br />
}<br />
<br />
group '''groupname/foo''' {<br />
cpu {<br />
'''cpu.shares''' = 100;<br />
}<br />
memory {<br />
'''memory.limit_in_bytes''' = 10000000;<br />
}<br />
}<br />
}}<br />
<br />
{{Note|1=<nowiki/><br />
* Comments should begin at the start of a line! The '''#''' character for comments must appear as the first character of a line. Else, ''cgconfigparser'' will have problem parsing it but will only report {{ic|cgroup change of group failed}} as the error, unless you started ''cgconfig'' with [[Systemd]]<br />
* The permissions section is optional.<br />
* The {{ic|/sys/fs/cgroup/}} hierarchy directory containing all ''controllers'' sub-directories is already created and mounted at boot as a virtual file system. This gives the ability to create a new group entry with the {{ic|''$CONTROLLER-NAME { }''}} command. If for any reason you want to create and mount hierachies in another place, you will then need to write a second entry in {{ic|/etc/cgconfig.conf}} following this way :<br />
<br />
mount { <br />
cpuset = /your/path/''groupname'';<br />
}<br />
<br />
This is equivalent to these shell commands:<br />
# mkdir /your/path/''groupname''<br />
# mount -t /your/path -o cpuset ''groupname'' /your/path/''groupname''<br />
<nowiki/>}}<br />
<br />
== With the cgroup virtual filesystem ==<br />
<br />
{{Expansion|this section must be integrated with the following sections}}<br />
<br />
Starting with systemd 232, the ''cgm'' method described in the next section, this section will instead describe a manual method to limit memory usage.<br />
<br />
Create a new cgroup named ''groupname'':<br />
<br />
# mkdir /sys/fs/cgroup/memory/''groupname''<br />
<br />
Example: set the maximum memory limit to 100MB:<br />
<br />
# echo 100000000 > /sys/fs/cgroup/memory/''groupname''/memory.limit_in_bytes<br />
<br />
Move a process to the cgroup:<br />
<br />
# echo ''pid'' > /sys/fs/cgroup/memory/''groupname''/cgroup.procs<br />
<br />
{{Note|In this last command, only one PID can be written at a time, so repeat this for each process that must be moved.}}<br />
<br />
== Examples ==<br />
<br />
=== Restrict memory or CPU use of a command ===<br />
<br />
The following example shows a ''cgroup'' that constrains a given command to 2GB of memory.<br />
<br />
$ systemd-run --scope -p MemoryMax=2G --user ''command''<br />
<br />
The following example shows a command restricted to 20% of one CPU core.<br />
<br />
$ systemd-run --scope -p CPUQuota="20%" --user ''command''<br />
<br />
=== Matlab ===<br />
<br />
Doing large calculations in [[MATLAB]] can crash your system, because Matlab does not have any protection against taking all your machine's memory or CPU. The following examples show a ''cgroup'' that constrains Matlab to first 6 CPU cores and 5 GB of memory.<br />
<br />
==== With systemd ====<br />
<br />
{{hc|~/.config/systemd/user/matlab.slice|2=<br />
[Slice]<br />
AllowedCPUs=0-5<br />
MemoryHigh=6G<br />
}}<br />
<br />
Launch Matlab like this (be sure to use the right path):<br />
<br />
$ systemd-run --user --slice=matlab.slice /opt/MATLAB/2012b/bin/matlab -desktop<br />
<br />
==== With libcgroup ====<br />
<br />
{{hc|/etc/cgconfig.conf|2=<br />
group matlab {<br />
perm {<br />
admin {<br />
uid = ''username'';<br />
}<br />
task {<br />
uid = ''username'';<br />
}<br />
}<br />
<br />
cpuset {<br />
cpuset.mems="0";<br />
cpuset.cpus="0-5";<br />
}<br />
memory {<br />
memory.limit_in_bytes = 5000000000;<br />
}<br />
}<br />
}}<br />
<br />
Change {{ic|''username''}} to the user Matlab is run as.<br />
<br />
You can also restrict the CPU share with the {{ic|cpu}} constraint.<br />
<br />
Launch Matlab like this (be sure to use the right path):<br />
<br />
$ cgexec -g memory,cpuset:matlab /opt/MATLAB/2012b/bin/matlab -desktop<br />
<br />
== Documentation ==<br />
<br />
* For information on controllers and what certain switches and tunables mean, refer to kernel's documentation [https://docs.kernel.org/admin-guide/cgroup-v1/index.html v1] or [https://docs.kernel.org/admin-guide/cgroup-v2.html v2] (or install {{Pkg|linux-docs}} and see {{ic|/usr/src/linux/Documentation/cgroup}})<br />
* Linux manual page: {{man|7|cgroups}}<br />
* A detailed and complete Resource Management Guide can be found in the [https://access.redhat.com/documentation/en-us/red_hat_enterprise_linux/6/html/resource_management_guide/ch01#sec-How_Control_Groups_Are_Organized Red Hat Enterprise Linux documentation].<br />
For commands and configuration files, see relevant man pages, e.g. {{man|1|cgcreate|url=https://github.com/libcgroup/libcgroup/blob/main/doc/man/cgcreate.1}} or {{man|5|cgrules.conf|url=https://github.com/libcgroup/libcgroup/blob/main/doc/man/cgrules.conf.5}}<br />
<br />
== Tips and tricks ==<br />
<br />
=== Enable cgroup v1 ===<br />
<br />
Cgroup v2 is now enabled by default. If you want to switch to cgroup v1 instead, you need to set the following [[kernel parameter]]:<br />
<br />
systemd.unified_cgroup_hierarchy=0<br />
<br />
== See also ==<br />
<br />
* [https://systemd.io/CGROUP_DELEGATION/ systemd cgroups hacker guide]<br />
* [https://chrisdown.name/talks/cgroupv2/cgroupv2-fosdem.pdf cgroupv2: Linux's new unified control group system]</div>Phireskyhttps://wiki.archlinux.org/index.php?title=Bcachefs&diff=797158Bcachefs2024-01-15T16:05:08Z<p>Phiresky: 6.7 is in core now</p>
<hr />
<div>[[Category:File systems]]<br />
[[ja:Bcachefs]]<br />
[[zh-hans:Bcachefs]]<br />
{{Related articles start}}<br />
{{Related|File systems}}<br />
{{Related articles end}}<br />
<br />
[https://bcachefs.org/ Bcachefs] is a next-generation CoW filesystem that aims to provide features from [[Btrfs]] and [[ZFS]] with a cleaner codebase, more stability, greater speed and a GPL-compatible license.<br />
<br />
It is built upon [[Bcache]] and is mainly developed by Kent Overstreet.<br />
<br />
== Installation ==<br />
<br />
As of [https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=9e87705289667a6c5185c619ea32f3d39314eb1b kernel 6.7 (January 2024)] Bcachefs has been merged into the upstream [[Kernel]] so it is available in the {{Pkg|linux}} package. Other kernel packages may be based on older versions than 6.7 and need special patches for Bcachefs.<br />
<br />
The Bcachefs userspace tools are available from {{Pkg|bcachefs-tools}}.<br />
<br />
== Setup ==<br />
<br />
=== Single drive ===<br />
<br />
# bcachefs format /dev/sda<br />
# mount -t bcachefs /dev/sda /mnt<br />
<br />
=== Multiple drives ===<br />
<br />
Bcachefs stripes data by default, similar to RAID0. Redundancy is handled via the '''replicas''' option. 2 drives with {{ic|--replicas{{=}}2}} is equivalent to RAID1, 4 drives with {{ic|--replicas{{=}}2}} is equivalent to RAID10, etc.<br />
<br />
# bcachefs format /dev/sda /dev/sdb --replicas=''n''<br />
# mount -t bcachefs /dev/sda:/dev/sdb /mnt<br />
<br />
Heterogeneous drives are supported. If they are different sizes, larger stripes will be used on some, so that they all fill up at the same rate. If they are different speeds, reads for replicated data will be sent to the ones with the lowest IO latency. If some are more reliable than others (a hardware raid device, for example) you can set {{ic|--durability{{=}}2 ''device''}} to count each copy of data on that device as 2 replicas.<br />
<br />
=== SSD caching ===<br />
<br />
Bcachefs has 3 storage targets: background, foreground, and promote. Writes to the filesystem prioritize the foreground drives, which are then moved to the background over time. Reads are cached on the promote drives.<br />
<br />
{{Note|These are only priority guidelines for a single large pool. Writes will go directly to the background if the foreground is full, or to promote if they both are. Metadata will prefer the foreground, but can be written to any of them. Be careful when removing a cache drive, as it may still contain data. see [[#Removing a device]]}}<br />
<br />
A recommended configuration is to use an ssd group for the foreground and promote, and an hdd group for the background (a writeback cache).<br />
<br />
# bcachefs format \<br />
--label=ssd.ssd1 /dev/sda \<br />
--label=ssd.ssd2 /dev/sdb \<br />
--label=hdd.hdd1 /dev/sdc \<br />
--label=hdd.hdd2 /dev/sdd \<br />
--label=hdd.hdd3 /dev/sde \<br />
--label=hdd.hdd4 /dev/sdf \<br />
--replicas=2 \<br />
--foreground_target=ssd \<br />
--promote_target=ssd \<br />
--background_target=hdd<br />
# mount -t bcachefs /dev/sda:/dev/sdb:/dev/sdc:/dev/sdd:/dev/sde:/dev/sdf /mnt<br />
<br />
For a writethrough cache, do the same as above, but set {{ic|--durability{{=}}0 ''device''}} on each of the ssd devices.<br />
For a writearound cache, foreground target to the hdd group, and promote target to the ssd group.<br />
<br />
== Configuration ==<br />
<br />
{{Expansion|Missing details on which options should be used}}<br />
<br />
Most options can be set at either during {{ic|bcachefs format}}, at mount time ({{ic|1=mount -o option=value}}), or through sysfs ({{ic|echo X > /sys/fs/bcachefs/''UUID''/options/''option''}}). Setting the option during format or changing it through sysfs saves it in the filesystem's superblock, making it the default for those drives. Mount options override those defaults.<br />
<br />
{{Note|The filesystem must be mounted for sysfs to be available. All operations except fsck are possible on a live filesystem.}}<br />
<br />
* data_checksum, metadata_checksum (none, crc32c, crc64)<br />
* (foreground) compression, background_compression (none, lz4, gzip, zstd)<br />
* foreground_target, background_target, promote_target<br />
<br />
The following can also be set on a per directory or per file basis with {{ic|1=bcachefs setattr ''file'' --option=value}}<br />
<br />
* data_replicas<br />
* data_checksum<br />
* compression, background_compression<br />
* foreground_target, background_target, promote_target<br />
<br />
{{Note|Disk usage reporting currently shows uncompressed size. Compression is otherwise complete.}}<br />
<br />
=== Changing a device's group ===<br />
<br />
# echo ''group.drive_name'' > /sys/fs/bcachefs/''filesystem_uuid''/dev-''X''/label<br />
<br />
{{Note|This requires a remount to take effect.}}<br />
<br />
=== Adding a device ===<br />
<br />
# bcachefs device add --label=''group.drive_name'' /mnt /dev/''device''<br />
<br />
If this is the first drive in a group, you will need to change the target settings to make use of it. This example is for adding a cache drive.<br />
<br />
# echo ''new_group'' > /sys/fs/bcachefs/''filesystem_uuid''/options/promote_target<br />
# echo ''new_group'' > /sys/fs/bcachefs/''filesystem_uuid''/options/foreground_target<br />
# echo ''old_group'' > /sys/fs/bcachefs/''filesystem_uuid''/options/background_target<br />
<br />
{{Note|Only new writes will be striped across added devices. Existing ones will be unchanged until disk usage reaches a certain threshold, when the disk rebalance is triggered. It is not currently possible to manually trigger a rebalance/restripe.}}<br />
<br />
=== Removing a device ===<br />
<br />
First make sure there are at least 2 metadata replicas (Evacuate does not appear to work for metadata). If your data and metadata are already replicated, you may skip this step.<br />
<br />
# echo 2 > /sys/fs/bcachefs/''UUID''/options/metadata_replicas<br />
# bcachefs data rereplicate /mnt<br />
# bcachefs device set-state ''device'' readonly<br />
# bcachefs device evacuate ''device''<br />
<br />
To remove the device:<br />
<br />
# bcachefs device remove ''device''<br />
# bcachefs data rereplicate /mnt<br />
<br />
== Tips and tricks ==<br />
<br />
{{Expansion|Information on auto-mounting would be useful}}<br />
<br />
Check the [[journal]] for more useful error messages.<br />
<br />
== See also ==<br />
<br />
* [https://bcachefs.org/bcachefs-principles-of-operation.pdf Official Manual]<br />
* [https://www.patreon.com/bcachefs Kent Overstreet's Patreon page]<br />
* [[Wikipedia:Bcachefs]]</div>Phireskyhttps://wiki.archlinux.org/index.php?title=Pacman/Tips_and_tricks&diff=728746Pacman/Tips and tricks2022-05-07T11:53:12Z<p>Phiresky: add a bit more info about why orphan packages can accumulate</p>
<hr />
<div>{{Lowercase title}}<br />
[[Category:Package manager]]<br />
[[de:Pacman-Tipps]]<br />
[[es:Pacman (Español)/Tips and tricks]]<br />
[[fa:Pacman tips]]<br />
[[fr:Pacman (Français)/Tips and tricks]]<br />
[[ja:Pacman ヒント]]<br />
[[pt:Pacman (Português)/Tips and tricks]]<br />
[[ru:Pacman (Русский)/Tips and tricks]]<br />
[[zh-hans:Pacman (简体中文)/Tips and tricks]]<br />
{{Related articles start}}<br />
{{Related|Mirrors}}<br />
{{Related|Creating packages}}<br />
{{Related articles end}}<br />
For general methods to improve the flexibility of the provided tips or ''pacman'' itself, see [[Core utilities]] and [[Bash]].<br />
<br />
== Maintenance ==<br />
<br />
{{Expansion|{{ic|1=Usage=}} introduced with ''pacman'' 4.2, see [http://allanmcrae.com/2014/12/pacman-4-2-released/]}}<br />
<br />
{{Note|Instead of using ''comm'' (which requires sorted input with ''sort'') in the sections below, you may also use {{ic|grep -Fxf}} or {{ic|grep -Fxvf}}.}}<br />
<br />
See also [[System maintenance]].<br />
<br />
=== Listing packages ===<br />
<br />
==== With version ====<br />
<br />
You may want to get the list of installed packages with their version, which is useful when reporting bugs or discussing installed packages.<br />
<br />
* List all explicitly installed packages: {{ic|pacman -Qe}}.<br />
* List all packages in the [[package group]] named {{ic|''group''}}: {{ic|pacman -Sg ''group''}}<br />
* List all foreign packages (typically manually downloaded and installed or packages removed from the repositories): {{ic|pacman -Qm}}.<br />
* List all native packages (installed from the sync database): {{ic|pacman -Qn}}.<br />
* List all explicitly installed native packages (i.e. present in the sync database) that are not direct or optional dependencies: {{ic|pacman -Qent}}.<br />
* List packages by regex: {{ic|pacman -Qs ''regex''}}.<br />
* List packages by regex with custom output format (needs {{Pkg|expac}}): {{ic|expac -s "%-30n %v" ''regex''}}.<br />
<br />
==== With size ====<br />
<br />
Figuring out which packages are largest can be useful when trying to free space on your hard drive. There are two options here: get the size of individual packages, or get the size of packages and their dependencies.<br />
<br />
===== Individual packages =====<br />
<br />
The following command will list all installed packages and their individual sizes:<br />
<br />
$ LC_ALL=C pacman -Qi | awk '/^Name/{name=$3} /^Installed Size/{print $4$5, name}' | sort -h<br />
<br />
===== Packages and dependencies =====<br />
<br />
To list package sizes with their dependencies,<br />
<br />
* Install {{Pkg|expac}} and run {{ic|expac -H M '%m\t%n' {{!}} sort -h}}.<br />
* Run {{Pkg|pacgraph}} with the {{ic|-c}} option.<br />
<br />
To list the download size of several packages (leave {{ic|''packages''}} blank to list all packages):<br />
<br />
$ expac -S -H M '%k\t%n' ''packages''<br />
<br />
To list explicitly installed packages not in the [[meta package]] {{Pkg|base}} nor [[package group]] {{Grp|base-devel}} with size and description:<br />
<br />
$ expac -H M "%011m\t%-20n\t%10d" $(comm -23 <(pacman -Qqen | sort) <({ pacman -Qqg base-devel; expac -l '\n' '%E' base; } | sort -u)) | sort -n<br />
<br />
To list the packages marked for upgrade with their download size:<br />
<br />
$ expac -S -H M '%k\t%n' $(pacman -Qqu) | sort -sh<br />
<br />
To list optional dependencies only:<br />
<br />
$ expac -S "%o" ''package''<br />
<br />
==== By date ====<br />
<br />
To list the 20 last installed packages with {{Pkg|expac}}, run:<br />
<br />
$ expac --timefmt='%Y-%m-%d %T' '%l\t%n' | sort | tail -n 20<br />
<br />
or, with seconds since the epoch (1970-01-01 UTC):<br />
<br />
$ expac --timefmt=%s '%l\t%n' | sort -n | tail -n 20<br />
<br />
==== Not in a specified group, repository or meta package ====<br />
<br />
{{Note|To get a list of packages installed as dependencies but no longer required by any installed package, see [[#Removing unused packages (orphans)]].<br />
}}<br />
<br />
List explicitly installed packages not in the {{Pkg|base}} [[meta package]]:<br />
<br />
$ comm -23 <(pacman -Qqe | sort) <(expac -l '\n' '%E' base | sort)<br />
<br />
List explicitly installed packages not in the {{Pkg|base}} meta package or {{Grp|base-devel}} [[package group]]:<br />
<br />
$ comm -23 <(pacman -Qqe | sort) <({ pacman -Qqg base-devel; expac -l '\n' '%E' base; } | sort -u)<br />
<br />
List all installed packages unrequired by other packages, and which are not in the {{Pkg|base}} meta package or {{Grp|base-devel}} package group:<br />
<br />
$ comm -23 <(pacman -Qqt | sort) <({ pacman -Qqg base-devel; echo base; } | sort -u)<br />
<br />
As above, but with descriptions:<br />
<br />
$ expac -H M '%-20n\t%10d' $(comm -23 <(pacman -Qqt | sort) <({ pacman -Qqg base-devel; echo base; } | sort -u))<br />
<br />
List all installed packages that are ''not'' in the specified repository ''repo_name''<br />
<br />
$ comm -23 <(pacman -Qq | sort) <(pacman -Sql ''repo_name'' | sort)<br />
<br />
List all installed packages that are in the ''repo_name'' repository:<br />
<br />
$ comm -12 <(pacman -Qq | sort) <(pacman -Sql ''repo_name'' | sort)<br />
<br />
List all packages on the Arch Linux ISO that are not in the {{Pkg|base}} meta package:<br />
<br />
<nowiki>$ comm -23 <(curl https://gitlab.archlinux.org/archlinux/archiso/-/raw/master/configs/releng/packages.x86_64) <(expac -l '\n' '%E' base | sort)</nowiki><br />
<br />
{{Tip|Alternatively, use {{ic|combine}} (instead of {{ic|comm}}) from the {{Pkg|moreutils}} package which has a syntax that is easier to remember. See {{man|1|combine}}.}}<br />
<br />
==== Development packages ====<br />
<br />
To list all development/unstable packages, run:<br />
<br />
$ pacman -Qq | grep -Ee '-(bzr|cvs|darcs|git|hg|svn)$'<br />
<br />
=== Browsing packages ===<br />
<br />
To browse all installed packages with an instant preview of each package:<br />
<br />
$ pacman -Qq | fzf --preview 'pacman -Qil {}' --layout=reverse --bind 'enter:execute(pacman -Qil {} | less)'<br />
<br />
This uses [[fzf]] to present a two-pane view listing all packages with package info shown on the right.<br />
<br />
Enter letters to filter the list of packages; use arrow keys (or {{ic|Ctrl-j}}/{{ic|Ctrl-k}}) to navigate; press {{ic|Enter}} to see package info under ''less''.<br />
<br />
To browse all packages currently known to ''pacman'' (both installed and not yet installed) in a similar way, using fzf, use:<br />
<br />
$ pacman -Slq | fzf --preview 'pacman -Si {}' --layout=reverse<br />
<br />
The navigational keybindings are the same, although {{ic|Enter}} will not work in the same way.<br />
<br />
=== Listing files owned by a package with size ===<br />
<br />
This one might come in handy if you have found that a specific package uses a huge amount of space and you want to find out which files make up the most of that.<br />
<br />
$ pacman -Qlq ''package'' | grep -v '/$' | xargs -r du -h | sort -h<br />
<br />
=== Identify files not owned by any package ===<br />
<br />
If your system has stray files not owned by any package (a common case if you do not [[Enhance system stability#Use the package manager to install software|use the package manager to install software]]), you may want to find such files in order to clean them up.<br />
<br />
One method is to use {{ic|pacreport --unowned-files}} as the root user from {{Pkg|pacutils}} which will list unowned files among other details.<br />
<br />
Another is to list all files of interest and check them against ''pacman'':<br />
<br />
# find /etc /usr /opt | LC_ALL=C pacman -Qqo - 2>&1 >&- >/dev/null | cut -d ' ' -f 5-<br />
<br />
{{Tip|The {{Pkg|lostfiles}} script performs similar steps, but also includes an extensive blacklist to remove common false positives from the output.}}<br />
<br />
=== Tracking unowned files created by packages ===<br />
<br />
Most systems will slowly collect several [http://ftp.rpm.org/max-rpm/s1-rpm-inside-files-list-directives.html#S3-RPM-INSIDE-FLIST-GHOST-DIRECTIVE ghost] files such as state files, logs, indexes, etc. through the course of usual operation.<br />
<br />
{{ic|pacreport}} from {{Pkg|pacutils}} can be used to track these files and their associations via {{ic|/etc/pacreport.conf}} (see {{man|1|pacreport|FILES}}).<br />
<br />
An example may look something like this (abridged):<br />
<br />
{{hc|/etc/pacreport.conf|2=<br />
[Options]<br />
IgnoreUnowned = usr/share/applications/mimeinfo.cache<br />
<br />
[PkgIgnoreUnowned]<br />
alsa-utils = var/lib/alsa/asound.state<br />
bluez = var/lib/bluetooth<br />
ca-certificates = etc/ca-certificates/trust-source/*<br />
dbus = var/lib/dbus/machine-id<br />
glibc = etc/ld.so.cache<br />
grub = boot/grub/*<br />
linux = boot/initramfs-linux.img<br />
pacman = var/lib/pacman/local<br />
update-mime-database = usr/share/mime/magic<br />
}}<br />
<br />
Then, when using {{ic|pacreport --unowned-files}} as the root user, any unowned files will be listed if the associated package is no longer installed (or if any new files have been created).<br />
<br />
Additionally, [https://github.com/CyberShadow/aconfmgr aconfmgr] ({{AUR|aconfmgr-git}}) allows tracking modified and orphaned files using a configuration script.<br />
<br />
=== Removing unused packages (orphans) ===<br />
<br />
Orphans are packages that were installed as a dependency and are no longer required by any package.<br />
<br />
They can accumulate on your system over time due to uninstalling packages using {{ic|pacman -R <pkg>}} instead of {{ic|pacman -Rs <pkg>}}, due to packages installed as [[PKGBUILD#makedepends|makedepends]], and due to packages removing dependencies in newer versions.<br />
<br />
For recursively removing orphans and their configuration files:<br />
<br />
# pacman -Qtdq | pacman -Rns -<br />
<br />
If no orphans were found, the output is {{ic|error: argument '-' specified with empty stdin}}. This is expected as no arguments were passed to {{ic|pacman -Rns}}.<br />
<br />
<br />
{{Note|The arguments {{ic|-Qt}} list only true orphans. To include packages which are ''optionally'' required by another package, pass the {{ic|-t}} flag twice (''i.e.'', {{ic|-Qtt}}).}}<br />
<br />
<br />
The package {{AUR|pacman-log-orphans-hook}} adds a [[Pacman#Hooks|hook]] that logs orphan packages during pacman operations.<br />
<br />
=== Removing everything but essential packages ===<br />
<br />
If it is ever necessary to remove all packages except the essentials packages, one method is to set the installation reason of the non-essential ones as dependency and then remove all unnecessary dependencies.<br />
<br />
First, for all the packages "explicitly installed", change their installation reason to "installed as a dependency":<br />
<br />
# pacman -D --asdeps $(pacman -Qqe)<br />
<br />
Then, change the installation reason to "explicitly installed" of only the essential packages, those you '''do not''' want to remove, in order to avoid targeting them:<br />
<br />
# pacman -D --asexplicit base linux linux-firmware<br />
<br />
{{Note|<br />
* Additional packages can be added to the above command in order to avoid being removed. See [[Installation guide#Install essential packages]] for more info on other packages that may be necessary for a fully functional base system.<br />
* This will also select the bootloader's package for removal. The system should still be bootable, but the boot parameters might not be changeable without it.<br />
}}<br />
<br />
Finally, follow the instructions in [[#Removing unused packages (orphans)]] to remove all packages that are "installed as a dependency".<br />
<br />
=== Getting the dependencies list of several packages ===<br />
<br />
Dependencies are alphabetically sorted and doubles are removed.<br />
<br />
{{Note|To only show the tree of local installed packages, use {{ic|pacman -Qi}}.}}<br />
<br />
$ LC_ALL=C pacman -Si ''packages'' | awk -F'[:<=>]' '/^Depends/ {print $2}' | xargs -n1 | sort -u<br />
<br />
Alternatively, with {{Pkg|expac}}: <br />
<br />
$ expac -l '\n' %E -S ''packages'' | sort -u<br />
<br />
=== Listing changed backup files ===<br />
<br />
{{Accuracy|What is the connection of this section to [[System backup]]? Listing modified "backup files" does not show files which are not tracked by ''pacman''.|section=Warning about listing changed backup files}}<br />
<br />
If you want to back up your system configuration files, you could copy all files in {{ic|/etc/}} but usually you are only interested in the files that you have changed. Modified [[Pacnew and Pacsave files#Package backup files|backup files]] can be viewed with the following command:<br />
<br />
# pacman -Qii | awk '/^MODIFIED/ {print $2}'<br />
<br />
Running this command with root permissions will ensure that files readable only by root (such as {{ic|/etc/sudoers}}) are included in the output.<br />
<br />
{{Tip|See [[#Listing all changed files from packages]] to list all changed files ''pacman'' knows about, not only backup files.}}<br />
<br />
=== Back up the pacman database ===<br />
<br />
The following command can be used to back up the local ''pacman'' database:<br />
<br />
$ tar -cjf pacman_database.tar.bz2 /var/lib/pacman/local<br />
<br />
Store the backup ''pacman'' database file on one or more offline media, such as a USB stick, external hard drive, or CD-R.<br />
<br />
The database can be restored by moving the {{ic|pacman_database.tar.bz2}} file into the {{ic|/}} directory and executing the following command:<br />
<br />
# tar -xjvf pacman_database.tar.bz2<br />
<br />
{{Note|If the ''pacman'' database files are corrupted, and there is no backup file available, there exists some hope of rebuilding the ''pacman'' database. Consult [[#Restore pacman's local database]].}}<br />
<br />
{{Tip|The {{AUR|pakbak-git}} package provides a script and a [[systemd]] service to automate the task. Configuration is possible in {{ic|/etc/pakbak.conf}}.}}<br />
<br />
=== Check changelogs easily ===<br />
<br />
When maintainers update packages, commits are often commented in a useful fashion. Users can quickly check these from the command line by installing {{AUR|pacolog}}. This utility lists recent commit messages for packages from the official repositories or the AUR, by using {{ic|pacolog ''package''}}.<br />
<br />
== Installation and recovery ==<br />
<br />
Alternative ways of getting and restoring packages.<br />
<br />
=== Installing packages from a CD/DVD or USB stick ===<br />
<br />
{{Merge|#Custom local repository|Use as an example and avoid duplication}}<br />
<br />
To download packages, or groups of packages:<br />
<br />
# cd ~/Packages<br />
# pacman -Syw --cachedir . base base-devel grub-bios xorg gimp<br />
# repo-add ./custom.db.tar.gz ./*<br />
<br />
Pacman, which will reference the host installation by default, will not properly resolve and download existing dependencies. In cases where all packages and dependencies are wanted, it is recommended to create a temporary blank DB and reference it with {{ic|--dbpath}}:<br />
<br />
# mkdir /tmp/blankdb<br />
# pacman -Syw --cachedir . --dbpath /tmp/blankdb base base-devel grub-bios xorg gimp<br />
# repo-add ./custom.db.tar.gz ./*<br />
<br />
Then you can burn the "Packages" directory to a CD/DVD or transfer it to a USB stick, external HDD, etc.<br />
<br />
To install:<br />
<br />
'''1.''' Mount the media:<br />
<br />
# mkdir /mnt/repo<br />
# mount /dev/sr0 /mnt/repo #For a CD/DVD.<br />
# mount /dev/sdxY /mnt/repo #For a USB stick.<br />
<br />
'''2.''' Edit {{ic|pacman.conf}} and add this repository ''before'' the other ones (e.g. extra, core, etc.). This is important. Do not just uncomment the one on the bottom. This way it ensures that the files from the CD/DVD/USB take precedence over those in the standard repositories:<br />
<br />
{{hc|/etc/pacman.conf|2=<br />
[custom]<br />
SigLevel = PackageRequired<br />
Server = file:///mnt/repo/Packages}}<br />
<br />
'''3.''' Finally, synchronize the ''pacman'' database to be able to use the new repository:<br />
<br />
# pacman -Syu<br />
<br />
=== Custom local repository ===<br />
<br />
Use the ''repo-add'' script included with ''pacman'' to generate a database for a personal repository. Use {{ic|repo-add --help}} for more details on its usage. <br />
A package database is a tar file, optionally compressed. Valid extensions are ''.db'' or ''.files'' followed by an archive extension of ''.tar'', ''.tar.gz'', ''.tar.bz2'', ''.tar.xz'', ''.tar.zst'', or ''.tar.Z''. The file does not need to exist, but all parent directories must exist.<br />
<br />
To add a new package to the database, or to replace the old version of an existing package in the database, run:<br />
<br />
$ repo-add ''/path/to/repo.db.tar.gz /path/to/package-1.0-1-x86_64.pkg.tar.zst''<br />
<br />
The database and the packages do not need to be in the same directory when using ''repo-add'', but keep in mind that when using ''pacman'' with that database, they should be together. Storing all the built packages to be included in the repository in one directory also allows to use shell glob expansion to add or update multiple packages at once:<br />
<br />
$ repo-add ''/path/to/repo.db.tar.gz /path/to/*.pkg.tar.zst''<br />
<br />
{{Warning|''repo-add'' adds the entries into the database in the same order as passed on the command line. If multiple versions of the same package are involved, care must be taken to ensure that the correct version is added last. In particular, note that lexical order used by the shell depends on the locale and differs from the {{man|8|vercmp}} ordering used by ''pacman''.}}<br />
<br />
If you are looking to support multiple architectures then precautions should be taken to prevent errors from occurring. Each architecture should have its own directory tree:<br />
<br />
{{hc|$ tree ~/customrepo/ {{!}} sed "s/$(uname -m)/''arch''/g"|<br />
/home/archie/customrepo/<br />
└── ''arch''<br />
├── customrepo.db -> customrepo.db.tar.xz<br />
├── customrepo.db.tar.xz<br />
├── customrepo.files -> customrepo.files.tar.xz<br />
├── customrepo.files.tar.xz<br />
└── personal-website-git-b99cce0-1-''arch''.pkg.tar.zst<br />
<br />
1 directory, 5 files<br />
}}<br />
<br />
The ''repo-add'' executable checks if the package is appropriate. If this is not the case you will be running into error messages similar to this:<br />
<br />
==> ERROR: '/home/archie/customrepo/''arch''/foo-''arch''.pkg.tar.zst' does not have a valid database archive extension.<br />
<br />
''repo-remove'' is used to remove packages from the package database, except that only package names are specified on the command line.<br />
<br />
$ repo-remove ''/path/to/repo.db.tar.gz pkgname''<br />
<br />
Once the local repository database has been created, add the repository to {{ic|pacman.conf}} for each system that is to use the repository. An example of a custom repository is in {{ic|pacman.conf}}. The repository's name is the database filename with the file extension omitted. In the case of the example above the repository's name would simply be ''repo''. Reference the repository's location using a {{ic|file://}} URL, or via FTP using ftp://localhost/path/to/directory.<br />
<br />
If willing, add the custom repository to the [[Unofficial user repositories|list of unofficial user repositories]], so that the community can benefit from it.<br />
<br />
=== Network shared pacman cache ===<br />
<br />
{{Merge|Package_Proxy_Cache|Same topic}}<br />
If you happen to run several Arch boxes on your LAN, you can share packages so that you can greatly decrease your download times. Keep in mind you should not share between different architectures (i.e. i686 and x86_64) or you will run into problems.<br />
<br />
==== Read-only cache ====<br />
<br />
{{Note|1=If pacman fails to download 3 packages from the server, it will use another mirror instead. See https://bbs.archlinux.org/viewtopic.php?id=268066.}}<br />
<br />
If you are looking for a quick solution, you can simply run a [https://gist.github.com/willurd/5720255 basic temporary webserver] which other computers can use as their first mirror.<br />
<br />
First of all, make pacman databases available into the directory you will serve:<br />
<br />
# ln -s /var/lib/pacman/sync/*.db /var/cache/pacman/pkg/<br />
<br />
Then start serving this directory. For example, with [[Python]] [https://docs.python.org/3/library/http.server.html#http-server-cli http.server] module:<br />
$ python -m http.server -d /var/cache/pacman/pkg/<br />
<br />
{{Tip|By default, Python {{ic|http.server}} listens on port {{ic|8000}}. To use another port, simply add it as an argument:<br />
<br />
$ python -m http.server -d /var/cache/pacman/pkg/ 8080<br />
}}<br />
<br />
Then [[textedit|edit]] {{ic|/etc/pacman.d/mirrorlist}} on each client machine to add this server as the top entry:<br />
<br />
{{hc|/etc/pacman.d/mirrorlist|2=<br />
Server = http://''server-ip'':''port''<br />
...<br />
}}<br />
<br />
{{Warning|Do '''not''' append {{ic|/repos/$repo/os/$arch}} to this custom server like for other entries, as this hierarchy does not exist and therefore queries will fail.}}<br />
<br />
If looking for a more standalone solution, {{Pkg|darkhttpd}} offers a very minimal webserver. Replace the previous {{ic|python}} command with e.g.:<br />
<br />
$ sudo -u http darkhttpd /var/cache/pacman/pkg --no-server-id<br />
<br />
You could also run darkhttpd as a ''systemd'' service for convenience: see [[Systemd#Writing unit files]].<br />
<br />
{{Pkg|miniserve}}, a web small server written in Rust, can also be used:<br />
<br />
$ miniserve /var/cache/pacman/pkg<br />
<br />
Then edit {{ic|/etc/pacman.d/mirrorlist}} as above with the first url miniserve is available at.<br />
<br />
If you are already running a web server for some other purpose, you might wish to reuse that as your local repository server instead. For example, if you already serve a site with [[nginx]], you can add an ''nginx'' server block listening on port 8080:<br />
<br />
{{hc|/etc/nginx/nginx.conf|<br />
server {<br />
listen 8080;<br />
root /var/cache/pacman/pkg;<br />
server_name myarchrepo.localdomain;<br />
try_files $uri $uri/;<br />
}<br />
}}<br />
<br />
Remember to [[restart]] {{ic|nginx.service}} after making this change.<br />
<br />
{{Tip|Whichever web server you use, make sure the firewall configuration (if any) allows the configured port to be reached by the desired traffic, and disallows any undesired traffic. See [[Security#Network and firewalls]].}}<br />
<br />
==== Overlay mount of read-only cache ====<br />
<br />
It is possible to use one machine on a local network as a read-only package cache by [[Overlay_filesystem|overlay mounting]] its {{ic|/var/cache/pacman/pkg}} directory. Such a configuration is advantageous if this server has installed on it a reasonably comprehensive selection of up-to-date packages which are also used by other boxes. This is useful for maintaining a number of machines at the end of a low bandwidth upstream connection.<br />
<br />
As an example, to use this method:<br />
<br />
# mkdir /tmp/remote_pkg /mnt/workdir_pkg /tmp/pacman_pkg<br />
# sshfs ''remote_username''@''remote_pkgcache_addr'':/var/cache/pacman/pkg /tmp/remote_pkg -C<br />
# mount -t overlay overlay -o lowerdir=/tmp/remote_pkg,upperdir=/var/cache/pacman/pkg,workdir=/mnt/workdir_pkg /tmp/pacman_pkg<br />
<br />
{{Note|The working directory must be an empty directory on the same mounted device as the upper directory. See [[Overlay filesystem#Usage]].}}<br />
<br />
{{Tip|1=If listing the {{ic|/tmp/pacman_pkg}} overlay directory gives errors, e.g., "Stale file handle", try overlay mounting with options {{ic|1=-o redirect_dir=off -o index=off}}. }}<br />
<br />
After this, run ''pacman'' using the option {{ic|--cachedir /tmp/pacman_pkg}}, e.g.:<br />
<br />
# pacman -Syu --cachedir /tmp/pacman_pkg<br />
<br />
==== Distributed read-only cache ====<br />
<br />
There are Arch-specific tools for automatically discovering other computers on your network offering a package cache. Try {{Pkg|pacredir}}, [[pacserve]], {{AUR|pkgdistcache}}, or {{AUR|paclan}}. pkgdistcache uses Avahi instead of plain UDP which may work better in certain home networks that route instead of bridge between WiFi and Ethernet.<br />
<br />
Historically, there was [https://bbs.archlinux.org/viewtopic.php?id=64391 PkgD] and [https://github.com/toofishes/multipkg multipkg], but they are no longer maintained.<br />
<br />
==== Read-write cache ====<br />
<br />
In order to share packages between multiple computers, simply share {{ic|/var/cache/pacman/}} using any network-based mount protocol. This section shows how to use [[SSHFS]] to share a package cache plus the related library-directories between multiple computers on the same local network. Keep in mind that a network shared cache can be slow depending on the file-system choice, among other factors.<br />
<br />
First, install any network-supporting filesystem packages: {{Pkg|sshfs}}, {{Pkg|curlftpfs}}, {{Pkg|samba}} or {{Pkg|nfs-utils}}.<br />
<br />
{{Tip|<br />
* To use ''sshfs'', consider reading [[Using SSH Keys]].<br />
* By default, ''smbfs'' does not serve filenames that contain colons, which results in the client downloading the offending package afresh. To prevent this, use the {{ic|mapchars}} mount option on the client.<br />
}}<br />
<br />
Then, to share the actual packages, mount {{ic|/var/cache/pacman/pkg}} from the server to {{ic|/var/cache/pacman/pkg}} on every client machine.<br />
<br />
{{Warning|Do not make {{ic|/var/cache/pacman/pkg}} or any of its ancestors (e.g., {{ic|/var}}) a symlink. Pacman expects these to be directories. When ''pacman'' re-installs or upgrades itself, it will remove the symlinks and create empty directories instead. However during the transaction ''pacman'' relies on some files residing there, hence breaking the update process. Refer to {{Bug|50298}} for further details.}}<br />
<br />
==== two-way with rsync ====<br />
<br />
Another approach in a local environment is [[rsync]]. Choose a server for caching and enable the [[Rsync#As a daemon|rsync daemon]]. On clients synchronize two-way with this share via the rsync protocol. Filenames that contain colons are no problem for the rsync protocol.<br />
<br />
Draft example for a client, using {{ic|uname -m}} within the share name ensures an architecture-dependent sync:<br />
# rsync rsync://server/share_$(uname -m)/ /var/cache/pacman/pkg/ ...<br />
# pacman ...<br />
# paccache ...<br />
# rsync /var/cache/pacman/pkg/ rsync://server/share_$(uname -m)/ ...<br />
<br />
==== Dynamic reverse proxy cache using nginx ====<br />
<br />
[[nginx]] can be used to proxy package requests to official upstream mirrors and cache the results to the local disk. All subsequent requests for that package will be served directly from the local cache, minimizing the amount of internet traffic needed to update a large number of computers. <br />
<br />
In this example, the cache server will run at {{ic|<nowiki>http://cache.domain.example:8080/</nowiki>}} and store the packages in {{ic|/srv/http/pacman-cache/}}. <br />
<br />
Install [[nginx]] on the computer that is going to host the cache. Create the directory for the cache and adjust the permissions so nginx can write files to it:<br />
<br />
# mkdir /srv/http/pacman-cache<br />
# chown http:http /srv/http/pacman-cache<br />
<br />
Use the [https://github.com/nastasie-octavian/nginx_pacman_cache_config/blob/c54eca4776ff162ab492117b80be4df95880d0e2/nginx.conf nginx pacman cache config] as a starting point for {{ic|/etc/nginx/nginx.conf}}. Check that the {{ic|resolver}} directive works for your needs. In the upstream server blocks, configure the {{ic|proxy_pass}} directives with addresses of official mirrors, see examples in the configuration file about the expected format. Once you are satisfied with the configuration file [[Nginx#Running|start and enable nginx]].<br />
<br />
In order to use the cache each Arch Linux computer (including the one hosting the cache) must have the following line at the top of the {{ic|mirrorlist}} file:<br />
<br />
{{hc|/etc/pacman.d/mirrorlist|<nowiki><br />
Server = http://cache.domain.example:8080/$repo/os/$arch<br />
...<br />
</nowiki>}}<br />
<br />
{{Note| You will need to create a method to clear old packages, as the cache directory will continue to grow over time. {{ic|paccache}} (which is provided by {{Pkg|pacman-contrib}}) can be used to automate this using retention criteria of your choosing. For example, {{ic|find /srv/http/pacman-cache/ -type d -exec paccache -v -r -k 2 -c {} \;}} will keep the last 2 versions of packages in your cache directory.}}<br />
<br />
==== Pacoloco proxy cache server ====<br />
<br />
[https://github.com/anatol/pacoloco Pacoloco] is an easy-to-use proxy cache server for ''pacman'' repositories. It also allows [https://github.com/anatol/pacoloco/commit/048b09956b0d8ef71c0ed1f804fd332d9ab5e3c8 automatic prefetching] of the cached packages.<br />
<br />
It can be installed as {{Pkg|pacoloco}}. Open the configuration file and add ''pacman'' mirrors:<br />
<br />
{{hc|/etc/pacoloco.yaml|<nowiki><br />
port: 9129<br />
repos:<br />
mycopy:<br />
urls:<br />
- http://mirror.lty.me/archlinux<br />
- http://mirrors.kernel.org/archlinux<br />
</nowiki>}}<br />
<br />
[[Restart]] {{ic|pacoloco.service}} and the proxy repository will be available at {{ic|http://''myserver'':9129/repo/mycopy}}.<br />
<br />
==== Flexo proxy cache server ====<br />
<br />
[https://github.com/nroi/flexo Flexo] is yet another proxy cache server for ''pacman'' repositories. Flexo is available as {{AUR|flexo-git}}. Once installed, [[start]] the {{ic|flexo.service}} unit.<br />
<br />
Flexo runs on port {{ic|7878}} by default. Enter {{ic|1=Server = http://''myserver'':7878/$repo/os/$arch}} to the top of your {{ic|/etc/pacman.d/mirrorlist}} so that ''pacman'' downloads packages via Flexo.<br />
<br />
==== Synchronize pacman package cache using synchronization programs ====<br />
<br />
Use [[Syncthing]] or [[Resilio Sync]] to synchronize the ''pacman'' cache directories (i.e. {{ic|/var/cache/pacman/pkg}}).<br />
<br />
==== Preventing unwanted cache purges ====<br />
<br />
By default, {{ic|pacman -Sc}} removes package tarballs from the cache that correspond to packages that are not installed on the machine the command was issued on. Because ''pacman'' cannot predict what packages are installed on all machines that share the cache, it will end up deleting files that should not be.<br />
<br />
To clean up the cache so that only ''outdated'' tarballs are deleted, add this entry in the {{ic|[options]}} section of {{ic|/etc/pacman.conf}}:<br />
<br />
CleanMethod = KeepCurrent<br />
<br />
=== Recreate a package from the file system ===<br />
<br />
To recreate a package from the file system, use {{AUR|fakepkg}}. Files from the system are taken as they are, hence any modifications will be present in the assembled package. Distributing the recreated package is therefore discouraged; see [[ABS]] and [[Arch Linux Archive]] for alternatives.<br />
<br />
=== List of installed packages ===<br />
<br />
Keeping a list of all explicitly installed packages can be useful to backup a system or quicken the installation of a new one:<br />
<br />
$ pacman -Qqe > pkglist.txt<br />
<br />
{{Note|<br />
* With option {{ic|-t}}, the packages already required by other explicitly installed packages are not mentioned. If reinstalling from this list they will be installed but as dependencies only.<br />
* With option {{ic|-n}}, foreign packages (e.g. from [[AUR]]) would be omitted from the list.<br />
* Use {{ic|comm -13 <(pacman -Qqdt {{!}} sort) <(pacman -Qqdtt {{!}} sort) > optdeplist.txt}} to also create a list of the installed optional dependencies which can be reinstalled with {{ic|--asdeps}}.<br />
* Use {{ic|pacman -Qqem > foreignpkglist.txt}} to create the list of AUR and other foreign packages that have been explicitly installed.}}<br />
<br />
To keep an up-to-date list of explicitly installed packages (e.g. in combination with a versioned {{ic|/etc/}}), you can set up a [[Pacman#Hooks|hook]]. Example:<br />
<br />
[Trigger]<br />
Operation = Install<br />
Operation = Remove<br />
Type = Package<br />
Target = *<br />
<br />
[Action]<br />
When = PostTransaction<br />
Exec = /bin/sh -c '/usr/bin/pacman -Qqe > /etc/pkglist.txt'<br />
<br />
=== Install packages from a list ===<br />
<br />
To install packages from a previously saved list of packages, while not reinstalling previously installed packages that are already up-to-date, run:<br />
<br />
# pacman -S --needed - < pkglist.txt<br />
<br />
However, it is likely foreign packages such as from the AUR or installed locally are present in the list. To filter out from the list the foreign packages, the previous command line can be enriched as follows:<br />
<br />
# pacman -S --needed $(comm -12 <(pacman -Slq | sort) <(sort pkglist.txt))<br />
<br />
Eventually, to make sure the installed packages of your system match the list and remove all the packages that are not mentioned in it:<br />
<br />
# pacman -Rsu $(comm -23 <(pacman -Qq | sort) <(sort pkglist.txt))<br />
<br />
{{Tip|These tasks can be automated. See {{AUR|bacpac}}, {{AUR|packup}}, {{AUR|pacmanity}}, and {{AUR|pug}} for examples.}}<br />
<br />
=== Listing all changed files from packages ===<br />
<br />
If you are suspecting file corruption (e.g. by software/hardware failure), but are unsure if files were corrupted, you might want to compare with the hash sums in the packages. This can be done with {{Pkg|pacutils}}:<br />
<br />
# paccheck --md5sum --quiet<br />
<br />
For recovery of the database see [[#Restore pacman's local database]]. The {{ic|mtree}} files can also be [[#Viewing a single file inside a .pkg file|extracted as {{ic|.MTREE}} from the respective package files]].<br />
<br />
{{Note|This should '''not''' be used as is when suspecting malicious changes! In this case security precautions such as using a live medium and an independent source for the hash sums are advised.}}<br />
<br />
=== Reinstalling all packages ===<br />
<br />
To reinstall all native packages, use:<br />
<br />
# pacman -Qqn | pacman -S -<br />
<br />
Foreign (AUR) packages must be reinstalled separately; you can list them with {{ic|pacman -Qqm}}.<br />
<br />
Pacman preserves the [[installation reason]] by default.<br />
<br />
{{Warning|To force all packages to be overwritten, use {{ic|1=--overwrite=*}}, though this should be an absolute last resort. See [[System maintenance#Avoid certain pacman commands]].}}<br />
<br />
=== Restore pacman's local database ===<br />
<br />
See [[pacman/Restore local database]].<br />
<br />
=== Recovering a USB key from existing install ===<br />
<br />
If you have Arch installed on a USB key and manage to mess it up (e.g. removing it while it is still being written to), then it is possible to re-install all the packages and hopefully get it back up and working again (assuming USB key is mounted in {{ic|/newarch}})<br />
<br />
# pacman -S $(pacman -Qq --dbpath /newarch/var/lib/pacman) --root /newarch --dbpath /newarch/var/lib/pacman<br />
<br />
=== Viewing a single file inside a .pkg file ===<br />
<br />
For example, if you want to see the contents of {{ic|/etc/systemd/logind.conf}} supplied within the {{Pkg|systemd}} package:<br />
<br />
$ bsdtar -xOf /var/cache/pacman/pkg/systemd-250.4-2-x86_64.pkg.tar.zst etc/systemd/logind.conf<br />
<br />
Or you can use {{Pkg|vim}} to browse the archive:<br />
<br />
$ vim /var/cache/pacman/pkg/systemd-250.4-2-x86_64.pkg.tar.zst<br />
<br />
=== Find applications that use libraries from older packages ===<br />
<br />
Already running processes do not automatically notice changes caused by updates. Instead, they continue using old library versions. That may be undesirable, due to potential issues related to security vulnerabilities or other bugs, and version incompatibility.<br />
<br />
Processes depending on updated libraries may be found using either {{pkg|htop}}, which highlights the names of the affected programs, or with a snippet based on {{pkg|lsof}}, which also prints the names of the libraries:<br />
<br />
# lsof +c 0 | grep -w DEL | awk '1 { print $1 ": " $NF }' | sort -u<br />
<br />
This solution will only detect files, that are normally kept opened by running processes, which basically limits it to shared libraries ({{ic|.so}} files). It may miss some dependencies, like those of Java or Python applications.<br />
<br />
=== Installing only content in required languages ===<br />
<br />
Many packages attempt to install documentation and translations in several languages. Some programs are designed to remove such unnecessary files, such as {{AUR|localepurge}}, which runs after a package is installed to delete the unneeded locale files. A more direct approach is provided through the {{ic|NoExtract}} directive in {{ic|pacman.conf}}, which prevent these files from ever being installed.<br />
<br />
{{Warning|1=Some users noted that removing locales has resulted in [[Special:Permalink/460285#Dangerous NoExtract example|unintended consequences]], even under [https://bbs.archlinux.org/viewtopic.php?id=250846 Xorg].}}<br />
<br />
The example below installs English (US) files, or none at all:<br />
<br />
{{hc|/etc/pacman.conf|2=<br />
NoExtract = usr/share/help/* !usr/share/help/C/*<br />
NoExtract = usr/share/gtk-doc/html/*<br />
NoExtract = usr/share/locale/* usr/share/X11/locale/*/* usr/share/i18n/locales/* opt/google/chrome/locales/* !usr/share/X11/locale/C/*<br />
NoExtract = !*locale*/en*/* !usr/share/*locale*/locale.*<br />
NoExtract = !usr/share/*locales/en_?? !usr/share/*locales/i18n* !usr/share/*locales/iso*<br />
NoExtract = usr/share/i18n/charmaps/* !usr/share/i18n/charmaps/UTF-8.gz<br />
NoExtract = !usr/share/*locales/trans*<br />
NoExtract = usr/share/man/* !usr/share/man/man*<br />
NoExtract = usr/share/vim/vim*/lang/*<br />
NoExtract = usr/lib/libreoffice/help/en-US/*<br />
NoExtract = usr/share/kbd/locale/*<br />
NoExtract = usr/share/*/translations/*.qm usr/share/*/nls/*.qm usr/share/qt/translations/*.pak !*/en-US.pak # Qt apps<br />
NoExtract = usr/share/*/locales/*.pak opt/*/locales/*.pak usr/lib/*/locales/*.pak !*/en-US.pak # Electron apps<br />
NoExtract = opt/onlyoffice/desktopeditors/dictionaries/* !opt/onlyoffice/desktopeditors/dictionaries/en_US/*<br />
NoExtract = opt/onlyoffice/desktopeditors/editors/web-apps/apps/*/main/locale/* !*/en.json<br />
NoExtract = opt/onlyoffice/desktopeditors/editors/web-apps/apps/*/main/resources/help/* !*/help/en/*<br />
NoExtract = opt/onlyoffice/desktopeditors/converter/empty/*/*<br />
NoExtract = usr/share/ibus/dicts/emoji-*.dict !usr/share/ibus/dicts/emoji-en.dict<br />
}}<br />
<br />
=== Installing packages on bad connection ===<br />
<br />
When trying to install a package from a bad connection (e.g. a train using a cell phone), use the {{ic|--disable-download-timeout}} option to lessen the chance of receiving errors such as: <br />
<br />
error: failed retrieving file […] Operation too slow. Less than 1 bytes/sec transferred the last 10 seconds<br />
<br />
or<br />
<br />
error: failed retrieving file […] Operation timed out after 10014 milliseconds with 0 out of 0 bytes received<br />
<br />
== Performance ==<br />
<br />
=== Download speeds ===<br />
<br />
When downloading packages ''pacman'' uses the mirrors in the order they are in {{ic|/etc/pacman.d/mirrorlist}}. The mirror which is at the top of the list by default however may not be the fastest for you. To select a faster mirror, see [[Mirrors]].<br />
<br />
Pacman's speed in downloading packages can also be improved by using a different application to download packages, instead of ''pacman''<nowiki/>'s built-in file downloader, or by [[pacman#Enabling parallel downloads|enabling parallel downloads]].<br />
<br />
In all cases, make sure you have the latest ''pacman'' before doing any modifications.<br />
<br />
# pacman -Syu<br />
<br />
==== Powerpill ====<br />
<br />
[[Powerpill]] is a ''pacman'' wrapper that uses parallel and segmented downloading to try to speed up downloads for ''pacman''.<br />
<br />
==== wget ====<br />
<br />
This is also very handy if you need more powerful proxy settings than ''pacman''<nowiki/>'s built-in capabilities. <br />
<br />
To use {{ic|wget}}, first [[install]] the {{Pkg|wget}} package then modify {{ic|/etc/pacman.conf}} by uncommenting the following line in the {{ic|[options]}} section:<br />
<br />
XferCommand = /usr/bin/wget --passive-ftp --show-progress -c -q -N %u<br />
<br />
Instead of uncommenting the {{ic|wget}} parameters in {{ic|/etc/pacman.conf}}, you can also modify the {{ic|wget}} configuration file directly (the system-wide file is {{ic|/etc/wgetrc}}, per user files are {{ic|$HOME/.wgetrc}}).<br />
<br />
==== aria2 ====<br />
<br />
[[aria2]] is a lightweight download utility with support for resumable and segmented HTTP/HTTPS and FTP downloads. aria2 allows for multiple and simultaneous HTTP/HTTPS and FTP connections to an Arch mirror, which should result in an increase in download speeds for both file and package retrieval.<br />
<br />
{{Note|Using aria2c in ''pacman''<nowiki/>'s XferCommand will '''not''' result in parallel downloads of multiple packages. Pacman invokes the XferCommand with a single package at a time and waits for it to complete before invoking the next. To download multiple packages in parallel, see [[Powerpill]].}}<br />
<br />
Install {{Pkg|aria2}}, then edit {{ic|/etc/pacman.conf}} by adding the following line to the {{ic|[options]}} section:<br />
<br />
XferCommand = /usr/bin/aria2c --allow-overwrite=true --continue=true --file-allocation=none --log-level=error --max-tries=2 --max-connection-per-server=2 --max-file-not-found=5 --min-split-size=5M --no-conf --remote-time=true --summary-interval=60 --timeout=5 --dir=/ --out %o %u<br />
<br />
{{Tip|1=[https://bbs.archlinux.org/viewtopic.php?pid=1491879#p1491879 This alternative configuration for using pacman with aria2] tries to simplify configuration and adds more configuration options.}}<br />
<br />
See {{man|1|aria2c|OPTIONS}} for used aria2c options.<br />
<br />
* {{ic|-d, --dir}}: The directory to store the downloaded file(s) as specified by ''pacman''.<br />
* {{ic|-o, --out}}: The output file name(s) of the downloaded file(s). <br />
* {{ic|%o}}: Variable which represents the local filename(s) as specified by ''pacman''.<br />
* {{ic|%u}}: Variable which represents the download URL as specified by ''pacman''.<br />
<br />
==== Other applications ====<br />
<br />
There are other downloading applications that you can use with ''pacman''. Here they are, and their associated XferCommand settings:<br />
<br />
* {{ic|snarf}}: {{ic|1=XferCommand = /usr/bin/snarf -N %u}}<br />
* {{ic|lftp}}: {{ic|1=XferCommand = /usr/bin/lftp -c pget %u}}<br />
* {{ic|axel}}: {{ic|1=XferCommand = /usr/bin/axel -n 2 -v -a -o %o %u}}<br />
* {{ic|hget}}: {{ic|1=XferCommand = /usr/bin/hget %u -n 2 -skip-tls false}} (please read the [https://github.com/huydx/hget documentation on the Github project page] for more info)<br />
* {{ic|saldl}}: {{ic|1=XferCommand = /usr/bin/saldl -c6 -l4 -s2m -o %o %u}} (please read the [https://saldl.github.io documentation on the project page] for more info)<br />
<br />
== Utilities ==<br />
<br />
* {{App|Lostfiles|Script that identifies files not owned by any package.|https://github.com/graysky2/lostfiles|{{Pkg|lostfiles}}}}<br />
* {{App|pacutils|Helper library for libalpm based programs.|https://github.com/andrewgregory/pacutils|{{Pkg|pacutils}}}}<br />
* {{App|[[pkgfile]]|Tool that finds what package owns a file.|https://github.com/falconindy/pkgfile|{{Pkg|pkgfile}}}}<br />
* {{App|pkgtools|Collection of scripts for Arch Linux packages.|https://github.com/Daenyth/pkgtools|{{AUR|pkgtools}}}}<br />
* {{App|pkgtop|Interactive package manager and resource monitor designed for the GNU/Linux.|https://github.com/orhun/pkgtop|{{AUR|pkgtop-git}}}}<br />
* {{App|[[Powerpill]]|Uses parallel and segmented downloading through [[aria2]] and [[Reflector]] to try to speed up downloads for ''pacman''.|https://xyne.dev/projects/powerpill/|{{AUR|powerpill}}}}<br />
* {{App|repoctl|Tool to help manage local repositories.|https://github.com/cassava/repoctl|{{AUR|repoctl}}}}<br />
* {{App|repose|An Arch Linux repository building tool.|https://github.com/vodik/repose|{{Pkg|repose}}}}<br />
* {{App|[[Snapper#Wrapping_pacman_transactions_in_snapshots|snap-pac]]|Make ''pacman'' automatically use snapper to create pre/post snapshots like openSUSE's YaST.|https://github.com/wesbarnett/snap-pac|{{Pkg|snap-pac}}}}<br />
* {{App|vrms-arch|A virtual Richard M. Stallman to tell you which non-free packages are installed.|https://github.com/orospakr/vrms-arch|{{AUR|vrms-arch-git}}}}<br />
<br />
=== Graphical ===<br />
<br />
{{Warning|PackageKit opens up system permissions by default, and is otherwise not recommended for general usage. See {{Bug|50459}} and {{Bug|57943}}.}}<br />
<br />
* {{App|Apper|Qt 5 application and package manager using PackageKit written in C++. Supports [https://www.freedesktop.org/wiki/Distributions/AppStream/ AppStream metadata].|https://userbase.kde.org/Apper|{{Pkg|apper}}}}<br />
* {{App|Deepin App Store|Third party app store for DDE built with DTK, using PackageKit. Supports [https://www.freedesktop.org/wiki/Distributions/AppStream/ AppStream metadata].|https://github.com/dekzi/dde-store|{{Pkg|deepin-store}}}}<br />
* {{App|Discover|Qt 5 application manager using PackageKit written in C++/QML. Supports [https://www.freedesktop.org/wiki/Distributions/AppStream/ AppStream metadata], [[Flatpak]] and [[fwupd|firmware updates]]. |https://userbase.kde.org/Discover|{{Pkg|discover}}}}<br />
* {{App|GNOME PackageKit|GTK 3 package manager using PackageKit written in C.|https://freedesktop.org/software/PackageKit/|{{Pkg|gnome-packagekit}}}}<br />
* {{App|GNOME Software|GTK 3 application manager using PackageKit written in C. Supports [https://www.freedesktop.org/wiki/Distributions/AppStream/ AppStream metadata], [[Flatpak]] and [[fwupd|firmware updates]]. |https://wiki.gnome.org/Apps/Software|{{Pkg|gnome-software}}}}<br />
* {{App|pcurses|Curses TUI ''pacman'' wrapper written in C++.|https://github.com/schuay/pcurses|{{AUR|pcurses}}}}<br />
* {{App|tkPacman|Tk pacman wrapper written in Tcl.|https://sourceforge.net/projects/tkpacman|{{AUR|tkpacman}}}}</div>Phireskyhttps://wiki.archlinux.org/index.php?title=X2Go&diff=703520X2Go2021-11-26T03:38:16Z<p>Phiresky: warning a bit more specific</p>
<hr />
<div>[[Category:Remote desktop]]<br />
[[de:x2go]]<br />
[[ja:X2Go]]<br />
[[zh-hans:X2Go]]<br />
[https://wiki.x2go.org X2Go] enables to access a graphical desktop of a computer over the network. The protocol is tunneled through the [[Secure Shell]] protocol, so it is encrypted.<br />
{{Warning|X2Go is not compatible with all desktop environments. You can check [https://wiki.x2go.org/doku.php/doc:de-compat X2Go desktop environment compatibility] first. Compatibility is especially important if you want to connect to an existing [[Xorg]] session. In addition, X2Go does not support GLX 1.4 which is required for many graphical applications, including Firefox. Making those work requires [https://wiki.x2go.org/doku.php/wiki:development:glx-xlib-workaround further hacks.] }}<br />
<br />
== Installation ==<br />
<br />
Two parts are available in [[official repositories]]. They can be [[Pacman|installed]] with the following packages:<br />
* {{Pkg|x2goserver}} - X2Go server<br />
* {{Pkg|x2goclient}} - X2Go client based on Qt5<br />
<br />
== Server side ==<br />
<br />
=== Configure Secure Shell daemon ===<br />
<br />
X2Go uses [[Secure Shell]] in order to work, so you need to configure sshd daemon to allow X11 forwarding. Follow the instructions at [[OpenSSH#X11 forwarding]] and [[OpenSSH#Daemon management]].<br />
<br />
=== Check fuse kernel module is loaded ===<br />
<br />
In order for the server to be able to access files on the client computer, the fuse module is needed. <br />
One can check that {{ic|lsmod {{!}} grep fuse}} returns a match, otherwise load the {{ic|fuse}} [[kernel module]].<br />
<br />
=== Setup SQLite database ===<br />
<br />
Run the following command on the server to initialize the SQLite database (which is required in order for the x2go server to work):<br />
# x2godbadmin --createdb<br />
<br />
=== Control published applications ===<br />
<br />
X2Go can publish the installed applications in a menu to the client.<br />
This is controlled by the files in {{ic|/etc/x2go/applications/}}. This location however is not created by default and can be created by creating a symlink to {{ic|/usr/share/applications/}}.<br />
Alternatively instead of creating a symlink one could also create a folder and link only the desired applications instead.<br />
<br />
See [https://wiki.x2go.org/doku.php/wiki:advanced:published-applications] for more information.<br />
<br />
=== Start X2Go server daemon ===<br />
<br />
Now all you need to do is [[start]] the system {{ic|x2goserver.service}}.<br />
<br />
== Client side ==<br />
<br />
Run ''X2Go Client'' on the client computer, the one that wants to access the server:<br />
$ x2goclient<br />
For the list of available options, see {{man|1|x2goclient}}.<br />
{{Note|Make sure the client computer can open a SSH session to the server by checking from the client that {{ic|ssh ''username@host''}} is successful.}}<br />
<br />
You can now create several sessions, which then appear on the right side and can be selected by a mouse click. Each entry consists of your username, hostname, IP, and port for SSH connection. Furthermore you can define several speed profiles (coming from modem up to LAN) and the desktop environment you want to start remotely.<br />
<br />
=== Access the local desktop ===<br />
<br />
To access the local desktop, the one currently running on the server rather than a new one, one can choose the option "X2Go/X11 Desktop Sharing" or "Connection to local desktop" (depending on the version of your client) in "session type" in the ''X2Go Client'' as long as the users match, if it is user ''foo'' accessing the session of user ''foo''.<br />
<br />
However to access the local desktop of a different user, one needs to install {{Aur|x2godesktopsharing}} and launch {{ic|x2godesktopsharing}}.<br />
<br />
=== Exchange data between client and server (desktop) ===<br />
<br />
On the X2Go client (e.g. laptop) local directories could be shared. The server will use [[fuse]] and [[SSHFS]] to access this directory and mount it to a subdirectory media of your home directory on the server. This enables you to have access to laptop data on your server or to exchange files. It is also possible to mount these shares automatically at each session start.<br />
<br />
=== Leave a session temporarily ===<br />
<br />
Another special feature of X2Go is the possibility of suspending a session. This means you can leave a session on one client and reopen it even from another client at the same point. This can be used to to start a session in the LAN and to reopen it later on a laptop. The session data are stored and administered in a [[SQLite]] database on the server in the meanwhile. The state of the sessions is protocolled by a process named ''x2gocleansessions''.<br />
<br />
== Troubleshooting ==<br />
<br />
=== The desktop environment does not start ===<br />
<br />
==== Local session prevents X2Go new session ====<br />
<br />
It happens that when a desktop session already runs locally and X2Go tries to start a new one, it fails.<br />
This is typically an issue related to ''D-Bus'', see [https://bugzilla.redhat.com/show_bug.cgi?id=1350004] for details.<br />
<br />
If ''D-Bus'' fails to start, try using a ''Custom desktop'' command instead of the default session type. For the command, use the desktop starter as an option of {{ic|dbus-launch}}, for example {{ic|dbus-launch startxfce4}}. This is a way to launch a session bus instance, set the appropriate environment variables so that the new session can find the bus.<br />
<br />
==== Path issue ====<br />
<br />
It may be that the desktop environment's executable, ''startkde'', ''startgnome'' or ''startxfce4'' is not in the {{ic|$PATH}} when logging in using SSH. In this case, do not simply choose the defaults of KDE, Gnome or XFCE but use the full paths to the executable, for example {{ic|/usr/bin/startxfce4}}. You can also start [[openbox]] or another window manager.<br />
You should be asked for your server's password and user name, now and after login you will see the X2Go logo for a short time, and the desktop.<br />
<br />
=== No selection screen in x2goclient ===<br />
<br />
A regression in {{Pkg|iproute2}} causes ''ss'' to show no result when specifying the {{ic|-u}} flag, as done in {{ic|/usr/bin/x2golistdesktops}}. [https://lore.kernel.org/netdev/553EE0D9.3050601@ionic.de/]<br />
<br />
See [https://bugs.x2go.org/cgi-bin/bugreport.cgi?bug=799], [https://bbs.archlinux.org/viewtopic.php?pid=1541035] for more information.<br />
<br />
=== Sessions do not logoff correctly ===<br />
<br />
Due to [https://bugs.x2go.org/cgi-bin/bugreport.cgi?bug=914 this bug] the X2Go sessions might not logoff correctly. The script that initiates the session spits out many log lines that might confuse X2go. A simple workarround is to create a custom session script and redirect the log output either to a file or to {{ic|/dev/null}} and then point your X2Go-client to this custom script.<br />
<br />
Here is a sample script for an XFCE session:<br />
<br />
#!/bin/sh<br />
#<br />
#xfce4-session spits out quite a bit of text during logout, which I guess<br />
#confuses x2go so we would get a black screen and session hang.<br />
#adding redirect to a logfile like "~/logfile" or "/dev/null" nicely solved it<br />
# see https://bugs.x2go.org/cgi-bin/bugreport.cgi?bug=914<br />
/usr/bin/xfce4-session > /dev/null<br />
<br />
=== Notification area disappeared ===<br />
<br />
If you log in, but the notification area is missing, you can use exactly the same fix as for [[#Local session prevents X2Go new session]].<br />
<br />
=== Shared folders do not mount (Windows Clients) ===<br />
<br />
The ssh-daemon used by the X2go windows client uses depreceated ssh-dss keys by default and because Arch does not accept them your shared folders will not mount. Check out this [https://bugs.x2go.org/cgi-bin/bugreport.cgi?bug=1009 bug report] for more information.<br />
<br />
This can be solved on the windows side by generating different type of key:<br />
<br />
C:\Program Files (x86)\x2goclient\ssh-keygen -b 2048 -t rsa<br />
<br />
And simply replace {{ic|c:\Users\User\.x2go\etc\ssh_host_dsa_key}} and {{ic|c:\Users\User\.x2go\etc\ssh_host_dsa_key.pub}} with the newly generated key files.<br />
<br />
Other workarrounds from [https://bugs.x2go.org/cgi-bin/bugreport.cgi?bug=1009] might help, too.<br />
<br />
=== Workaround for failing compositing window manager for remote session ===<br />
<br />
{{style}}<br />
<br />
This is useful for situations, when the computer running x2goserver is used also for local sessions with e.g. compiz as the window manager. For remote connections with x2goclient, compiz fails to load and metacity should be used instead. The following is for GNOME, but could be modified for other desktop environments. (Getting compiz ready is not part of this how-to.)<br />
<br />
Create /usr/local/share/applications/gnome-wm-test.desktop:<br />
<br />
[Desktop Entry]<br />
Type=Application<br />
Encoding=UTF-8<br />
Name=gnome-wm-test<br />
Exec=/usr/local/bin/gnome-wm-test.sh<br />
NoDisplay=true<br />
<br />
Create script /usr/local/bin/gnome-wm-test.sh:<br />
<br />
#!/bin/sh<br />
# Script for choosing compiz when possible, otherwise metacity<br />
# Proper way to use this script is to set the key to mk-gnome-wm<br />
# /desktop/gnome/session/required_components/windowmanager<br />
xdpyinfo 2> /dev/null | grep -q "^ *Composite$" 2> /dev/null<br />
IS_X_COMPOSITED=$?<br />
if [ $IS_X_COMPOSITED -eq 0 ] ; then<br />
gtk-window-decorator &<br />
WM="compiz ccp --indirect-rendering --sm-client-id $DESKTOP_AUTOSTART_ID"<br />
else<br />
WM="metacity --sm-client-id=$DESKTOP_AUTOSTART_ID"<br />
fi<br />
exec bash -c "$WM"<br />
<br />
Modify the following gconf key to start the session with gnome-wm-test window manager:<br />
<br />
$ gconftool-2 --type string --set /desktop/gnome/session/required_components/windowmanager "gnome-wm-test"<br />
<br />
=== /bin/bash: No such file or directory when connect (or what ever shell you use) ===<br />
<br />
In you ssh configuration, if you chroot a user, this user need to have his own /bin directory inside his chrooted directory. If not, you will not be able to connect.<br />
<br />
=== X2Go client: Cannot connect to remote X2Go server from local Wayland session: "The connection with the remote server was shut down" ===<br />
<br />
X2Go client launches as expected from a local Wayland session but attempting to connect to a remote X2Go server fails. This problem is usually caused by Qt on Wayland.<br />
<br />
Follow the instructions in [[Desktop entries#Modify environment variables]] and change the lines starting with<br />
<br />
Exec=x2goclient<br />
<br />
to<br />
<br />
Exec=env QT_QPA_PLATFORM=xcb x2goclient<br />
<br />
=== X2Go client: Cannot connect to xyz:22 - Could not apply options ===<br />
<br />
x2go client can not parse custom user ssh config files with tokens.<br />
<br />
This ssh config<br />
<br />
Host xyz<br />
IdentitiesOnly yes<br />
IdentityFile %d/.ssh/%L.key<br />
<br />
will result in connection failure with debug output:<br />
<br />
x2go-DEBUG-../src/sshmasterconnection.cpp:622> "Cannot connect to xyz:22" - "Couldn't apply options"<br />
<br />
This ssh config works:<br />
<br />
Host xyz<br />
IdentitiesOnly yes<br />
IdentityFile ~/.ssh/mysupersecret.key<br />
<br />
=== X2Go server: localhost ssh tunnel on server fails with sshd error: error: connect_to localhost port xyz: failed. ===<br />
<br />
X2Go server requires IPv4 for localhost ssh tunnel on the server. If you configure the sshd server to use IPv6 only<br />
<br />
AddressFamily inet6<br />
<br />
X2Go client connections will fail.<br />
<br />
=== Performance issues ===<br />
<br />
In case of performance issues (applications are unresponsive or freeze), try to turn off sound support, printing support and file share tunneling.<br />
<br />
== See also ==<br />
<br />
* [https://wiki.archlinux.de/?title=Bild:X2go-1.png Screenshot KDE-Session]<br />
* [https://wiki.archlinux.de/?title=Bild:X2go-2.png Screenshot configuration dialog]</div>Phireskyhttps://wiki.archlinux.org/index.php?title=Gamepad&diff=695948Gamepad2021-09-15T19:01:00Z<p>Phiresky: /* PlayStation 3 controller tip */</p>
<hr />
<div>[[Category:Input devices]]<br />
[[Category:Gaming]]<br />
[[ja:ゲームパッド]]<br />
[[ru:Gamepad]]<br />
Many gamepads are working out-of-the-box nowadays, but there are still many potential problems and sources for errors since gamepad support in applications varies by a lot.<br />
<br />
== Gamepad input systems ==<br />
<br />
{{Expansion|Need info about differences between API, how to switch between them.|section=Joystick API vibration support}}<br />
Linux has two different input systems for Gamepads – the original Joystick interface and the newer evdev-based interface.<br />
<br />
{{ic|1=/dev/input/jsX}} maps to the Joystick API interface and {{ic|/dev/input/event*}} maps to the evdev ones (this also includes other input devices such as mice and keyboards). Symbolic links to those devices are also available in {{ic|/dev/input/by-id/}} and {{ic|/dev/input/by-path/}} where the legacy Joystick API has names ending with {{ic|-joystick}} while the evdev have names ending with {{ic|-event-joystick}}.<br />
<br />
Most new games will default to the evdev interface as it gives more detailed information about the buttons and axes available and also adds support for force feedback.<br />
<br />
While SDL1 defaults to evdev interface you can force it to use the old Joystick API by setting the environment variable {{ic|1=SDL_JOYSTICK_DEVICE=/dev/input/js0}}. This can help many games such as X3. SDL2 supports only the new evdev interface.<br />
<br />
== Determining which modules you need ==<br />
<br />
Unless you are using very old joystick that uses [[Wikipedia:Game port|Gameport]] or a proprietary USB protocol, you will need just the generic USB Human Interface Device (HID) modules.<br />
<br />
For an extensive overview of all joystick related modules in Linux, you will need access to the Linux kernel sources -- specifically the Documentation section. Unfortunately, pacman kernel packages do not include what we need. If you have the kernel sources downloaded, have a look at {{ic|Documentation/input/joydev/}}. You can browse the kernel source tree at [https://kernel.org/ kernel.org] by clicking the "browse" (cgit - the git frontend) link for the kernel that you are using, then clicking the "tree" link near the top. Here is a link to the [https://www.kernel.org/doc/html/latest/input/joydev/joystick.html documentation from the latest kernel].<br />
<br />
Some joysticks need specific modules, such as the Microsoft Sidewinder controllers ({{ic|sidewinder}}), or the Logitech digital controllers ({{ic|adi}}). Many older joysticks will work with the simple {{ic|analog}} module. If your joystick is plugging in to a gameport provided by your soundcard, you will need your soundcard drivers loaded - however, some cards, like the Soundblaster Live, have a specific gameport driver ({{ic|emu10k1-gp}}). Older ISA soundcards may need the {{ic|ns558}} module, which is a standard gameport module.<br />
<br />
As you can see, there are many different modules related to getting your joystick working in Linux, so I could not possibly cover everything here. Please have a look at the documentation mentioned above for details.<br />
<br />
=== Loading the modules for analogue devices ===<br />
<br />
You need to load a module for your gameport ({{ic|ns558}}, {{ic|emu10k1-gp}}, {{ic|cs461x}}, etc...), a module for your joystick ({{ic|analog}}, {{ic|sidewinder}}, {{ic|adi}}, etc...), and finally the kernel joystick device driver ({{ic|joydev}}). Add these to a new file in {{ic|/etc/modules-load.d/}}, or simply modprobe them. The {{ic|gameport}} module should load automatically, as this is a dependency of the other modules.<br />
<br />
=== USB gamepads ===<br />
<br />
You need to get USB working, and then modprobe your gamepad driver, which is {{ic|usbhid}}, as well as {{ic|joydev}}. <br />
If you use a usb mouse or keyboard, {{ic|usbhid}} will be loaded already and you just have to load the {{ic|joydev}} module.<br />
<br />
==== Troubleshooting ====<br />
<br />
If your Xbox 360 gamepad is connected with the Play&Charge USB cable it will show up in {{ic|lsusb}} but it will not show up as an input device in {{ic|/dev/input/js*}}, see note below.<br />
<br />
== Testing your configuration ==<br />
<br />
Once the modules are loaded, you should be able to find a new device: {{ic|/dev/input/js0}} and a file ending with {{ic|-event-joystick}} in {{ic|/dev/input/by-id}} directory. You can simply {{ic|cat}} those devices to see if the joystick works - move the stick around, press all the buttons - you should see mojibake printed when you move the sticks or press buttons. <br />
<br />
Both interfaces are also supported in wine and reported as separate devices. You can test them (including vibration feedback) with {{ic|1=wine control joy.cpl}}.<br />
<br />
{{Tip| Input devices by default have '''input''' group; for example, {{pkg|pcsx2}} have no access to gamepad without rights. Make sure your user is in the '''input''' group.}}<br />
<br />
=== Joystick API ===<br />
<br />
There are a lot of applications that can test this old API, {{ic|jstest}} from the {{pkg|joyutils}} package is the simplest one. If the output is unreadable because the line printed is too long you can also use graphical tools. KDE Plasma has a built in one in System Settings -&gt; Input Devices -&gt; Game Controller. There is {{AUR|jstest-gtk-git}} as an alternative.<br />
<br />
Use of {{ic|jstest}} is fairly simple, you just run {{ic|jstest /dev/input/js0}} and it will print a line with state of all the axes (normalised to {-32767,32767}) and buttons.<br />
<br />
After you start {{ic|jstest-gtk}}, it will just show you a list of joysticks available, you just need to select one and press Properties.<br />
<br />
=== evdev API ===<br />
<br />
The new 'evdev' API can be tested using the SDL2 joystick test application or using {{ic|evtest}} from community repository. Install {{AUR|sdl2-jstest-git}} and then run {{ic|sdl2-jstest --test 0}}. Use {{ic|sdl2-jstest --list}} to get IDs of other controllers if you have multiple ones connected.<br />
<br />
To test force feedback on the device, use {{ic|fftest}} from {{ic|linuxconsole}} package:<br />
$ fftest /dev/input/by-id/usb-*event-joystick<br />
<br />
=== HTML5 Gamepad API ===<br />
<br />
Go to https://gamepad-tester.com/. To test vibration, click on the word Vibration. Note, that vibration is currently supported in Chromium browser, but not in Firefox. Also, in Firefox you cannot see a gamepad image, but in Chromium you can.<br />
<br />
== Setting up deadzones and calibration ==<br />
<br />
If you want to set up the deadzones (or remove them completely) of your analog input you have to do it separately for the xorg (for mouse and keyboard emulation), Joystick API and evdev API.<br />
<br />
=== Wine deadzones ===<br />
<br />
Add the following registry entry and set it to a string from 0 to 10000 (affects all axes):<br />
HKEY_CURRENT_USER\Software\Wine\DirectInput\DefaultDeadZone<br />
Source: [https://wiki.winehq.org/UsefulRegistryKeys UsefulRegistryKeys]<br />
<br />
=== Xorg deadzones ===<br />
<br />
Add a similar line to {{ic|/etc/X11/xorg.conf.d/51-joystick.conf}} (create if it does not exist):<br />
{{hc|1=/etc/X11/xorg.conf.d/51-joystick.conf|2=<nowiki><br />
Section "InputClass"<br />
Option "MapAxis1" "deadzone=1000"<br />
EndSection<br />
</nowiki>}}<br />
1000 is the default value, but you can set anything between 0 and 30 000. To get the axis number see the "Testing Your Configuration" section of this article.<br />
If you already have an option with a specific axis just type in the {{ic|1=deadzone=value}} at the end of the parameter separated by a space.<br />
<br />
=== Joystick API deadzones ===<br />
<br />
The easiest way is using {{ic|jstest-gtk}} from {{AUR|jstest-gtk-git}}. Select the controller you want to edit, then click the Calibration button at the bottom of the dialog ('''do not''' click Start Calibration there). You can then set the CenterMin and CenterMax values (which control the center deadzone), RangeMin and RangeMax which control the end of throw deadzones. Note that the calibration settings are applied when the application opens the device, so you need to restart your game or test application to see updated calibration settings.<br />
<br />
After you set the deadzones use {{ic|jscal}} to dump the new values into a shell script:<br />
$ jscal -p /dev/input/jsX > jscal.sh # replace X with your joystick's number <br />
$ chmod +x jscal.sh<br />
<br />
Now you need to make a [[udev]] rule (for example {{ic|/etc/udev/rules.d and name it 85-jscal.rules}}) so the script will automatically run when you connect the controller:<br />
SUBSYSTEM=="input", ATTRS{idVendor}=="054c", ATTRS{idProduct}=="c268", ACTION=="add", RUN+="/usr/bin/bash /usr/bin/jscal.sh"<br />
To get the idVendor and idProduct use {{ic|udevadm info --attribute-walk --name /dev/input/jsX}}<br />
<br />
Use the `/dev/input/by-id/*-joystick` device names in case you use multiple controllers.<br />
<br />
=== evdev API deadzones ===<br />
<br />
The {{ic|evdev-joystick}} tool from the {{pkg|linuxconsole}} package can be used to view and change deadzones and calibration for {{ic|evdev}} API devices.<br />
<br />
To view your device configuration:<br />
$ evdev-joystick --showcal /dev/input/by-id/usb-*-event-joystick<br />
<br />
To change the deadzone for a particular axis, use a command like:<br />
$ evdev-joystick --evdev /dev/input/by-id/usb-*-event-joystick --axis 0 --deadzone 0<br />
<br />
To set the same deadzone for all axes at once, omit the "--axis 0" option.<br />
<br />
Use udev rules file to set them automatically when the controller is connected.<br />
<br />
Note that inside the kernel, the value is called {{ic|flatness}} and is set using the {{ic|EVIOCSABS}} {{ic|ioctl}}.<br />
<br />
Default configuration will look like similar to this:<br />
{{hc|$ evdev-joystick --showcal /dev/input/by-id/usb-Madcatz_Saitek_Pro_Flight_X-55_Rhino_Stick_G0000090-event-joystick|2= Supported Absolute axes:<br />
Absolute axis 0x00 (0) (X Axis) (min: 0, max: 65535, flatness: 4095 (=6.25%), fuzz: 255)<br />
Absolute axis 0x01 (1) (Y Axis) (min: 0, max: 65535, flatness: 4095 (=6.25%), fuzz: 255)<br />
Absolute axis 0x05 (5) (Z Rate Axis) (min: 0, max: 4095, flatness: 255 (=6.23%), fuzz: 15)<br />
Absolute axis 0x10 (16) (Hat zero, x axis) (min: -1, max: 1, flatness: 0 (=0.00%), fuzz: 0)<br />
Absolute axis 0x11 (17) (Hat zero, y axis) (min: -1, max: 1, flatness: 0 (=0.00%), fuzz: 0)}}<br />
<br />
While a more reasonable setting would be achieved with something like this (repeat for other axes):<br />
{{hc|$ evdev-joystick --evdev /dev/input/by-id/usb-Madcatz_Saitek_Pro_Flight_X-55_Rhino_Stick_G0000090-event-joystick --axis 0 --deadzone 512|2= Event device file: /dev/input/by-id/usb-Madcatz_Saitek_Pro_Flight_X-55_Rhino_Stick_G0000090-event-joystick<br />
Axis index to deal with: 0<br />
New dead zone value: 512<br />
Trying to set axis 0 deadzone to: 512<br />
Absolute axis 0x00 (0) (X Axis) Setting deadzone value to : 512<br />
(min: 0, max: 65535, flatness: 512 (=0.78%), fuzz: 255)}}<br />
<br />
=== Configuring curves and responsiveness ===<br />
<br />
In case your game requires just limited amount of buttons or has good support for multiple controllers, you may have good results with using {{ic|xboxdrv}} to change response curves of the joystick.<br />
<br />
Below are the setups I use for Saitek X-55 HOTAS:<br />
$ xboxdrv --evdev /dev/input/by-id/usb-Madcatz_Saitek_Pro_Flight_X-55_Rhino_Throttle_G0000021-event-joystick \<br />
--evdev-no-grab --evdev-absmap 'ABS_#40=x1,ABS_#41=y1,ABS_X=x2,ABS_Y=y2' --device-name 'Hat and throttle' \<br />
--ui-axismap 'x2^cal:-32000:0:32000=,y2^cal:-32000:0:32000=' --silent<br />
<br />
this maps the EV_ABS event with id of 40 and 41 (use xboxdrv with --evdev-debug to see the events registered), which is the normally inaccessible "mouse pointer" on the throttle, to first gamepad joystick and throttles to second joystick, it also clamps the top and lower ranges as they not always register fully.<br />
<br />
A bit more interesting is the setup for the stick:<br />
$ xboxdrv --evdev /dev/input/by-id/usb-Madcatz_Saitek_Pro_Flight_X-55_Rhino_Stick_G0000090-event-joystick \<br />
--evdev-no-grab --evdev-absmap 'ABS_X=x1' --evdev-absmap 'ABS_Y=y1' --device-name 'Joystick' \<br />
--ui-axismap 'x1^cal:-32537:-455:32561=,x1^dead:-900:700:1=,x1^resp:-32768:-21845:-2000:0:2000:21485:32767=' \<br />
--ui-axismap 'y1^cal:-32539:-177:32532=,y1^dead:-700:2500:1=,y1^resp:-32768:-21845:-2000:0:2000:21485:32767=' \<br />
--evdev-absmap 'ABS_RZ=x2' --ui-axismap 'x2^cal:-32000:-100:32000,x2^dead:-1500:1000:1=,x2^resp:-32768:-21845:-2000:0:2000:21485:32767=' \<br />
--silent<br />
<br />
this maps the 3 joystick axes to gamepad axes and changes the calibration (min value, centre value, max value), dead zones (negative side, positive side, flag to turn smoothing) and finally change of response curve to a more flat one in the middle.<br />
<br />
You can also modify the responsiveness by setting the 'sen' (sensitivity). Setting it to value of 0 will give you a linear sensitivity, value of -1 will give very insensitive axis while value of 1 will give very sensitive axis. You can use intermediate values to make it less or more sensitive. Internally xboxdrv uses a quadratic formula to calculate the resulting value, so this setting gives a more smooth result than 'resp' shown above.<br />
<br />
Nice thing about xboxdrv is that it exports resulting device as both old Joystick API and new style evdev API so it should be compatible with basically any application.<br />
<br />
== Disable joystick from controlling mouse ==<br />
<br />
If you want to play games with your gamepad, you might want to disable its joystick control over mouse cursor. To do this, edit {{ic|/etc/X11/xorg.conf.d/51-joystick.conf}} (create if it does not exists) so that it looks like this:<br />
{{hc|/etc/X11/xorg.conf.d/51-joystick.conf |<br />
Section "InputClass"<br />
Identifier "joystick catchall"<br />
MatchIsJoystick "on"<br />
MatchDevicePath "/dev/input/event*"<br />
Driver "joystick"<br />
Option "StartKeysEnabled" "False" #Disable mouse<br />
Option "StartMouseEnabled" "False" #support<br />
EndSection}}<br />
<br />
== Using gamepad to send keystrokes ==<br />
<br />
A couple gamepad buttons to keystroke programs exist like {{AUR|qjoypad}} or {{AUR|antimicrox-git}}, all work well without the need for X.org configuration.<br />
<br />
=== Xorg configuration example ===<br />
<br />
This is a good solution for systems where restarting Xorg is a rare event because it is a static configuration loaded only on X startup. The example runs on a [[Kodi]] media PC, controlled with a Logitech Cordless RumblePad 2. Due to a problem with the d-pad (a.k.a. "hat") being recognized as another axis, [[Joy2key]] was used as a workaround. Since upgrade to Kodi version 11.0 and joy2key 1.6.3-1, this setup no longer worked and the following was created for letting Xorg handle joystick events.<br />
<br />
First, [[install]] the {{AUR|xf86-input-joystick}} package. Then, create {{ic|/etc/X11/xorg.conf.d/51-joystick.conf}} like so:<br />
{{bc|<nowiki><br />
Section "InputClass"<br />
Identifier "Joystick hat mapping"<br />
Option "StartKeysEnabled" "True"<br />
#MatchIsJoystick "on"<br />
Option "MapAxis5" "keylow=113 keyhigh=114"<br />
Option "MapAxis6" "keylow=111 keyhigh=116"<br />
EndSection<br />
</nowiki>}}<br />
{{Note|The {{ic|MatchIsJoystick "on"}} line does not seem to be required for the setup to work, but you may want to uncomment it.}}<br />
<br />
== Specific devices ==<br />
<br />
While most gamepads, especially USB based ones should just work, some may require (or give better results) if you use alternative drivers. If it does not work the first time, do not give up, and read those docs thoroughly!<br />
<br />
=== Dance pads ===<br />
<br />
Most dance pads should work. However some pads, especially those used from a video game console via an adapter, have a tendency to map the directional buttons as axis buttons. This prevents hitting left-right or up-down simultaneously. This behavior can be fixed for devices recognized by xpad via a module option:<br />
<br />
# modprobe -r xpad<br />
# modprobe xpad dpad_to_buttons=1<br />
<br />
If that did not work, you can try {{AUR|axisfix-git}} or patching the {{ic|joydev}} kernel module (https://github.com/adiel-mittmann/dancepad).<br />
<br />
=== Logitech Thunderpad Digital ===<br />
<br />
Logitech Thunderpad Digital will not show all the buttons if you use the {{ic|analog}} module. Use the device specific {{ic|adi}} module for this controller.<br />
<br />
=== Nintendo Gamecube Controller ===<br />
<br />
Dolphin Emulator has a [https://wiki.dolphin-emu.org/index.php?title=How_to_use_the_Official_GameCube_Controller_Adapter_for_Wii_U_in_Dolphin page on their wiki] that explains how to use the official Nintendo USB adapter with a Gamecube controller. This configuration also works with the Mayflash Controller Adapter if the switch is set to "Wii U".<br />
<br />
=== Nintendo Switch Pro Controller and Joy-Cons ===<br />
<br />
==== Kernel Nintendo HID Driver ====<br />
<br />
The hid-nintendo kernel HID driver is currently in review on the linux-input mailing list. The most recent version of the changeset is currently being maintained in this[https://github.com/DanielOgorchock/linux] git repository, or its staging branch[https://git.kernel.org/pub/scm/linux/kernel/git/hid/hid.git/log/?h=for-5.10/nintendo] for kernel 5.10. It's also available as a dkms module named {{AUR|hid-nintendo-dkms}}. The driver provides support for rumble, battery level, and control of the player and home LEDs. It supports the Nintendo Switch Pro Controller over both USB and bluetooth in addition to the joy-cons.<br />
<br />
An alternate dkms module named {{AUR|hid-nintendo-nso-dkms}} patches in support for the Switch Online NES and SNES controllers.<br />
<br />
===== joycond Userspace Daemon =====<br />
<br />
The hid-nintendo kernel driver does not handle the combination of two joy-cons into one virtual input device. That functionality has been left up to userspace. {{AUR|joycond-git}} is a userspace daemon that combines two kernel joy-con evdev devices into one virtual input device using uinput. An application can use two joy-cons as if they are a single controller. When the daemon is active, switch controllers will be placed in a pseudo pairing mode, and the LEDs will start flashing. Holding the triggers can be used to pair controllers and make them usable. To pair two joy-cons together, press one trigger on each joy-con.<br />
<br />
===== Using hid-nintendo pro controller with Steam Games (without joycond) =====<br />
<br />
The hid-nintendo driver currently conflicts with steam using hidraw to implement its own pro controller driver. If you wish to use the Steam implementation, the hid-nintendo driver can be blacklisted. Alternatively if you want to use hid-nintendo with a Steam game directly, Steam can be started without access to hidraw using firejail:<br />
<br />
$ firejail --noprofile --blacklist=/sys/class/hidraw/ steam<br />
<br />
An [https://github.com/ValveSoftware/steam-for-linux/issues/6651 issue] has been opened on the steam-for-linux github repo.<br />
<br />
===== Using hid-nintendo pro controller with Steam Games (with joycond) =====<br />
<br />
A workaround has been added to joycond to have udev rules to block steam from accessing hidraw for the pro controller.<br />
<br />
That alone isn't enough though, since steam won't recognize the pro controller when access to hidraw is blocked. To get around that issue, joycond supports creating a virtual pro controller with a different product id. Steam will view this as a typical controller. This gives the best of both worlds, where hid-nintendo can be used both inside and outside of steam simultaneously.<br />
<br />
To pair the pro controller in virtual mode, press the Plus and Minus buttons simultaneously (instead of using the triggers). You must be using a version of joycond including [https://github.com/DanielOgorchock/joycond/commit/e31db38eeae14c63331ea8dae972e3873d7ff6fa this commit].<br />
<br />
===== Using hid-nintendo with SDL2 Games =====<br />
<br />
To add a mapping for the joy-cons or the pro controller to an SDL2 game, {{AUR|controllermap}} can be run in the game's directory of games which have their own gamecontrollerdb.txt file.<br />
<br />
Alternatively, the mappings can be added to an environment variable:<br />
<br />
{{hc|1=~/.bashrc |2=# hid-nintendo SDL2 mappings<br />
export SDL_GAMECONTROLLERCONFIG="050000007e0500000920000001800000,Nintendo Switch Pro Controller,platform:Linux,a:b0,b:b1,x:b3,y:b2,back:b9,guide:b11,start:b10,leftstick:b12,rightstick:b13,leftshoulder:b5,rightshoulder:b6,dpup:h0.1,dpdown:h0.4,dpleft:h0.8,dpright:h0.2,leftx:a0,lefty:a1,rightx:a2,righty:a3,lefttrigger:b7,righttrigger:b8,<br />
030000007e0500000920000011810000,Nintendo Switch Pro Controller,platform:Linux,a:b0,b:b1,x:b3,y:b2,back:b9,guide:b11,start:b10,leftstick:b12,rightstick:b13,leftshoulder:b5,rightshoulder:b6,dpup:h0.1,dpdown:h0.4,dpleft:h0.8,dpright:h0.2,leftx:a0,lefty:a1,rightx:a2,righty:a3,lefttrigger:b7,righttrigger:b8,<br />
060000007e0500000620000000000000,Nintendo Switch Combined Joy-Cons,platform:Linux,a:b0,b:b1,x:b3,y:b2,back:b9,guide:b11,start:b10,leftstick:b12,rightstick:b13,leftshoulder:b5,rightshoulder:b6,dpup:b14,dpdown:b15,dpleft:b16,dpright:b17,leftx:a0,lefty:a1,rightx:a2,righty:a3,lefttrigger:b7,righttrigger:b8,<br />
"}}<br />
<br />
==== Dolphin (Gamecube Controller Emulation) ====<br />
<br />
Shinyquagsire23 made a git repo called "HID Joy-Con Whispering"[https://github.com/shinyquagsire23/HID-Joy-Con-Whispering], which contains a userspace driver for the Joy Cons and the Switch Pro Controller over USB. Currently, it does not support rumble or gyroscope. For rumble support, see the hid-nintendo kernel driver section above.<br />
<br />
After running make, load the uinput module:<br />
<br />
# modprobe uinput<br />
<br />
Then to activate the driver:<br />
<br />
# ./uinputdriver > /dev/null<br />
<br />
Over on Dolphin's controller configuration menu, there should be an entry for evdev/0/joycon (not Nintendo Switch Pro Controller). Select it, and you should now be able to configure the controls.<br />
<br />
==== Steam ====<br />
<br />
While the controller works for native Linux games, this controller is not detected by Steam. To fix this, we will need to add a line to 70-steam-controller.rules.<br />
<br />
{{hc<br />
|head=/lib/udev/rules.d/70-steam-controller.rules<br />
|output=<nowiki># NS PRO Controller USB<br />
KERNEL=="hidraw*", ATTRS{idVendor}=="20d6", ATTRS{idProduct}=="a711", MODE="0660", TAG+="uaccess"</nowiki><br />
}}<br />
<br />
udev can be reloaded with the new configuration by executing<br />
<br />
# udevadm control --reload-rules<br />
<br />
=== iPEGA-9017s and other Bluetooth gamepads ===<br />
<br />
If you want to use one of the widely available bluetooth gamepads, such as iPEGA-9017s designed mostly for Android and iOS devices you would need {{AUR|xboxdrv}}, {{Pkg|bluez}}, {{Pkg|bluez-plugins}}, and {{Pkg|bluez-utils}}. You should connect it in gamepad mode (if there are different modes, choose the gamepad one). Technically it's ready to be used, but in most cases games would not recognize it, and you would have to map it individually for all application. The best way to simplify it and make it work with all applications is to mimic Microsoft X360 controller with {{AUR|xboxdrv}}.<br />
Once connected you can create a udev rule to give it a persistent name, that would come in handy when setting it up.<br />
<br />
{{hc<br />
|/etc/udev/rules.d/99-btjoy.rules|2=<br />
#Create a symlink to appropriate /dev/input/eventX at /dev/btjoy<br />
ACTION=="add", SUBSYSTEM=="input", ATTRS{name}=="Bluetooth Gamepad", ATTRS{uniq}=="00:17:02:01:ae:2a", SYMLINK+="btjoy"<br />
}}<br />
<br />
Replace "Bluetooth Gampad" with your device name and "00:17:02:01:ae:2a" with your device's address.<br />
<br />
Next, create a configuration for {{AUR|xboxdrv}} somewhere, for example:<br />
<br />
{{hc<br />
|~/.config/xboxdrv/ipega.conf|2=<br />
#iPEGA PG-9017S Config <br />
<br />
[xboxdrv]<br />
evdev-debug = true<br />
evdev-grab = true<br />
rumble = false<br />
mimic-xpad = true<br />
<br />
[evdev-absmap]<br />
ABS_HAT0X = dpad_x<br />
ABS_HAT0Y = dpad_y<br />
<br />
ABS_X = X1<br />
ABS_Y = Y1<br />
<br />
ABS_Z = X2<br />
ABS_RZ = Y2<br />
<br />
[axismap]<br />
-Y1 = Y1<br />
-Y2 = Y2<br />
<br />
[evdev-keymap]<br />
BTN_EAST=a<br />
BTN_C=b<br />
BTN_NORTH=y<br />
BTN_SOUTH=x<br />
BTN_TR2=start<br />
BTN_TL2=back<br />
BTN_Z=rt<br />
BTN_WEST=lt<br />
<br />
BTN_MODE = guide<br />
}}<br />
<br />
Refer to the xboxdrv man to see all the options.<br />
<br />
Now when you have the config and your device is connected you can start the {{AUR|xboxdrv}} like so:<br />
<br />
# xboxdrv --evdev /dev/btjoy --config .config/xboxdrv/ipega.conf<br />
<br />
Your games will now work with bluetooth gamepad as long as xboxdrv is running.<br />
<br />
==== iPEGA-9068 and 9087 ====<br />
<br />
For this model, use the same procedures as above, but with the configs:<br />
<br />
{{hc<br />
|~/.config/xboxdrv/ipega.conf|2=<br />
#iPEGA PG-9068 and PG-9087 Config <br />
<br />
[xboxdrv]<br />
evdev-debug = true<br />
evdev-grab = true<br />
rumble = false<br />
mimic-xpad = true<br />
<br />
[evdev-absmap]<br />
ABS_HAT0X = dpad_x<br />
ABS_HAT0Y = dpad_y<br />
<br />
ABS_X = X1<br />
ABS_Y = Y1<br />
<br />
ABS_Z = X2<br />
ABS_RZ = Y2<br />
<br />
[axismap]<br />
-Y1 = Y1<br />
-Y2 = Y2<br />
<br />
[evdev-keymap]<br />
BTN_A=a<br />
BTN_B=b<br />
BTN_Y=y<br />
BTN_X=x<br />
BTN_TR=rb<br />
BTN_TL=lb<br />
BTN_TR2=rt<br />
BTN_TL2=lt<br />
BTN_THUMBL=tl<br />
BTN_THUMBR=tr<br />
BTN_START=start<br />
BTN_SELECT=back<br />
<br />
BTN_MODE = guide<br />
}}<br />
<br />
=== Steam Controller ===<br />
<br />
{{Note|Kernel 4.18 [https://lkml.org/lkml/2018/4/16/380 provides a kernel driver] for wired/wireless use of the steam controller as a controller input device without [[Steam]].}}<br />
<br />
The [[Steam]] client will recognize the controller and provide keyboard/mouse/gamepad emulation while Steam is running. The in-game Steam overlay needs to be enabled and working in order for gamepad emulation to work. You may need to run {{ic|udevadm trigger}} with root privileges or plug the dongle out and in again, if the controller does not work immediately after installing and running Steam. If all else fails, try restarting the computer while the dongle is plugged in.<br />
<br />
For Steam client to be able to emulate other gamepads in games, you will need to follow this [https://steamcommunity.com/app/353370/discussions/2/1735465524711324558/ post] from Valve. Note that name of the file with udev rules may be different, for example {{ic|/usr/lib/udev/rules.d/70-steam-input.rules}}. A reboot may be required after making changes.<br />
<br />
If you are using the controller connected via Bluetooth LE, make sure the user is part of the {{ic|input}} group.<br />
<br />
If you cannot get the Steam Controller to work, see [[#Steam Controller not pairing]].<br />
<br />
Alternatively you can install {{AUR|python-steamcontroller-git}} to have controller and mouse emulation without Steam or {{AUR|sc-controller}} for a versatile graphical configuration tool simillar to what is provided by the Steam client.<br />
<br />
{{Note|If you do not use the [[Steam runtime]], you might actually need to disable the overlay for the controller to work in certain games (Rocket Wars, Rocket League, Binding of Isaac, etc.). Right click on a game in your library, select "Properties", and uncheck "Enable Steam Overlay".}}<br />
<br />
==== Wine ====<br />
<br />
{{AUR|python-steamcontroller-git}} can also be used to make the Steam Controller work for games running under Wine. You need to find and download the application {{ic|xbox360cemu.v.3.0}} (e.g. from [https://github.com/jacobmischka/ds4-in-wine/tree/master/xbox360cemu.v.3.0 here]). Then copy the files {{ic|dinput8.dll}}, {{ic|xbox360cemu.ini}}, {{ic|xinput1_3.dll}} and {{ic|xinput_9_1_0.dll}} to the directory that contains your game executable. Edit {{ic|xbox360cemu.ini}} and only change the following values under {{ic|[PAD1]}} to remap the Steam Controller correctly to a XBox controller.<br />
<br />
{{hc|xbox360cemu.ini|2= Right Analog X=4<br />
Right Analog Y=-5<br />
A=1<br />
B=2<br />
X=3<br />
Y=4<br />
Back=7<br />
Start=8<br />
Left Thumb=10<br />
Right Thumb=11<br />
Left Trigger=a3<br />
Right Trigger=a6}}<br />
<br />
Now start python-steamcontroller in Xbox360 mode ({{ic|sc-xbox.py start}}). You might also want to copy {{ic|XInputTest.exe}} from {{ic|xbox360cemu.v.3.0}} to the same directory and run it with Wine in order to test if the mappings work correctly. However neither mouse nor keyboard emulation work with this method.<br />
<br />
Alternatively you can use {{AUR|sc-controller}} for a similar graphical setup as Steam's own configurator. As of writing, it's a bit buggy here and there but offers an easy click and go way of configuring the controller.<br />
<br />
=== Xbox 360 controller ===<br />
<br />
Both the wired and wireless (with the ''Xbox 360 Wireless Receiver for Windows'') controllers are supported by the {{ic|xpad}} kernel module and should work without additional packages. Note that using a wireless Xbox360 controller with the Play&Charge USB cable will not work. The cable is for recharging only and does not transmit any input data over the wire.<br />
<br />
It has been reported that the default xpad driver has some issues with a few newer wired and wireless controllers, such as:<br />
* incorrect button mapping. ([https://github.com/ValveSoftware/steam-for-linux/issues/95#issuecomment-14009081 discussion in Steam bugtracker])<br />
* not-working sync. All four LEDs keep blinking, but controller works. ([https://bbs.archlinux.org/viewtopic.php?id=156028 discussion in Arch Forum])<br />
<br />
If you experience such issues, you can use [[#xboxdrv]] as the default {{ic|xpad}} driver instead.<br />
<br />
If you wish to use the controller for controlling the mouse, or mapping buttons to keys, etc. you should use the {{AUR|xf86-input-joystick}} package (configuration help can be found using {{man|4|joystick|url=}}). If the mouse locks itself in a corner, it might help changing the {{ic|MatchDevicePath}} in {{ic|/etc/X11/xorg.conf.d/50-joystick.conf}} from {{ic|/dev/input/event*}} to {{ic|/dev/input/js*}}.<br />
<br />
{{Tip|If you use the [[TLP]] power management tool, you may experience connection issues with your Microsoft wireless adapter (e.g. the indicator LED will go out after the adapter has been connected for a few seconds, and controller connection attempts fail). This is due to TLP's USB autosuspend functionality, and the solution is to add the Microsoft wireless adapter's device ID to this feature's blacklist (USB_BLACKLIST, check [https://linrunner.de/en/tlp/docs/tlp-configuration.html#usb TLP configuration] for more details).}}<br />
<br />
In order to connect via Bluetooth using KDE, add the following [[kernel parameter]] {{ic|1=bluetooth.disable_ertm=1}}.<br />
<br />
==== xboxdrv ====<br />
<br />
[https://gitlab.com/xboxdrv/xboxdrv xboxdrv] is an alternative to {{ic|xpad}} which provides more functionality and might work better with certain controllers. It works in userspace and can be launched as system service. <br />
<br />
Install it with the {{AUR|xboxdrv}} package. Then [[start]]/[[enable]] {{ic|xboxdrv.service}}.<br />
<br />
If you have issues with the controller being recognized but not working in steam games or working but with incorrect mappings, it may be required to modify you config as such:<br />
{{hc<br />
|/etc/default/xboxdrv|2=<br />
[xboxdrv]<br />
silent = true<br />
device-name = "Xbox 360 Wireless Receiver"<br />
mimic-xpad = true<br />
deadzone = 4000<br />
<br />
[xboxdrv-daemon]<br />
dbus = disabled<br />
}}<br />
<br />
Then [[restart]] {{ic|xboxdrv.service}}.<br />
<br />
===== Multiple controllers =====<br />
<br />
xboxdrv supports a multitude of controllers, but they need to be set up in {{ic|/etc/default/xboxdrv}}. For each extra controller, add an {{ic|1=next-controller = true}} line. For example, when using 4 controllers, add it 3 times:<br />
<br />
{{bc|<nowiki><br />
[xboxdrv]<br />
silent = true<br />
next-controller = true<br />
next-controller = true<br />
next-controller = true<br />
[xboxdrv-daemon]<br />
dbus = disabled<br />
</nowiki>}}<br />
<br />
Then [[restart]] {{ic|xboxdrv.service}}.<br />
<br />
===== Mimic Xbox 360 controller with other controllers =====<br />
<br />
xboxdrv can be used to make any controller register as an Xbox 360 controller with the {{ic|--mimic-xpad}} switch. This may be desirable for games that support Xbox 360 controllers out of the box, but have trouble detecting or working with other gamepads.<br />
<br />
First, you need to find out what each button and axis on the controller is called. You can use {{Pkg|evtest}} for this. Run {{ic|evtest}} and select the device event ID number ({{ic|/dev/input/event*}}) that corresponds to your controller. Press the buttons on the controller and move the axes to read the names of each button and axis.<br />
<br />
Here is an example of the output:<br />
{{bc|<nowiki><br />
<br />
Event: time 1380985017.964843, type 4 (EV_MSC), code 4 (MSC_SCAN), value 90003<br />
Event: time 1380985017.964843, type 1 (EV_KEY), code 290 (BTN_THUMB2), value 1<br />
Event: time 1380985017.964843, -------------- SYN_REPORT ------------<br />
Event: time 1380985018.076843, type 4 (EV_MSC), code 4 (MSC_SCAN), value 90003<br />
Event: time 1380985018.076843, type 1 (EV_KEY), code 290 (BTN_THUMB2), value 0<br />
Event: time 1380985018.076843, -------------- SYN_REPORT ------------<br />
Event: time 1380985018.460841, type 4 (EV_MSC), code 4 (MSC_SCAN), value 90002<br />
Event: time 1380985018.460841, type 1 (EV_KEY), code 289 (BTN_THUMB), value 1<br />
Event: time 1380985018.460841, -------------- SYN_REPORT ------------<br />
Event: time 1380985018.572835, type 4 (EV_MSC), code 4 (MSC_SCAN), value 90002<br />
Event: time 1380985018.572835, type 1 (EV_KEY), code 289 (BTN_THUMB), value 0<br />
Event: time 1380985018.572835, -------------- SYN_REPORT ------------<br />
Event: time 1380985019.980824, type 4 (EV_MSC), code 4 (MSC_SCAN), value 90006<br />
Event: time 1380985019.980824, type 1 (EV_KEY), code 293 (BTN_PINKIE), value 1<br />
Event: time 1380985019.980824, -------------- SYN_REPORT ------------<br />
Event: time 1380985020.092835, type 4 (EV_MSC), code 4 (MSC_SCAN), value 90006<br />
Event: time 1380985020.092835, type 1 (EV_KEY), code 293 (BTN_PINKIE), value 0<br />
Event: time 1380985020.092835, -------------- SYN_REPORT ------------<br />
Event: time 1380985023.596806, type 3 (EV_ABS), code 3 (ABS_RX), value 18<br />
Event: time 1380985023.596806, -------------- SYN_REPORT ------------<br />
Event: time 1380985023.612811, type 3 (EV_ABS), code 3 (ABS_RX), value 0<br />
Event: time 1380985023.612811, -------------- SYN_REPORT ------------<br />
Event: time 1380985023.708768, type 3 (EV_ABS), code 3 (ABS_RX), value 14<br />
Event: time 1380985023.708768, -------------- SYN_REPORT ------------<br />
Event: time 1380985023.724772, type 3 (EV_ABS), code 3 (ABS_RX), value 128<br />
Event: time 1380985023.724772, -------------- SYN_REPORT ------------<br />
</nowiki>}}<br />
<br />
In this case, {{ic|BTN_THUMB}}, {{ic|BTN_THUMB2}} and {{ic|BTN_PINKIE}} are buttons and {{ic|ABS_RX}} is the X axis of the right analogue stick.<br />
You can now mimic an Xbox 360 controller with the following command:<br />
<br />
$ xboxdrv --evdev /dev/input/event* --evdev-absmap ABS_RX=X2 --evdev-keymap BTN_THUMB2=a,BTN_THUMB=b,BTN_PINKIE=rt --mimic-xpad<br />
<br />
The above example is incomplete. It only maps one axis and 3 buttons for demonstration purposes. Use {{ic|xboxdrv --help-button}} to see the names of the Xbox controller buttons and axes and bind them accordingly by expanding the command above. Axes mappings should go after {{ic|--evdev-absmap}} and button mappings follow {{ic|--evdev-keymap}} (comma separated list; no spaces).<br />
<br />
By default, xboxdrv outputs all events to the terminal. You can use this to test that the mappings are correct. Append the {{ic|--silent}} option to keep it quiet.<br />
<br />
==== Using generic/clone controllers ====<br />
<br />
Some clone gamepads might require a specific initialization sequence in order to work ([https://superuser.com/a/1380235 Super User answer]). For that you should run the following python script as the root user:<br />
<br />
{{bc|<nowiki><br />
#!/usr/bin/env python3<br />
<br />
import usb.core<br />
<br />
dev = usb.core.find(idVendor=0x045e, idProduct=0x028e)<br />
<br />
if dev is None:<br />
raise ValueError('Device not found')<br />
else:<br />
dev.ctrl_transfer(0xc1, 0x01, 0x0100, 0x00, 0x14) <br />
</nowiki>}}<br />
<br />
=== Xbox Wireless Controller / Xbox One Wireless Controller ===<br />
<br />
==== Connect Xbox Wireless Controller with usb cable ====<br />
<br />
This is supported by the kernel and should work without additional packages.<br />
<br />
==== Connect Xbox Wireless Controller with Bluetooth ====<br />
<br />
===== Update controller firmware via Windows 10 =====<br />
<br />
The firmware of the Xbox Wireless Controller used to cause loops of connecting / disconnecting with Bluez. The best workaround for now is to plug (via a USB cord) the controller to a Windows 10 computer, download the xbox accessories application, and update the firmware of the controller.<br />
<br />
===== Old workaround =====<br />
<br />
{{Out of date|Disabling ERTM is not required for kernel 5.12+ and there seems to be some connection tricks that may not be necessary in the future}}<br />
<br />
{{Note|Disabling ERTM is not required since kernel version 5.12.}}<br />
<br />
You may have to disable Enhanced Retransmission Mode (ERTM) to make it work. Use either:<br />
<br />
# echo 1 > /sys/module/bluetooth/parameters/disable_ertm<br />
<br />
Or add this file to your module configuration:<br />
<br />
{{hc|/etc/modprobe.d/xbox_bt.conf|2=<br />
options bluetooth disable_ertm=1<br />
}}<br />
<br />
===== xpadneo =====<br />
<br />
A relatively new driver which does support the Xbox One S and Xbox Series X|S controller via Bluetooth is called [https://github.com/atar-axis/xpadneo/ xpadneo]. In addition to these two models, it has also basic support for the Xbox Elite Series 2 Wireless controller. In exchange for fully supporting just two controllers so far, it enables one to read out the correct battery level, supports rumble (even the one on the trigger buttons - L2/R2), corrects the (sometimes wrong) button mapping and more.<br />
<br />
Installation is done using DKMS: {{AUR|xpadneo-dkms-git}}.<br />
<br />
{{Note|Pairing a new Xbox One S controller for the first time may prove difficult, from not pairing at all to entering a connect/disconnect loop. These problems are described [https://github.com/atar-axis/xpadneo/issues/295 there]. The best way to reliably pair the controller is to first pair it in Windows 10. However, this needs be done using the same Bluetooth adapter. A solution is to install a free copy of Windows 10 Evaluation on a Virtual machine (using [[QEMU]] or [[VirtualBox]], taking care of the Bluetooth adapter passthrough requirements, ''e.g.'' as an USB device) using Archlinux as your host, and pair in Windows 10 first, then do the same again under your Arch Linux system. Then pairing will succeed and there will be no need of further Windows 10 use.}}<br />
<br />
==== Connect Xbox Wireless Controller with Microsoft Xbox Wireless Adapter ====<br />
<br />
===== xow =====<br />
<br />
[https://github.com/medusalix/xow xow] is a project that allows connection with a wireless dongle. It is currently in very early stages of development. It can be installed via {{AUR|xow-git}}<br />
<br />
=== Logitech Dual Action ===<br />
<br />
The Logitech Dual Action gamepad has a very similar mapping to the PS2 pad, but some buttons and triggers need to be swapped to mimic the Xbox controller.<br />
<br />
# xboxdrv --evdev /dev/input/event* \<br />
--evdev-absmap ABS_X=x1,ABS_Y=y1,ABS_RZ=x2,ABS_Z=y2,ABS_HAT0X=dpad_x,ABS_HAT0Y=dpad_y \<br />
--axismap -Y1=Y1,-Y2=Y2 \<br />
--evdev-keymap BTN_TRIGGER=x,BTN_TOP=y,BTN_THUMB=a,BTN_THUMB2=b,BTN_BASE3=back,BTN_BASE4=start,BTN_BASE=lt,BTN_BASE2=rt,BTN_TOP2=lb,BTN_PINKIE=rb,BTN_BASE5=tl,BTN_BASE6=tr \<br />
--mimic-xpad --silent<br />
<br />
=== PlayStation 2 controller via USB adapter ===<br />
<br />
To fix the button mapping of PS2 dual adapters and mimic the Xbox controller you can run the following command:<br />
<br />
# xboxdrv --evdev /dev/input/event* \<br />
--evdev-absmap ABS_X=x1,ABS_Y=y1,ABS_RZ=x2,ABS_Z=y2,ABS_HAT0X=dpad_x,ABS_HAT0Y=dpad_y \<br />
--axismap -Y1=Y1,-Y2=Y2 \<br />
--evdev-keymap BTN_TOP=x,BTN_TRIGGER=y,BTN_THUMB2=a,BTN_THUMB=b,BTN_BASE3=back,BTN_BASE4=start,BTN_BASE=lb,BTN_BASE2=rb,BTN_TOP2=lt,BTN_PINKIE=rt,BTN_BASE5=tl,BTN_BASE6=tr \<br />
--mimic-xpad --silent<br />
<br />
=== PlayStation 3 controller ===<br />
<br />
==== Pairing via USB ====<br />
<br />
If you own a PS3 controller and can connect with USB, plug it to your computer and press the PS button. The controller will power up and one of the four LEDs should light up indicating the controller's number.<br />
<br />
==== Pairing via Bluetooth ====<br />
<br />
Install {{Pkg|bluez}} {{Pkg|bluez-utils}} {{Pkg|bluez-plugins}}. Make sure bluetooth is working by following the first five steps of [[Bluetooth#Pairing]] and leave the bluetoothctl command running, then turn on the controller by pressing the middle 'PS' button(all 4 leds should be blinking quickly ~4 hz) and connect to your computer using usb. Lastly, type yes in the bluetoothctl prompt when asked '{{ic|Authorize service 00001124-0000-1000-8000-00805f9b34fb (yes/no)}}'.<br />
<br />
Alternative instructions:<br />
To connect your PS3 controller to your computer using Bluetooth, you first need to install {{Pkg|bluez}} and {{Pkg|bluez-plugins}} then connect your controller via USB. A pop-up should appear asking for pairing. Click on Trust & Authorize. You can now unplug your controller and press the PS button. The controller will connect and a LED will remain solid. You can now use it to play games. Connecting using the USB cable is only needed after the controller has been connected to another system.<br />
<br />
{{Tip|There are many complicated instructions on the internet on setting up a PS3 controller that require many steps such as compiling and installing qtsixa or sixpair and setting up the controller manually, or patching bluez with some specific patches. None of this is necessary on a modern Linux kernel and after installing bluez-plugins.}}<br />
<br />
=== PlayStation 4 controller ===<br />
<br />
==== Pairing via USB ====<br />
<br />
Connect your controller via USB and press the {{ic|PS}} button.<br />
<br />
==== Pairing via Bluetooth ====<br />
<br />
If you want to use bluetooth mode, press {{ic|PS}} button and {{ic|Share}} button together, white led of gamepad should blink very quickly, then add wireless controller with your bluetooth manager (bluez, gnome-bluetooth.<br />
<br />
==== Button mapping ====<br />
<br />
To fix the button mapping of PS4 controller you can use the following command with xboxdrv (or try with the [https://github.com/chrippa/ds4drv ds4drv] program, {{AUR|ds4drv}}):<br />
<br />
# xboxdrv \<br />
--evdev /dev/input/by-id/usb-Sony_Computer_Entertainment_Wireless_Controller-event-joystick\<br />
--evdev-absmap ABS_X=x1,ABS_Y=y1 \<br />
--evdev-absmap ABS_Z=x2,ABS_RZ=y2 \<br />
--evdev-absmap ABS_HAT0X=dpad_x,ABS_HAT0Y=dpad_y \<br />
--evdev-keymap BTN_A=x,BTN_B=a \<br />
--evdev-keymap BTN_C=b,BTN_X=y \<br />
--evdev-keymap BTN_Y=lb,BTN_Z=rb \<br />
--evdev-keymap BTN_TL=lt,BTN_TR=rt \<br />
--evdev-keymap BTN_SELECT=tl,BTN_START=tr \<br />
--evdev-keymap BTN_TL2=back,BTN_TR2=start \<br />
--evdev-keymap BTN_MODE=guide \<br />
--axismap -y1=y1,-y2=y2 \<br />
--mimic-xpad \<br />
--silent<br />
<br />
==== Fix Motion control conflict (gamepad will not work on some applications) ====<br />
<br />
Dualshock 4 V1 and V2 are both like 3 devices, touchpad, motion control, and joypad.<br />
<br />
With somes softwares like Parsec and Shadow cloud gaming streaming apps, motion control is in conflict with joypad, you can disable touchpad and motion control by adding the following udev rule:<br />
<br />
{{hc|1=/etc/udev/rules.d/51-disable-DS3-and-DS4-motion-controls.rules|2=<br />
SUBSYSTEM=="input", ATTRS{name}=="*Controller Motion Sensors", RUN+="/bin/rm %E{DEVNAME}", ENV{ID_INPUT_JOYSTICK}=""<br />
SUBSYSTEM=="input", ATTRS{name}=="*Controller Touchpad", RUN+="/bin/rm %E{DEVNAME}", ENV{ID_INPUT_JOYSTICK}=""<br />
}}<br />
<br />
This should work in USB and Bluetooth mode.<br />
<br />
==== Disable touchpad acting as mouse ====<br />
<br />
This fixes conflicts with games that actually use touchpad as part of the gamepad, such as Rise of the Tomb Raider.<br />
<br />
Edit {{ic|/etc/X11/xorg.conf.d/30ds4.conf}}.<br />
<br />
And then paste the following and restart X11:<br />
Section "InputClass"<br />
Identifier "ds4-touchpad"<br />
Driver "libinput"<br />
MatchProduct "Wireless Controller Touchpad"<br />
Option "Ignore" "True"<br />
EndSection<br />
<br />
=== Playstation 5 (Dualsense) controller ===<br />
<br />
Configure button mapping (thanks to [https://github.com/yoyossef/ds360 yoyossef]):<br />
xboxdrv \<br />
--evdev /dev/input/by-id/usb-Sony_Interactive_Entertainment_Wireless_Controller-if03-event-joystick \<br />
--evdev-absmap ABS_HAT0X=dpad_x,ABS_HAT0Y=dpad_y,ABS_X=X1,ABS_Y=Y1,ABS_RX=X2,ABS_RY=Y2,ABS_Z=LT,ABS_RZ=RT \<br />
--evdev-keymap BTN_SOUTH=A,BTN_EAST=B,BTN_NORTH=Y,BTN_WEST=X,BTN_START=start,BTN_MODE=guide,BTN_SELECT=back \<br />
--evdev-keymap BTN_TL=LB,BTN_TR=RB,BTN_TL2=LT,BTN_TR2=RT,BTN_THUMBL=TL,BTN_THUMBR=TR \<br />
--axismap -y1=y1,-y2=y2 \<br />
--mimic-xpad \<br />
--silent<br />
<br />
Some apps, for example, Steam inside Geforce NOW inside web browser, may be confused with original joystick events, which shadow the newly created event source.<br />
Simply deletion of /dev/input/js0 works this around.<br />
<br />
The playstation and mode buttons still don't work, however.<br />
<br />
=== PlayStation 3/4 controller ===<br />
<br />
The DualShock 3, DualShock 4 and Sixaxis controllers work out of the box when plugged in via USB (the PS button will need to be pushed to begin). They can also be used wirelessly via Bluetooth.<br />
<br />
Steam properly recognizes it as a PS3 pad and Big Picture can be launched with the PS button. Big Picture and some games may act as if it was a 360 controller. Gamepad control over mouse is on by default. You may want to turn it off before playing games, see [[#Joystick moving mouse]].<br />
<br />
==== Connecting via Bluetooth ====<br />
<br />
Install the {{Pkg|bluez}}, {{Pkg|bluez-plugins}}, and {{Pkg|bluez-utils}} packages, which includes the ''sixaxis'' plugin. Then [[start]] the [[bluetooth]] service and ensure bluetooth is powered on. If using ''bluetoothctl'' start it in a terminal and then plug the controller in via USB. You should be prompted to trust the controller in bluetoothctl. A graphical bluetooth front-end may program your PC's bluetooth address into the controller automatically. Hit the PlayStation button and check that the controller works while plugged in.<br />
<br />
You can now disconnect your controller. The next time you hit the PlayStation button it will connect without asking anything else.<br />
<br />
Alternatively, on a PS4 controller you can hold the share button and the PlayStation button simultaneously (for a few seconds) to put the gamepad in pairing mode, and pair as you would normally.<br />
<br />
GNOME's Settings also provides a graphical interface to pair sixaxis controllers when connected by wire.<br />
<br />
Remember to disconnect the controller when you are done as the controller will stay on when connected and drain the battery.<br />
<br />
{{Note|If the controller does not connect, make sure the bluetooth interface is turned on and the controllers have been trusted. (See [[Bluetooth]])}}<br />
<br />
==== Using Playstation 3 controllers with Steam ====<br />
<br />
{{out of date|Tested connecting PS3 controller via bluetooth and it worked in Steam version 1575605714 and some Steam games without performing the following steps.}}<br />
<br />
For Steam to recognize your controllers, it needs to be able to read their device file. The {{Pkg|steam}} package sets up udev rules for lots of controllers, but not the PlayStation 3 controller (aka. DualShock 3 controller). You can add the rules yourself:<br />
<br />
{{hc<br />
|/etc/udev/rules.d/99-dualshock-3.rules|2=<br />
# DualShock 3 controller, Bluetooth<br />
KERNEL=="hidraw*", KERNELS=="*054C:0268*", MODE="0660", TAG+="uaccess"<br />
<br />
# DualShock 3 controller, USB<br />
KERNEL=="hidraw*", ATTRS{idVendor}=="054c", ATTRS{idProduct}=="0268", MODE="0660", TAG+="uaccess"<br />
}}<br />
<br />
Make sure your user is in the "input" group:<br />
<br />
{{ic | usermod -aG input yourusername}}<br />
<br />
Reboot your system. Steam should now detect your PlayStation 3 controller.<br />
<br />
==== Using generic/clone controllers ====<br />
<br />
Using generic/clone Dualshock controllers is possible, however there is an issue that may require to install a patched package. The default Bluetooth protocol stack does not detect some of the clone controllers. The {{AUR|bluez-ps3}} package is a version patched to be able to detect them.<br />
<br />
== Tips and tricks ==<br />
<br />
=== Gamepad over network ===<br />
<br />
If you want to use your gamepad with another computer over a network, you can use [https://github.com/moslevin/netstick netstick] to easily do this.<br />
<br />
== Troubleshooting ==<br />
<br />
=== Joystick moving mouse ===<br />
<br />
Sometimes USB gamepad can be recognized as HID mouse (only in X, it is still being installed as {{ic|/dev/input/js0}} as well). Known issue is cursor being moved by the joystick, or escaping to en edge of a screen right after plugin. If your application can detect gamepad by itself, you can remove the {{AUR|xf86-input-joystick}} package.<br />
<br />
A more gentle solution is described in [[#Disable joystick from controlling mouse]].<br />
<br />
=== Gamepad is not working in FNA/SDL based games ===<br />
<br />
If you are using a generic non-widely used gamepad you may encounter issues getting the gamepad recognized in games based on SDL. Since [https://github.com/flibitijibibo/FNA/commit/e55742cfe7e38b778a21ed8a12cb2f2081490d8d 14 May 2015], FNA supports dropping a {{ic|gamecontrollerdb.txt}} into the executable folder of the game, for example the [https://github.com/gabomdq/SDL_GameControllerDB SDL_GameControllerDB].<br />
<br />
As an alternative and for older versions of FNA or for SDL you can generate a mapping yourself by downloading the SDL source code via https://libsdl.org/, navigating to {{ic|/test/}}, compile the {{ic|controllermap.c}} program (alternatively install {{AUR|controllermap}}) and run the test. After completing the controllermap test, a guid will be generated that you can put in the {{ic|SDL_GAMECONTROLLERCONFIG}} environment variable which will then be picked up by SDL/FNA games. For example:<br />
<br />
$ export SDL_GAMECONTROLLERCONFIG="030000008f0e00000300000010010000,GreenAsia Inc. USB Joystick ,platform:Linux,x:b3,a:b2,b:b1,y:b0,back:b8,start:b9,dpleft:h0.8,dpdown:h0.0,dpdown:h0.4,dpright:h0.0,dpright:h0.2,dpup:h0.0,dpup:h0.1,leftshoulder:h0.0,leftshoulder:b6,lefttrigger:b4,rightshoulder:b7,righttrigger:b5,leftstick:b10,rightstick:b11,leftx:a0,lefty:a1,rightx:a3,righty:a2,"<br />
<br />
=== Gamepad is not recognized by all programs ===<br />
<br />
Some software, Steam for example, will only recognize the first gamepad it encounters. Due to a bug in the driver for Microsoft wireless periphery devices this can in fact be the bluetooth dongle. If you find you have a {{ic|/dev/input/js*}} and {{ic|/dev/input/event*}} belonging to you keyboard's bluetooth transceiver you can get automatically get rid of it by creating according udev rules.<br />
Create a {{ic|/}}:<br />
{{hc|/etc/udev/rules.d/99-btcleanup.rules|<nowiki><br />
ACTION=="add", KERNEL=="js[0-9]*", SUBSYSTEM=="input", KERNELS=="...", ATTRS{bInterfaceSubClass}=="00", ATTRS{bInterfaceProtocol}=="00", ATTRS{bInterfaceNumber}=="02", RUN+="/usr/bin/rm /dev/input/js%n"<br />
ACTION=="add", KERNEL=="event*", SUBSYSTEM=="input", KERNELS=="...", ATTRS{bInterfaceSubClass}=="00", ATTRS{bInterfaceProtocol}=="00", ATTRS{bInterfaceNumber}=="02", RUN+="/usr/bin/rm /dev/input/event%n"<br />
</nowiki>}}<br />
Correct the {{ic|<nowiki>KERNELS=="..."</nowiki>}} to match your device. The correct value can be found by running<br />
<br />
# udevadm info -an /dev/input/js0<br />
<br />
Assuming the device in question is {{ic|/dev/input/js0}}. After you placed the rule reload the rules with<br />
<br />
# udevadm control --reload<br />
<br />
Then replug the device making you trouble. The joystick and event devices should be gone, although their number will still be reserved. But the files are out of the way.<br />
<br />
=== Steam Controller not pairing ===<br />
<br />
There are some unknown cases where the packaged udev rule for the Steam controller does not work ({{bug|47330}}). The most reliable workaround is to make the controller world readable. Copy the rule {{ic|/usr/lib/udev/rules.d/70-steam-controller.rules}} to {{ic|/etc/udev/rules.d}} with a later prioritiy and change anything that says {{ic|1=MODE="0660"}} to {{ic|1=MODE="066'''6'''"}} e.g.<br />
<br />
{{hc|/etc/udev/rules.d/99-steam-controller-perms.rules|<nowiki><br />
...<br />
SUBSYSTEM=="usb", ATTRS{idVendor}=="28de", MODE="0666"<br />
...<br />
</nowiki>}}<br />
<br />
You may have to reboot in order for the change to take effect.<br />
<br />
=== Steam Controller makes a game crash or not recognized ===<br />
<br />
If your Steam Controller is working well in Steam Big Picture mode, but not recognized by a game or the game starts crashing when you plug in the controller, this may be because of the native driver that has been added to the Linux kernel 4.18. Try to unload it, restart Steam and replug the controller.<br />
<br />
The module name of the driver is ''hid_steam'', so to unload it you may perform:<br />
# rmmod hid_steam</div>Phireskyhttps://wiki.archlinux.org/index.php?title=Dm-crypt/Device_encryption&diff=674309Dm-crypt/Device encryption2021-05-29T13:53:39Z<p>Phiresky: /* Re-encrypting devices */ tip</p>
<hr />
<div>{{Lowercase title}}<br />
[[Category:Data-at-rest encryption]]<br />
[[es:Dm-crypt (Español)/Device encryption]]<br />
[[ja:Dm-crypt/デバイスの暗号化]]<br />
[[pl:Dm-crypt (Polski)/Device encryption]]<br />
[[pt:Dm-crypt (Português)/Device encryption]]<br />
This section covers how to manually utilize ''dm-crypt'' from the command line to encrypt a system.<br />
<br />
== Preparation ==<br />
<br />
Before using {{pkg|cryptsetup}}, always make sure the {{ic|dm_crypt}} [[kernel module]] is loaded.<br />
<br />
== Cryptsetup usage ==<br />
<br />
''Cryptsetup'' is the command line tool to interface with ''dm-crypt'' for creating, accessing and managing encrypted devices. The tool was later expanded to support different encryption types that rely on the Linux kernel '''d'''evice-'''m'''apper and the '''crypt'''ographic modules. The most notable expansion was for the Linux Unified Key Setup (LUKS) extension, which stores all of the needed setup information for dm-crypt on the disk itself and abstracts partition and key management in an attempt to improve ease of use. Devices accessed via the device-mapper are called blockdevices. For further information see [[Data-at-rest encryption#Block device encryption]]. <br />
<br />
The tool is used as follows: <br />
<br />
# cryptsetup <OPTIONS> <action> <action-specific-options> <device> <dmname><br />
<br />
It has compiled-in defaults for the options and the encryption mode, which will be used if no others are specified on the command line. Have a look at <br />
<br />
$ cryptsetup --help <br />
<br />
which lists options, actions and the default parameters for the encryption modes in that order. A full list of options can be found on the man page.<br />
Since different parameters are required or optional, depending on encryption mode and action, the following sections point out differences further. Blockdevice encryption is fast, but speed matters a lot too. Since changing an encryption cipher of a blockdevice after setup is difficult, it is important to check ''dm-crypt'' performance for the individual parameters in advance: <br />
<br />
$ cryptsetup benchmark <br />
<br />
can give guidance on deciding for an algorithm and key-size prior to installation. If certain AES ciphers excel with a considerable higher throughput, these are probably the ones with hardware support in the CPU.<br />
<br />
{{Tip|You may want to practise encrypting a virtual hard drive in a [[virtual machine]] when learning.}}<br />
<br />
=== Cryptsetup passphrases and keys ===<br />
<br />
An encrypted blockdevice is protected by a key. A key is either: <br />
<br />
* a passphrase: see [[Security#Passwords]].<br />
* a keyfile, see [[#Keyfiles]].<br />
<br />
Both key types have default maximum sizes: passphrases can be up to 512 characters and keyfiles up to 8192kiB. <br />
<br />
An important distinction of ''LUKS'' to note at this point is that the key is used to unlock the master-key of a LUKS-encrypted device and can be changed with root access. Other encryption modes do not support changing the key after setup, because they do not employ a master-key for the encryption. See [[Data-at-rest encryption#Block device encryption]] for details.<br />
<br />
== Encryption options with dm-crypt ==<br />
<br />
''Cryptsetup'' supports different encryption operating modes to use with ''dm-crypt'': <br />
<br />
* {{ic|--type luks}} for using the default LUKS format version (LUKS1 with {{Pkg|cryptsetup}} < 2.1.0, LUKS2 with {{Pkg|cryptsetup}} ≥ 2.1.0),<br />
* {{ic|--type luks1}} for using LUKS1, the most common version of LUKS,<br />
* {{ic|--type luks2}} for using LUKS2, the latest available version of LUKS that allows additional extensions,<br />
* {{ic|--type plain}} for using dm-crypt plain mode, <br />
* {{ic|--type loopaes}} for a loopaes legacy mode, <br />
* {{ic|--type tcrypt}} for a [[TrueCrypt]] compatibility mode.<br />
<br />
The basic cryptographic options for encryption cipher and hashes available can be used for all modes and rely on the kernel cryptographic backend features. All that are loaded and available to use as options at runtime can be viewed with:<br />
<br />
$ less /proc/crypto <br />
<br />
{{Tip|If the list is short, execute {{ic|$ cryptsetup benchmark}} which will trigger loading available modules.}}<br />
<br />
The following introduces encryption options for the {{ic|luks}}, {{ic|luks1}}, {{ic|luks2}} and {{ic|plain}} modes. Note that the tables list options used in the respective examples in this article and not all available ones.<br />
<br />
=== Encryption options for LUKS mode ===<br />
<br />
The ''cryptsetup'' action to set up a new dm-crypt device in LUKS encryption mode is {{ic|luksFormat}}. Unlike what the name implies, it does not format the device, but sets up the LUKS device header and encrypts the master-key with the desired cryptographic options. <br />
<br />
In order to create a new LUKS container with the compiled-in defaults listed by {{ic|cryptsetup --help}}, simply execute:<br />
<br />
# cryptsetup luksFormat ''device''<br />
<br />
As of cryptsetup 2.3.4, this is equivalent to: <br />
<br />
# cryptsetup --type luks2 --cipher aes-xts-plain64 --hash sha256 --iter-time 2000 --key-size 256 --pbkdf argon2i --sector-size 512 --use-urandom --verify-passphrase luksFormat ''device''<br />
<br />
Defaults are compared with a cryptographically higher specification example in the table below, with accompanying comments: <br />
<br />
{| class="wikitable"<br />
! Options !! Cryptsetup 2.1.0 defaults !! Example !! Comment<br />
|-<br />
! style="text-align:right;white-space:nowrap;" | --cipher<br />
-c<br />
| {{ic|aes-xts-plain64}}<br />
| {{ic|aes-xts-plain64}}<br />
| [https://www.kernel.org/pub/linux/utils/cryptsetup/v1.6/v1.6.0-ReleaseNotes Release 1.6.0] changed the defaults to an AES [[Data-at-rest encryption#Ciphers and modes of operation|cipher]] in [[wikipedia:Disk encryption theory#XEX-based tweaked-codebook mode with ciphertext stealing (XTS)|XTS]] mode (see item 5.16 [https://gitlab.com/cryptsetup/cryptsetup/wikis/FrequentlyAskedQuestions#5-security-aspects of the FAQ]). It is advised against using the previous default {{ic|--cipher aes-cbc-essiv}} because of its known [[wikipedia:Disk encryption theory#Cipher-block chaining (CBC)|issues]] and practical [https://www.jakoblell.com/blog/2013/12/22/practical-malleability-attack-against-cbc-encrypted-luks-partitions/ attacks] against them.<br />
|-<br />
! style="text-align:right;white-space:nowrap;" | --key-size<br />
-s<br />
| {{ic|256}} ({{ic|512}} for XTS)<br />
| {{ic|512}}<br />
| By default a 512 bit key-size is used for XTS ciphers. Note however that [[wikipedia:Disk encryption theory#XEX-based tweaked-codebook mode with ciphertext stealing (XTS)|XTS splits the supplied key in half]], so this results in AES-256 being used.<br />
|-<br />
! style="text-align:right;white-space:nowrap;" | --hash<br />
-h<br />
| {{ic|sha256}}<br />
| {{ic|sha512}}<br />
| Hash algorithm used for [[Data-at-rest encryption#Cryptographic metadata|key derivation]]. Release 1.7.0 changed defaults from {{ic|sha1}} to {{ic|sha256}} "''not for security reasons [but] mainly to prevent compatibility problems on hardened systems where SHA1 is already [being] phased out''"[https://www.kernel.org/pub/linux/utils/cryptsetup/v1.7/v1.7.0-ReleaseNotes]. The former default of {{ic|sha1}} can still be used for compatibility with older versions of ''cryptsetup'' since it is [https://gitlab.com/cryptsetup/cryptsetup/wikis/FrequentlyAskedQuestions#5-security-aspects considered secure] (see item 5.20). <br />
|-<br />
! style="text-align:right;white-space:nowrap;" | --iter-time<br />
-i<br />
| {{ic|2000}}<br />
| {{ic|5000}}<br />
| Number of milliseconds to spend with PBKDF2 passphrase processing. Release 1.7.0 changed defaults from {{ic|1000}} to {{ic|2000}} to "''try to keep PBKDF2 iteration count still high enough and also still acceptable for users.''"[https://www.kernel.org/pub/linux/utils/cryptsetup/v1.7/v1.7.0-ReleaseNotes]. This option is only relevant for LUKS operations that set or change passphrases, such as {{ic|luksFormat}} or {{ic|luksAddKey}}. Specifying 0 as parameter selects the compiled-in default..<br />
|-<br />
! style="text-align:right;white-space:nowrap;" | --use-urandom<br />
| {{ic|--use-urandom}}<br />
| {{ic|--use-random}}<br />
| Selects which [[random number generator]] to use. Note that [https://lwn.net/Articles/808575/ /dev/random blocking pool has been removed]. Therefore, {{ic|--use-random}} flag is now equivalent to {{ic|--use-urandom}}.<br />
|-<br />
! style="text-align:right;white-space:nowrap;" | --verify-passphrase<br />
-y<br />
| Yes<br />
| -<br />
| Enabled by default in Arch Linux for {{ic|luksFormat}} and {{ic|luksAddKey}}.<br />
|}<br />
<br />
The properties of LUKS features and options are described in the [https://gitlab.com/cryptsetup/cryptsetup/wikis/Specification LUKS1] (pdf) and [https://gitlab.com/cryptsetup/cryptsetup/blob/master/docs/on-disk-format-luks2.pdf LUKS2] (pdf) specifications.<br />
<br />
{{Tip|The project developers' [https://mbroz.fedorapeople.org/talks/DevConf2016/devconf2016-luks2.pdf devconfcz2016] (pdf) presentation summarizes the motivation for the major specification update to LUKS2.}}<br />
<br />
==== Choice of argon2 variant for PBKDF ====<br />
<br />
Although argon2id is in [https://crypto.stackexchange.com/questions/48935/why-use-argon2i-or-argon2d-if-argon2id-exists#49969 most cases the preferred variant] of the [[Wikipedia:Argon2|Argon2 family]], the [https://tools.ietf.org/html/draft-irtf-cfrg-argon2-12 quoted RFC] does indicate that:<br />
<br />
:''Argon2i uses data-independent memory access, '''which is preferred for password hashing and password-based key derivation'''.''<br />
<br />
We can also see this reflected in this [https://www.saout.de/pipermail/dm-crypt/2018-September/005968.html mailing list answer] by cryptsetup maintainer Milan Broz:<br />
<br />
:''You can use Argon2id though (combination of data dependent and independent processing). I prefer Argon2i for key derivation, but opinions differ here.''<br />
<br />
Therefore it is recommended to keep the default as is so as not to inadvertently decrease security.<br />
<br />
==== Iteration time ====<br />
<br />
From [https://gitlab.com/cryptsetup/cryptsetup/-/wikis/FrequentlyAskedQuestions#2-setup cryptsetup FAQ§2.1] and [https://gitlab.com/cryptsetup/cryptsetup/-/wikis/FrequentlyAskedQuestions#3-common-problems §3.4]:<br />
<br />
:The unlock time for a key-slot [...] is calculated when setting a passphrase. By default it is 1 second (2 seconds for LUKS2). [...]<br />
:Passphrase iteration count is based on time and hence security level depends on CPU power of the system the LUKS container is created on. [...]<br />
:If you set a passphrase on a fast machine and then unlock it on a slow machine, the unlocking time can be much longer.<br />
<br />
As such, it is better to always create a container on the machine where it will be most often accessed.<br />
<br />
Read the rest of those sections for advice on how to correctly adjust the iteration count should the need arise.<br />
<br />
==== Sector size ====<br />
<br />
{{Warning|The filesystem to be used inside the LUKS container must also be formatted with the same sector size.}}<br />
For drives with [[Advanced Format]] (also known as ''AF'', ''4K'', ''4096 byte sector'' drives) sectors it is recommended to use the appropriate physical sector size. In particular ''shingled magnetic recording'' (SMR) drives that are firmware-managed are negatively impacted if using a logical sector size of 512 bytes if their physical sector size is of 4096 bytes.<br />
<br />
{{man|8|cryptsetup|OPTIONS}} description for {{ic|--sector-size}} indicates that: "''Increasing sector size from 512 bytes to 4096 bytes can provide better performance on most of the modern storage devices and also with some hw encryption accelerators''" (see the rest of the description for caveats to be aware of).<br />
<br />
In a comment from 2019[https://lwn.net/Articles/777071/], the following reason is given for why the default was not switched over to 4K sector:<br />
<br />
:# ''Compatibility with old kernels and cryptsetup versions. The 4K encryption sector support is still fairly new, after all.''<br />
:# ''It’s not guaranteed safe on disks with 512-byte sectors, as it can break atomicity guarantees that might be assumed by software. I don’t believe this is a problem on modern disks or flash storage, nor on ext4 or f2fs. But the cryptsetup default needs to be more conservative.''<br />
<br />
To create a LUKS2 container with a 4K sector size and otherwise default options:<br />
<br />
# cryptsetup luksFormat --sector-size 4096 ''device''<br />
<br />
The command will abort on an error if the requested size does not match your device:<br />
<br />
{{bc|# cryptsetup luksFormat --sector-size 4096 ''device''<br />
(...)<br />
Verify passphrase: <br />
Device size is not aligned to requested sector size.<br />
}}<br />
<br />
{{Note|See [https://gitlab.com/cryptsetup/cryptsetup/-/issues/585] for why the command may fail while the underlying drive does use 4K physical sectors.}}<br />
<br />
Afterwards format the mapped container using the same sector size, for example with [[btrfs]]: <br />
<br />
# mkfs.btrfs -s 4096 /dev/mapper/''mapped_device''<br />
<br />
=== Encryption options for plain mode ===<br />
<br />
In dm-crypt ''plain'' mode, there is no master-key on the device, hence, there is no need to set it up. Instead the encryption options to be employed are used directly to create the mapping between an encrypted disk and a named device. The mapping can be created against a partition or a full device. In the latter case not even a partition table is needed. <br />
<br />
To create a ''plain'' mode mapping with cryptsetup's default parameters: <br />
<br />
# cryptsetup <options> open --type plain <device> <dmname><br />
<br />
Executing it will prompt for a password, which should have very high entropy. <br />
Below a comparison of default parameters with the example in [[dm-crypt/Encrypting an entire system#Plain dm-crypt]]<br />
<br />
{| class="wikitable"<br />
! Option !! Cryptsetup 2.1.0 defaults !! Example !! Comment<br />
|-<br />
! style="text-align:right;white-space:nowrap;" | --hash<br />
-h<br />
| {{ic|ripemd160}}<br />
| -<br />
| The hash is used to create the key from the passphrase; it is not used on a keyfile. <br />
|-<br />
! style="text-align:right;white-space:nowrap;" | --cipher<br />
-c<br />
| {{ic|aes-cbc-essiv:sha256}}<br />
| {{ic|aes-xts-plain64}}<br />
| The cipher consists of three parts: cipher-chainmode-IV generator. Please see [[Data-at-rest encryption#Ciphers and modes of operation]] for an explanation of these settings, and the [https://gitlab.com/cryptsetup/cryptsetup/wikis/DMCrypt DMCrypt documentation] for some of the options available. <br />
|-<br />
! style="text-align:right;white-space:nowrap;" | --key-size<br />
-s<br />
|{{ic|256}}<br />
|{{ic|512}}<br />
|The key size (in bits). The size will depend on the cipher being used and also the chainmode in use. Xts mode requires twice the key size of cbc. <br />
|-<br />
! style="text-align:right;white-space:nowrap;" | --size<br />
-b<br />
|real size of target disk<br />
|{{ic|2048}} (mapped device will be 512B×2048=1MiB)<br />
|Limit the maximum size of the device (in 512-byte sectors).<br />
|-<br />
! style="text-align:right;white-space:nowrap;" | --offset<br />
-o<br />
|{{ic|0}}<br />
|{{ic|0}}<br />
|The offset from the beginning of the target disk (in 512-byte sectors) from which to start the mapping.<br />
|-<br />
! style="text-align:right;white-space:nowrap;" | --skip<br />
-p<br />
|{{ic|0}}<br />
|{{ic|2048}} (512B×2048=1MiB will be skipped)<br />
|The number of 512-byte sectors of encrypted data to skip at the beginning.<br />
|-<br />
! style="text-align:right;white-space:nowrap;" | --key-file<br />
-d<br />
| default uses a passphrase<br />
| {{ic|/dev/sd''Z''}} (or e.g. {{ic|/boot/keyfile.enc}})<br />
|The device or file to be used as a key. See [[#Keyfiles]] for further details.<br />
|-<br />
! style="text-align:right;white-space:nowrap;" | --keyfile-offset<br />
| {{ic|0}}<br />
| {{ic|0}}<br />
| Offset from the beginning of the file where the key starts (in bytes). This option is supported from ''cryptsetup'' 1.6.7 onwards. <br />
|-<br />
! style="text-align:right;white-space:nowrap;" | --keyfile-size<br />
-l<br />
|{{ic|8192kB}}<br />
| - (default applies)<br />
| Limits the bytes read from the key file. This option is supported from ''cryptsetup'' 1.6.7 onwards. <br />
|}<br />
<br />
Using the device {{ic|/dev/sd''X''}}, the above right column example results in:<br />
<br />
# cryptsetup --cipher=aes-xts-plain64 --offset=0 --key-file=/dev/sd''Z'' --key-size=512 open --type=plain /dev/sdX enc<br />
<br />
Unlike encrypting with LUKS, the above command must be executed ''in full'' whenever the mapping needs to be re-established, so it is important to remember the cipher, hash and key file details.<br />
We can now check that the mapping has been made:<br />
<br />
# fdisk -l<br />
<br />
An entry should now exist for {{ic|/dev/mapper/enc}}.<br />
<br />
== Encrypting devices with cryptsetup ==<br />
<br />
This section shows how to employ the options for creating new encrypted blockdevices and accessing them manually. <br />
<br />
{{Warning|GRUB does not support LUKS2 headers; see [https://savannah.gnu.org/bugs/?55093 GRUB bug #55093]. Therefore, if you plan to [[GRUB#Encrypted /boot|unlock an encrypted boot partition with GRUB]], specify {{ic|--type luks1}} on encrypted devices that GRUB will need to access.}}<br />
<br />
=== Encrypting devices with LUKS mode ===<br />
<br />
==== Formatting LUKS partitions ====<br />
<br />
In order to setup a partition as an encrypted LUKS partition execute:<br />
<br />
# cryptsetup luksFormat ''device''<br />
<br />
You will then be prompted to enter a password and verify it.<br />
<br />
See [[#Encryption options for LUKS mode]] for command line options.<br />
<br />
You can check the results with:<br />
<br />
# cryptsetup luksDump ''device''<br />
<br />
You will note that the dump not only shows the cipher header information, but also the key-slots in use for the LUKS partition. <br />
<br />
The following example will create an encrypted root partition on {{ic|/dev/sda1}} using the default AES cipher in XTS mode with an effective 256-bit encryption <br />
<br />
# cryptsetup -s 512 luksFormat /dev/sda1<br />
<br />
===== Using LUKS to format partitions with a keyfile =====<br />
<br />
When creating a new LUKS encrypted partition, a keyfile may be associated with the partition on its creation using:<br />
<br />
# cryptsetup luksFormat ''device'' ''/path/to/mykeyfile''<br />
<br />
See [[#Keyfiles]] for instructions on how to generate and manage keyfiles.<br />
<br />
==== Unlocking/Mapping LUKS partitions with the device mapper ====<br />
<br />
Once the LUKS partitions have been created, they can then be unlocked.<br />
<br />
The unlocking process will map the partitions to a new device name using the device mapper. This alerts the kernel that {{ic|''device''}} is actually an encrypted device and should be addressed through LUKS using the {{ic|/dev/mapper/''dm_name''}} so as not to overwrite the encrypted data. To guard against accidental overwriting, read about the possibilities to [[#Backup and restore|backup the cryptheader]] after finishing setup.<br />
<br />
In order to open an encrypted LUKS partition execute:<br />
<br />
# cryptsetup open ''device'' ''dm_name''<br />
<br />
You will then be prompted for the password to unlock the partition. Usually the device mapped name is descriptive of the function of the partition that is mapped. For example the following unlocks a luks partition {{ic|/dev/sda1}} and maps it to device mapper named {{ic|cryptroot}}:<br />
<br />
# cryptsetup open /dev/sda1 cryptroot <br />
<br />
Once opened, the root partition device address would be {{ic|/dev/mapper/cryptroot}} instead of the partition (e.g. {{ic|/dev/sda1}}). <br />
<br />
For setting up LVM ontop the encryption layer the device file for the decrypted volume group would be anything like {{ic|/dev/mapper/cryptroot}} instead of {{ic|/dev/sda1}}. LVM will then give additional names to all logical volumes created, e.g. {{ic|/dev/lvmpool/root}} and {{ic|/dev/lvmpool/swap}}.<br />
<br />
In order to write encrypted data into the partition it must be accessed through the device mapped name. The first step of access will typically be to [[File systems#Create a file system|create a filesystem]]. For example:<br />
<br />
# mkfs -t ext4 /dev/mapper/cryptroot<br />
<br />
The device {{ic|/dev/mapper/cryptroot}} can then be [[mount]]ed like any other partition.<br />
<br />
To close the LUKS container, unmount the partition and do:<br />
<br />
# cryptsetup close cryptroot<br />
<br />
==== Using a TPM to store keys ====<br />
<br />
See [[Trusted Platform Module#Data-at-rest encryption with LUKS]].<br />
<br />
=== Encrypting devices with plain mode ===<br />
<br />
The creation and subsequent access of a ''dm-crypt'' plain mode encryption both require not more than using the ''cryptsetup'' {{ic|open}} action with correct [[#Encryption options for plain mode|parameters]]. The following shows that with two examples of non-root devices, but adds a quirk by stacking both (i.e. the second is created inside the first). Obviously, stacking the encryption doubles overhead. The usecase here is simply to illustrate another example of the cipher option usage. <br />
<br />
A first mapper is created with ''cryptsetup's'' plain-mode defaults, as described in the table's left column above <br />
<br />
{{hc|# cryptsetup --type plain -v open /dev/sdaX plain1|<br />
Enter passphrase: <br />
Command successful.<br />
}}<br />
<br />
Now we add the second blockdevice inside it, using different encryption parameters and with an (optional) offset, create a filesystem and mount it <br />
<br />
{{hc|1=# cryptsetup --type plain --cipher=serpent-xts-plain64 --hash=sha256 --key-size=256 --offset=10 open /dev/mapper/plain1 plain2|2=<br />
Enter passphrase:<br />
}}<br />
<br />
{{hc|# lsblk -p|<br />
NAME <br />
/dev/sda <br />
├─/dev/sdaX <br />
│ └─/dev/mapper/plain1 <br />
│ └─/dev/mapper/plain2 <br />
...<br />
}}<br />
<br />
# mkfs -t ext2 /dev/mapper/plain2<br />
# mount -t ext2 /dev/mapper/plain2 /mnt<br />
# echo "This is stacked. one passphrase per foot to shoot." > /mnt/stacked.txt<br />
<br />
We close the stack to check access works<br />
<br />
# cryptsetup close plain2<br />
# cryptsetup close plain1<br />
<br />
First, let us try to open the filesystem directly: <br />
<br />
# cryptsetup --type plain --cipher=serpent-xts-plain64 --hash=sha256 --key-size=256 --offset=10 open /dev/sdaX plain2<br />
<br />
{{hc|# mount -t ext2 /dev/mapper/plain2 /mnt|<br />
mount: wrong fs type, bad option, bad superblock on /dev/mapper/plain2,<br />
missing codepage or helper program, or other error<br />
}}<br />
<br />
Why that did not work? Because the "plain2" starting block ({{ic|10}}) is still encrypted with the cipher from "plain1". It can only be accessed via the stacked mapper. The error is arbitrary though, trying a wrong passphrase or wrong options will yield the same. For ''dm-crypt'' plain mode, the {{ic|open}} action will not error out itself. <br />
<br />
Trying again in correct order: <br />
<br />
# cryptsetup close plain2 # dysfunctional mapper from previous try<br />
<br />
{{hc|# cryptsetup --type plain open /dev/sdaX plain1|<br />
Enter passphrase:<br />
}}<br />
<br />
{{hc|1=# cryptsetup --type plain --cipher=serpent-xts-plain64 --hash=sha256 --key-size=256 --offset=10 open /dev/mapper/plain1 plain2|2=<br />
Enter passphrase:<br />
}}<br />
<br />
{{hc|# mount /dev/mapper/plain2 /mnt && cat /mnt/stacked.txt|<br />
This is stacked. one passphrase per foot to shoot.<br />
}}<br />
<br />
''dm-crypt'' will handle stacked encryption with some mixed modes too. For example LUKS mode could be stacked on the "plain1" mapper. Its header would then be encrypted inside "plain1" when that is closed.<br />
<br />
Available for plain mode only is the option {{ic|--shared}}. With it a single device can be segmented into different non-overlapping mappers. We do that in the next example, using a ''loopaes'' compatible cipher mode for "plain2" this time: <br />
<br />
{{hc|# cryptsetup --type plain --offset 0 --size 1000 open /dev/sdaX plain1|2=<br />
Enter passphrase:<br />
}}<br />
<br />
{{hc|1=# cryptsetup --type plain --offset 1000 --size 1000 --shared --cipher=aes-cbc-lmk --hash=sha256 open /dev/sdaX plain2|2=<br />
Enter passphrase:<br />
}}<br />
<br />
{{hc|# lsblk -p|<br />
NAME <br />
dev/sdaX <br />
├─/dev/sdaX <br />
│ ├─/dev/mapper/plain1 <br />
│ └─/dev/mapper/plain2 <br />
...<br />
}}<br />
<br />
As the devicetree shows both reside on the same level, i.e. are not stacked and "plain2" can be opened individually.<br />
<br />
== Cryptsetup actions specific for LUKS ==<br />
<br />
=== Key management ===<br />
<br />
It is possible to define addition keys for the LUKS partition. This enables the user to create access keys for safe backup storage In so-called key escrow, one key is used for daily usage, another kept in escrow to gain access to the partition in case the daily passphrase is forgotten or a keyfile is lost/damaged. A different key-slot could also be used to grant access to a partition to a user by issuing a second key and later revoking it again. <br />
<br />
Once an encrypted partition has been created, the initial keyslot 0 is created (if no other was specified manually). Additional keyslots are numbered from 1 to 7. Which keyslots are used can be seen by issuing <br />
<br />
# cryptsetup luksDump /dev/<device><br />
<br />
Where <device> is the volume containing the LUKS header. This and all the following commands in this section work on header backup files as well.<br />
<br />
==== Adding LUKS keys ====<br />
<br />
Adding new keyslots is accomplished using cryptsetup with the {{ic|luksAddKey}} action. For safety it will always, i.e. also for already unlocked devices, ask for a valid existing key ("any passphrase") before a new one may be entered:<br />
<br />
{{hc|# cryptsetup luksAddKey /dev/<device> (/path/to/<additionalkeyfile>)|<br />
Enter any passphrase:<br />
Enter new passphrase for key slot:<br />
Verify passphrase: <br />
}}<br />
<br />
If {{ic|/path/to/<additionalkeyfile>}} is given, cryptsetup will add a new keyslot for <additionalkeyfile>. Otherwise a new passphrase will be prompted for twice. For using an existing ''keyfile'' to authorize the action, the {{ic|--key-file}} or {{ic|-d}} option followed by the "old" <keyfile> will try to unlock all available keyfile keyslots:<br />
<br />
# cryptsetup luksAddKey /dev/<device> (/path/to/<additionalkeyfile>) -d /path/to/<keyfile><br />
<br />
If it is intended to use multiple keys and change or revoke them, the {{ic|--key-slot}} or {{ic|-S}} option may be used to specify the slot: <br />
<br />
{{hc|# cryptsetup luksAddKey /dev/<device> -S 6|<br />
Enter any passphrase: <br />
Enter new passphrase for key slot: <br />
Verify passphrase:<br />
}}<br />
<br />
{{hc|# cryptsetup luksDump /dev/sda8 {{!}} grep 'Slot 6'|<br />
Key Slot 6: ENABLED<br />
}}<br />
<br />
To show an associated action in this example, we decide to change the key right away: <br />
<br />
{{hc|# cryptsetup luksChangeKey /dev/<device> -S 6|<br />
Enter LUKS passphrase to be changed: <br />
Enter new LUKS passphrase:<br />
}}<br />
<br />
before continuing to remove it.<br />
<br />
==== Removing LUKS keys ====<br />
<br />
There are three different actions to remove keys from the header: <br />
<br />
* {{ic|luksRemoveKey}} is used to remove a key by specifying its passphrase/key-file. <br />
* {{ic|luksKillSlot}} may be used to remove a key from a specific key slot (using another key). Obviously, this is extremely useful if you have forgotten a passphrase, lost a key-file, or have no access to it. <br />
* {{ic|luksErase}} is used to quickly remove '''all''' active keys. <br />
<br />
{{warning|<br />
* All above actions can be used to irrevocably delete the last active key for an encrypted device! <br />
* The {{ic|luksErase}} command was added in version 1.6.4 to quickly nuke access to the device. This action '''will not''' prompt for a valid passphrase! It will not [[dm-crypt/Drive preparation#Wipe LUKS header|wipe the LUKS header]], but all keyslots at once and you will, therefore, not be able to regain access unless you have a valid backup of the LUKS header.<br />
}}<br />
<br />
For above warning it is good to know the key we want to '''keep''' is valid. An easy check is to unlock the device with the {{ic|-v}} option, which will specify which slot it occupies: <br />
<br />
{{hc|# cryptsetup -v open /dev/<device> testcrypt|<br />
Enter passphrase for /dev/<device>: <br />
Key slot 1 unlocked.<br />
Command successful.<br />
}}<br />
<br />
Now we can remove the key added in the previous subsection using its passphrase: <br />
<br />
{{hc|# cryptsetup luksRemoveKey /dev/<device>|<br />
Enter LUKS passphrase to be deleted:<br />
}}<br />
<br />
If we had used the same passphrase for two keyslots, the first slot would be wiped now. Only executing it again would remove the second one. <br />
<br />
Alternatively, we can specify the key slot: <br />
<br />
{{hc|# cryptsetup luksKillSlot /dev/<device> 6|<br />
Enter any remaining LUKS passphrase:<br />
}}<br />
<br />
Note that in both cases, no confirmation was required.<br />
<br />
{{hc|# cryptsetup luksDump /dev/sda8 {{!}} grep 'Slot 6'|<br />
Key Slot 6: DISABLED<br />
}}<br />
<br />
To re-iterate the warning above: If the same passphrase had been used for key slots 1 and 6, both would be gone now.<br />
<br />
=== Backup and restore ===<br />
<br />
If the header of a LUKS encrypted partition gets destroyed, you will not be able to decrypt your data. It is just as much of a dilemma as forgetting the passphrase or damaging a key-file used to unlock the partition. Damage may occur by your own fault while re-partitioning the disk later or by third-party programs misinterpreting the partition table. Therefore, having a backup of the header and storing it on another disk might be a good idea.<br />
<br />
{{Note|If one of the LUKS-encrypted partitions' passphrases becomes compromised, you must revoke it on ''every'' copy of the cryptheader, even those you have backed up. Otherwise, a copy of the backed-up cryptheader that uses the compromised passphrase can be used to determine the master key which in turn can be used to decrypt the associated partition (even your actual partition, not only the backed-up version). On the other hand, if the master key gets compromised, you have to reencrypt your whole partition. See [https://gitlab.com/cryptsetup/cryptsetup/wikis/FrequentlyAskedQuestions#6-backup-and-data-recovery LUKS FAQ] for further details.}}<br />
<br />
==== Backup using cryptsetup ====<br />
<br />
Cryptsetup's {{ic|luksHeaderBackup}} action stores a binary backup of the LUKS header and keyslot area:<br />
<br />
# cryptsetup luksHeaderBackup /dev/<device> --header-backup-file /mnt/<backup>/<file>.img<br />
<br />
where <device> is the partition containing the LUKS volume.<br />
<br />
You can also back up the plaintext header into ramfs and encrypt it with e.g. [[GPG]] before writing it to persistent storage:<br />
<br />
# mkdir /root/<tmp>/<br />
# mount ramfs /root/<tmp>/ -t ramfs<br />
# cryptsetup luksHeaderBackup /dev/<device> --header-backup-file /root/<tmp>/<file>.img<br />
# gpg2 --recipient <User ID> --encrypt /root/<tmp>/<file>.img <br />
# cp /root/<tmp>/<file>.img.gpg /mnt/<backup>/<br />
# umount /root/<tmp><br />
<br />
{{Warning|Tmpfs can swap to harddisk if low on memory so it is not recommended here.}}<br />
<br />
==== Restore using cryptsetup ====<br />
<br />
{{Warning|Restoring the wrong header or restoring to an unencrypted partition will cause data loss! The action can not perform a check whether the header is actually the ''correct'' one for that particular device.}} <br />
<br />
In order to evade restoring a wrong header, you can ensure it does work by using it as a remote {{ic|--header}} first: <br />
<br />
{{hc|# cryptsetup -v --header /mnt/<backup>/<file>.img open /dev/<device> test|<br />
Key slot 0 unlocked.<br />
Command successful.<br />
}}<br />
<br />
# mount /dev/mapper/test /mnt/test && ls /mnt/test <br />
# umount /mnt/test <br />
# cryptsetup close test <br />
<br />
Now that the check succeeded, the restore may be performed: <br />
<br />
# cryptsetup luksHeaderRestore /dev/<device> --header-backup-file ./mnt/<backup>/<file>.img<br />
<br />
Now that all the keyslot areas are overwritten; only active keyslots from the backup file are available after issuing the command.<br />
<br />
==== Manual backup and restore ====<br />
<br />
The header always resides at the beginning of the device and a backup can be performed without access to ''cryptsetup'' as well. First you have to find out the payload offset of the crypted partition:<br />
<br />
{{hc|# cryptsetup luksDump /dev/<device> {{!}} grep "Payload offset"|<br />
Payload offset: 4040<br />
}}<br />
<br />
Second check the sector size of the drive<br />
<br />
{{hc|# fdisk -l /dev/<device> {{!}} grep "Sector size"|<br />
Sector size (logical/physical): 512 bytes / 512 bytes<br />
}}<br />
<br />
Now that you know the values, you can backup the header with a simple [[dd]] command:<br />
<br />
# dd if=/dev/<device> of=/path/to/<file>.img bs=512 count=4040<br />
<br />
and store it safely.<br />
<br />
A restore can then be performed using the same values as when backing up:<br />
<br />
# dd if=./<file>.img of=/dev/<device> bs=512 count=4040<br />
<br />
=== Re-encrypting devices ===<br />
<br />
{{Expansion|cryptsetup 2.2 using LUKS2 (with a 16 MiB header) supports online encryption/decryption/reencryption.[https://mirrors.edge.kernel.org/pub/linux/utils/cryptsetup/v2.2/v2.2.0-ReleaseNotes]}}<br />
<br />
The {{Pkg|cryptsetup}} package features two options for re-encryption.<br />
<br />
; cryptsetup reencrypt: Argument to {{ic|cryptsetup}} itself: Preferred method. Currently LUKS2 devices only. Actions can be performed online. Supports multiple parallel re-encryption jobs. Resilient to system failures. See {{man|8|cryptsetup}} for more information.<br />
<br />
; cryptsetup-reencrypt: Legacy tool, supports LUKS1 in addition to LUKS2. Actions can be performed on unmounted devices only. Single process at a time. Sensitive to system failures. See {{man|8|cryptsetup-reencrypt}} for more information. <br />
<br />
Both can be used to convert an existing unencrypted filesystem to a LUKS encrypted one or permanently remove LUKS encryption from a device (using {{ic|--decrypt}}). As its name suggests it can also be used to re-encrypt an existing LUKS encrypted device, though, re-encryption is not possible for a detached LUKS header or other encryption modes (e.g. plain-mode). For re-encryption it is possible to change the [[#Encryption options for LUKS mode]]. <br />
<br />
One application of re-encryption may be to secure the data again after a passphrase or [[#Keyfiles|keyfile]] has been compromised ''and'' one cannot be certain that no copy of the LUKS header has been obtained. For example, if only a passphrase has been shoulder-surfed but no physical/logical access to the device happened, it would be enough to change the respective passphrase/key only ([[#Key management]]). <br />
<br />
{{Warning|Always make sure a '''reliable backup''' is available and double-check options you specify before using the tool!}}<br />
<br />
The following shows an example to encrypt an unencrypted filesystem partition and a re-encryption of an existing LUKS device.<br />
<br />
==== Encrypt an existing unencrypted filesystem ====<br />
<br />
A LUKS encryption header is always stored at the beginning of the device. Since an existing filesystem will usually be allocated all partition sectors, the first step is to shrink it to make space for the LUKS header.<br />
<br />
{{Expansion|cryptsetup man pages suggest using twice the LUKS2 header size. That implies 32MiB and using {{ic|--reduce-device-size 32M}}}}<br />
The [[#Encryption options for LUKS mode|default]] LUKS2 header requires 16 MiB. If the current filesystem occupies all the available space, we will have to shrink it at least that much.<br />
To shrink an existing {{ic|ext4}} filesystem on {{ic|/dev/sdaX}} to its current possible minimum:<br />
<br />
# umount /mnt<br />
<br />
{{hc|# e2fsck -f /dev/sdaX|<br />
e2fsck 1.43-WIP (18-May-2015)<br />
Pass 1: Checking inodes, blocks, and sizes<br />
...<br />
/dev/sda6: 12/166320 files (0.0% non-contiguous), 28783/665062 blocks<br />
}}<br />
<br />
{{hc|# resize2fs -p -M /dev/sdaX|<br />
resize2fs 1.43-WIP (18-May-2015)<br />
Resizing the filesystem on /dev/sdaX to 26347 (4k) blocks.<br />
The filesystem on /dev/sdaX is now 26347 (4k) blocks long.<br />
}}<br />
<br />
{{Tip|Shrinking to the minimum size with -M might take very long. You might want to calculate a size just 32MiB smaller than the current size instead of using -M, or use a graphical tool like gparted.}}<br />
<br />
Now we encrypt it, using the default cipher we do not have to specify it explicitly:<br />
{{hc|# cryptsetup reencrypt --encrypt --reduce-device-size 16M /dev/sdaX|<nowiki><br />
<br />
WARNING!<br />
<br />
========<br />
<br />
This will overwrite data on LUKS2-temp-12345678-9012-3456-7890-123456789012.new irrevocably.<br />
<br />
Are you sure? (Type 'yes' in capital letters): YES<br />
Enter passphrase for LUKS2-temp-12345678-9012-3456-7890-123456789012.new: <br />
Verify passphrase: <br />
</nowiki>}}<br />
<br />
After it finished, the whole {{ic|/dev/sdaX}} partition is encrypted, not only the space the filesystem was shrunk to. As a final step we extend the original {{ic|ext4}} filesystem to occupy all available space again, on the now encrypted partition:<br />
<br />
{{hc|# cryptsetup open /dev/sdaX recrypt|<br />
Enter passphrase for /dev/sdaX: <br />
...<br />
}}<br />
<br />
{{hc|# resize2fs /dev/mapper/recrypt|<br />
resize2fs 1.43-WIP (18-May-2015)<br />
Resizing the filesystem on /dev/mapper/recrypt to 664807 (4k) blocks.<br />
The filesystem on /dev/mapper/recrypt is now 664807 (4k) blocks long.<br />
}}<br />
<br />
# mount /dev/mapper/recrypt /mnt<br />
<br />
The filesystem is now ready to use. You may want to add it to your [[crypttab]].<br />
<br />
==== Re-encrypting an existing LUKS partition ====<br />
<br />
In this example an existing LUKS device is re-encrypted. <br />
<br />
{{Warning|Double-check you specify encryption options for correctly and ''never'' re-encrypt without a '''reliable backup'''!}}<br />
<br />
In order to re-encrypt a device with its existing encryption options, they do not need to be specified:<br />
{{bc|# cryptsetup reencrypt /dev/sdaX}}<br />
{{Note|For LUKS1 we will need to use the legacy tool:{{bc|# cryptsetup-reencrypt /dev/sdaX}}}}<br />
<br />
Existing keys are retained when re-encrypting a device with a different cipher and/or hash. Another use case is to re-encrypt LUKS devices which have non-current encryption options. Apart from above warning on specifying options correctly, the ability to change the LUKS header may also be limited by its size. For example, if the device was initially encrypted using a CBC mode cipher and 128 bit key-size, the LUKS header will be half the size of above mentioned {{ic|4096}} sectors: <br />
<br />
{{hc|# cryptsetup luksDump /dev/sdaX <nowiki>|</nowiki>grep -e "mode" -e "Payload" -e "MK bits"|<br />
Cipher mode: cbc-essiv:sha256<br />
Payload offset: '''2048'''<br />
MK bits: 128<br />
}}<br />
<br />
While it is possible to upgrade the encryption of such a device, it is currently only feasible in two steps. First, re-encrypting with the same encryption options, but using the {{ic|--reduce-device-size}} option to make further space for the larger LUKS header. Second, re-encypt the whole device again with the desired cipher. For this reason and the fact that a backup should be created in any case, creating a new, fresh encrypted device to restore into is always the faster option.<br />
<br />
== Resizing encrypted devices ==<br />
<br />
{{Expansion|This section should be rewritten to introduce resizing more generically. Perhaps work on it together with [[Resizing LVM-on-LUKS]].}}<br />
<br />
If a storage device encrypted with dm-crypt is being cloned (with a tool like dd) to another larger device, the underlying dm-crypt device must be resized to use the whole space. <br />
<br />
The destination device is /dev/sdX2 in this example, the whole available space adjacent to the partition will be used:<br />
<br />
# cryptsetup luksOpen /dev/sdX2 sdX2<br />
# cryptsetup resize sdX2<br />
<br />
Then the underlying filesystem must be resized.<br />
<br />
=== Loopback filesystem ===<br />
<br />
Assume that an encrypted loopback filesystem is stored in a file {{ic|/bigsecret}}, looped to {{ic|/dev/loop0}}, mapped to {{ic|secret}} and mounted on {{ic|/mnt/secret}}, as in the example at [[dm-crypt/Encrypting a non-root file system#File container]].<br />
<br />
If the container file is currently mapped and/or mounted, unmount and/or close it:<br />
<br />
# umount /mnt/secret<br />
# cryptsetup close secret<br />
# losetup -d /dev/loop0<br />
<br />
Next, expand the container file with the size of the data you want to add. In this example, the file will be expanded with 1M * 1024, which is 1G.<br />
<br />
{{Warning|Make absolutely sure to use '''two''' {{ic|>}}, instead of just one, or else you will overwrite the file instead of appending to it. Making a backup before this step is strongly recommended.}}<br />
<br />
# dd if=/dev/urandom bs=1M count=1024 | cat - >> /bigsecret<br />
<br />
Now map the container to the loop device:<br />
<br />
# losetup /dev/loop0 /bigsecret<br />
# cryptsetup open /dev/loop0 secret<br />
<br />
After this, resize the encrypted part of the container to the new maximum size of the container file:<br />
<br />
# cryptsetup resize secret<br />
<br />
Finally, perform a filesystem check and, if it is ok, resize it (example for ext2/3/4):<br />
<br />
# e2fsck -f /dev/mapper/secret<br />
# resize2fs /dev/mapper/secret<br />
<br />
You can now mount the container again:<br />
<br />
# mount /dev/mapper/secret /mnt/secret<br />
<br />
=== Integrity protected device ===<br />
<br />
If the device was formatted with integrity support (e.g., {{ic|--integrity hmac-sha256}}) and the backing block device is shrinked, it cannot be opened with this error: {{ic|device-mapper: reload ioctl on failed: Invalid argument}}.<br />
<br />
To fix this issue without wiping the device again, it can be formatted with the previous master key (keeping the per-sector tags valid).<br />
<br />
# cryptsetup luksDump /dev/sdX2 --dump-master-key --master-key-file=/tmp/masterkey-in-tmpfs.key<br />
# cryptsetup luksFormat /dev/sdX2 --type luks2 --integrity hmac-sha256 --master-key-file=/tmp/masterkey-in-tmpfs.key --integrity-no-wipe<br />
# rm /tmp/masterkey-in-tmpfs.key<br />
<br />
== Keyfiles ==<br />
<br />
{{Note|This section describes using a plaintext keyfile. If you want to encrypt your keyfile giving you two factor authentication see [[dm-crypt/Specialties#Using GPG, LUKS, or OpenSSL Encrypted Keyfiles|Using GPG or OpenSSL Encrypted Keyfiles]] for details, but please still read this section.}}<br />
<br />
'''What is a keyfile?'''<br />
<br />
A keyfile is a file whose data is used as the passphrase to unlock an encrypted volume.<br />
That means if such a file is lost or changed, decrypting the volume may no longer be possible.<br />
<br />
{{Tip|Define a passphrase in addition to the keyfile for backup access to encrypted volumes in the event the defined keyfile is lost or changed.}}<br />
<br />
'''Why use a keyfile?'''<br />
<br />
There are many kinds of keyfiles. Each type of keyfile used has benefits and disadvantages summarized below:<br />
<br />
=== Types of keyfiles ===<br />
<br />
==== passphrase ====<br />
<br />
This is a keyfile containing a simple passphrase. The benefit of this type of keyfile is that if the file is lost the data it contained is known and hopefully easily remembered by the owner of the encrypted volume. However the disadvantage is that this does not add any security over entering a passphrase during the initial system start.<br />
<br />
Example: {{ic|1234}}<br />
<br />
{{Note|The keyfile containing the passphrase must not have a newline in it. One option is to create it using <br />
<br />
# echo -n 'your_passphrase' > /path/to/<keyfile><br />
# chown root:root /path/to/<keyfile>; chmod 400 /path/to/<keyfile><br />
<br />
If the file contains special characters such as a backslash, rather than escaping these, it is recommended to simply edit the key file directly entering or pasting the passphrase and then remove the trailing newline with a handy perl one-liner:<br />
<br />
# perl -pi -e 'chomp if eof' /path/to/<keyfile><br />
}}<br />
<br />
==== randomtext ====<br />
<br />
This is a keyfile containing a block of random characters. The benefit of this type of keyfile is that it is much more resistant to dictionary attacks than a simple passphrase. An additional strength of keyfiles can be utilized in this situation which is the length of data used. Since this is not a string meant to be memorized by a person for entry, it is trivial to create files containing thousands of random characters as the key. The disadvantage is that if this file is lost or changed, it will most likely not be possible to access the encrypted volume without a backup passphrase.<br />
<br />
Example: {{ic|fjqweifj830149-57 819y4my1-38t1934yt8-91m 34co3;t8y;9p3y-}}<br />
<br />
==== binary ====<br />
<br />
This is a binary file that has been defined as a keyfile. When identifying files as candidates for a keyfile, it is recommended to choose files that are relatively static such as photos, music, video clips. The benefit of these files is that they serve a dual function which can make them harder to identify as keyfiles. Instead of having a text file with a large amount of random text, the keyfile would look like a regular image file or music clip to the casual observer. The disadvantage is that if this file is lost or changed, it will most likely not be possible to access the encrypted volume without a backup passphrase. Additionally, there is a theoretical loss of randomness when compared to a randomly generated text file. This is due to the fact that images, videos and music have some intrinsic relationship between neighboring bits of data that does not exist for a random text file. However this is controversial and has never been exploited publicly.<br />
<br />
Example: images, text, video, ...<br />
<br />
=== Creating a keyfile with random characters ===<br />
<br />
==== Storing the keyfile on a filesystem ====<br />
<br />
A keyfile can be of arbitrary content and size. <br />
<br />
Here [[dd]] is used to generate a keyfile of 2048 random bytes, storing it in the file {{ic|/etc/mykeyfile}}:<br />
<br />
# dd bs=512 count=4 if=/dev/random of=/etc/mykeyfile iflag=fullblock<br />
<br />
If you are planning to store the keyfile on an external device, you can also simply change the outputfile to the corresponding directory:<br />
<br />
# dd bs=512 count=4 if=/dev/random of=/media/usbstick/mykeyfile iflag=fullblock<br />
<br />
To deny any access for other users than {{ic|root}}:<br />
<br />
# chmod 600 /etc/mykeyfile<br />
<br />
===== Securely overwriting stored keyfiles =====<br />
<br />
If you stored your temporary keyfile on a physical storage device, and want to delete it, remember to not just remove the keyfile later on, but use something like<br />
<br />
# shred --remove --zero mykeyfile<br />
<br />
to securely overwrite it. For overaged filesystems like FAT or ext2 this will suffice while in the case of journaling filesystems, flash memory hardware and other cases it is highly recommended to [[Securely wipe disk|wipe the entire device]].<br />
<br />
==== Storing the keyfile in ramfs ====<br />
<br />
Alternatively, you can mount a ramfs for storing the keyfile temporarily:<br />
<br />
# mkdir /root/myramfs<br />
# mount ramfs /root/myramfs/ -t ramfs<br />
# cd /root/myramfs<br />
<br />
The advantage is that it resides in RAM and not on a physical disk, therefore it can not be recovered after unmounting the ramfs. After copying the keyfile to another secure and persistent filesystem, unmount the ramfs again with<br />
<br />
# umount /root/myramfs<br />
<br />
=== Configuring LUKS to make use of the keyfile ===<br />
<br />
Add a keyslot for the keyfile to the LUKS header:<br />
<br />
{{hc|# cryptsetup luksAddKey /dev/sda2 /etc/mykeyfile|<br />
Enter any LUKS passphrase:<br />
key slot 0 unlocked.<br />
Command successful.<br />
}}<br />
<br />
=== Manually unlocking a partition using a keyfile ===<br />
<br />
Use the {{ic|--key-file}} option when opening the LUKS device:<br />
<br />
# cryptsetup open /dev/sda2 ''dm_name'' --key-file /etc/mykeyfile<br />
<br />
=== Unlocking the root partition at boot ===<br />
<br />
This is simply a matter of configuring [[mkinitcpio]] to include the necessary modules or files and configuring the [[dm-crypt/System configuration#cryptkey|cryptkey]] [[kernel parameter]] to know where to find the keyfile.<br />
<br />
Two cases are covered below:<br />
<br />
# Using a keyfile stored on an external medium (e.g. a USB stick)<br />
# Using a keyfile embedded in the initramfs<br />
<br />
==== With a keyfile stored on an external media ====<br />
<br />
===== Configuring mkinitcpio =====<br />
<br />
You have to add the kernel module for the drive's [[file system]] to the [[mkinitcpio#MODULES|MODULES array]] in {{ic|/etc/mkinitcpio.conf}}. For example, add {{ic|ext4}} if the file system is [[Ext4]] or {{ic|vfat}} in case it is [[FAT]]:<br />
<br />
MODULES=(vfat)<br />
<br />
If there are messages about bad superblock and bad codepage at boot, then you need an extra codepage module to be loaded. For instance, you may need {{ic|nls_iso8859-1}} module for {{ic|iso8859-1}} codepage.<br />
<br />
[[Regenerate the initramfs]].<br />
<br />
===== Configuring the kernel parameters =====<br />
<br />
* For a busybox-based initramfs using the [[dm-crypt/System configuration#Using encrypt hook|encrypt]] hook, see [[dm-crypt/System configuration#cryptkey]].<br />
* For a systemd based initramfs using the [[sd-encrypt]] hook, see [[dm-crypt/System configuration#rd.luks.key]].<br />
<br />
==== With a keyfile embedded in the initramfs ====<br />
<br />
{{Expansion|Add method for [[sd-encrypt]].}}<br />
<br />
{{Warning|Use an embedded keyfile '''only''' if you protect the keyfile sufficiently by:<br />
* Using some form of authentication earlier in the boot process. Otherwise auto-decryption will occur, defeating completely the purpose of block device encryption.<br />
* {{ic|/boot}} is encrypted. Otherwise root on a different installation (including the [[Installation guide#Boot the live environment|live environment]]) can extract your key from the initramfs, and unlock the device without any other authentication.}}<br />
<br />
This method allows to use a specially named keyfile that will be embedded in the [[initramfs]] and picked up by the {{ic|encrypt}} [[Mkinitcpio#HOOKS|hook]] to unlock the root filesystem ({{ic|cryptdevice}}) automatically. It may be useful to apply when using the [[GRUB#Encrypted /boot|GRUB early cryptodisk]] feature, in order to avoid entering two passphrases during boot.<br />
<br />
The {{ic|encrypt}} hook lets the user specify a keyfile with the {{ic|cryptkey}} kernel parameter: in the case of initramfs, the syntax is {{ic|rootfs:''path''}}. See [[dm-crypt/System configuration#cryptkey]]. Besides, this kernel parameter defaults to use {{ic|/crypto_keyfile.bin}}, and if the initramfs contains a valid key with this name, decryption will occur automatically without the need to configure the {{ic|cryptkey}} parameter.<br />
<br />
If using {{ic|sd-encrypt}} instead of {{ic|encrypt}}, specify the location of the keyfile with the {{ic|rd.luks.key}} kernel parameter. See [[dm-crypt/System configuration#rd.luks.key]].<br />
<br />
[[#Creating a keyfile with random characters|Generate the keyfile]], give it suitable permissions and [[#Adding LUKS keys|add it as a LUKS key]]:<br />
<br />
# dd bs=512 count=4 if=/dev/random of=/crypto_keyfile.bin iflag=fullblock<br />
# chmod 600 /crypto_keyfile.bin<br />
# chmod 600 /boot/initramfs-linux*<br />
# cryptsetup luksAddKey /dev/sdX# /crypto_keyfile.bin<br />
<br />
{{Warning|When initramfs' permissions are set to 644 (by default), then all users will be able to dump the keyfile. Make sure the permissions are still 600 if you install a new kernel.}}<br />
<br />
Include the key in [[mkinitcpio#BINARIES and FILES|mkinitcpio's FILES array]]:<br />
<br />
{{hc|/etc/mkinitcpio.conf|2=<br />
FILES=(/crypto_keyfile.bin)<br />
}}<br />
<br />
Finally [[regenerate the initramfs]].<br />
<br />
On the next reboot you should only have to enter your container decryption passphrase once.<br />
<br />
([https://www.pavelkogan.com/2014/05/23/luks-full-disk-encryption/#bonus-login-once source])</div>Phireskyhttps://wiki.archlinux.org/index.php?title=Dm-crypt/Device_encryption&diff=648271Dm-crypt/Device encryption2021-01-05T17:03:07Z<p>Phiresky: /* change title of encrypt an unencrypted filesystem */</p>
<hr />
<div>{{Lowercase title}}<br />
[[Category:Data-at-rest encryption]]<br />
[[es:Dm-crypt (Español)/Device encryption]]<br />
[[ja:Dm-crypt/デバイスの暗号化]]<br />
[[pl:Dm-crypt (Polski)/Device encryption]]<br />
[[pt:Dm-crypt (Português)/Device encryption]]<br />
This section covers how to manually utilize ''dm-crypt'' from the command line to encrypt a system. <br />
<br />
== Preparation ==<br />
<br />
Before using {{pkg|cryptsetup}}, always make sure the {{ic|dm_crypt}} [[kernel module]] is loaded.<br />
<br />
== Cryptsetup usage ==<br />
<br />
''Cryptsetup'' is the command line tool to interface with ''dm-crypt'' for creating, accessing and managing encrypted devices. The tool was later expanded to support different encryption types that rely on the Linux kernel '''d'''evice-'''m'''apper and the '''crypt'''ographic modules. The most notable expansion was for the Linux Unified Key Setup (LUKS) extension, which stores all of the needed setup information for dm-crypt on the disk itself and abstracts partition and key management in an attempt to improve ease of use. Devices accessed via the device-mapper are called blockdevices. For further information see [[Data-at-rest encryption#Block device encryption]]. <br />
<br />
The tool is used as follows: <br />
<br />
# cryptsetup <OPTIONS> <action> <action-specific-options> <device> <dmname><br />
<br />
It has compiled-in defaults for the options and the encryption mode, which will be used if no others are specified on the command line. Have a look at <br />
<br />
$ cryptsetup --help <br />
<br />
which lists options, actions and the default parameters for the encryption modes in that order. A full list of options can be found on the man page.<br />
Since different parameters are required or optional, depending on encryption mode and action, the following sections point out differences further. Blockdevice encryption is fast, but speed matters a lot too. Since changing an encryption cipher of a blockdevice after setup is difficult, it is important to check ''dm-crypt'' performance for the individual parameters in advance: <br />
<br />
$ cryptsetup benchmark <br />
<br />
can give guidance on deciding for an algorithm and key-size prior to installation. If certain AES ciphers excel with a considerable higher throughput, these are probably the ones with hardware support in the CPU.<br />
<br />
{{Tip|You may want to practise encrypting a virtual hard drive in a [[virtual machine]] when learning.}}<br />
<br />
=== Cryptsetup passphrases and keys ===<br />
<br />
An encrypted blockdevice is protected by a key. A key is either: <br />
<br />
* a passphrase: see [[Security#Passwords]].<br />
* a keyfile, see [[#Keyfiles]].<br />
<br />
Both key types have default maximum sizes: passphrases can be up to 512 characters and keyfiles up to 8192kiB. <br />
<br />
An important distinction of ''LUKS'' to note at this point is that the key is used to unlock the master-key of a LUKS-encrypted device and can be changed with root access. Other encryption modes do not support changing the key after setup, because they do not employ a master-key for the encryption. See [[Data-at-rest encryption#Block device encryption]] for details.<br />
<br />
== Encryption options with dm-crypt ==<br />
<br />
''Cryptsetup'' supports different encryption operating modes to use with ''dm-crypt'': <br />
<br />
* {{ic|--type luks}} for using the default LUKS format version (LUKS1 with {{Pkg|cryptsetup}} < 2.1.0, LUKS2 with {{Pkg|cryptsetup}} ≥ 2.1.0),<br />
* {{ic|--type luks1}} for using LUKS1, the most common version of LUKS,<br />
* {{ic|--type luks2}} for using LUKS2, the latest available version of LUKS that allows additional extensions,<br />
* {{ic|--type plain}} for using dm-crypt plain mode, <br />
* {{ic|--type loopaes}} for a loopaes legacy mode, <br />
* {{ic|--type tcrypt}} for a [[TrueCrypt]] compatibility mode.<br />
<br />
The basic cryptographic options for encryption cipher and hashes available can be used for all modes and rely on the kernel cryptographic backend features. All that are loaded and available to use as options at runtime can be viewed with:<br />
<br />
$ less /proc/crypto <br />
<br />
{{Tip|If the list is short, execute {{ic|$ cryptsetup benchmark}} which will trigger loading available modules.}}<br />
<br />
The following introduces encryption options for the {{ic|luks}}, {{ic|luks1}}, {{ic|luks2}} and {{ic|plain}} modes. Note that the tables list options used in the respective examples in this article and not all available ones. <br />
<br />
=== Encryption options for LUKS mode ===<br />
<br />
The ''cryptsetup'' action to set up a new dm-crypt device in LUKS encryption mode is ''luksFormat''. Unlike the name implies, it does not format the device, but sets up the LUKS device header and encrypts the master-key with the desired cryptographic options. <br />
<br />
In order to create a new LUKS container with the compiled-in defaults listed by {{ic|cryptsetup --help}}, simply execute:<br />
<br />
# cryptsetup luksFormat ''device''<br />
<br />
As of cryptsetup 2.3.4, this is equivalent to: <br />
<br />
# cryptsetup --type luks2 --cipher aes-xts-plain64 --hash sha256 --iter-time 2000 --key-size 256 --pbkdf argon2i --sector-size 512 --use-urandom --verify-passphrase luksFormat ''device''<br />
<br />
Defaults are compared with a cryptographically higher specification example in the table below, with accompanying comments: <br />
<br />
{| class="wikitable"<br />
! Options !! Cryptsetup 2.1.0 defaults !! Example !! Comment<br />
|-<br />
! style="text-align:right;white-space:nowrap;" | --cipher<br />
-c<br />
| {{ic|aes-xts-plain64}}<br />
| {{ic|aes-xts-plain64}}<br />
| [https://www.kernel.org/pub/linux/utils/cryptsetup/v1.6/v1.6.0-ReleaseNotes Release 1.6.0] changed the defaults to an AES [[Data-at-rest encryption#Ciphers and modes of operation|cipher]] in [[wikipedia:Disk encryption theory#XEX-based tweaked-codebook mode with ciphertext stealing (XTS)|XTS]] mode (see item 5.16 [https://gitlab.com/cryptsetup/cryptsetup/wikis/FrequentlyAskedQuestions#5-security-aspects of the FAQ]). It is advised against using the previous default {{ic|--cipher aes-cbc-essiv}} because of its known [[wikipedia:Disk encryption theory#Cipher-block chaining (CBC)|issues]] and practical [http://www.jakoblell.com/blog/2013/12/22/practical-malleability-attack-against-cbc-encrypted-luks-partitions/ attacks] against them.<br />
|-<br />
! style="text-align:right;white-space:nowrap;" | --key-size<br />
-s<br />
| {{ic|256}} ({{ic|512}} for XTS)<br />
| {{ic|512}}<br />
| By default a 512 bit key-size is used for XTS ciphers. Note however that [[wikipedia:Disk encryption theory#XEX-based tweaked-codebook mode with ciphertext stealing (XTS)|XTS splits the supplied key in half]], so this results in AES-256 being used.<br />
|-<br />
! style="text-align:right;white-space:nowrap;" | --hash<br />
-h<br />
| {{ic|sha256}}<br />
| {{ic|sha512}}<br />
| Hash algorithm used for [[Data-at-rest encryption#Cryptographic metadata|key derivation]]. Release 1.7.0 changed defaults from {{ic|sha1}} to {{ic|sha256}} "''not for security reasons [but] mainly to prevent compatibility problems on hardened systems where SHA1 is already [being] phased out''"[https://www.kernel.org/pub/linux/utils/cryptsetup/v1.7/v1.7.0-ReleaseNotes]. The former default of {{ic|sha1}} can still be used for compatibility with older versions of ''cryptsetup'' since it is [https://gitlab.com/cryptsetup/cryptsetup/wikis/FrequentlyAskedQuestions#5-security-aspects considered secure] (see item 5.20). <br />
|-<br />
! style="text-align:right;white-space:nowrap;" | --iter-time<br />
-i<br />
| {{ic|2000}}<br />
| {{ic|5000}}<br />
| Number of milliseconds to spend with PBKDF2 passphrase processing. Release 1.7.0 changed defaults from {{ic|1000}} to {{ic|2000}} to "''try to keep PBKDF2 iteration count still high enough and also still acceptable for users.''"[https://www.kernel.org/pub/linux/utils/cryptsetup/v1.7/v1.7.0-ReleaseNotes]. This option is only relevant for LUKS operations that set or change passphrases, such as ''luksFormat'' or ''luksAddKey''. Specifying 0 as parameter selects the compiled-in default..<br />
|-<br />
! style="text-align:right;white-space:nowrap;" | --use-{u,}random<br />
| {{ic|--use-urandom}}<br />
| {{ic|--use-random}}<br />
| Selects which [[random number generator]] to use. Quoting the cryptsetup manual page: "In a low-entropy situation (e.g. in an embedded system), both selections are problematic. Using /dev/urandom can lead to weak keys. Using /dev/random can block a long time, potentially forever, if not enough entropy can be harvested by the kernel."<br />
|-<br />
! style="text-align:right;white-space:nowrap;" | --verify-passphrase<br />
-y<br />
| Yes<br />
| -<br />
| Enabled by default in Arch Linux for luksFormat and luksAddKey.<br />
|}<br />
<br />
The properties of LUKS features and options are described in the [https://gitlab.com/cryptsetup/cryptsetup/wikis/Specification LUKS1] (pdf) and [https://gitlab.com/cryptsetup/cryptsetup/blob/master/docs/on-disk-format-luks2.pdf LUKS2] (pdf) specifications.<br />
<br />
{{Tip|The project developers' [https://mbroz.fedorapeople.org/talks/DevConf2016/devconf2016-luks2.pdf devconfcz2016] (pdf) presentation summarizes the motivation for the major specification update to LUKS2.}}<br />
<br />
==== Choice of argon2 variant for PBKDF ====<br />
<br />
Although argon2id is in [https://crypto.stackexchange.com/questions/48935/why-use-argon2i-or-argon2d-if-argon2id-exists#49969 most cases the preferred variant] of the [[Wikipedia:Argon2|Argon2 family]], the [https://tools.ietf.org/html/draft-irtf-cfrg-argon2-12 quoted RFC] does indicate that:<br />
<br />
:''Argon2i uses data-independent memory access, '''which is preferred for password hashing and password-based key derivation'''.''<br />
<br />
We can also see this reflected in this [https://www.saout.de/pipermail/dm-crypt/2018-September/005968.html mailing list answer] by cryptsetup maintainer Milan Broz:<br />
<br />
:''You can use Argon2id though (combination of data dependent and independent processing). I prefer Argon2i for key derivation, but opinions differ here.''<br />
<br />
Therefore it is recommended to keep the default as is so as not to inadvertently decrease security.<br />
<br />
==== Iteration time ====<br />
<br />
From [https://gitlab.com/cryptsetup/cryptsetup/-/wikis/FrequentlyAskedQuestions#2-setup cryptsetup FAQ§2.1] and [https://gitlab.com/cryptsetup/cryptsetup/-/wikis/FrequentlyAskedQuestions#3-common-problems §3.4]:<br />
<br />
:The unlock time for a key-slot [...] is calculated when setting a passphrase. By default it is 1 second (2 seconds for LUKS2). [...]<br />
:Passphrase iteration count is based on time and hence security level depends on CPU power of the system the LUKS container is created on. [...]<br />
:If you set a passphrase on a fast machine and then unlock it on a slow machine, the unlocking time can be much longer.<br />
<br />
As such, it is better to always create a container on the machine where it will be most often accessed.<br />
<br />
Read the rest of those sections for advice on how to correctly adjust the iteration count should the need arise.<br />
<br />
==== Sector size ====<br />
{{Warning|The filesystem to be used inside the LUKS container must also be formatted with the same sector size.}}<br />
For drives with [[Advanced Format]] (also known as ''AF'', ''4K'', ''4096 byte sector'' drives) sectors it is recommended to use the appropriate physical sector size. In particular ''shingled magnetic recording'' (SMR) drives that are firmware-managed are negatively impacted if using a logical sector size of 512 bytes if their physical sector size is of 4096 bytes.<br />
<br />
{{man|8|cryptsetup|OPTIONS}} description for {{ic|--sector-size}} indicates that: "''Increasing sector size from 512 bytes to 4096 bytes can provide better performance on most of the modern storage devices and also with some hw encryption accelerators''" (see the rest of the description for caveats to be aware of).<br />
<br />
In a comment from 2019[https://lwn.net/Articles/777071/], the following reason is given for why the default was not switched over to 4K sector:<br />
<br />
:# ''Compatibility with old kernels and cryptsetup versions. The 4K encryption sector support is still fairly new, after all.''<br />
:# ''It’s not guaranteed safe on disks with 512-byte sectors, as it can break atomicity guarantees that might be assumed by software. I don’t believe this is a problem on modern disks or flash storage, nor on ext4 or f2fs. But the cryptsetup default needs to be more conservative.''<br />
<br />
To create a LUKS2 container with a 4K sector size and otherwise default options:<br />
<br />
# cryptsetup luksFormat --sector-size 4096 ''device''<br />
<br />
The command will abort on an error if the requested size does not match your device:<br />
<br />
{{bc|# cryptsetup luksFormat --sector-size 4096 ''device''<br />
(...)<br />
Verify passphrase: <br />
Device size is not aligned to requested sector size.<br />
}}<br />
<br />
Afterwards format the mapped container using the same sector size, for example with [[btrfs]]: <br />
<br />
# mkfs.btrfs -s 4096 /dev/mapper/''mapped_device''<br />
<br />
=== Encryption options for plain mode ===<br />
<br />
In dm-crypt ''plain'' mode, there is no master-key on the device, hence, there is no need to set it up. Instead the encryption options to be employed are used directly to create the mapping between an encrypted disk and a named device. The mapping can be created against a partition or a full device. In the latter case not even a partition table is needed. <br />
<br />
To create a ''plain'' mode mapping with cryptsetup's default parameters: <br />
<br />
# cryptsetup <options> open --type plain <device> <dmname><br />
<br />
Executing it will prompt for a password, which should have very high entropy. <br />
Below a comparison of default parameters with the example in [[dm-crypt/Encrypting an entire system#Plain dm-crypt]]<br />
<br />
{| class="wikitable"<br />
! Option !! Cryptsetup 2.1.0 defaults !! Example !! Comment<br />
|-<br />
! style="text-align:right;white-space:nowrap;" | --hash<br />
-h<br />
| {{ic|ripemd160}}<br />
| -<br />
| The hash is used to create the key from the passphrase; it is not used on a keyfile. <br />
|-<br />
! style="text-align:right;white-space:nowrap;" | --cipher<br />
-c<br />
| {{ic|aes-cbc-essiv:sha256}}<br />
| {{ic|aes-xts-plain64}}<br />
| The cipher consists of three parts: cipher-chainmode-IV generator. Please see [[Data-at-rest encryption#Ciphers and modes of operation]] for an explanation of these settings, and the [https://gitlab.com/cryptsetup/cryptsetup/wikis/DMCrypt DMCrypt documentation] for some of the options available. <br />
|-<br />
! style="text-align:right;white-space:nowrap;" | --key-size<br />
-s<br />
|{{ic|256}}<br />
|{{ic|512}}<br />
|The key size (in bits). The size will depend on the cipher being used and also the chainmode in use. Xts mode requires twice the key size of cbc. <br />
|-<br />
! style="text-align:right;white-space:nowrap;" | --size<br />
-b<br />
|real size of target disk<br />
|{{ic|2048}} (mapped device will be 512B×2048=1MiB)<br />
|Limit the maximum size of the device (in 512-byte sectors).<br />
|-<br />
! style="text-align:right;white-space:nowrap;" | --offset<br />
-o<br />
|{{ic|0}}<br />
|{{ic|0}}<br />
|The offset from the beginning of the target disk (in 512-byte sectors) from which to start the mapping.<br />
|-<br />
! style="text-align:right;white-space:nowrap;" | --skip<br />
-p<br />
|{{ic|0}}<br />
|{{ic|2048}} (512B×2048=1MiB will be skipped)<br />
|The number of 512-byte sectors of encrypted data to skip at the beginning.<br />
|-<br />
! style="text-align:right;white-space:nowrap;" | --key-file<br />
-d<br />
| default uses a passphrase<br />
| {{ic|/dev/sd''Z''}} (or e.g. {{ic|/boot/keyfile.enc}})<br />
|The device or file to be used as a key. See [[#Keyfiles]] for further details.<br />
|-<br />
! style="text-align:right;white-space:nowrap;" | --keyfile-offset<br />
| {{ic|0}}<br />
| {{ic|0}}<br />
| Offset from the beginning of the file where the key starts (in bytes). This option is supported from ''cryptsetup'' 1.6.7 onwards. <br />
|-<br />
! style="text-align:right;white-space:nowrap;" | --keyfile-size<br />
-l<br />
|{{ic|8192kB}}<br />
| - (default applies)<br />
| Limits the bytes read from the key file. This option is supported from ''cryptsetup'' 1.6.7 onwards. <br />
|}<br />
<br />
Using the device {{ic|/dev/sd''X''}}, the above right column example results in:<br />
<br />
# cryptsetup --cipher=aes-xts-plain64 --offset=0 --key-file=/dev/sd''Z'' --key-size=512 open --type=plain /dev/sdX enc<br />
<br />
Unlike encrypting with LUKS, the above command must be executed ''in full'' whenever the mapping needs to be re-established, so it is important to remember the cipher, hash and key file details.<br />
We can now check that the mapping has been made:<br />
<br />
# fdisk -l<br />
<br />
An entry should now exist for {{ic|/dev/mapper/enc}}.<br />
<br />
== Encrypting devices with cryptsetup ==<br />
<br />
This section shows how to employ the options for creating new encrypted blockdevices and accessing them manually. <br />
<br />
{{Warning|GRUB does not support LUKS2 headers; see [https://savannah.gnu.org/bugs/?55093 GRUB bug #55093]. Therefore, if you plan to [[GRUB#Encrypted /boot|unlock an encrypted boot partition with GRUB]], specify {{ic|--type luks1}} on encrypted devices that GRUB will need to access.}}<br />
<br />
=== Encrypting devices with LUKS mode ===<br />
<br />
==== Formatting LUKS partitions ====<br />
<br />
In order to setup a partition as an encrypted LUKS partition execute:<br />
<br />
# cryptsetup luksFormat ''device''<br />
<br />
You will then be prompted to enter a password and verify it.<br />
<br />
See [[#Encryption options for LUKS mode]] for command line options.<br />
<br />
You can check the results with:<br />
<br />
# cryptsetup luksDump ''device''<br />
<br />
You will note that the dump not only shows the cipher header information, but also the key-slots in use for the LUKS partition. <br />
<br />
The following example will create an encrypted root partition on {{ic|/dev/sda1}} using the default AES cipher in XTS mode with an effective 256-bit encryption <br />
<br />
# cryptsetup -s 512 luksFormat /dev/sda1<br />
<br />
=====Using LUKS to format partitions with a keyfile=====<br />
<br />
When creating a new LUKS encrypted partition, a keyfile may be associated with the partition on its creation using:<br />
<br />
# cryptsetup luksFormat ''device'' ''/path/to/mykeyfile''<br />
<br />
See [[#Keyfiles]] for instructions on how to generate and manage keyfiles.<br />
<br />
====Unlocking/Mapping LUKS partitions with the device mapper ====<br />
<br />
Once the LUKS partitions have been created, they can then be unlocked.<br />
<br />
The unlocking process will map the partitions to a new device name using the device mapper. This alerts the kernel that {{ic|''device''}} is actually an encrypted device and should be addressed through LUKS using the {{ic|/dev/mapper/''dm_name''}} so as not to overwrite the encrypted data. To guard against accidental overwriting, read about the possibilities to [[#Backup and restore|backup the cryptheader]] after finishing setup.<br />
<br />
In order to open an encrypted LUKS partition execute:<br />
<br />
# cryptsetup open ''device'' ''dm_name''<br />
<br />
You will then be prompted for the password to unlock the partition. Usually the device mapped name is descriptive of the function of the partition that is mapped. For example the following unlocks a luks partition {{ic|/dev/sda1}} and maps it to device mapper named {{ic|cryptroot}}:<br />
<br />
# cryptsetup open /dev/sda1 cryptroot <br />
<br />
Once opened, the root partition device address would be {{ic|/dev/mapper/cryptroot}} instead of the partition (e.g. {{ic|/dev/sda1}}). <br />
<br />
For setting up LVM ontop the encryption layer the device file for the decrypted volume group would be anything like {{ic|/dev/mapper/cryptroot}} instead of {{ic|/dev/sda1}}. LVM will then give additional names to all logical volumes created, e.g. {{ic|/dev/lvmpool/root}} and {{ic|/dev/lvmpool/swap}}.<br />
<br />
In order to write encrypted data into the partition it must be accessed through the device mapped name. The first step of access will typically be to [[File systems#Create a file system|create a filesystem]]. For example:<br />
<br />
# mkfs -t ext4 /dev/mapper/cryptroot<br />
<br />
The device {{ic|/dev/mapper/cryptroot}} can then be [[mount]]ed like any other partition.<br />
<br />
To close the luks container, unmount the partition and do:<br />
<br />
# cryptsetup close cryptroot<br />
<br />
=== Encrypting devices with plain mode ===<br />
<br />
The creation and subsequent access of a ''dm-crypt'' plain mode encryption both require not more than using the ''cryptsetup'' {{ic|open}} action with correct [[#Encryption options for plain mode|parameters]]. The following shows that with two examples of non-root devices, but adds a quirk by stacking both (i.e. the second is created inside the first). Obviously, stacking the encryption doubles overhead. The usecase here is simply to illustrate another example of the cipher option usage. <br />
<br />
A first mapper is created with ''cryptsetup's'' plain-mode defaults, as described in the table's left column above <br />
<br />
{{hc|# cryptsetup --type plain -v open /dev/sdaX plain1|<br />
Enter passphrase: <br />
Command successful.<br />
}}<br />
<br />
Now we add the second blockdevice inside it, using different encryption parameters and with an (optional) offset, create a filesystem and mount it <br />
<br />
{{hc|1=# cryptsetup --type plain --cipher=serpent-xts-plain64 --hash=sha256 --key-size=256 --offset=10 open /dev/mapper/plain1 plain2|2=<br />
Enter passphrase:<br />
}}<br />
<br />
{{hc|# lsblk -p|<br />
NAME <br />
/dev/sda <br />
├─/dev/sdaX <br />
│ └─/dev/mapper/plain1 <br />
│ └─/dev/mapper/plain2 <br />
...<br />
}}<br />
<br />
# mkfs -t ext2 /dev/mapper/plain2<br />
# mount -t ext2 /dev/mapper/plain2 /mnt<br />
# echo "This is stacked. one passphrase per foot to shoot." > /mnt/stacked.txt<br />
<br />
We close the stack to check access works<br />
<br />
# cryptsetup close plain2<br />
# cryptsetup close plain1<br />
<br />
First, let us try to open the filesystem directly: <br />
<br />
# cryptsetup --type plain --cipher=serpent-xts-plain64 --hash=sha256 --key-size=256 --offset=10 open /dev/sdaX plain2<br />
<br />
{{hc|# mount -t ext2 /dev/mapper/plain2 /mnt|<br />
mount: wrong fs type, bad option, bad superblock on /dev/mapper/plain2,<br />
missing codepage or helper program, or other error<br />
}}<br />
<br />
Why that did not work? Because the "plain2" starting block ({{ic|10}}) is still encrypted with the cipher from "plain1". It can only be accessed via the stacked mapper. The error is arbitrary though, trying a wrong passphrase or wrong options will yield the same. For ''dm-crypt'' plain mode, the {{ic|open}} action will not error out itself. <br />
<br />
Trying again in correct order: <br />
<br />
# cryptsetup close plain2 # dysfunctional mapper from previous try<br />
<br />
{{hc|# cryptsetup --type plain open /dev/sdaX plain1|<br />
Enter passphrase:<br />
}}<br />
<br />
{{hc|1=# cryptsetup --type plain --cipher=serpent-xts-plain64 --hash=sha256 --key-size=256 --offset=10 open /dev/mapper/plain1 plain2|2=<br />
Enter passphrase:<br />
}}<br />
<br />
{{hc|# mount /dev/mapper/plain2 /mnt && cat /mnt/stacked.txt|<br />
This is stacked. one passphrase per foot to shoot.<br />
}}<br />
<br />
''dm-crypt'' will handle stacked encryption with some mixed modes too. For example LUKS mode could be stacked on the "plain1" mapper. Its header would then be encrypted inside "plain1" when that is closed.<br />
<br />
Available for plain mode only is the option {{ic|--shared}}. With it a single device can be segmented into different non-overlapping mappers. We do that in the next example, using a ''loopaes'' compatible cipher mode for "plain2" this time: <br />
<br />
{{hc|# cryptsetup --type plain --offset 0 --size 1000 open /dev/sdaX plain1|2=<br />
Enter passphrase:<br />
}}<br />
<br />
{{hc|1=# cryptsetup --type plain --offset 1000 --size 1000 --shared --cipher=aes-cbc-lmk --hash=sha256 open /dev/sdaX plain2|2=<br />
Enter passphrase:<br />
}}<br />
<br />
{{hc|# lsblk -p|<br />
NAME <br />
dev/sdaX <br />
├─/dev/sdaX <br />
│ ├─/dev/mapper/plain1 <br />
│ └─/dev/mapper/plain2 <br />
...<br />
}}<br />
<br />
As the devicetree shows both reside on the same level, i.e. are not stacked and "plain2" can be opened individually.<br />
<br />
== Cryptsetup actions specific for LUKS ==<br />
<br />
=== Key management ===<br />
<br />
It is possible to define addition keys for the LUKS partition. This enables the user to create access keys for safe backup storage In so-called key escrow, one key is used for daily usage, another kept in escrow to gain access to the partition in case the daily passphrase is forgotten or a keyfile is lost/damaged. A different key-slot could also be used to grant access to a partition to a user by issuing a second key and later revoking it again. <br />
<br />
Once an encrypted partition has been created, the initial keyslot 0 is created (if no other was specified manually). Additional keyslots are numbered from 1 to 7. Which keyslots are used can be seen by issuing <br />
<br />
# cryptsetup luksDump /dev/<device><br />
<br />
Where <device> is the volume containing the LUKS header. This and all the following commands in this section work on header backup files as well. <br />
<br />
==== Adding LUKS keys ====<br />
<br />
Adding new keyslots is accomplished using cryptsetup with the {{ic|luksAddKey}} action. For safety it will always, i.e. also for already unlocked devices, ask for a valid existing key ("any passphrase") before a new one may be entered:<br />
<br />
{{hc|# cryptsetup luksAddKey /dev/<device> (/path/to/<additionalkeyfile>)|<br />
Enter any passphrase:<br />
Enter new passphrase for key slot:<br />
Verify passphrase: <br />
}}<br />
<br />
If {{ic|/path/to/<additionalkeyfile>}} is given, cryptsetup will add a new keyslot for <additionalkeyfile>. Otherwise a new passphrase will be prompted for twice. For using an existing ''keyfile'' to authorize the action, the {{ic|--key-file}} or {{ic|-d}} option followed by the "old" <keyfile> will try to unlock all available keyfile keyslots:<br />
<br />
# cryptsetup luksAddKey /dev/<device> (/path/to/<additionalkeyfile>) -d /path/to/<keyfile><br />
<br />
If it is intended to use multiple keys and change or revoke them, the {{ic|--key-slot}} or {{ic|-S}} option may be used to specify the slot: <br />
<br />
{{hc|# cryptsetup luksAddKey /dev/<device> -S 6|<br />
Enter any passphrase: <br />
Enter new passphrase for key slot: <br />
Verify passphrase:<br />
}}<br />
<br />
{{hc|# cryptsetup luksDump /dev/sda8 {{!}} grep 'Slot 6'|<br />
Key Slot 6: ENABLED<br />
}}<br />
<br />
To show an associated action in this example, we decide to change the key right away: <br />
<br />
{{hc|# cryptsetup luksChangeKey /dev/<device> -S 6|<br />
Enter LUKS passphrase to be changed: <br />
Enter new LUKS passphrase:<br />
}}<br />
<br />
before continuing to remove it.<br />
<br />
==== Removing LUKS keys ====<br />
<br />
There are three different actions to remove keys from the header: <br />
<br />
* {{ic|luksRemoveKey}} is used to remove a key by specifying its passphrase/key-file. <br />
* {{ic|luksKillSlot}} may be used to remove a key from a specific key slot (using another key). Obviously, this is extremely useful if you have forgotten a passphrase, lost a key-file, or have no access to it. <br />
* {{ic|luksErase}} is used to quickly remove '''all''' active keys. <br />
<br />
{{warning|<br />
* All above actions can be used to irrevocably delete the last active key for an encrypted device! <br />
* The {{ic|luksErase}} command was added in version 1.6.4 to quickly nuke access to the device. This action '''will not''' prompt for a valid passphrase! It will not [[dm-crypt/Drive preparation#Wipe LUKS header|wipe the LUKS header]], but all keyslots at once and you will, therefore, not be able to regain access unless you have a valid backup of the LUKS header.<br />
}}<br />
<br />
For above warning it is good to know the key we want to '''keep''' is valid. An easy check is to unlock the device with the {{ic|-v}} option, which will specify which slot it occupies: <br />
<br />
{{hc|# cryptsetup -v open /dev/<device> testcrypt|<br />
Enter passphrase for /dev/<device>: <br />
Key slot 1 unlocked.<br />
Command successful.<br />
}}<br />
<br />
Now we can remove the key added in the previous subsection using its passphrase: <br />
<br />
{{hc|# cryptsetup luksRemoveKey /dev/<device>|<br />
Enter LUKS passphrase to be deleted:<br />
}}<br />
<br />
If we had used the same passphrase for two keyslots, the first slot would be wiped now. Only executing it again would remove the second one. <br />
<br />
Alternatively, we can specify the key slot: <br />
<br />
{{hc|# cryptsetup luksKillSlot /dev/<device> 6|<br />
Enter any remaining LUKS passphrase:<br />
}}<br />
<br />
Note that in both cases, no confirmation was required.<br />
<br />
{{hc|# cryptsetup luksDump /dev/sda8 {{!}} grep 'Slot 6'|<br />
Key Slot 6: DISABLED<br />
}}<br />
<br />
To re-iterate the warning above: If the same passphrase had been used for key slots 1 and 6, both would be gone now.<br />
<br />
=== Backup and restore ===<br />
<br />
If the header of a LUKS encrypted partition gets destroyed, you will not be able to decrypt your data. It is just as much of a dilemma as forgetting the passphrase or damaging a key-file used to unlock the partition. Damage may occur by your own fault while re-partitioning the disk later or by third-party programs misinterpreting the partition table. Therefore, having a backup of the header and storing it on another disk might be a good idea.<br />
<br />
{{Note|If one of the LUKS-encrypted partitions' passphrases becomes compromised, you must revoke it on ''every'' copy of the cryptheader, even those you have backed up. Otherwise, a copy of the backed-up cryptheader that uses the compromised passphrase can be used to determine the master key which in turn can be used to decrypt the associated partition (even your actual partition, not only the backed-up version). On the other hand, if the master key gets compromised, you have to reencrypt your whole partition. See [https://gitlab.com/cryptsetup/cryptsetup/wikis/FrequentlyAskedQuestions#6-backup-and-data-recovery LUKS FAQ] for further details.}}<br />
<br />
==== Backup using cryptsetup ====<br />
<br />
Cryptsetup's {{ic|luksHeaderBackup}} action stores a binary backup of the LUKS header and keyslot area:<br />
<br />
# cryptsetup luksHeaderBackup /dev/<device> --header-backup-file /mnt/<backup>/<file>.img<br />
<br />
where <device> is the partition containing the LUKS volume.<br />
<br />
You can also back up the plaintext header into ramfs and encrypt it with e.g. [[GPG]] before writing it to persistent storage:<br />
<br />
# mkdir /root/<tmp>/<br />
# mount ramfs /root/<tmp>/ -t ramfs<br />
# cryptsetup luksHeaderBackup /dev/<device> --header-backup-file /root/<tmp>/<file>.img<br />
# gpg2 --recipient <User ID> --encrypt /root/<tmp>/<file>.img <br />
# cp /root/<tmp>/<file>.img.gpg /mnt/<backup>/<br />
# umount /root/<tmp><br />
<br />
{{Warning|Tmpfs can swap to harddisk if low on memory so it is not recommended here.}}<br />
<br />
==== Restore using cryptsetup ====<br />
<br />
{{Warning|Restoring the wrong header or restoring to an unencrypted partition will cause data loss! The action can not perform a check whether the header is actually the ''correct'' one for that particular device.}} <br />
<br />
In order to evade restoring a wrong header, you can ensure it does work by using it as a remote {{ic|--header}} first: <br />
<br />
{{hc|# cryptsetup -v --header /mnt/<backup>/<file>.img open /dev/<device> test|<br />
Key slot 0 unlocked.<br />
Command successful.<br />
}}<br />
<br />
# mount /dev/mapper/test /mnt/test && ls /mnt/test <br />
# umount /mnt/test <br />
# cryptsetup close test <br />
<br />
Now that the check succeeded, the restore may be performed: <br />
<br />
# cryptsetup luksHeaderRestore /dev/<device> --header-backup-file ./mnt/<backup>/<file>.img<br />
<br />
Now that all the keyslot areas are overwritten; only active keyslots from the backup file are available after issuing the command.<br />
<br />
==== Manual backup and restore ====<br />
<br />
The header always resides at the beginning of the device and a backup can be performed without access to ''cryptsetup'' as well. First you have to find out the payload offset of the crypted partition:<br />
<br />
{{hc|# cryptsetup luksDump /dev/<device> {{!}} grep "Payload offset"|<br />
Payload offset: 4040<br />
}}<br />
<br />
Second check the sector size of the drive<br />
<br />
{{hc|# fdisk -l /dev/<device> {{!}} grep "Sector size"|<br />
Sector size (logical/physical): 512 bytes / 512 bytes<br />
}}<br />
<br />
Now that you know the values, you can backup the header with a simple [[dd]] command:<br />
<br />
# dd if=/dev/<device> of=/path/to/<file>.img bs=512 count=4040<br />
<br />
and store it safely.<br />
<br />
A restore can then be performed using the same values as when backing up:<br />
<br />
# dd if=./<file>.img of=/dev/<device> bs=512 count=4040<br />
<br />
=== Re-encrypting devices ===<br />
<br />
{{Expansion|cryptsetup 2.2 using LUKS2 (with a 16 MiB header) supports online encryption/decryption/reencryption.[https://mirrors.edge.kernel.org/pub/linux/utils/cryptsetup/v2.2/v2.2.0-ReleaseNotes]}}<br />
<br />
The {{Pkg|cryptsetup}} package features two options for re-encryption.<br />
<br />
; cryptsetup reencrypt: Argument to {{ic|cryptsetup}} itself: Preferred method. Currently LUKS2 devices only. Actions can be performed online. Supports multiple parallel re-encryption jobs. Resilient to system failures. See {{man|8|cryptsetup}} for more information.<br />
<br />
; cryptsetup-reencrypt: Legacy tool, supports LUKS1 in addition to LUKS2. Actions can be performed on unmounted devices only. Single process at a time. Sensitive to system failures. See {{man|8|cryptsetup-reencrypt}} for more information. <br />
<br />
Both can be used to convert an existing unencrypted filesystem to a LUKS encrypted one or permanently remove LUKS encryption from a device (using {{ic|--decrypt}}). As its name suggests it can also be used to re-encrypt an existing LUKS encrypted device, though, re-encryption is not possible for a detached LUKS header or other encryption modes (e.g. plain-mode). For re-encryption it is possible to change the [[#Encryption options for LUKS mode]]. <br />
<br />
One application of re-encryption may be to secure the data again after a passphrase or [[#Keyfiles|keyfile]] has been compromised ''and'' one cannot be certain that no copy of the LUKS header has been obtained. For example, if only a passphrase has been shoulder-surfed but no physical/logical access to the device happened, it would be enough to change the respective passphrase/key only ([[#Key management]]). <br />
<br />
{{Warning|Always make sure a '''reliable backup''' is available and double-check options you specify before using the tool!}}<br />
<br />
The following shows an example to encrypt an unencrypted filesystem partition and a re-encryption of an existing LUKS device. <br />
<br />
==== Encrypt an existing unencrypted filesystem ====<br />
<br />
A LUKS encryption header is always stored at the beginning of the device. Since an existing filesystem will usually be allocated all partition sectors, the first step is to shrink it to make space for the LUKS header.<br />
<br />
{{Expansion|cryptsetup man pages suggest using twice the LUKS2 header size. That implies 32MiB and using {{ic|--reduce-device-size 32M}}}}<br />
The [[#Encryption options for LUKS mode|default]] LUKS2 header requires 16 MiB. If the current filesystem occupies all the available space, we'll have to shrink it at least that much.<br />
To shrink an existing {{ic|ext4}} filesystem on {{ic|/dev/sdaX}} to its current possible minimum:<br />
<br />
# umount /mnt<br />
<br />
{{hc|# e2fsck -f /dev/sdaX|<br />
e2fsck 1.43-WIP (18-May-2015)<br />
Pass 1: Checking inodes, blocks, and sizes<br />
...<br />
/dev/sda6: 12/166320 files (0.0% non-contiguous), 28783/665062 blocks<br />
}}<br />
<br />
{{hc|# resize2fs -p -M /dev/sdaX|<br />
resize2fs 1.43-WIP (18-May-2015)<br />
Resizing the filesystem on /dev/sdaX to 26347 (4k) blocks.<br />
The filesystem on /dev/sdaX is now 26347 (4k) blocks long.<br />
}}<br />
<br />
Note that shrinking to the minimum size might take very long. You might want to calculate a size just 32MiB smaller than the current size instead of using -M.<br />
<br />
Now we encrypt it, using the default cipher we do not have to specify it explicitly:<br />
{{hc|# cryptsetup reencrypt --encrypt --reduce-device-size 16M /dev/sdaX|<nowiki><br />
<br />
WARNING!<br />
========<br />
This will overwrite data on LUKS2-temp-12345678-9012-3456-7890-123456789012.new irrevocably.<br />
<br />
Are you sure? (Type 'yes' in capital letters): YES<br />
Enter passphrase for LUKS2-temp-12345678-9012-3456-7890-123456789012.new: <br />
Verify passphrase: <br />
</nowiki>}}<br />
<br />
After it finished, the whole {{ic|/dev/sdaX}} partition is encrypted, not only the space the filesystem was shrunk to. As a final step we extend the original {{ic|ext4}} filesystem to occupy all available space again, on the now encrypted partition:<br />
<br />
{{hc|# cryptsetup open /dev/sdaX recrypt|<br />
Enter passphrase for /dev/sdaX: <br />
...<br />
}}<br />
<br />
{{hc|# resize2fs /dev/mapper/recrypt|<br />
resize2fs 1.43-WIP (18-May-2015)<br />
Resizing the filesystem on /dev/mapper/recrypt to 664807 (4k) blocks.<br />
The filesystem on /dev/mapper/recrypt is now 664807 (4k) blocks long.<br />
}}<br />
<br />
# mount /dev/mapper/recrypt /mnt<br />
<br />
The filesystem is now ready to use. You may want to add it to your [[crypttab]].<br />
<br />
==== Re-encrypting an existing LUKS partition ====<br />
<br />
In this example an existing LUKS device is re-encrypted. <br />
<br />
{{Warning|Double-check you specify encryption options for correctly and ''never'' re-encrypt without a '''reliable backup'''!}}<br />
<br />
In order to re-encrypt a device with its existing encryption options, they do not need to be specified:<br />
{{bc|# cryptsetup reencrypt /dev/sdaX}}<br />
{{Note|For LUKS1 we'll need to use the legacy tool:{{bc|# cryptsetup-reencrypt /dev/sdaX}}}}<br />
<br />
Existing keys are retained when re-encrypting a device with a different cipher and/or hash. Another use case is to re-encrypt LUKS devices which have non-current encryption options. Apart from above warning on specifying options correctly, the ability to change the LUKS header may also be limited by its size. For example, if the device was initially encrypted using a CBC mode cipher and 128 bit key-size, the LUKS header will be half the size of above mentioned {{ic|4096}} sectors: <br />
<br />
{{hc|# cryptsetup luksDump /dev/sdaX <nowiki>|</nowiki>grep -e "mode" -e "Payload" -e "MK bits"|<br />
Cipher mode: cbc-essiv:sha256<br />
Payload offset: '''2048'''<br />
MK bits: 128<br />
}}<br />
<br />
While it is possible to upgrade the encryption of such a device, it is currently only feasible in two steps. First, re-encrypting with the same encryption options, but using the {{ic|--reduce-device-size}} option to make further space for the larger LUKS header. Second, re-encypt the whole device again with the desired cipher. For this reason and the fact that a backup should be created in any case, creating a new, fresh encrypted device to restore into is always the faster option.<br />
<br />
== Resizing encrypted devices ==<br />
<br />
{{Expansion|This section should be rewritten to introduce resizing more generically. Perhaps work on it together with [[Resizing LVM-on-LUKS]].}}<br />
<br />
If a storage device encrypted with dm-crypt is being cloned (with a tool like dd) to another larger device, the underlying dm-crypt device must be resized to use the whole space. <br />
<br />
The destination device is /dev/sdX2 in this example, the whole available space adjacent to the partition will be used:<br />
<br />
# cryptsetup luksOpen /dev/sdX2 sdX2<br />
# cryptsetup resize sdX2<br />
<br />
Then the underlying filesystem must be resized.<br />
<br />
=== Loopback filesystem ===<br />
<br />
Assume that an encrypted loopback filesystem is stored in a file {{ic|/bigsecret}}, looped to {{ic|/dev/loop0}}, mapped to {{ic|secret}} and mounted on {{ic|/mnt/secret}}, as in the example at [[dm-crypt/Encrypting a non-root file system#File container]].<br />
<br />
If the container file is currently mapped and/or mounted, unmount and/or close it:<br />
<br />
# umount /mnt/secret<br />
# cryptsetup close secret<br />
# losetup -d /dev/loop0<br />
<br />
Next, expand the container file with the size of the data you want to add. In this example, the file will be expanded with 1M * 1024, which is 1G.<br />
<br />
{{Warning|Make absolutely sure to use '''two''' {{ic|>}}, instead of just one, or else you will overwrite the file instead of appending to it. Making a backup before this step is strongly recommended.}}<br />
<br />
# dd if=/dev/urandom bs=1M count=1024 | cat - >> /bigsecret<br />
<br />
Now map the container to the loop device:<br />
<br />
# losetup /dev/loop0 /bigsecret<br />
# cryptsetup open /dev/loop0 secret<br />
<br />
After this, resize the encrypted part of the container to the new maximum size of the container file:<br />
<br />
# cryptsetup resize secret<br />
<br />
Finally, perform a filesystem check and, if it is ok, resize it (example for ext2/3/4):<br />
<br />
# e2fsck -f /dev/mapper/secret<br />
# resize2fs /dev/mapper/secret<br />
<br />
You can now mount the container again:<br />
<br />
# mount /dev/mapper/secret /mnt/secret<br />
<br />
=== Integrity protected device ===<br />
<br />
If the device was formatted with integrity support (e.g., {{ic|--integrity hmac-sha256}}) and the backing block device is shrinked, it cannot be opened with this error: {{ic|device-mapper: reload ioctl on failed: Invalid argument}}.<br />
<br />
To fix this issue without wiping the device again, it can be formatted with the previous master key (keeping the per-sector tags valid).<br />
<br />
# cryptsetup luksDump /dev/sdX2 --dump-master-key --master-key-file=/tmp/masterkey-in-tmpfs.key<br />
# cryptsetup luksFormat /dev/sdX2 --type luks2 --integrity hmac-sha256 --master-key-file=/tmp/masterkey-in-tmpfs.key --integrity-no-wipe<br />
# rm /tmp/masterkey-in-tmpfs.key<br />
<br />
== Keyfiles ==<br />
<br />
{{Note|This section describes using a plaintext keyfile. If you want to encrypt your keyfile giving you two factor authentication see [[dm-crypt/Specialties#Using GPG, LUKS, or OpenSSL Encrypted Keyfiles|Using GPG or OpenSSL Encrypted Keyfiles]] for details, but please still read this section.}}<br />
<br />
'''What is a keyfile?'''<br />
<br />
A keyfile is a file whose data is used as the passphrase to unlock an encrypted volume.<br />
That means if such a file is lost or changed, decrypting the volume may no longer be possible.<br />
<br />
{{Tip|Define a passphrase in addition to the keyfile for backup access to encrypted volumes in the event the defined keyfile is lost or changed.}}<br />
<br />
'''Why use a keyfile?'''<br />
<br />
There are many kinds of keyfiles. Each type of keyfile used has benefits and disadvantages summarized below:<br />
<br />
=== Types of keyfiles ===<br />
<br />
==== passphrase ====<br />
<br />
This is a keyfile containing a simple passphrase. The benefit of this type of keyfile is that if the file is lost the data it contained is known and hopefully easily remembered by the owner of the encrypted volume. However the disadvantage is that this does not add any security over entering a passphrase during the initial system start.<br />
<br />
Example: {{ic|1234}}<br />
<br />
{{Note|The keyfile containing the passphrase must not have a newline in it. One option is to create it using <br />
<br />
# echo -n 'your_passphrase' > /path/to/<keyfile><br />
# chown root:root /path/to/<keyfile>; chmod 400 /path/to/<keyfile><br />
<br />
If the file contains special characters such as a backslash, rather than escaping these, it is recommended to simply edit the key file directly entering or pasting the passphrase and then remove the trailing newline with a handy perl one-liner:<br />
<br />
# perl -pi -e 'chomp if eof' /path/to/<keyfile><br />
}}<br />
<br />
==== randomtext ====<br />
<br />
This is a keyfile containing a block of random characters. The benefit of this type of keyfile is that it is much more resistant to dictionary attacks than a simple passphrase. An additional strength of keyfiles can be utilized in this situation which is the length of data used. Since this is not a string meant to be memorized by a person for entry, it is trivial to create files containing thousands of random characters as the key. The disadvantage is that if this file is lost or changed, it will most likely not be possible to access the encrypted volume without a backup passphrase.<br />
<br />
Example: {{ic|fjqweifj830149-57 819y4my1-38t1934yt8-91m 34co3;t8y;9p3y-}}<br />
<br />
==== binary ====<br />
<br />
This is a binary file that has been defined as a keyfile. When identifying files as candidates for a keyfile, it is recommended to choose files that are relatively static such as photos, music, video clips. The benefit of these files is that they serve a dual function which can make them harder to identify as keyfiles. Instead of having a text file with a large amount of random text, the keyfile would look like a regular image file or music clip to the casual observer. The disadvantage is that if this file is lost or changed, it will most likely not be possible to access the encrypted volume without a backup passphrase. Additionally, there is a theoretical loss of randomness when compared to a randomly generated text file. This is due to the fact that images, videos and music have some intrinsic relationship between neighboring bits of data that does not exist for a random text file. However this is controversial and has never been exploited publicly.<br />
<br />
Example: images, text, video, ...<br />
<br />
=== Creating a keyfile with random characters ===<br />
<br />
==== Storing the keyfile on a filesystem ====<br />
<br />
A keyfile can be of arbitrary content and size. <br />
<br />
Here [[dd]] is used to generate a keyfile of 2048 random bytes, storing it in the file {{ic|/etc/mykeyfile}}:<br />
<br />
# dd bs=512 count=4 if=/dev/random of=/etc/mykeyfile iflag=fullblock<br />
<br />
If you are planning to store the keyfile on an external device, you can also simply change the outputfile to the corresponding directory:<br />
<br />
# dd bs=512 count=4 if=/dev/random of=/media/usbstick/mykeyfile iflag=fullblock<br />
<br />
To deny any access for other users than {{ic|root}}:<br />
<br />
# chmod 600 /etc/mykeyfile<br />
<br />
===== Securely overwriting stored keyfiles =====<br />
<br />
If you stored your temporary keyfile on a physical storage device, and want to delete it, remember to not just remove the keyfile later on, but use something like<br />
<br />
# shred --remove --zero mykeyfile<br />
<br />
to securely overwrite it. For overaged filesystems like FAT or ext2 this will suffice while in the case of journaling filesystems, flash memory hardware and other cases it is highly recommended to [[Securely wipe disk|wipe the entire device]].<br />
<br />
==== Storing the keyfile in ramfs ====<br />
<br />
Alternatively, you can mount a ramfs for storing the keyfile temporarily:<br />
<br />
# mkdir /root/myramfs<br />
# mount ramfs /root/myramfs/ -t ramfs<br />
# cd /root/myramfs<br />
<br />
The advantage is that it resides in RAM and not on a physical disk, therefore it can not be recovered after unmounting the ramfs. After copying the keyfile to another secure and persistent filesystem, unmount the ramfs again with<br />
<br />
# umount /root/myramfs<br />
<br />
=== Configuring LUKS to make use of the keyfile ===<br />
<br />
Add a keyslot for the keyfile to the LUKS header:<br />
<br />
{{hc|# cryptsetup luksAddKey /dev/sda2 /etc/mykeyfile|<br />
Enter any LUKS passphrase:<br />
key slot 0 unlocked.<br />
Command successful.<br />
}}<br />
<br />
=== Manually unlocking a partition using a keyfile ===<br />
<br />
Use the {{ic|--key-file}} option when opening the LUKS device:<br />
<br />
# cryptsetup open /dev/sda2 ''dm_name'' --key-file /etc/mykeyfile<br />
<br />
=== Unlocking the root partition at boot ===<br />
<br />
This is simply a matter of configuring [[mkinitcpio]] to include the necessary modules or files and configuring the [[dm-crypt/System configuration#cryptkey|cryptkey]] [[kernel parameter]] to know where to find the keyfile.<br />
<br />
Two cases are covered below:<br />
<br />
# Using a keyfile stored on an external medium (e.g. a USB stick)<br />
# Using a keyfile embedded in the initramfs <br />
<br />
==== With a keyfile stored on an external media ====<br />
<br />
===== Configuring mkinitcpio =====<br />
<br />
You have to add the kernel module for the drive's [[file system]] to the [[mkinitcpio#MODULES|MODULES array]] in {{ic|/etc/mkinitcpio.conf}}. For example, add {{ic|ext4}} if the file system is [[Ext4]] or {{ic|vfat}} in case it is [[FAT]]:<br />
<br />
MODULES=(vfat)<br />
<br />
If there are messages about bad superblock and bad codepage at boot, then you need an extra codepage module to be loaded. For instance, you may need {{ic|nls_iso8859-1}} module for {{ic|iso8859-1}} codepage.<br />
<br />
[[Regenerate the initramfs]].<br />
<br />
===== Configuring the kernel parameters =====<br />
<br />
* For a busybox-based initramfs using the [[dm-crypt/System configuration#Using encrypt hook|encrypt]] hook, see [[dm-crypt/System configuration#cryptkey]].<br />
* For a systemd based initramfs using the [[sd-encrypt]] hook, see [[dm-crypt/System configuration#rd.luks.key]].<br />
<br />
==== With a keyfile embedded in the initramfs ====<br />
<br />
{{Expansion|Add method for [[sd-encrypt]].}}<br />
<br />
{{Warning|Use an embedded keyfile '''only''' if you protect the keyfile sufficiently by:<br />
* Using some form of authentication earlier in the boot process. Otherwise auto-decryption will occur, defeating completely the purpose of block device encryption.<br />
* {{ic|/boot}} is encrypted. Otherwise root on a different installation (including the [[Installation guide#Boot the live environment|live environment]]) can extract your key from the initramfs, and unlock the device without any other authentication.}}<br />
<br />
This method allows to use a specially named keyfile that will be embedded in the [[initramfs]] and picked up by the {{ic|encrypt}} [[Mkinitcpio#HOOKS|hook]] to unlock the root filesystem ({{ic|cryptdevice}}) automatically. It may be useful to apply when using the [[GRUB#Encrypted /boot|GRUB early cryptodisk]] feature, in order to avoid entering two passphrases during boot.<br />
<br />
The {{ic|encrypt}} hook lets the user specify a keyfile with the {{ic|cryptkey}} kernel parameter: in the case of initramfs, the syntax is {{ic|rootfs:''path''}}. See [[dm-crypt/System configuration#cryptkey]]. Besides, this kernel parameter defaults to use {{ic|/crypto_keyfile.bin}}, and if the initramfs contains a valid key with this name, decryption will occur automatically without the need to configure the {{ic|cryptkey}} parameter.<br />
<br />
If using {{ic|sd-encrypt}} instead of {{ic|encrypt}}, specify the location of the keyfile with the {{ic|rd.luks.key}} kernel parameter. See [[dm-crypt/System configuration#rd.luks.key]].<br />
<br />
[[#Creating a keyfile with random characters|Generate the keyfile]], give it suitable permissions and [[#Adding LUKS keys|add it as a LUKS key]]:<br />
<br />
# dd bs=512 count=4 if=/dev/random of=/crypto_keyfile.bin iflag=fullblock<br />
# chmod 600 /crypto_keyfile.bin<br />
# chmod 600 /boot/initramfs-linux*<br />
# cryptsetup luksAddKey /dev/sdX# /crypto_keyfile.bin<br />
<br />
{{Warning|When initramfs' permissions are set to 644 (by default), then all users will be able to dump the keyfile. Make sure the permissions are still 600 if you install a new kernel.}}<br />
<br />
Include the key in [[mkinitcpio#BINARIES and FILES|mkinitcpio's FILES array]]:<br />
<br />
{{hc|/etc/mkinitcpio.conf|2=<br />
FILES=(/crypto_keyfile.bin)<br />
}}<br />
<br />
Finally [[regenerate the initramfs]].<br />
<br />
On the next reboot you should only have to enter your container decryption passphrase once.<br />
<br />
([http://www.pavelkogan.com/2014/05/23/luks-full-disk-encryption/#bonus-login-once source])</div>Phireskyhttps://wiki.archlinux.org/index.php?title=Dm-crypt/Device_encryption&diff=645894Dm-crypt/Device encryption2020-12-18T20:32:38Z<p>Phiresky: add progress flag, and add note that resizing to minimum takes very long</p>
<hr />
<div>{{Lowercase title}}<br />
[[Category:Data-at-rest encryption]]<br />
[[es:Dm-crypt (Español)/Device encryption]]<br />
[[ja:Dm-crypt/デバイスの暗号化]]<br />
[[pl:Dm-crypt (Polski)/Device encryption]]<br />
[[pt:Dm-crypt (Português)/Device encryption]]<br />
This section covers how to manually utilize ''dm-crypt'' from the command line to encrypt a system. <br />
<br />
== Preparation ==<br />
<br />
Before using {{pkg|cryptsetup}}, always make sure the {{ic|dm_crypt}} [[kernel module]] is loaded.<br />
<br />
== Cryptsetup usage ==<br />
<br />
''Cryptsetup'' is the command line tool to interface with ''dm-crypt'' for creating, accessing and managing encrypted devices. The tool was later expanded to support different encryption types that rely on the Linux kernel '''d'''evice-'''m'''apper and the '''crypt'''ographic modules. The most notable expansion was for the Linux Unified Key Setup (LUKS) extension, which stores all of the needed setup information for dm-crypt on the disk itself and abstracts partition and key management in an attempt to improve ease of use. Devices accessed via the device-mapper are called blockdevices. For further information see [[Data-at-rest encryption#Block device encryption]]. <br />
<br />
The tool is used as follows: <br />
<br />
# cryptsetup <OPTIONS> <action> <action-specific-options> <device> <dmname><br />
<br />
It has compiled-in defaults for the options and the encryption mode, which will be used if no others are specified on the command line. Have a look at <br />
<br />
$ cryptsetup --help <br />
<br />
which lists options, actions and the default parameters for the encryption modes in that order. A full list of options can be found on the man page.<br />
Since different parameters are required or optional, depending on encryption mode and action, the following sections point out differences further. Blockdevice encryption is fast, but speed matters a lot too. Since changing an encryption cipher of a blockdevice after setup is difficult, it is important to check ''dm-crypt'' performance for the individual parameters in advance: <br />
<br />
$ cryptsetup benchmark <br />
<br />
can give guidance on deciding for an algorithm and key-size prior to installation. If certain AES ciphers excel with a considerable higher throughput, these are probably the ones with hardware support in the CPU.<br />
<br />
{{Tip|You may want to practise encrypting a virtual hard drive in a [[virtual machine]] when learning.}}<br />
<br />
=== Cryptsetup passphrases and keys ===<br />
<br />
An encrypted blockdevice is protected by a key. A key is either: <br />
<br />
* a passphrase: see [[Security#Passwords]].<br />
* a keyfile, see [[#Keyfiles]].<br />
<br />
Both key types have default maximum sizes: passphrases can be up to 512 characters and keyfiles up to 8192kiB. <br />
<br />
An important distinction of ''LUKS'' to note at this point is that the key is used to unlock the master-key of a LUKS-encrypted device and can be changed with root access. Other encryption modes do not support changing the key after setup, because they do not employ a master-key for the encryption. See [[Data-at-rest encryption#Block device encryption]] for details.<br />
<br />
== Encryption options with dm-crypt ==<br />
<br />
''Cryptsetup'' supports different encryption operating modes to use with ''dm-crypt'': <br />
<br />
* {{ic|--type luks}} for using the default LUKS format version (LUKS1 with {{Pkg|cryptsetup}} < 2.1.0, LUKS2 with {{Pkg|cryptsetup}} ≥ 2.1.0),<br />
* {{ic|--type luks1}} for using LUKS1, the most common version of LUKS,<br />
* {{ic|--type luks2}} for using LUKS2, the latest available version of LUKS that allows additional extensions,<br />
* {{ic|--type plain}} for using dm-crypt plain mode, <br />
* {{ic|--type loopaes}} for a loopaes legacy mode, <br />
* {{ic|--type tcrypt}} for a [[TrueCrypt]] compatibility mode.<br />
<br />
The basic cryptographic options for encryption cipher and hashes available can be used for all modes and rely on the kernel cryptographic backend features. All that are loaded and available to use as options at runtime can be viewed with:<br />
<br />
$ less /proc/crypto <br />
<br />
{{Tip|If the list is short, execute {{ic|$ cryptsetup benchmark}} which will trigger loading available modules.}}<br />
<br />
The following introduces encryption options for the {{ic|luks}}, {{ic|luks1}}, {{ic|luks2}} and {{ic|plain}} modes. Note that the tables list options used in the respective examples in this article and not all available ones. <br />
<br />
=== Encryption options for LUKS mode ===<br />
<br />
The ''cryptsetup'' action to set up a new dm-crypt device in LUKS encryption mode is ''luksFormat''. Unlike the name implies, it does not format the device, but sets up the LUKS device header and encrypts the master-key with the desired cryptographic options. <br />
<br />
In order to create a new LUKS container with the compiled-in defaults listed by {{ic|cryptsetup --help}}, simply execute:<br />
<br />
# cryptsetup luksFormat ''device''<br />
<br />
As of cryptsetup 2.3.4, this is equivalent to: <br />
<br />
# cryptsetup --type luks2 --cipher aes-xts-plain64 --hash sha256 --iter-time 2000 --key-size 256 --pbkdf argon2i --sector-size 512 --use-urandom --verify-passphrase luksFormat ''device''<br />
<br />
Defaults are compared with a cryptographically higher specification example in the table below, with accompanying comments: <br />
<br />
{| class="wikitable"<br />
! Options !! Cryptsetup 2.1.0 defaults !! Example !! Comment<br />
|-<br />
! style="text-align:right;white-space:nowrap;" | --cipher<br />
-c<br />
| {{ic|aes-xts-plain64}}<br />
| {{ic|aes-xts-plain64}}<br />
| [https://www.kernel.org/pub/linux/utils/cryptsetup/v1.6/v1.6.0-ReleaseNotes Release 1.6.0] changed the defaults to an AES [[Data-at-rest encryption#Ciphers and modes of operation|cipher]] in [[wikipedia:Disk encryption theory#XEX-based tweaked-codebook mode with ciphertext stealing (XTS)|XTS]] mode (see item 5.16 [https://gitlab.com/cryptsetup/cryptsetup/wikis/FrequentlyAskedQuestions#5-security-aspects of the FAQ]). It is advised against using the previous default {{ic|--cipher aes-cbc-essiv}} because of its known [[wikipedia:Disk encryption theory#Cipher-block chaining (CBC)|issues]] and practical [http://www.jakoblell.com/blog/2013/12/22/practical-malleability-attack-against-cbc-encrypted-luks-partitions/ attacks] against them.<br />
|-<br />
! style="text-align:right;white-space:nowrap;" | --key-size<br />
-s<br />
| {{ic|256}} ({{ic|512}} for XTS)<br />
| {{ic|512}}<br />
| By default a 512 bit key-size is used for XTS ciphers. Note however that [[wikipedia:Disk encryption theory#XEX-based tweaked-codebook mode with ciphertext stealing (XTS)|XTS splits the supplied key in half]], so this results in AES-256 being used.<br />
|-<br />
! style="text-align:right;white-space:nowrap;" | --hash<br />
-h<br />
| {{ic|sha256}}<br />
| {{ic|sha512}}<br />
| Hash algorithm used for [[Data-at-rest encryption#Cryptographic metadata|key derivation]]. Release 1.7.0 changed defaults from {{ic|sha1}} to {{ic|sha256}} "''not for security reasons [but] mainly to prevent compatibility problems on hardened systems where SHA1 is already [being] phased out''"[https://www.kernel.org/pub/linux/utils/cryptsetup/v1.7/v1.7.0-ReleaseNotes]. The former default of {{ic|sha1}} can still be used for compatibility with older versions of ''cryptsetup'' since it is [https://gitlab.com/cryptsetup/cryptsetup/wikis/FrequentlyAskedQuestions#5-security-aspects considered secure] (see item 5.20). <br />
|-<br />
! style="text-align:right;white-space:nowrap;" | --iter-time<br />
-i<br />
| {{ic|2000}}<br />
| {{ic|5000}}<br />
| Number of milliseconds to spend with PBKDF2 passphrase processing. Release 1.7.0 changed defaults from {{ic|1000}} to {{ic|2000}} to "''try to keep PBKDF2 iteration count still high enough and also still acceptable for users.''"[https://www.kernel.org/pub/linux/utils/cryptsetup/v1.7/v1.7.0-ReleaseNotes]. This option is only relevant for LUKS operations that set or change passphrases, such as ''luksFormat'' or ''luksAddKey''. Specifying 0 as parameter selects the compiled-in default..<br />
|-<br />
! style="text-align:right;white-space:nowrap;" | --use-{u,}random<br />
| {{ic|--use-urandom}}<br />
| {{ic|--use-random}}<br />
| Selects which [[random number generator]] to use. Quoting the cryptsetup manual page: "In a low-entropy situation (e.g. in an embedded system), both selections are problematic. Using /dev/urandom can lead to weak keys. Using /dev/random can block a long time, potentially forever, if not enough entropy can be harvested by the kernel."<br />
|-<br />
! style="text-align:right;white-space:nowrap;" | --verify-passphrase<br />
-y<br />
| Yes<br />
| -<br />
| Enabled by default in Arch Linux for luksFormat and luksAddKey.<br />
|}<br />
<br />
The properties of LUKS features and options are described in the [https://gitlab.com/cryptsetup/cryptsetup/wikis/Specification LUKS1] (pdf) and [https://gitlab.com/cryptsetup/cryptsetup/blob/master/docs/on-disk-format-luks2.pdf LUKS2] (pdf) specifications.<br />
<br />
{{Tip|The project developers' [https://mbroz.fedorapeople.org/talks/DevConf2016/devconf2016-luks2.pdf devconfcz2016] (pdf) presentation summarizes the motivation for the major specification update to LUKS2.}}<br />
<br />
==== Choice of argon2 variant for PBKDF ====<br />
<br />
Although argon2id is in [https://crypto.stackexchange.com/questions/48935/why-use-argon2i-or-argon2d-if-argon2id-exists#49969 most cases the preferred variant] of the [[Wikipedia:Argon2|Argon2 family]], the [https://tools.ietf.org/html/draft-irtf-cfrg-argon2-12 quoted RFC] does indicate that:<br />
<br />
:''Argon2i uses data-independent memory access, '''which is preferred for password hashing and password-based key derivation'''.''<br />
<br />
We can also see this reflected in this [https://www.saout.de/pipermail/dm-crypt/2018-September/005968.html mailing list answer] by cryptsetup maintainer Milan Broz:<br />
<br />
:''You can use Argon2id though (combination of data dependent and independent processing). I prefer Argon2i for key derivation, but opinions differ here.''<br />
<br />
Therefore it is recommended to keep the default as is so as not to inadvertently decrease security.<br />
<br />
==== Iteration time ====<br />
<br />
From [https://gitlab.com/cryptsetup/cryptsetup/-/wikis/FrequentlyAskedQuestions#2-setup cryptsetup FAQ§2.1] and [https://gitlab.com/cryptsetup/cryptsetup/-/wikis/FrequentlyAskedQuestions#3-common-problems §3.4]:<br />
<br />
:The unlock time for a key-slot [...] is calculated when setting a passphrase. By default it is 1 second (2 seconds for LUKS2). [...]<br />
:Passphrase iteration count is based on time and hence security level depends on CPU power of the system the LUKS container is created on. [...]<br />
:If you set a passphrase on a fast machine and then unlock it on a slow machine, the unlocking time can be much longer.<br />
<br />
As such, it is better to always create a container on the machine where it will be most often accessed.<br />
<br />
Read the rest of those sections for advice on how to correctly adjust the iteration count should the need arise.<br />
<br />
==== Sector size ====<br />
<br />
{{man|8|cryptsetup|OPTIONS}} description for {{ic|--sector-size}} indicates that: "''Increasing sector size from 512 bytes to 4096 bytes can provide better performance on most of the modern storage devices and also with some hw encryption accelerators''" (see the rest of the description for caveats to be aware of).<br />
<br />
In a comment from 2019[https://lwn.net/Articles/777071/], the following reason is given for why the default was not switched over to 4K sector:<br />
<br />
:# ''Compatibility with old kernels and cryptsetup versions. The 4K encryption sector support is still fairly new, after all.''<br />
:# ''It’s not guaranteed safe on disks with 512-byte sectors, as it can break atomicity guarantees that might be assumed by software. I don’t believe this is a problem on modern disks or flash storage, nor on ext4 or f2fs. But the cryptsetup default needs to be more conservative.''<br />
<br />
To create a LUKS2 container with a 4K sector size and otherwise default options:<br />
<br />
# cryptsetup luksFormat --sector-size 4096 ''device''<br />
<br />
The command will abort on an error if the requested size does not match your device:<br />
<br />
{{bc|# cryptsetup luksFormat --sector-size 4096 ''device''<br />
(...)<br />
Verify passphrase: <br />
Device size is not aligned to requested sector size.<br />
}}<br />
<br />
=== Encryption options for plain mode ===<br />
<br />
In dm-crypt ''plain'' mode, there is no master-key on the device, hence, there is no need to set it up. Instead the encryption options to be employed are used directly to create the mapping between an encrypted disk and a named device. The mapping can be created against a partition or a full device. In the latter case not even a partition table is needed. <br />
<br />
To create a ''plain'' mode mapping with cryptsetup's default parameters: <br />
<br />
# cryptsetup <options> open --type plain <device> <dmname><br />
<br />
Executing it will prompt for a password, which should have very high entropy. <br />
Below a comparison of default parameters with the example in [[dm-crypt/Encrypting an entire system#Plain dm-crypt]]<br />
<br />
{| class="wikitable"<br />
! Option !! Cryptsetup 2.1.0 defaults !! Example !! Comment<br />
|-<br />
! style="text-align:right;white-space:nowrap;" | --hash<br />
-h<br />
| {{ic|ripemd160}}<br />
| -<br />
| The hash is used to create the key from the passphrase; it is not used on a keyfile. <br />
|-<br />
! style="text-align:right;white-space:nowrap;" | --cipher<br />
-c<br />
| {{ic|aes-cbc-essiv:sha256}}<br />
| {{ic|aes-xts-plain64}}<br />
| The cipher consists of three parts: cipher-chainmode-IV generator. Please see [[Data-at-rest encryption#Ciphers and modes of operation]] for an explanation of these settings, and the [https://gitlab.com/cryptsetup/cryptsetup/wikis/DMCrypt DMCrypt documentation] for some of the options available. <br />
|-<br />
! style="text-align:right;white-space:nowrap;" | --key-size<br />
-s<br />
|{{ic|256}}<br />
|{{ic|512}}<br />
|The key size (in bits). The size will depend on the cipher being used and also the chainmode in use. Xts mode requires twice the key size of cbc. <br />
|-<br />
! style="text-align:right;white-space:nowrap;" | --size<br />
-b<br />
|real size of target disk<br />
|{{ic|2048}} (mapped device will be 512B×2048=1MiB)<br />
|Limit the maximum size of the device (in 512-byte sectors).<br />
|-<br />
! style="text-align:right;white-space:nowrap;" | --offset<br />
-o<br />
|{{ic|0}}<br />
|{{ic|0}}<br />
|The offset from the beginning of the target disk (in 512-byte sectors) from which to start the mapping.<br />
|-<br />
! style="text-align:right;white-space:nowrap;" | --skip<br />
-p<br />
|{{ic|0}}<br />
|{{ic|2048}} (512B×2048=1MiB will be skipped)<br />
|The number of 512-byte sectors of encrypted data to skip at the beginning.<br />
|-<br />
! style="text-align:right;white-space:nowrap;" | --key-file<br />
-d<br />
| default uses a passphrase<br />
| {{ic|/dev/sd''Z''}} (or e.g. {{ic|/boot/keyfile.enc}})<br />
|The device or file to be used as a key. See [[#Keyfiles]] for further details.<br />
|-<br />
! style="text-align:right;white-space:nowrap;" | --keyfile-offset<br />
| {{ic|0}}<br />
| {{ic|0}}<br />
| Offset from the beginning of the file where the key starts (in bytes). This option is supported from ''cryptsetup'' 1.6.7 onwards. <br />
|-<br />
! style="text-align:right;white-space:nowrap;" | --keyfile-size<br />
-l<br />
|{{ic|8192kB}}<br />
| - (default applies)<br />
| Limits the bytes read from the key file. This option is supported from ''cryptsetup'' 1.6.7 onwards. <br />
|}<br />
<br />
Using the device {{ic|/dev/sd''X''}}, the above right column example results in:<br />
<br />
# cryptsetup --cipher=aes-xts-plain64 --offset=0 --key-file=/dev/sd''Z'' --key-size=512 open --type=plain /dev/sdX enc<br />
<br />
Unlike encrypting with LUKS, the above command must be executed ''in full'' whenever the mapping needs to be re-established, so it is important to remember the cipher, hash and key file details.<br />
We can now check that the mapping has been made:<br />
<br />
# fdisk -l<br />
<br />
An entry should now exist for {{ic|/dev/mapper/enc}}.<br />
<br />
== Encrypting devices with cryptsetup ==<br />
<br />
This section shows how to employ the options for creating new encrypted blockdevices and accessing them manually. <br />
<br />
{{Warning|GRUB does not support LUKS2 headers; see [https://savannah.gnu.org/bugs/?55093 GRUB bug #55093]. Therefore, if you plan to [[GRUB#Encrypted /boot|unlock an encrypted boot partition with GRUB]], specify {{ic|--type luks1}} on encrypted devices that GRUB will need to access.}}<br />
<br />
=== Encrypting devices with LUKS mode ===<br />
<br />
==== Formatting LUKS partitions ====<br />
<br />
In order to setup a partition as an encrypted LUKS partition execute:<br />
<br />
# cryptsetup luksFormat ''device''<br />
<br />
You will then be prompted to enter a password and verify it.<br />
<br />
See [[#Encryption options for LUKS mode]] for command line options.<br />
<br />
You can check the results with:<br />
<br />
# cryptsetup luksDump ''device''<br />
<br />
You will note that the dump not only shows the cipher header information, but also the key-slots in use for the LUKS partition. <br />
<br />
The following example will create an encrypted root partition on {{ic|/dev/sda1}} using the default AES cipher in XTS mode with an effective 256-bit encryption <br />
<br />
# cryptsetup -s 512 luksFormat /dev/sda1<br />
<br />
=====Using LUKS to format partitions with a keyfile=====<br />
<br />
When creating a new LUKS encrypted partition, a keyfile may be associated with the partition on its creation using:<br />
<br />
# cryptsetup luksFormat ''device'' ''/path/to/mykeyfile''<br />
<br />
See [[#Keyfiles]] for instructions on how to generate and manage keyfiles.<br />
<br />
====Unlocking/Mapping LUKS partitions with the device mapper ====<br />
<br />
Once the LUKS partitions have been created, they can then be unlocked.<br />
<br />
The unlocking process will map the partitions to a new device name using the device mapper. This alerts the kernel that {{ic|''device''}} is actually an encrypted device and should be addressed through LUKS using the {{ic|/dev/mapper/''dm_name''}} so as not to overwrite the encrypted data. To guard against accidental overwriting, read about the possibilities to [[#Backup and restore|backup the cryptheader]] after finishing setup.<br />
<br />
In order to open an encrypted LUKS partition execute:<br />
<br />
# cryptsetup open ''device'' ''dm_name''<br />
<br />
You will then be prompted for the password to unlock the partition. Usually the device mapped name is descriptive of the function of the partition that is mapped. For example the following unlocks a luks partition {{ic|/dev/sda1}} and maps it to device mapper named {{ic|cryptroot}}:<br />
<br />
# cryptsetup open /dev/sda1 cryptroot <br />
<br />
Once opened, the root partition device address would be {{ic|/dev/mapper/cryptroot}} instead of the partition (e.g. {{ic|/dev/sda1}}). <br />
<br />
For setting up LVM ontop the encryption layer the device file for the decrypted volume group would be anything like {{ic|/dev/mapper/cryptroot}} instead of {{ic|/dev/sda1}}. LVM will then give additional names to all logical volumes created, e.g. {{ic|/dev/lvmpool/root}} and {{ic|/dev/lvmpool/swap}}.<br />
<br />
In order to write encrypted data into the partition it must be accessed through the device mapped name. The first step of access will typically be to [[File systems#Create a file system|create a filesystem]]. For example:<br />
<br />
# mkfs -t ext4 /dev/mapper/cryptroot<br />
<br />
The device {{ic|/dev/mapper/cryptroot}} can then be [[mount]]ed like any other partition.<br />
<br />
To close the luks container, unmount the partition and do:<br />
<br />
# cryptsetup close cryptroot<br />
<br />
=== Encrypting devices with plain mode ===<br />
<br />
The creation and subsequent access of a ''dm-crypt'' plain mode encryption both require not more than using the ''cryptsetup'' {{ic|open}} action with correct [[#Encryption options for plain mode|parameters]]. The following shows that with two examples of non-root devices, but adds a quirk by stacking both (i.e. the second is created inside the first). Obviously, stacking the encryption doubles overhead. The usecase here is simply to illustrate another example of the cipher option usage. <br />
<br />
A first mapper is created with ''cryptsetup's'' plain-mode defaults, as described in the table's left column above <br />
<br />
{{hc|# cryptsetup --type plain -v open /dev/sdaX plain1|<br />
Enter passphrase: <br />
Command successful.<br />
}}<br />
<br />
Now we add the second blockdevice inside it, using different encryption parameters and with an (optional) offset, create a filesystem and mount it <br />
<br />
{{hc|1=# cryptsetup --type plain --cipher=serpent-xts-plain64 --hash=sha256 --key-size=256 --offset=10 open /dev/mapper/plain1 plain2|2=<br />
Enter passphrase:<br />
}}<br />
<br />
{{hc|# lsblk -p|<br />
NAME <br />
/dev/sda <br />
├─/dev/sdaX <br />
│ └─/dev/mapper/plain1 <br />
│ └─/dev/mapper/plain2 <br />
...<br />
}}<br />
<br />
# mkfs -t ext2 /dev/mapper/plain2<br />
# mount -t ext2 /dev/mapper/plain2 /mnt<br />
# echo "This is stacked. one passphrase per foot to shoot." > /mnt/stacked.txt<br />
<br />
We close the stack to check access works<br />
<br />
# cryptsetup close plain2<br />
# cryptsetup close plain1<br />
<br />
First, let us try to open the filesystem directly: <br />
<br />
# cryptsetup --type plain --cipher=serpent-xts-plain64 --hash=sha256 --key-size=256 --offset=10 open /dev/sdaX plain2<br />
<br />
{{hc|# mount -t ext2 /dev/mapper/plain2 /mnt|<br />
mount: wrong fs type, bad option, bad superblock on /dev/mapper/plain2,<br />
missing codepage or helper program, or other error<br />
}}<br />
<br />
Why that did not work? Because the "plain2" starting block ({{ic|10}}) is still encrypted with the cipher from "plain1". It can only be accessed via the stacked mapper. The error is arbitrary though, trying a wrong passphrase or wrong options will yield the same. For ''dm-crypt'' plain mode, the {{ic|open}} action will not error out itself. <br />
<br />
Trying again in correct order: <br />
<br />
# cryptsetup close plain2 # dysfunctional mapper from previous try<br />
<br />
{{hc|# cryptsetup --type plain open /dev/sdaX plain1|<br />
Enter passphrase:<br />
}}<br />
<br />
{{hc|1=# cryptsetup --type plain --cipher=serpent-xts-plain64 --hash=sha256 --key-size=256 --offset=10 open /dev/mapper/plain1 plain2|2=<br />
Enter passphrase:<br />
}}<br />
<br />
{{hc|# mount /dev/mapper/plain2 /mnt && cat /mnt/stacked.txt|<br />
This is stacked. one passphrase per foot to shoot.<br />
}}<br />
<br />
''dm-crypt'' will handle stacked encryption with some mixed modes too. For example LUKS mode could be stacked on the "plain1" mapper. Its header would then be encrypted inside "plain1" when that is closed.<br />
<br />
Available for plain mode only is the option {{ic|--shared}}. With it a single device can be segmented into different non-overlapping mappers. We do that in the next example, using a ''loopaes'' compatible cipher mode for "plain2" this time: <br />
<br />
{{hc|# cryptsetup --type plain --offset 0 --size 1000 open /dev/sdaX plain1|2=<br />
Enter passphrase:<br />
}}<br />
<br />
{{hc|1=# cryptsetup --type plain --offset 1000 --size 1000 --shared --cipher=aes-cbc-lmk --hash=sha256 open /dev/sdaX plain2|2=<br />
Enter passphrase:<br />
}}<br />
<br />
{{hc|# lsblk -p|<br />
NAME <br />
dev/sdaX <br />
├─/dev/sdaX <br />
│ ├─/dev/mapper/plain1 <br />
│ └─/dev/mapper/plain2 <br />
...<br />
}}<br />
<br />
As the devicetree shows both reside on the same level, i.e. are not stacked and "plain2" can be opened individually.<br />
<br />
== Cryptsetup actions specific for LUKS ==<br />
<br />
=== Key management ===<br />
<br />
It is possible to define addition keys for the LUKS partition. This enables the user to create access keys for safe backup storage In so-called key escrow, one key is used for daily usage, another kept in escrow to gain access to the partition in case the daily passphrase is forgotten or a keyfile is lost/damaged. A different key-slot could also be used to grant access to a partition to a user by issuing a second key and later revoking it again. <br />
<br />
Once an encrypted partition has been created, the initial keyslot 0 is created (if no other was specified manually). Additional keyslots are numbered from 1 to 7. Which keyslots are used can be seen by issuing <br />
<br />
# cryptsetup luksDump /dev/<device><br />
<br />
Where <device> is the volume containing the LUKS header. This and all the following commands in this section work on header backup files as well. <br />
<br />
==== Adding LUKS keys ====<br />
<br />
Adding new keyslots is accomplished using cryptsetup with the {{ic|luksAddKey}} action. For safety it will always, i.e. also for already unlocked devices, ask for a valid existing key ("any passphrase") before a new one may be entered:<br />
<br />
{{hc|# cryptsetup luksAddKey /dev/<device> (/path/to/<additionalkeyfile>)|<br />
Enter any passphrase:<br />
Enter new passphrase for key slot:<br />
Verify passphrase: <br />
}}<br />
<br />
If {{ic|/path/to/<additionalkeyfile>}} is given, cryptsetup will add a new keyslot for <additionalkeyfile>. Otherwise a new passphrase will be prompted for twice. For using an existing ''keyfile'' to authorize the action, the {{ic|--key-file}} or {{ic|-d}} option followed by the "old" <keyfile> will try to unlock all available keyfile keyslots:<br />
<br />
# cryptsetup luksAddKey /dev/<device> (/path/to/<additionalkeyfile>) -d /path/to/<keyfile><br />
<br />
If it is intended to use multiple keys and change or revoke them, the {{ic|--key-slot}} or {{ic|-S}} option may be used to specify the slot: <br />
<br />
{{hc|# cryptsetup luksAddKey /dev/<device> -S 6|<br />
Enter any passphrase: <br />
Enter new passphrase for key slot: <br />
Verify passphrase:<br />
}}<br />
<br />
{{hc|# cryptsetup luksDump /dev/sda8 {{!}} grep 'Slot 6'|<br />
Key Slot 6: ENABLED<br />
}}<br />
<br />
To show an associated action in this example, we decide to change the key right away: <br />
<br />
{{hc|# cryptsetup luksChangeKey /dev/<device> -S 6|<br />
Enter LUKS passphrase to be changed: <br />
Enter new LUKS passphrase:<br />
}}<br />
<br />
before continuing to remove it.<br />
<br />
==== Removing LUKS keys ====<br />
<br />
There are three different actions to remove keys from the header: <br />
<br />
* {{ic|luksRemoveKey}} is used to remove a key by specifying its passphrase/key-file. <br />
* {{ic|luksKillSlot}} may be used to remove a key from a specific key slot (using another key). Obviously, this is extremely useful if you have forgotten a passphrase, lost a key-file, or have no access to it. <br />
* {{ic|luksErase}} is used to quickly remove '''all''' active keys. <br />
<br />
{{warning|<br />
* All above actions can be used to irrevocably delete the last active key for an encrypted device! <br />
* The {{ic|luksErase}} command was added in version 1.6.4 to quickly nuke access to the device. This action '''will not''' prompt for a valid passphrase! It will not [[dm-crypt/Drive preparation#Wipe LUKS header|wipe the LUKS header]], but all keyslots at once and you will, therefore, not be able to regain access unless you have a valid backup of the LUKS header.<br />
}}<br />
<br />
For above warning it is good to know the key we want to '''keep''' is valid. An easy check is to unlock the device with the {{ic|-v}} option, which will specify which slot it occupies: <br />
<br />
{{hc|# cryptsetup -v open /dev/<device> testcrypt|<br />
Enter passphrase for /dev/<device>: <br />
Key slot 1 unlocked.<br />
Command successful.<br />
}}<br />
<br />
Now we can remove the key added in the previous subsection using its passphrase: <br />
<br />
{{hc|# cryptsetup luksRemoveKey /dev/<device>|<br />
Enter LUKS passphrase to be deleted:<br />
}}<br />
<br />
If we had used the same passphrase for two keyslots, the first slot would be wiped now. Only executing it again would remove the second one. <br />
<br />
Alternatively, we can specify the key slot: <br />
<br />
{{hc|# cryptsetup luksKillSlot /dev/<device> 6|<br />
Enter any remaining LUKS passphrase:<br />
}}<br />
<br />
Note that in both cases, no confirmation was required.<br />
<br />
{{hc|# cryptsetup luksDump /dev/sda8 {{!}} grep 'Slot 6'|<br />
Key Slot 6: DISABLED<br />
}}<br />
<br />
To re-iterate the warning above: If the same passphrase had been used for key slots 1 and 6, both would be gone now.<br />
<br />
=== Backup and restore ===<br />
<br />
If the header of a LUKS encrypted partition gets destroyed, you will not be able to decrypt your data. It is just as much of a dilemma as forgetting the passphrase or damaging a key-file used to unlock the partition. Damage may occur by your own fault while re-partitioning the disk later or by third-party programs misinterpreting the partition table. Therefore, having a backup of the header and storing it on another disk might be a good idea.<br />
<br />
{{Note|If one of the LUKS-encrypted partitions' passphrases becomes compromised, you must revoke it on ''every'' copy of the cryptheader, even those you have backed up. Otherwise, a copy of the backed-up cryptheader that uses the compromised passphrase can be used to determine the master key which in turn can be used to decrypt the associated partition (even your actual partition, not only the backed-up version). On the other hand, if the master key gets compromised, you have to reencrypt your whole partition. See [https://gitlab.com/cryptsetup/cryptsetup/wikis/FrequentlyAskedQuestions#6-backup-and-data-recovery LUKS FAQ] for further details.}}<br />
<br />
==== Backup using cryptsetup ====<br />
<br />
Cryptsetup's {{ic|luksHeaderBackup}} action stores a binary backup of the LUKS header and keyslot area:<br />
<br />
# cryptsetup luksHeaderBackup /dev/<device> --header-backup-file /mnt/<backup>/<file>.img<br />
<br />
where <device> is the partition containing the LUKS volume.<br />
<br />
You can also back up the plaintext header into ramfs and encrypt it with e.g. [[GPG]] before writing it to persistent storage:<br />
<br />
# mkdir /root/<tmp>/<br />
# mount ramfs /root/<tmp>/ -t ramfs<br />
# cryptsetup luksHeaderBackup /dev/<device> --header-backup-file /root/<tmp>/<file>.img<br />
# gpg2 --recipient <User ID> --encrypt /root/<tmp>/<file>.img <br />
# cp /root/<tmp>/<file>.img.gpg /mnt/<backup>/<br />
# umount /root/<tmp><br />
<br />
{{Warning|Tmpfs can swap to harddisk if low on memory so it is not recommended here.}}<br />
<br />
==== Restore using cryptsetup ====<br />
<br />
{{Warning|Restoring the wrong header or restoring to an unencrypted partition will cause data loss! The action can not perform a check whether the header is actually the ''correct'' one for that particular device.}} <br />
<br />
In order to evade restoring a wrong header, you can ensure it does work by using it as a remote {{ic|--header}} first: <br />
<br />
{{hc|# cryptsetup -v --header /mnt/<backup>/<file>.img open /dev/<device> test|<br />
Key slot 0 unlocked.<br />
Command successful.<br />
}}<br />
<br />
# mount /dev/mapper/test /mnt/test && ls /mnt/test <br />
# umount /mnt/test <br />
# cryptsetup close test <br />
<br />
Now that the check succeeded, the restore may be performed: <br />
<br />
# cryptsetup luksHeaderRestore /dev/<device> --header-backup-file ./mnt/<backup>/<file>.img<br />
<br />
Now that all the keyslot areas are overwritten; only active keyslots from the backup file are available after issuing the command.<br />
<br />
==== Manual backup and restore ====<br />
<br />
The header always resides at the beginning of the device and a backup can be performed without access to ''cryptsetup'' as well. First you have to find out the payload offset of the crypted partition:<br />
<br />
{{hc|# cryptsetup luksDump /dev/<device> {{!}} grep "Payload offset"|<br />
Payload offset: 4040<br />
}}<br />
<br />
Second check the sector size of the drive<br />
<br />
{{hc|# fdisk -l /dev/<device> {{!}} grep "Sector size"|<br />
Sector size (logical/physical): 512 bytes / 512 bytes<br />
}}<br />
<br />
Now that you know the values, you can backup the header with a simple [[dd]] command:<br />
<br />
# dd if=/dev/<device> of=/path/to/<file>.img bs=512 count=4040<br />
<br />
and store it safely.<br />
<br />
A restore can then be performed using the same values as when backing up:<br />
<br />
# dd if=./<file>.img of=/dev/<device> bs=512 count=4040<br />
<br />
=== Re-encrypting devices ===<br />
<br />
{{Expansion|cryptsetup 2.2 using LUKS2 (with a 16 MiB header) supports online encryption/decryption/reencryption.[https://mirrors.edge.kernel.org/pub/linux/utils/cryptsetup/v2.2/v2.2.0-ReleaseNotes]}}<br />
<br />
The {{Pkg|cryptsetup}} package features two options for re-encryption.<br />
<br />
; cryptsetup reencrypt: Argument to {{ic|cryptsetup}} itself: Preferred method. Currently LUKS2 devices only. Actions can be performed online. Supports multiple parallel re-encryption jobs. Resilient to system failures. See {{man|8|cryptsetup}} for more information.<br />
<br />
; cryptsetup-reencrypt: Legacy tool, supports LUKS1 in addition to LUKS2. Actions can be performed on unmounted devices only. Single process at a time. Sensitive to system failures. See {{man|8|cryptsetup-reencrypt}} for more information. <br />
<br />
Both can be used to convert an existing unencrypted filesystem to a LUKS encrypted one or permanently remove LUKS encryption from a device (using {{ic|--decrypt}}). As its name suggests it can also be used to re-encrypt an existing LUKS encrypted device, though, re-encryption is not possible for a detached LUKS header or other encryption modes (e.g. plain-mode). For re-encryption it is possible to change the [[#Encryption options for LUKS mode]]. <br />
<br />
One application of re-encryption may be to secure the data again after a passphrase or [[#Keyfiles|keyfile]] has been compromised ''and'' one cannot be certain that no copy of the LUKS header has been obtained. For example, if only a passphrase has been shoulder-surfed but no physical/logical access to the device happened, it would be enough to change the respective passphrase/key only ([[#Key management]]). <br />
<br />
{{Warning|Always make sure a '''reliable backup''' is available and double-check options you specify before using the tool!}}<br />
<br />
The following shows an example to encrypt an unencrypted filesystem partition and a re-encryption of an existing LUKS device. <br />
<br />
==== Encrypt an unencrypted filesystem ====<br />
<br />
A LUKS encryption header is always stored at the beginning of the device. Since an existing filesystem will usually be allocated all partition sectors, the first step is to shrink it to make space for the LUKS header.<br />
<br />
{{Expansion|cryptsetup man pages suggest using twice the LUKS2 header size. That implies 32MiB and using {{ic|--reduce-device-size 32M}}}}<br />
The [[#Encryption options for LUKS mode|default]] LUKS2 header requires 16 MiB. If the current filesystem occupies all the available space, we'll have to shrink it at least that much.<br />
To shrink an existing {{ic|ext4}} filesystem on {{ic|/dev/sdaX}} to its current possible minimum:<br />
<br />
# umount /mnt<br />
<br />
{{hc|# e2fsck -f /dev/sdaX|<br />
e2fsck 1.43-WIP (18-May-2015)<br />
Pass 1: Checking inodes, blocks, and sizes<br />
...<br />
/dev/sda6: 12/166320 files (0.0% non-contiguous), 28783/665062 blocks<br />
}}<br />
<br />
{{hc|# resize2fs -p -M /dev/sdaX|<br />
resize2fs 1.43-WIP (18-May-2015)<br />
Resizing the filesystem on /dev/sdaX to 26347 (4k) blocks.<br />
The filesystem on /dev/sdaX is now 26347 (4k) blocks long.<br />
}}<br />
<br />
Note that shrinking to the minimum size might take very long. You might want to calculate a size just 32MiB smaller than the current size instead of using -M.<br />
<br />
Now we encrypt it, using the default cipher we do not have to specify it explicitly:<br />
{{hc|# cryptsetup reencrypt --encrypt --reduce-device-size 16M /dev/sdaX|<nowiki><br />
<br />
WARNING!<br />
========<br />
This will overwrite data on LUKS2-temp-12345678-9012-3456-7890-123456789012.new irrevocably.<br />
<br />
Are you sure? (Type 'yes' in capital letters): YES<br />
Enter passphrase for LUKS2-temp-12345678-9012-3456-7890-123456789012.new: <br />
Verify passphrase: <br />
</nowiki>}}<br />
<br />
After it finished, the whole {{ic|/dev/sdaX}} partition is encrypted, not only the space the filesystem was shrunk to. As a final step we extend the original {{ic|ext4}} filesystem to occupy all available space again, on the now encrypted partition:<br />
<br />
{{hc|# cryptsetup open /dev/sdaX recrypt|<br />
Enter passphrase for /dev/sdaX: <br />
...<br />
}}<br />
<br />
{{hc|# resize2fs /dev/mapper/recrypt|<br />
resize2fs 1.43-WIP (18-May-2015)<br />
Resizing the filesystem on /dev/mapper/recrypt to 664807 (4k) blocks.<br />
The filesystem on /dev/mapper/recrypt is now 664807 (4k) blocks long.<br />
}}<br />
<br />
# mount /dev/mapper/recrypt /mnt<br />
<br />
The filesystem is now ready to use. You may want to add it to your [[crypttab]].<br />
<br />
==== Re-encrypting an existing LUKS partition ====<br />
<br />
In this example an existing LUKS device is re-encrypted. <br />
<br />
{{Warning|Double-check you specify encryption options for correctly and ''never'' re-encrypt without a '''reliable backup'''!}}<br />
<br />
In order to re-encrypt a device with its existing encryption options, they do not need to be specified:<br />
{{bc|# cryptsetup reencrypt /dev/sdaX}}<br />
{{Note|For LUKS1 we'll need to use the legacy tool:{{bc|# cryptsetup-reencrypt /dev/sdaX}}}}<br />
<br />
Existing keys are retained when re-encrypting a device with a different cipher and/or hash. Another use case is to re-encrypt LUKS devices which have non-current encryption options. Apart from above warning on specifying options correctly, the ability to change the LUKS header may also be limited by its size. For example, if the device was initially encrypted using a CBC mode cipher and 128 bit key-size, the LUKS header will be half the size of above mentioned {{ic|4096}} sectors: <br />
<br />
{{hc|# cryptsetup luksDump /dev/sdaX <nowiki>|</nowiki>grep -e "mode" -e "Payload" -e "MK bits"|<br />
Cipher mode: cbc-essiv:sha256<br />
Payload offset: '''2048'''<br />
MK bits: 128<br />
}}<br />
<br />
While it is possible to upgrade the encryption of such a device, it is currently only feasible in two steps. First, re-encrypting with the same encryption options, but using the {{ic|--reduce-device-size}} option to make further space for the larger LUKS header. Second, re-encypt the whole device again with the desired cipher. For this reason and the fact that a backup should be created in any case, creating a new, fresh encrypted device to restore into is always the faster option.<br />
<br />
== Resizing encrypted devices ==<br />
<br />
{{Expansion|This section should be rewritten to introduce resizing more generically. Perhaps work on it together with [[Resizing LVM-on-LUKS]].}}<br />
<br />
If a storage device encrypted with dm-crypt is being cloned (with a tool like dd) to another larger device, the underlying dm-crypt device must be resized to use the whole space. <br />
<br />
The destination device is /dev/sdX2 in this example, the whole available space adjacent to the partition will be used:<br />
<br />
# cryptsetup luksOpen /dev/sdX2 sdX2<br />
# cryptsetup resize sdX2<br />
<br />
Then the underlying filesystem must be resized.<br />
<br />
=== Loopback filesystem ===<br />
<br />
Assume that an encrypted loopback filesystem is stored in a file {{ic|/bigsecret}}, looped to {{ic|/dev/loop0}}, mapped to {{ic|secret}} and mounted on {{ic|/mnt/secret}}, as in the example at [[dm-crypt/Encrypting a non-root file system#File container]].<br />
<br />
If the container file is currently mapped and/or mounted, unmount and/or close it:<br />
<br />
# umount /mnt/secret<br />
# cryptsetup close secret<br />
# losetup -d /dev/loop0<br />
<br />
Next, expand the container file with the size of the data you want to add. In this example, the file will be expanded with 1M * 1024, which is 1G.<br />
<br />
{{Warning|Make absolutely sure to use '''two''' {{ic|>}}, instead of just one, or else you will overwrite the file instead of appending to it. Making a backup before this step is strongly recommended.}}<br />
<br />
# dd if=/dev/urandom bs=1M count=1024 | cat - >> /bigsecret<br />
<br />
Now map the container to the loop device:<br />
<br />
# losetup /dev/loop0 /bigsecret<br />
# cryptsetup open /dev/loop0 secret<br />
<br />
After this, resize the encrypted part of the container to the new maximum size of the container file:<br />
<br />
# cryptsetup resize secret<br />
<br />
Finally, perform a filesystem check and, if it is ok, resize it (example for ext2/3/4):<br />
<br />
# e2fsck -f /dev/mapper/secret<br />
# resize2fs /dev/mapper/secret<br />
<br />
You can now mount the container again:<br />
<br />
# mount /dev/mapper/secret /mnt/secret<br />
<br />
=== Integrity protected device ===<br />
<br />
If the device was formatted with integrity support (e.g., {{ic|--integrity hmac-sha256}}) and the backing block device is shrinked, it cannot be opened with this error: {{ic|device-mapper: reload ioctl on failed: Invalid argument}}.<br />
<br />
To fix this issue without wiping the device again, it can be formatted with the previous master key (keeping the per-sector tags valid).<br />
<br />
# cryptsetup luksDump /dev/sdX2 --dump-master-key --master-key-file=/tmp/masterkey-in-tmpfs.key<br />
# cryptsetup luksFormat /dev/sdX2 --type luks2 --integrity hmac-sha256 --master-key-file=/tmp/masterkey-in-tmpfs.key --integrity-no-wipe<br />
# rm /tmp/masterkey-in-tmpfs.key<br />
<br />
== Keyfiles ==<br />
<br />
{{Note|This section describes using a plaintext keyfile. If you want to encrypt your keyfile giving you two factor authentication see [[dm-crypt/Specialties#Using GPG, LUKS, or OpenSSL Encrypted Keyfiles|Using GPG or OpenSSL Encrypted Keyfiles]] for details, but please still read this section.}}<br />
<br />
'''What is a keyfile?'''<br />
<br />
A keyfile is a file whose data is used as the passphrase to unlock an encrypted volume.<br />
That means if such a file is lost or changed, decrypting the volume may no longer be possible.<br />
<br />
{{Tip|Define a passphrase in addition to the keyfile for backup access to encrypted volumes in the event the defined keyfile is lost or changed.}}<br />
<br />
'''Why use a keyfile?'''<br />
<br />
There are many kinds of keyfiles. Each type of keyfile used has benefits and disadvantages summarized below:<br />
<br />
=== Types of keyfiles ===<br />
<br />
==== passphrase ====<br />
<br />
This is a keyfile containing a simple passphrase. The benefit of this type of keyfile is that if the file is lost the data it contained is known and hopefully easily remembered by the owner of the encrypted volume. However the disadvantage is that this does not add any security over entering a passphrase during the initial system start.<br />
<br />
Example: {{ic|1234}}<br />
<br />
{{Note|The keyfile containing the passphrase must not have a newline in it. One option is to create it using <br />
<br />
# echo -n 'your_passphrase' > /path/to/<keyfile><br />
# chown root:root /path/to/<keyfile>; chmod 400 /path/to/<keyfile><br />
<br />
If the file contains special characters such as a backslash, rather than escaping these, it is recommended to simply edit the key file directly entering or pasting the passphrase and then remove the trailing newline with a handy perl one-liner:<br />
<br />
# perl -pi -e 'chomp if eof' /path/to/<keyfile><br />
}}<br />
<br />
==== randomtext ====<br />
<br />
This is a keyfile containing a block of random characters. The benefit of this type of keyfile is that it is much more resistant to dictionary attacks than a simple passphrase. An additional strength of keyfiles can be utilized in this situation which is the length of data used. Since this is not a string meant to be memorized by a person for entry, it is trivial to create files containing thousands of random characters as the key. The disadvantage is that if this file is lost or changed, it will most likely not be possible to access the encrypted volume without a backup passphrase.<br />
<br />
Example: {{ic|fjqweifj830149-57 819y4my1-38t1934yt8-91m 34co3;t8y;9p3y-}}<br />
<br />
==== binary ====<br />
<br />
This is a binary file that has been defined as a keyfile. When identifying files as candidates for a keyfile, it is recommended to choose files that are relatively static such as photos, music, video clips. The benefit of these files is that they serve a dual function which can make them harder to identify as keyfiles. Instead of having a text file with a large amount of random text, the keyfile would look like a regular image file or music clip to the casual observer. The disadvantage is that if this file is lost or changed, it will most likely not be possible to access the encrypted volume without a backup passphrase. Additionally, there is a theoretical loss of randomness when compared to a randomly generated text file. This is due to the fact that images, videos and music have some intrinsic relationship between neighboring bits of data that does not exist for a random text file. However this is controversial and has never been exploited publicly.<br />
<br />
Example: images, text, video, ...<br />
<br />
=== Creating a keyfile with random characters ===<br />
<br />
==== Storing the keyfile on a filesystem ====<br />
<br />
A keyfile can be of arbitrary content and size. <br />
<br />
Here [[dd]] is used to generate a keyfile of 2048 random bytes, storing it in the file {{ic|/etc/mykeyfile}}:<br />
<br />
# dd bs=512 count=4 if=/dev/random of=/etc/mykeyfile iflag=fullblock<br />
<br />
If you are planning to store the keyfile on an external device, you can also simply change the outputfile to the corresponding directory:<br />
<br />
# dd bs=512 count=4 if=/dev/random of=/media/usbstick/mykeyfile iflag=fullblock<br />
<br />
To deny any access for other users than {{ic|root}}:<br />
<br />
# chmod 600 /etc/mykeyfile<br />
<br />
===== Securely overwriting stored keyfiles =====<br />
<br />
If you stored your temporary keyfile on a physical storage device, and want to delete it, remember to not just remove the keyfile later on, but use something like<br />
<br />
# shred --remove --zero mykeyfile<br />
<br />
to securely overwrite it. For overaged filesystems like FAT or ext2 this will suffice while in the case of journaling filesystems, flash memory hardware and other cases it is highly recommended to [[Securely wipe disk|wipe the entire device]].<br />
<br />
==== Storing the keyfile in ramfs ====<br />
<br />
Alternatively, you can mount a ramfs for storing the keyfile temporarily:<br />
<br />
# mkdir /root/myramfs<br />
# mount ramfs /root/myramfs/ -t ramfs<br />
# cd /root/myramfs<br />
<br />
The advantage is that it resides in RAM and not on a physical disk, therefore it can not be recovered after unmounting the ramfs. After copying the keyfile to another secure and persistent filesystem, unmount the ramfs again with<br />
<br />
# umount /root/myramfs<br />
<br />
=== Configuring LUKS to make use of the keyfile ===<br />
<br />
Add a keyslot for the keyfile to the LUKS header:<br />
<br />
{{hc|# cryptsetup luksAddKey /dev/sda2 /etc/mykeyfile|<br />
Enter any LUKS passphrase:<br />
key slot 0 unlocked.<br />
Command successful.<br />
}}<br />
<br />
=== Manually unlocking a partition using a keyfile ===<br />
<br />
Use the {{ic|--key-file}} option when opening the LUKS device:<br />
<br />
# cryptsetup open /dev/sda2 ''dm_name'' --key-file /etc/mykeyfile<br />
<br />
=== Unlocking the root partition at boot ===<br />
<br />
This is simply a matter of configuring [[mkinitcpio]] to include the necessary modules or files and configuring the [[dm-crypt/System configuration#cryptkey|cryptkey]] [[kernel parameter]] to know where to find the keyfile.<br />
<br />
Two cases are covered below:<br />
<br />
# Using a keyfile stored on an external medium (e.g. a USB stick)<br />
# Using a keyfile embedded in the initramfs <br />
<br />
==== With a keyfile stored on an external media ====<br />
<br />
===== Configuring mkinitcpio =====<br />
<br />
You have to add the kernel module for the drive's [[file system]] to the [[mkinitcpio#MODULES|MODULES array]] in {{ic|/etc/mkinitcpio.conf}}. For example, add {{ic|ext4}} if the file system is [[Ext4]] or {{ic|vfat}} in case it is [[FAT]]:<br />
<br />
MODULES=(vfat)<br />
<br />
If there are messages about bad superblock and bad codepage at boot, then you need an extra codepage module to be loaded. For instance, you may need {{ic|nls_iso8859-1}} module for {{ic|iso8859-1}} codepage.<br />
<br />
[[Regenerate the initramfs]].<br />
<br />
===== Configuring the kernel parameters =====<br />
<br />
* For a busybox-based initramfs using the [[dm-crypt/System configuration#Using encrypt hook|encrypt]] hook, see [[dm-crypt/System configuration#cryptkey]].<br />
* For a systemd based initramfs using the [[sd-encrypt]] hook, see [[dm-crypt/System configuration#rd.luks.key]].<br />
<br />
==== With a keyfile embedded in the initramfs ====<br />
<br />
{{Expansion|Add method for [[sd-encrypt]].}}<br />
<br />
{{Warning|Use an embedded keyfile '''only''' if you protect the keyfile sufficiently by:<br />
* Using some form of authentication earlier in the boot process. Otherwise auto-decryption will occur, defeating completely the purpose of block device encryption.<br />
* {{ic|/boot}} is encrypted. Otherwise root on a different installation (including the [[Installation guide#Boot the live environment|live environment]]) can extract your key from the initramfs, and unlock the device without any other authentication.}}<br />
<br />
This method allows to use a specially named keyfile that will be embedded in the [[initramfs]] and picked up by the {{ic|encrypt}} [[Mkinitcpio#HOOKS|hook]] to unlock the root filesystem ({{ic|cryptdevice}}) automatically. It may be useful to apply when using the [[GRUB#Encrypted /boot|GRUB early cryptodisk]] feature, in order to avoid entering two passphrases during boot.<br />
<br />
The {{ic|encrypt}} hook lets the user specify a keyfile with the {{ic|cryptkey}} kernel parameter: in the case of initramfs, the syntax is {{ic|rootfs:''path''}}. See [[dm-crypt/System configuration#cryptkey]]. Besides, this kernel parameter defaults to use {{ic|/crypto_keyfile.bin}}, and if the initramfs contains a valid key with this name, decryption will occur automatically without the need to configure the {{ic|cryptkey}} parameter.<br />
<br />
If using {{ic|sd-encrypt}} instead of {{ic|encrypt}}, specify the location of the keyfile with the {{ic|rd.luks.key}} kernel parameter. See [[dm-crypt/System configuration#rd.luks.key]].<br />
<br />
[[#Creating a keyfile with random characters|Generate the keyfile]], give it suitable permissions and [[#Adding LUKS keys|add it as a LUKS key]]:<br />
<br />
# dd bs=512 count=4 if=/dev/random of=/crypto_keyfile.bin iflag=fullblock<br />
# chmod 600 /crypto_keyfile.bin<br />
# chmod 600 /boot/initramfs-linux*<br />
# cryptsetup luksAddKey /dev/sdX# /crypto_keyfile.bin<br />
<br />
{{Warning|When initramfs' permissions are set to 644 (by default), then all users will be able to dump the keyfile. Make sure the permissions are still 600 if you install a new kernel.}}<br />
<br />
Include the key in [[mkinitcpio#BINARIES and FILES|mkinitcpio's FILES array]]:<br />
<br />
{{hc|/etc/mkinitcpio.conf|2=<br />
FILES=(/crypto_keyfile.bin)<br />
}}<br />
<br />
Finally [[regenerate the initramfs]].<br />
<br />
On the next reboot you should only have to enter your container decryption passphrase once.<br />
<br />
([http://www.pavelkogan.com/2014/05/23/luks-full-disk-encryption/#bonus-login-once source])</div>Phireskyhttps://wiki.archlinux.org/index.php?title=Zsh&diff=639603Zsh2020-10-24T13:35:50Z<p>Phiresky: /* "command not found" handler: print "not found as soon as possible so user knows what is happening since search may take a bit */</p>
<hr />
<div>[[Category:Command-line shells]]<br />
[[cs:Zsh]]<br />
[[de:Zsh]]<br />
[[es:Zsh]]<br />
[[fa:Zsh]]<br />
[[fr:Zsh]]<br />
[[ja:Zsh]]<br />
[[ru:Zsh]]<br />
[[zh-hans:Zsh]]<br />
[https://www.zsh.org/ Zsh] is a powerful [[shell]] that operates as both an interactive shell and as a scripting language interpreter. While being compatible with the POSIX sh (not by default, only if issuing {{ic|emulate sh}}), it offers advantages such as improved [http://zsh.sourceforge.net/Guide/zshguide06.html tab completion] and [http://zsh.sourceforge.net/Doc/Release/Expansion.html globbing].<br />
<br />
The [http://zsh.sourceforge.net/FAQ/zshfaq01.html#l4 Zsh FAQ] offers more reasons to use Zsh.<br />
<br />
== Installation ==<br />
<br />
Before starting, users may want to see what shell is currently being used:<br />
<br />
$ echo $SHELL<br />
<br />
[[Install]] the {{Pkg|zsh}} package. For additional completion definitions, install the {{pkg|zsh-completions}} package as well.<br />
<br />
=== Initial configuration ===<br />
<br />
Make sure that Zsh has been installed correctly by running the following in a terminal:<br />
<br />
$ zsh<br />
<br />
You should now see ''zsh-newuser-install'', which will walk you through some basic configuration. If you want to skip this, press {{ic|q}}. If you did not see it, you can invoke it manually with:<br />
<br />
$ autoload -Uz zsh-newuser-install<br />
$ zsh-newuser-install -f<br />
<br />
{{Note|Make sure your terminal's size is at least 72×15 otherwise ''zsh-newuser-install'' will not run.}}<br />
<br />
=== Making Zsh your default shell ===<br />
<br />
Change your shell to {{ic|/usr/bin/zsh}}. See [[Command-line shell#Changing your default shell]].<br />
<br />
{{Tip|If replacing {{Pkg|bash}}, users may want to move some code from {{ic|~/.bashrc}} to {{ic|~/.zshrc}} (e.g. the prompt and the [[Bash#Aliases|aliases]]) and from {{ic|~/.bash_profile}} to {{ic|~/.zprofile}} (e.g. [[xinit#Autostart X at login|the code that starts the X Window System]]).}}<br />
<br />
== Startup/Shutdown files ==<br />
<br />
{{Tip|<br />
* See [http://zsh.sourceforge.net/Guide/zshguide02.html A User's Guide to the Z-Shell] for explanation on interactive and login shells, and what to put in your startup files.<br />
* You could consider [[Command-line shell#Standardisation|implementing a standard path]] for your Zsh configuration files.<br />
}}<br />
<br />
{{Note|<br />
* If {{ic|$ZDOTDIR}} is not set, {{ic|$HOME}} is used instead.<br />
* If option {{ic|RCS}} is unset in any of the files, no configuration files will be read after that file.<br />
* If option {{ic|GLOBAL_RCS}} is unset in any of the files, no global configuration files ({{ic|/etc/zsh/*}}) will be read after that file.<br />
}}<br />
<br />
When starting, Zsh will read commands from the following files in this order by default, provided they exist. <br />
<br />
* {{ic|/etc/zsh/zshenv}} Used for setting [[environment variables]] for all users; it should not contain commands that produce output or assume the shell is attached to a TTY. When this file exists it will '''''always''''' be read, this cannot be overridden.<br />
* {{ic|$ZDOTDIR/.zshenv}} Used for setting user's environment variables; it should not contain commands that produce output or assume the shell is attached to a TTY. When this file exists it will '''''always''''' be read.<br />
* {{ic|/etc/zsh/zprofile}} Used for executing commands at start for all users, will be read when starting as a '''''login shell'''''. Please note that on Arch Linux, by default it contains [https://github.com/archlinux/svntogit-packages/blob/packages/zsh/trunk/zprofile one line] which sources {{ic|/etc/profile}}. See warning below before wanting to remove that!<br />
** {{ic|/etc/profile}} This file should be sourced by all POSIX sh-compatible shells upon login: it sets up {{ic|$PATH}} and other environment variables and application-specific ({{ic|/etc/profile.d/*.sh}}) settings upon login.<br />
* {{ic|$ZDOTDIR/.zprofile}} Used for executing user's commands at start, will be read when starting as a '''''login shell'''''. Typically used to autostart graphical sessions and to set session-wide environment variables.<br />
* {{ic|/etc/zsh/zshrc}} Used for setting interactive shell configuration and executing commands for all users, will be read when starting as an '''''interactive shell'''''.<br />
* {{ic|$ZDOTDIR/.zshrc}} Used for setting user's interactive shell configuration and executing commands, will be read when starting as an '''''interactive shell'''''.<br />
* {{ic|/etc/zsh/zlogin}} Used for executing commands for all users at ending of initial progress, will be read when starting as a '''''login shell'''''.<br />
* {{ic|$ZDOTDIR/.zlogin}} Used for executing user's commands at ending of initial progress, will be read when starting as a '''''login shell'''''. Typically used to autostart command line utilities. Should not be used to autostart graphical sessions, as at this point the session might contain configuration meant only for an interactive shell.<br />
* {{ic|$ZDOTDIR/.zlogout}} Used for executing commands when a '''''login shell''''' '''exits'''.<br />
* {{ic|/etc/zsh/zlogout}} Used for executing commands for all users when a '''''login shell''''' '''exits'''.<br />
<br />
See [https://blog.flowblok.id.au/2013-02/shell-startup-scripts.html#implementation the graphic representation].<br />
<br />
{{Note|<br />
* {{ic|$HOME/.profile}} is not a part of the Zsh startup files and '''is not sourced''' by Zsh unless Zsh is invoked as {{ic|sh}} or {{ic|ksh}} and started as a login shell. For more details about the sh and [[ksh]] compatibility modes refer to {{man|1|zsh|COMPATIBILITY}}.<br />
* The paths used in Arch's {{Pkg|zsh}} package are different from the default ones used in the [[man page]]s ({{Bug|48992}}).<br />
}}<br />
<br />
{{Warning|Do not remove the default [https://github.com/archlinux/svntogit-packages/blob/packages/zsh/trunk/zprofile one line] in {{ic|/etc/zsh/zprofile}}, otherwise it will break the integrity of other packages which provide some scripts in {{ic|/etc/profile.d/}}.}}<br />
<br />
== Configure Zsh ==<br />
<br />
Although Zsh is usable out of the box, it is almost certainly not set up the way most users would like to use it. But due to the sheer amount of customization available in Zsh, configuring Zsh can be a daunting and time-consuming experience.<br />
<br />
=== Simple .zshrc ===<br />
<br />
Included below is a sample configuration file. It provides a decent set of default options as well as giving examples of many ways that Zsh can be customized. In order to use this configuration save it as a file named {{ic|.zshrc}}.<br />
<br />
{{Tip|Apply the changes without needing to logout and then back in by running {{ic|source ~/.zshrc}}.}}<br />
<br />
Here is a simple {{ic|.zshrc}}:<br />
<br />
{{hc|~/.zshrc|<br />
autoload -Uz compinit promptinit<br />
compinit<br />
promptinit<br />
<br />
# This will set the default prompt to the walters theme<br />
prompt walters<br />
}}<br />
<br />
=== Configuring $PATH ===<br />
<br />
Zsh ties the {{ic|PATH}} variable to a {{ic|path}} array. They are automatically synchronized. This allows us to easily manipulate {{ic|PATH}} by simply modifying the array. See [http://zsh.sourceforge.net/Guide/zshguide02.html#l24 A User's Guide to the Z-Shell] for details.<br />
<br />
The line {{ic|typeset -U PATH path}}, where the {{ic|-U}} stands for unique, instructs the shell to discard duplicates from both {{ic|$PATH}} and {{ic|$path}}:<br />
<br />
{{hc|~/.zshenv|2=<br />
typeset -U PATH path<br />
path=("$HOME/.local/bin" ''/other/things/in/path'' "$path[@]")<br />
export PATH<br />
}}<br />
<br />
=== Command completion ===<br />
<br />
Perhaps the most compelling feature of Zsh is its advanced autocompletion abilities. At the very least, enable autocompletion in {{ic|.zshrc}}. To enable autocompletion, add the following to your {{ic|~/.zshrc}}:<br />
<br />
{{hc|~/.zshrc|<br />
autoload -Uz compinit<br />
compinit<br />
}}<br />
<br />
The above configuration includes ssh/scp/sftp hostnames completion but in order for this feature to work, users must not enable ssh's hostname hashing (i.e. option {{ic|HashKnownHosts}} in ssh client configuration).<br />
<br />
For autocompletion with an arrow-key driven interface, add the following to:<br />
<br />
{{hc|~/.zshrc|<br />
zstyle ':completion:*' menu select<br />
}}<br />
<br />
To activate the menu, press {{ic|Tab}} twice.<br />
<br />
For autocompletion of command line switches for aliases, add the following to:<br />
<br />
{{hc|~/.zshrc|<br />
setopt COMPLETE_ALIASES<br />
}}<br />
<br />
For enabling autocompletion of privileged environments in privileged commands (e.g. if you complete a command starting with [[sudo]], completion scripts will also try to determine your completions with sudo), include:<br />
<br />
{{hc|~/.zshrc|<br />
zstyle ':completion::complete:*' gain-privileges 1<br />
}}<br />
<br />
{{Warning|This will let Zsh completion scripts run commands with sudo privileges. You should not enable this if you use untrusted autocompletion scripts.}}<br />
<br />
{{Note|This special kind of context-aware completion is only available for a small number of commands.}}<br />
<br />
=== Key bindings ===<br />
<br />
Zsh does not use [[readline]], instead it uses its own and more powerful Zsh Line Editor (ZLE). It does not read {{ic|/etc/inputrc}} or {{ic|~/.inputrc}}. Read [https://sgeb.io/posts/2014/04/zsh-zle-custom-widgets/ A closer look at the zsh line editor and creating custom widgets] for an introduction to ZLE configuration.<br />
<br />
ZLE has an [[Emacs]] mode and a [[vi]] mode. If one of the {{ic|VISUAL}} or {{ic|EDITOR}} [[environment variables]] contain the string {{ic|vi}} then vi mode will be used; otherwise, it will default to Emacs mode. Set the mode explicitly with {{ic|bindkey -e}} or {{ic|bindkey -v}} respectively for Emacs mode or vi mode.<br />
<br />
Key bindings are assigned by mapping an escape sequence matching a keypress to a ZLE widget. The available widgets, with descriptions of their actions and their default keybindings, are listed in {{man|1|zshzle|STANDARD WIDGETS}} and {{man|1|zshcontrib|ZLE FUNCTIONS}}.<br />
<br />
The recommended way to set key bindings in Zsh is by using string capabilities from {{man|5|terminfo}}. For example[https://web.archive.org/web/20180704181216/http://zshwiki.org/home/zle/bindkeys][https://www.zsh.org/mla/users/2010/msg00065.html]:<br />
<br />
{{hc|~/.zshrc|2=<br />
# create a zkbd compatible hash;<br />
# to add other keys to this hash, see: man 5 terminfo<br />
typeset -g -A key<br />
<br />
key[Home]="${terminfo[khome]}"<br />
key[End]="${terminfo[kend]}"<br />
key[Insert]="${terminfo[kich1]}"<br />
key[Backspace]="${terminfo[kbs]}"<br />
key[Delete]="${terminfo[kdch1]}"<br />
key[Up]="${terminfo[kcuu1]}"<br />
key[Down]="${terminfo[kcud1]}"<br />
key[Left]="${terminfo[kcub1]}"<br />
key[Right]="${terminfo[kcuf1]}"<br />
key[PageUp]="${terminfo[kpp]}"<br />
key[PageDown]="${terminfo[knp]}"<br />
key[Shift-Tab]="${terminfo[kcbt]}"<br />
<br />
# setup key accordingly<br />
[[ -n "${key[Home]}" ]] && bindkey -- "${key[Home]}" beginning-of-line<br />
[[ -n "${key[End]}" ]] && bindkey -- "${key[End]}" end-of-line<br />
[[ -n "${key[Insert]}" ]] && bindkey -- "${key[Insert]}" overwrite-mode<br />
[[ -n "${key[Backspace]}" ]] && bindkey -- "${key[Backspace]}" backward-delete-char<br />
[[ -n "${key[Delete]}" ]] && bindkey -- "${key[Delete]}" delete-char<br />
[[ -n "${key[Up]}" ]] && bindkey -- "${key[Up]}" up-line-or-history<br />
[[ -n "${key[Down]}" ]] && bindkey -- "${key[Down]}" down-line-or-history<br />
[[ -n "${key[Left]}" ]] && bindkey -- "${key[Left]}" backward-char<br />
[[ -n "${key[Right]}" ]] && bindkey -- "${key[Right]}" forward-char<br />
[[ -n "${key[PageUp]}" ]] && bindkey -- "${key[PageUp]}" beginning-of-buffer-or-history<br />
[[ -n "${key[PageDown]}" ]] && bindkey -- "${key[PageDown]}" end-of-buffer-or-history<br />
[[ -n "${key[Shift-Tab]}" ]] && bindkey -- "${key[Shift-Tab]}" reverse-menu-complete<br />
<br />
# Finally, make sure the terminal is in application mode, when zle is<br />
# active. Only then are the values from $terminfo valid.<br />
if (( ${+terminfo[smkx]} && ${+terminfo[rmkx]} )); then<br />
autoload -Uz add-zle-hook-widget<br />
function zle_application_mode_start { echoti smkx }<br />
function zle_application_mode_stop { echoti rmkx }<br />
add-zle-hook-widget -Uz zle-line-init zle_application_mode_start<br />
add-zle-hook-widget -Uz zle-line-finish zle_application_mode_stop<br />
fi<br />
}}<br />
<br />
==== History search ====<br />
<br />
You need to set up the {{ic|key}} array and make sure that ZLE enters application mode to use the following instructions; see [[#Key bindings]].<br />
<br />
To enable history search add these lines to {{ic|.zshrc}} file:<br />
<br />
{{hc|~/.zshrc|<br />
autoload -Uz up-line-or-beginning-search down-line-or-beginning-search<br />
zle -N up-line-or-beginning-search<br />
zle -N down-line-or-beginning-search<br />
<br />
[[ -n "${key[Up]}" ]] && bindkey -- "${key[Up]}" up-line-or-beginning-search<br />
[[ -n "${key[Down]}" ]] && bindkey -- "${key[Down]}" down-line-or-beginning-search<br />
}}<br />
<br />
By doing this, only the past commands matching the current line up to the current cursor position will be shown when {{ic|Up}} or {{ic|Down}} keys are pressed.<br />
<br />
==== Shift, Alt, Ctrl and Meta modifiers ====<br />
<br />
xterm-compatible terminals can use extended key-definitions from {{man|5|user_caps}}. Those are combinations of {{ic|Shift}}, {{ic|Alt}}, {{ic|Ctrl}} and {{ic|Meta}} together with {{ic|Up}}, {{ic|Down}}, {{ic|Left}}, {{ic|Right}}, {{ic|PageUp}}, {{ic|PageDown}}, {{ic|Home}}, {{ic|End}} or {{ic|Del}}. Refer to the [https://sourceforge.net/p/zsh/code/ci/master/tree/Functions/Misc/zkbd zkbd source] for a list of recommended names for the modifier keys and key combinations.<br />
<br />
For example, for {{ic|Ctrl+Left}} to move to the beginning of the previous word and {{ic|Ctrl+Right}} to move to the beginning of the next word:<br />
<br />
{{hc|~/.zshrc|2=<br />
key[Control-Left]="${terminfo[kLFT5]}"<br />
key[Control-Right]="${terminfo[kRIT5]}"<br />
<br />
[[ -n "${key[Control-Left]}" ]] && bindkey -- "${key[Control-Left]}" backward-word<br />
[[ -n "${key[Control-Right]}" ]] && bindkey -- "${key[Control-Right]}" forward-word<br />
}}<br />
<br />
=== Prompts ===<br />
<br />
Zsh offers the options of using a prompt theme or, for users who are dissatisfied with the themes (or want to expand their usefulness), the possibility to build a custom prompt.<br />
<br />
==== Prompt themes ====<br />
<br />
Prompt themes are a quick and easy way to set up a colored prompt in Zsh. See {{man|1|zshcontrib|PROMPT THEMES}} for more information about them.<br />
<br />
To use a theme, make sure that prompt theme system is set to autoload in {{ic|.zshrc}}. This can be done by adding these lines to:<br />
<br />
{{hc|~/.zshrc|<br />
autoload -Uz promptinit<br />
promptinit<br />
}}<br />
<br />
Available prompt themes are listed by running the command:<br />
<br />
$ prompt -l<br />
<br />
For example, to use the {{ic|walters}} theme, enter:<br />
<br />
$ prompt walters<br />
<br />
To preview all available themes, use this command:<br />
<br />
$ prompt -p<br />
<br />
===== Manually installing prompt themes =====<br />
<br />
It is possible to install themes manually, without external configuration manager tools. For a local installation, first create a folder and add it to the {{ic|fpath}} array, eg:<br />
<br />
$ mkdir ~/.zprompts<br />
$ fpath=("$HOME/.zprompts" "$fpath[@]")<br />
<br />
Now create a symbolic link of your theme file in this folder:<br />
<br />
$ ln -s mytheme.zsh ~/.zprompts/prompt_mytheme_setup<br />
<br />
If instead you wish to install a theme globally, do:<br />
<br />
# ln -s mytheme.zsh /usr/share/zsh/functions/Prompts/prompt_mytheme_setup<br />
<br />
Now you should be able to activate it using:<br />
<br />
$ prompt mytheme<br />
<br />
If everything works, you can edit your {{ic|.zshrc}} accordingly.<br />
<br />
==== Customized prompt ====<br />
<br />
{{Expansion|Add a simple colorless {{ic|PROMPT}} example.}}<br />
<br />
Additionally to a primary left-sided prompt {{ic|PS1}} ({{ic|PROMPT}}, {{ic|prompt}}) that is common to all shells, Zsh also supports a right-sided prompt {{ic|RPS1}} ({{ic|RPROMPT}}). These two variables are the ones you will want to set to a custom value.<br />
<br />
Other special purpose prompts, such as {{ic|PS2}} ({{ic|PROMPT2}}), {{ic|PS3}} ({{ic|PROMPT3}}), {{ic|PS4}} ({{ic|PROMPT4}}), {{ic|RPS1}} ({{ic|RPROMPT}}), {{ic|RPS2}} ({{ic|RPROMPT2}}) and {{ic|SPROMPT}}, are explained in {{man|1|zshparam|PARAMETERS USED BY THE SHELL}}.<br />
<br />
All prompts can be customized with prompt escapes. The available prompt escapes are listed in {{man|1|zshmisc|EXPANSION OF PROMPT SEQUENCES}}.<br />
<br />
===== Colors =====<br />
<br />
Zsh sets colors differently than [[Bash/Prompt customization|Bash]], you do not need to use convoluted ANSI escape sequences or terminal capabilities from {{man|5|terminfo}}. Zsh provides convenient prompt escapes to set the foreground color, background color and other visual effects; see {{man|1|zshmisc|Visual effects}} for a list of them and their descriptions.<br />
<br />
[http://zsh.sourceforge.net/FAQ/zshfaq03.html#l42 Colors] can be specified using a decimal integer, the name of one of the eight most widely-supported colors or as a # followed by an RGB triplet in hexadecimal format. See the description of fg=colour in {{man|1|zshzle|CHARACTER HIGHLIGHTING}} for more details.<br />
<br />
Most terminals support the following colors by name:<br />
<br />
{| class="wikitable"<br />
|-<br />
! Name !! Number<br />
|-<br />
| {{ic|black}} || {{ic|0}}<br />
|-<br />
| {{ic|red}} || {{ic|1}}<br />
|-<br />
| {{ic|green}} || {{ic|2}}<br />
|-<br />
| {{ic|yellow}} || {{ic|3}}<br />
|-<br />
| {{ic|blue}} || {{ic|4}}<br />
|-<br />
| {{ic|magenta}} || {{ic|5}}<br />
|-<br />
| {{ic|cyan}} || {{ic|6}}<br />
|-<br />
| {{ic|white}} || {{ic|7}}<br />
|}<br />
<br />
Color numbers 0–255 for terminal emulators compatible with xterm 256 colors can be found in the [https://upload.wikimedia.org/wikipedia/commons/1/15/Xterm_256color_chart.svg xterm-256color chart].<br />
<br />
With a correctly set TERM environment variable, the terminal's supported maximum number of colors can be found from the {{man|5|terminfo}} database using {{ic|echoti colors}}. In the case of [https://gist.github.com/XVilka/8346728 24-bit colors], also check the COLORTERM environment variable with {{ic|print $COLORTERM}}. If it returns {{ic|24bit}} or {{ic|truecolor}} then your terminal supports 16777216 (2<sup>24</sup>) colors even if terminfo shows a smaller number.<br />
<br />
{{Note|<br />
* The colors 0–15 may differ between terminal emulators and their used color schemes.<br />
* Many terminal emulators display bold with a brighter color.<br />
}}<br />
<br />
{{Tip|<br />
* Prompt escapes can be tested with command {{ic|print -P ''"prompt escapes"''}}, for example: {{bc|$ print -P '%B%F{red}co%F{green}lo%F{blue}rs%f%b'}}<br />
* If you use 24-bit colors, you might want to load the {{ic|zsh/nearcolor}} module in terminals that do not support them. E.g.: {{bc|<nowiki>[[ "$COLORTERM" == (24bit|truecolor) || "${terminfo[colors]}" -eq '16777216' ]] || zmodload zsh/nearcolor</nowiki>}} See {{man|1|zshmodules|THE ZSH/NEARCOLOR MODULE}} for details about the {{ic|zsh/nearcolor}} module.<br />
}}<br />
<br />
===== Example =====<br />
<br />
{{Expansion|Add an example using a color from the 16–255 range and one with 24-bit color.}}<br />
<br />
This is an example of a two-sided prompt:<br />
<br />
PROMPT='%F{green}%n%f@%F{magenta}%m%f %F{blue}%B%~%b%f %# '<br />
RPROMPT='[%F{yellow}%?%f]'<br />
<br />
And here is how it will be displayed:<br />
<br />
<div style="font-family: monospace; white-space: pre; padding: 1em; background-color: #000; border: 1px solid #bcd; color: #c0c0c0; overflow: hidden;"><span style="float:left;"><span style="color: #008000;">username</span>@<span style="color: #800080;">host</span> <span style="color: #0000ff;">~</span> % </span><span style="float:right;">[<span style="color: #808000;">0</span>]</span></div><br />
<br />
=== Sample .zshrc files ===<br />
<br />
* To get the same setup as the [https://www.archlinux.org/download/ monthly ISO releases] (which use Zsh by default), install {{Pkg|grml-zsh-config}}. It includes the many tweaks and advanced optimizations from [https://grml.org/zsh/ grml].<br />
* https://github.com/MrElendig/dotfiles-alice/blob/master/.zshrc - basic setup, with dynamic prompt and window title/hardinfo.<br />
* https://github.com/slashbeast/conf-mgmt/blob/master/roles/home_files/files/DOTzshrc - zshrc with multiple features, be sure to check out comments into it. Notable features: confirm function to ensure that user want to run poweroff, reboot or hibernate, support for GIT in prompt (done without vcsinfo), tab completion with menu, printing current executed command into window's title bar and more.<br />
<br />
See [[dotfiles#User repositories]] for more.<br />
<br />
== Tips and tricks ==<br />
<br />
=== Autostart X at login ===<br />
<br />
See [[xinit#Autostart X at login]].<br />
<br />
=== Restore terminal settings after a program exits abnormally ===<br />
<br />
Many programs change the terminal state, and often do not restore terminal settings on exiting abnormally (e.g. when crashing or encountering SIGINT). <br />
<br />
This can typically be solved by executing {{man|1|reset}}:<br />
<br />
$ reset<br />
<br />
The following sections describe ways to avoid the need to manually reset the terminal.<br />
<br />
==== The ttyctl command ====<br />
<br />
The [http://zsh.sourceforge.net/Doc/Release/Shell-Builtin-Commands.html#index-tty_002c-freezing ttyctl] command can be used to "freeze/unfreeze" the terminal. To freeze the interactive shell on launch, use the following:<br />
<br />
{{hc|~/.zshrc|<br />
ttyctl -f<br />
}}<br />
<br />
==== Resetting the terminal with escape sequences ====<br />
<br />
[https://www.in-ulm.de/~mascheck/various/alternate_charset/ Alternate linedrawing character set] can screw up the terminal in a way which ttyctl cannot prevent.<br />
<br />
A simple solution is to output the escape sequences that reset the terminal from the {{ic|precmd}} hook function, so that they are executed every time before the prompt is drawn. For example, using [https://www.in-ulm.de/~mascheck/various/alternate_charset/#solution the escape sequence] {{ic|\e[0m\e(B\e)0\017\e[?5l\e7\e[0;0r\e8}}:<br />
<br />
{{hc|~/.zshrc|2=<br />
<br />
autoload -Uz add-zsh-hook<br />
<br />
function reset_broken_terminal () {<br />
printf '%b' '\e[0m\e(B\e)0\017\e[?5l\e7\e[0;0r\e8'<br />
}<br />
<br />
add-zsh-hook -Uz precmd reset_broken_terminal<br />
}}<br />
<br />
To test if it works, run: <br />
<br />
$ print '\e(0\e)B'<br />
<br />
=== Remembering recent directories ===<br />
<br />
==== Dirstack ====<br />
<br />
Zsh can be configured to remember the DIRSTACKSIZE last visited folders. This can then be used to ''cd'' them very quickly. You need to add some lines to your configuration file:<br />
<br />
{{hc|~/.zshrc|<nowiki><br />
autoload -Uz add-zsh-hook<br />
<br />
DIRSTACKFILE="${XDG_CACHE_HOME:-$HOME/.cache}/zsh/dirs"<br />
if [[ -f "$DIRSTACKFILE" ]] && (( ${#dirstack} == 0 )); then<br />
dirstack=("${(@f)"$(< "$DIRSTACKFILE")"}")<br />
[[ -d "${dirstack[1]}" ]] && cd -- "${dirstack[1]}"<br />
fi<br />
chpwd_dirstack() {<br />
print -l -- "$PWD" "${(u)dirstack[@]}" > "$DIRSTACKFILE"<br />
}<br />
add-zsh-hook -Uz chpwd chpwd_dirstack<br />
<br />
DIRSTACKSIZE='20'<br />
<br />
setopt AUTO_PUSHD PUSHD_SILENT PUSHD_TO_HOME<br />
<br />
## Remove duplicate entries<br />
setopt PUSHD_IGNORE_DUPS<br />
<br />
## This reverts the +/- operators.<br />
setopt PUSHD_MINUS<br />
</nowiki>}}<br />
<br />
Now use<br />
<br />
$ dirs -v<br />
<br />
to print the dirstack. Use {{ic|cd -<NUM>}} to go back to a visited folder. Use autocompletion after the dash. This proves very handy if using the autocompletion menu.<br />
<br />
{{Note|This will not work if you have more than one ''zsh'' session open, and attempt to {{ic|cd}}, due to a conflict in both sessions writing to the same file.}}<br />
<br />
==== cdr ====<br />
<br />
cdr allows you to change the working directory to a previous working directory from a list maintained automatically. It stores all entries in files that are maintained across sessions and (by default) between terminal emulators in the current session.<br />
<br />
See {{man|1|zshcontrib|REMEMBERING RECENT DIRECTORIES}} for setup instructions.<br />
<br />
=== Help command ===<br />
<br />
Unlike [[Bash]], Zsh does not enable a built in {{ic|help}} command, instead it provides {{ic|run-help}}. By default {{ic|run-help}} is an alias to {{ic|man}}, it can be either executed manually by prepending it to a command or it can be invoked for the currently typed command with the keyboard shortcuts {{ic|Alt+h}} or {{ic|Esc}} {{ic|h}}.<br />
<br />
Since by default it is just an alias to [[man]], it will only work on external commands. To improve its functionality, so that it works on shell builtins and other shell features, you need to use the {{ic|run-help}} function. See {{man|1|zshcontrib}} for more information on the {{ic|run-help}} and its assistant functions.<br />
<br />
First load the {{ic|run-help}} function and then remove the existing {{ic|run-help}} alias. For convenience {{ic|help}} can be aliased to {{ic|run-help}}. For example, add following to your {{ic|zshrc}}:<br />
<br />
{{bc|1=<br />
autoload -Uz run-help<br />
unalias run-help<br />
alias help=run-help<br />
}}<br />
<br />
Assistant functions have to be enabled separately:<br />
<br />
autoload -Uz run-help-git run-help-ip run-help-openssl run-help-p4 run-help-sudo run-help-svk run-help-svn<br />
<br />
For example, {{ic|run-help git commit}} command will now open the [[man page]] {{man|1|git-commit}} instead of {{man|1|git}}.<br />
<br />
=== Persistent rehash ===<br />
<br />
Typically, compinit will not automatically find new executables in the {{ic|$PATH}}. For example, after you install a new package, the files in {{ic|/usr/bin/}} would not be immediately or automatically included in the completion. Thus, to have these new executables included, one would run:<br />
<br />
$ rehash<br />
<br />
This 'rehash' can be set to happen automatically.[https://github.com/robbyrussell/oh-my-zsh/issues/3440] Simply include the following in your {{ic|zshrc}}:<br />
<br />
{{hc|~/.zshrc|<br />
zstyle ':completion:*' rehash true<br />
}}<br />
<br />
==== On-demand rehash ====<br />
<br />
As above, however [[pacman]] can be configured with [[Pacman#Hooks|hooks]] to automatically request a {{ic|rehash}}, which does not incur the performance penalty of constant rehashing as above. To enable this, create the {{ic|/etc/pacman.d/hooks}} directory, and a {{ic|/var/cache/zsh}} directory, then create a hook file:<br />
<br />
{{hc|head=/etc/pacman.d/hooks/zsh.hook|output=<br />
[Trigger]<br />
Operation = Install<br />
Operation = Upgrade<br />
Operation = Remove<br />
Type = Path<br />
Target = usr/bin/*<br />
[Action]<br />
Depends = zsh<br />
When = PostTransaction<br />
Exec = /usr/bin/install -Dm644 /dev/null /var/cache/zsh/pacman<br />
}}<br />
<br />
This keeps the modification date of the file {{ic|/var/cache/zsh/pacman}} consistent with the last time a package was installed, upgraded or removed. Then, {{ic|zsh}} must be coaxed into rehashing its own command cache when it goes out of date, by adding to your {{ic|~/.zshrc}}:<br />
<br />
{{hc|~/.zshrc|<nowiki><br />
zshcache_time="$(date +%s%N)"<br />
<br />
autoload -Uz add-zsh-hook<br />
<br />
rehash_precmd() {<br />
if [[ -a /var/cache/zsh/pacman ]]; then<br />
local paccache_time="$(date -r /var/cache/zsh/pacman +%s%N)"<br />
if (( zshcache_time < paccache_time )); then<br />
rehash<br />
zshcache_time="$paccache_time"<br />
fi<br />
fi<br />
}<br />
<br />
add-zsh-hook -Uz precmd rehash_precmd<br />
</nowiki>}}<br />
<br />
If the {{ic|precmd}} hook is triggered before {{ic|/var/cache/zsh/pacman}} is updated, completion may not work until a new prompt is initiated. Running an empty command, e.g. pressing {{ic|enter}}, should be sufficient.<br />
<br />
==== Alternative on-demand rehash using SIGUSR1 ====<br />
<br />
As above, however the hook file looks like this:<br />
<br />
{{hc|/etc/pacman.d/hooks/zsh-rehash.hook|output=<br />
[Trigger]<br />
Operation = Install<br />
Operation = Upgrade<br />
Operation = Remove<br />
Type = Path<br />
Target = usr/bin/*<br />
<br />
[Action]<br />
Depends = zsh<br />
Depends = procps-ng<br />
When = PostTransaction<br />
Exec = /usr/bin/pkill zsh --signal=USR1<br />
}}<br />
<br />
{{Warning|This sends SIGUSR1 to all running {{ic|zsh}} instances. Note that the default behavior for SIGUSR1 is terminate so when you first configure this all running {{ic|zsh}} instances of all users (including login shells) will terminate if they have not sourced the trap below.}}<br />
<br />
{{hc|~/.zshrc|<br />
TRAPUSR1() { rehash }<br />
}}<br />
<br />
The ''function trap'' above can be replaced with a ''list trap'' {{ic|trap 'rehash' USR1}}. See {{man|1|zshmisc|Trap Functions}} for differences between types of traps.<br />
<br />
This method will instantly {{ic|rehash}} all {{ic|zsh}} instances, removing the need to press enter to trigger {{ic|precmd}}.<br />
<br />
=== Bind key to ncurses application ===<br />
<br />
Bind a ncurses application to a keystroke, but it will not accept interaction. Use {{ic|BUFFER}} variable to make it work. The following example lets users open [[ncmpcpp]] using {{ic|Alt+\}}:<br />
<br />
{{hc|~/.zshrc|2=<br />
ncmpcppShow() {<br />
BUFFER="ncmpcpp"<br />
zle accept-line<br />
}<br />
zle -N ncmpcppShow<br />
bindkey '^[\' ncmpcppShow<br />
}}<br />
<br />
An alternate method, that will keep everything you entered in the line before calling application:<br />
<br />
{{hc|~/.zshrc|2=<br />
ncmpcppShow() {<br />
ncmpcpp <$TTY<br />
zle redisplay<br />
}<br />
zle -N ncmpcppShow<br />
bindkey '^[\' ncmpcppShow<br />
}}<br />
<br />
=== File manager key binds ===<br />
<br />
Key binds like those used in graphic file managers may come handy. The first comes back in directory history ({{ic|Alt+Left}}), the second let the user go to the parent directory ({{ic|Alt+Up}}). They also display the directory content.<br />
<br />
{{hc|~/.zshrc|<nowiki><br />
cdUndoKey() {<br />
popd<br />
zle reset-prompt<br />
print<br />
ls<br />
zle reset-prompt<br />
}<br />
<br />
cdParentKey() {<br />
pushd ..<br />
zle reset-prompt<br />
print<br />
ls<br />
zle reset-prompt<br />
}<br />
<br />
zle -N cdParentKey<br />
zle -N cdUndoKey<br />
bindkey '^[[1;3A' cdParentKey<br />
bindkey '^[[1;3D' cdUndoKey<br />
</nowiki>}}<br />
<br />
=== xterm title ===<br />
<br />
If your terminal emulator supports it, you can set its title from Zsh. This allows dynamically changing the title to display relevant information about the shell state, for example showing the user name and current directory or the currently executing command.<br />
<br />
The xterm title is set with the [https://www.tldp.org/HOWTO/Xterm-Title-3.html#ss3.1 xterm escape sequence] {{ic|\e]2;}}{{ic|\a}}. For example:<br />
<br />
$ print -n '\e]2;My xterm title\a'<br />
<br />
will set the title to<br />
<br />
My xterm title<br />
<br />
A simple way to have a dynamic title is to set the title in the {{ic|precmd}} and {{ic|preexec}} hook functions. See {{man|1|zshmisc|Hook Functions}} for a list of available hook functions and their descriptions.<br />
<br />
By using {{ic|print -P}} you can additionally take advantage of Zsh's prompt escapes.<br />
<br />
{{Tip|<br />
* Title printing can be split up in multiple commands as long as they are sequential.<br />
* [[GNU Screen]] sends the xterm title to the hardstatus ({{ic|%h}}). If you want to use Screen's [https://www.gnu.org/software/screen/manual/html_node/String-Escapes.html string escapes] (e.g. for colors) you should set the hardstatus with the {{ic|\e_}}{{ic|\e\\}} escape sequence. Otherwise, if string escapes are used in {{ic|\e]2;}}{{ic|\a}}, the terminal emulator will get a garbled title due to it being incapable of interpreting Screen's string escapes.<br />
}}<br />
<br />
{{Note|<br />
* Do not use the {{ic|-P}} option of {{ic|print}} when printing variables to prevent them from being parsed as prompt escapes.<br />
* Use the {{ic|q}} [http://zsh.sourceforge.net/Doc/Release/Expansion.html#Parameter-Expansion-Flags parameter expansion flag] when printing variables to prevent them from being parsed as escape sequences.<br />
}}<br />
<br />
{{hc|~/.zshrc|<nowiki><br />
autoload -Uz add-zsh-hook<br />
<br />
function xterm_title_precmd () {<br />
print -Pn -- '\e]2;%n@%m %~\a'<br />
[[ "$TERM" == 'screen'* ]] && print -Pn -- '\e_\005{g}%n\005{-}@\005{m}%m\005{-} \005{B}%~\005{-}\e\\'<br />
}<br />
<br />
function xterm_title_preexec () {<br />
print -Pn -- '\e]2;%n@%m %~ %# ' && print -n -- "${(q)1}\a"<br />
[[ "$TERM" == 'screen'* ]] && { print -Pn -- '\e_\005{g}%n\005{-}@\005{m}%m\005{-} \005{B}%~\005{-} %# ' && print -n -- "${(q)1}\e\\"; }<br />
}<br />
<br />
if [[ "$TERM" == (alacritty*|gnome*|konsole*|putty*|rxvt*|screen*|tmux*|xterm*) ]]; then<br />
add-zsh-hook -Uz precmd xterm_title_precmd<br />
add-zsh-hook -Uz preexec xterm_title_preexec<br />
fi<br />
</nowiki>}}<br />
<br />
==== Terminal emulator tab title ====<br />
<br />
Some terminal emulators and multiplexers support setting the title of the tab. The escape sequences depend on the terminal:<br />
<br />
{| class="wikitable sortable"<br />
! Terminal<br />
! Escape sequences<br />
! Description<br />
|-<br />
! [[GNU Screen]]<br />
| {{ic|\ek}}{{ic|\e\\}}<br />
| Screen's window title ({{ic|%t}}).<br />
|-<br />
! Konsole<br />
| {{ic|\e]30;}}{{ic|\a}}<br />
| Konsole's tab title.<br />
|}<br />
<br />
=== Shell environment detection ===<br />
<br />
See [https://gitlab.com/jdorel-documentation/shell-environment-detection a repository about shell environment detection] for tests to detect the shell environment. This includes login/interactive shell, Xorg session, TTY and SSH session.<br />
<br />
=== /dev/tcp equivalent: ztcp ===<br />
<br />
Use the {{ic|zsh/net/tcp}} module:<br />
<br />
$ zmodload zsh/net/tcp<br />
<br />
You can now establish TCP connections:<br />
<br />
$ ztcp example.com 80<br />
<br />
=== Shortcut to exit shell on partial command line ===<br />
<br />
By default, {{ic|Ctrl+d}} will not close your shell if the command line is filled, this fixes it:<br />
<br />
{{hc|.zshrc|<br />
exit_zsh() { exit }<br />
zle -N exit_zsh<br />
bindkey '^D' exit_zsh<br />
}}<br />
<br />
== Third-party extensions ==<br />
<br />
=== Configuration frameworks ===<br />
<br />
* {{App|oh-my-zsh|A popular, community-driven framework for managing your Zsh configuration. It comes bundled with a ton of helpful functions, helpers, plugins, themes.|https://github.com/robbyrussell/oh-my-zsh|{{AUR|oh-my-zsh-git}}}}<br />
* {{App|Prezto|A configuration framework for Zsh. It comes with modules, enriching the command line interface environment with sane defaults, aliases, functions, auto completion, and prompt themes.|https://github.com/sorin-ionescu/prezto|{{AUR|prezto-git}}}}<br />
* {{App|ZIM|A configuration framework with blazing speed and modular extensions. Zim is very easy to customize, and comes with a rich set of modules and features without compromising on speed or functionality.|https://github.com/zimfw/zimfw|{{AUR|zsh-zim-git}}}}<br />
<br />
=== Plugin managers ===<br />
<br />
* {{App|Antibody|A performance-focused plugin manager similar to Antigen.|https://github.com/getantibody/antibody|{{AUR|antibody}}}}<br />
* {{App|zinit (previously "zplugin")|Flexible Zsh plugin manager with clean fpath, reports, completion management, turbo mode|https://github.com/zdharma/zinit|{{AUR|zsh-zplugin-git}}}}<br />
* {{App|Antigen|A plugin manager for Zsh, inspired by oh-my-zsh and vundle. [https://github.com/zsh-users/antigen/issues/673 ABANDONED]|https://github.com/zsh-users/antigen|{{AUR|antigen-git}}}}<br />
* {{App|zgen|A lightweight and simple plugin manager for Zsh. [https://github.com/tarjoilija/zgen/issues/123 ABANDONED]|https://github.com/tarjoilija/zgen|{{AUR|zgen-git}}}}<br />
* {{App|zplug|A next-generation plugin manager for Zsh. [https://github.com/zplug/zplug/issues/403#issuecomment-477520784 ABANDONED]|https://github.com/zplug/zplug|{{AUR|zplug}}}}<br />
<br />
=== Fish-like syntax highlighting and autosuggestions ===<br />
<br />
[[Fish]] provides very powerful shell syntax highlighting and autosuggestions. To use both in Zsh, you can install {{pkg|zsh-syntax-highlighting}}, {{pkg|zsh-autosuggestions}}, and finally [[source]] one or both of the provided scripts from your zshrc:<br />
<br />
source /usr/share/zsh/plugins/zsh-syntax-highlighting/zsh-syntax-highlighting.zsh<br />
source /usr/share/zsh/plugins/zsh-autosuggestions/zsh-autosuggestions.zsh<br />
<br />
=== The "command not found" handler ===<br />
<br />
[[pacman]] includes functionality to search for packages containing a file. The following command-not-found handler will use pacman directly to search for matching packages when an unknown command is executed.<br />
<br />
{{hc|1=~/.zshrc|2=<br />
command_not_found_handler() {<br />
local pkgs cmd="$1" files=()<br />
printf 'zsh: command not found: %s' "$cmd" # print command not found asap, then search for packages<br />
files=(${(f)"$(pacman -F --machinereadable -- "/usr/bin/${cmd}")"})<br />
if (( ${#files[@]} )); then<br />
printf '\r%s may be found in the following packages:\n' "$cmd"<br />
local res=() repo package version file<br />
for file in "$files[@]"; do<br />
res=("${(0)file}")<br />
repo="$res[1]"<br />
package="$res[2]"<br />
version="$res[3]"<br />
file="$res[4]"<br />
printf ' %s/%s %s: /%s\n' "$repo" "$package" "$version" "$file"<br />
done<br />
else<br />
printf '\n'<br />
fi<br />
return 127<br />
}<br />
}}<br />
<br />
{{Note|The files database of pacman is separate from the normal sync database and it needs to be fetched using {{ic|pacman -Fy}}. See [[pacman#Search for a package that contains a specific file]] for details.}}<br />
<br />
Alternatively, [[pkgfile]] includes a Zsh script file that provides a {{ic|command_not_found_handler}} function that will automatically search the pkgfile database when entering an unrecognized command.<br />
<br />
You need to [[source]] the script to enable it. For example:<br />
<br />
{{hc|~/.zshrc|<br />
source /usr/share/doc/pkgfile/command-not-found.zsh<br />
}}<br />
<br />
{{Note|The pkgfile database may need to be updated before this will work. See [[pkgfile#Installation]] for details.}}<br />
<br />
== Uninstallation ==<br />
<br />
Change the default shell before removing the {{Pkg|zsh}} package.<br />
<br />
{{Warning|Failure to follow the below procedure may result in users no longer having access to a working shell.}}<br />
<br />
Run following command:<br />
<br />
$ chsh -s /bin/bash ''user''<br />
<br />
Use it for every user with ''zsh'' set as their login shell (including root if needed). When completed, the {{Pkg|zsh}} package can be removed.<br />
<br />
Alternatively, change the default shell back to Bash by editing {{ic|/etc/passwd}} as root.<br />
<br />
{{Warning|It is strongly recommended to use {{man|8|vipw}} when editing {{ic|/etc/passwd}} as it helps prevent invalid entries and/or syntax errors.}}<br />
<br />
For example, change the following:<br />
<br />
''username'':x:1000:1000:''Full Name'',,,:/home/''username'':/bin/zsh<br />
<br />
To this:<br />
<br />
''username'':x:1000:1000:''Full Name'',,,:/home/''username'':/bin/bash<br />
<br />
== See also ==<br />
<br />
* [[Wikipedia:Zsh]]<br />
* [http://zsh.sourceforge.net/Intro/intro_toc.html An Introduction to the Z Shell]<br />
* [http://zsh.sourceforge.net/Guide/zshguide.html A User's Guide to ZSH]<br />
* [http://zsh.sourceforge.net/Doc/Release/index-frame.html The Z Shell Manual] (different format available [http://zsh.sourceforge.net/Doc/ here])<br />
* [http://zsh.sourceforge.net/FAQ/zshfaq01.html Zsh FAQ]<br />
* [http://zshwiki.org/home/ Zsh Wiki]<br />
* {{man|1|zsh-lovers}} (available as {{pkg|zsh-lovers}} package)<br />
* [[Gentoo: Zsh/Guide]]<br />
* [http://www.bash2zsh.com/zsh_refcard/refcard.pdf Bash2Zsh Reference Card]</div>Phireskyhttps://wiki.archlinux.org/index.php?title=Flatpak&diff=639213Flatpak2020-10-19T19:02:29Z<p>Phiresky: clarify XDG_DATA_DIRS</p>
<hr />
<div>[[Category:Development]]<br />
[[de:Flatpak]]<br />
[[es:Flatpak]]<br />
[[ja:Flatpak]]<br />
[[pt:Flatpak]]<br />
[[zh-hans:Flatpak]]<br />
{{Related articles start}}<br />
{{Related|Snapd}}<br />
{{Related|bubblewrap}}<br />
{{Related articles end}}<br />
From the project [https://github.com/flatpak/flatpak/blob/master/README.md README]: "''[http://flatpak.org Flatpak] is a system for building, distributing and running sandboxed desktop applications on Linux.''"<br />
<br />
From {{man|1|flatpak}}:<br />
<br />
:''Flatpak is a tool for managing applications and the runtimes they use. In the Flatpak model, applications can be built and distributed independently from the host system they are used on, and they are isolated from the host system ('sandboxed') to some degree, at runtime.''<br />
:''Flatpak uses [https://ostree.readthedocs.io/en/latest/ OSTree] to distribute and deploy data. The repositories it uses are OSTree repositories and can be manipulated with the ostree utility. Installed runtimes and applications are OSTree checkouts.''<br />
<br />
== Installation ==<br />
<br />
[[Install]] the {{pkg|flatpak}} package.<br />
<br />
{{Note|If you want to build flatpaks with {{ic|flatpak-builder}} you will need to install the optional dependencies of {{pkg|elfutils}} and {{pkg|patch}}.}}<br />
<br />
== Managing repositories ==<br />
<br />
{{Note|By default, every flatpak command works system-wide, i.e. packages are installed for all users in the computer and flatpak requires the user to supply root's password. To install packages and work with repositories on a single user (with no need of superuser rights) you can add the option {{ic|--user}} to each command. If you want, for example, to add a repository only visible to you, you should run {{ic|flatpak remote-add --user ''name'' ''location''}}. To install a package visible only to you, run {{ic|flatpack install --user ''package-name''}}.}}<br />
<br />
=== Add a repository ===<br />
<br />
To add a remote flatpak repository do:<br />
<br />
$ flatpak remote-add ''name'' ''location''<br />
<br />
where ''name'' is the name for the new remote, and ''location'' is the path or URL for the repository.<br />
<br />
For example to add the official [https://flathub.org/ Flathub repository]:<br />
<br />
$ flatpak remote-add --if-not-exists flathub <nowiki>https://dl.flathub.org/repo/flathub.flatpakrepo</nowiki><br />
<br />
=== Delete a repository ===<br />
<br />
To delete a remote flatpak repository do:<br />
<br />
$ flatpak remote-delete ''name''<br />
<br />
where ''name'' is the name of the remote repository to be deleted.<br />
<br />
=== List repositories ===<br />
<br />
To list all the added repositories do:<br />
<br />
$ flatpak remotes<br />
<br />
== Managing runtimes and applications ==<br />
<br />
=== Search for a remote runtime or application ===<br />
<br />
Before being able to search for a runtime or application in a newly added remote repository, we need to retrieve the appstream data for it:<br />
<br />
{{hc|$ flatpak update|<br />
Looking for updates...<br />
Updating appstream data for remote ''name''<br />
}}<br />
<br />
Then we can proceed to search for a package with {{ic|flatpak search ''packagename''}}, ''e.g.'' to look for the package {{ic|libreoffice}} with the {{ic|flathub}} remote configured:<br />
<br />
{{hc|$ flatpak search libreoffice|<br />
Application ID Version Branch Remotes Description <br />
org.libreoffice.LibreOffice stable flathub The LibreOffice productivity suite<br />
}}<br />
<br />
=== List all available runtimes and applications ===<br />
<br />
To list all available runtimes and applications in a remote repository named ''remote'' do:<br />
<br />
$ flatpak remote-ls ''remote''<br />
<br />
=== Install a runtime or application ===<br />
<br />
To install a runtime or application do:<br />
<br />
$ flatpak install ''remote'' ''name''<br />
<br />
where ''remote'' is the name of the remote repository, and ''name'' is the name of the application or runtime to install.<br />
<br />
{{Tip|You can use partial identifiers {{ic|flatpak install ''partial-name''}} (for example {{ic|flatpak install libreoffice}}).}}<br />
<br />
=== List installed runtimes and applications ===<br />
<br />
To list installed runtimes and applications do:<br />
<br />
$ flatpak list<br />
<br />
=== Run applications ===<br />
<br />
Binaries are available in {{ic|/var/lib/flatpak/exports/bin}}, which is automatically added to $PATH by {{ic|/etc/profile.d/flatpak-bindir.sh}}. You may have to re-login to apply the change.<br />
<br />
Flatpak applications can also be run with the command line:<br />
<br />
$ flatpak run ''name''<br />
<br />
=== Update a runtime or application ===<br />
<br />
To update a runtime or application named ''name'' do:<br />
<br />
$ flatpak update ''name''<br />
<br />
=== Uninstall a runtime or application ===<br />
<br />
To uninstall a runtime or application named ''name'' do:<br />
<br />
$ flatpak uninstall ''name''<br />
<br />
{{Tip|You can uninstall unused flatpak "refs" (aka orphans with no application/runtime) with {{ic|flatpak uninstall --unused}}.}}<br />
<br />
=== Adding Flatpak .desktop files to your menu ===<br />
Flatpak expects window managers to respect the XDG_DATA_DIRS environment variable to discover applications. This variable is set by the script {{ic|/etc/profile.d/flatpak.sh}}. Updating the environment may require restarting the session. If the launcher does not support XDG_DATA_DIRS, you can edit the list of directories scanned and add these to it:<br />
<br />
~/.local/share/flatpak/exports/share/applications<br />
/var/lib/flatpak/exports/share/applications<br />
<br />
This is known to be necessary in Awesome.<br />
<br />
=== Viewing sandbox permissions of application ===<br />
Flatpak applications come with predefined sandbox rules which defines the resources and file system paths the application is allowed to access.<br />
To view the specific application permissions do:<br />
$ flatpak info --show-permissions ''name''<br />
The reference of the sandbox permission names can be found on [https://docs.flatpak.org/en/latest/sandbox-permissions-reference.html official flatpak documentation].<br />
<br />
=== Overriding sandbox permissions of applications ===<br />
If you find the predefined permissions of the application too lax or too restrictive you can change to anything you want using {{ic|flatpak override}} command.<br />
For example:<br />
flatpak override --nofilesystem=home ''name''<br />
This will prevent the application access to your home folder.<br />
<br />
Every type of permission such as device, filesystem or socket has an command line option that allows that particular permission and a separated option that denies. For example, in case of device access {{ic|1=--device=''device_name''}} allows access, {{ic|1=--nodevice=''device_name''}} denies the permission to access device.<br />
<br />
For all permission types commands consult the manual page: {{man|1|flatpak-override}}<br />
<br />
Permission overrides can be reset to defaults with command:<br />
$ flatpak override --reset ''name''<br />
<br />
[https://flathub.org/apps/details/com.github.tchx84.Flatseal Flatseal] is a GUI permissions manager which offer simple point-and-click permissions operations.<br />
<br />
== Creating a custom base runtime ==<br />
<br />
{{Expansion|This can certainly be improved. It also has problems with D-Bus for GNOME apps.}}<br />
<br />
{{Warning|If you want to release your software to the public as a Flatpak, an Arch-based runtime is unsuitable. In this case, you will want to follow [http://docs.flatpak.org official documentation] to integrate your software into the proper Flatpak ecosystem using the [http://flatpak.org/runtimes.html common runtimes].}}<br />
<br />
{{Note|<br />
* You may want to use an untrusted, unprivileged user account for bundling untrusted software because the software is not sandboxed during app and runtime creation.<br />
* When distributing bundles to others, you may be legally obliged to provide the source code of some of the bundled software upon request. You may want to use [[ABS]] to build these packages from source.<br />
}}<br />
<br />
You can create a custom Arch-based base runtime and base SDK for Flatpak using pacman. You can then use it for building and packaging applications. This is an alternative for personal use to the default {{ic|org.freedesktop.BasePlatform}} and {{ic|org.freedesktop.BaseSdk}} runtimes.<br />
<br />
In addition to {{Pkg|flatpak}}, you need to have installed {{Pkg|fakeroot}} and for pacman hooks support also {{Pkg|fakechroot}}.<br />
<br />
First, start by creating a directory for building the runtime and possibly applications.<br />
<br />
$ mkdir ''myflatpakbuilddir''<br />
$ cd ''myflatpakbuilddir''<br />
<br />
You can then prepare a directory for building the runtime base platform. The files subdirectory will contain what will later be the {{ic|/usr}} directory in the sandbox. Therefore you will need to create symbolic links so the default {{ic|/usr/share}} etc. from Arch can still be accessed at the usual path.<br />
<br />
$ mkdir -p ''myruntime''/files/var/lib/pacman<br />
$ touch ''myruntime''/files/.ref<br />
$ ln -s /usr/usr/share ''myruntime''/files/share<br />
$ ln -s /usr/usr/include ''myruntime''/files/include<br />
$ ln -s /usr/usr/local ''myruntime''/files/local<br />
<br />
Make your host OS fonts available to the Arch runtime:<br />
<br />
$ mkdir -p ''myruntime''/files/usr/share/fonts<br />
$ ln -s /run/host/fonts ''myruntime''/files/usr/share/fonts/flatpakhostfonts<br />
<br />
You need and may want to adapt your {{ic|pacman.conf}} before installing packages to the runtime. Copy {{ic|/etc/pacman.conf}} to your build directory and then make the following changes:<br />
<br />
* Remove the {{ic|CheckSpace}} option so pacman will not complain about errors finding the root filesystem for checking disk space.<br />
* Remove any undesired custom repositories and {{ic|IgnorePkg}}, {{ic|IgnoreGroup}}, {{ic|NoUpgrade}} and {{ic|NoExtract}} settings that are needed only for the host system.<br />
<br />
Now install the packages for the runtime.<br />
<br />
$ fakechroot fakeroot pacman -Syu --root ''myruntime''/files --dbpath ''myruntime''/files/var/lib/pacman --config pacman.conf base<br />
$ mv pacman.conf ''myruntime''/files/etc/pacman.conf<br />
<br />
Set up the [[Locale|locales]] to be used by editing {{ic|''myruntime''/files/etc/locale.gen}}. Then regenerate the runtime’s locales.<br />
<br />
$ fakechroot chroot ''myruntime''/files locale-gen<br />
<br />
The base SDK can be created from the base runtime with added applications needed for building packages and running pacman.<br />
<br />
$ cp -r ''myruntime'' mysdk<br />
$ fakechroot fakeroot pacman -S --root mysdk/files --dbpath mysdk/files/var/lib/pacman --config mysdk/files/etc/pacman.conf base-devel fakeroot fakechroot --needed<br />
<br />
Insert metadata about runtime and SDK.<br />
<br />
{{hc|''myruntime''/metadata|2=<br />
[Runtime]<br />
name=org.mydomain.BasePlatform<br />
runtime=org.mydomain.BasePlatform/x86_64/2016-06-26<br />
sdk=org.mydomain.BaseSdk/x86_64/2016-06-26<br />
}}<br />
<br />
{{hc|mysdk/metadata|2=<br />
[Runtime]<br />
name=org.mydomain.BaseSdk<br />
runtime=org.mydomain.BasePlatform/x86_64/2016-06-26<br />
sdk=org.mydomain.BaseSdk/x86_64/2016-06-26<br />
}}<br />
<br />
Add base runtime and SDK to a local repository in the current directory. You may want to give them appropriate commit messages such as “My Arch base runtime” and “My Arch base SDK”.<br />
<br />
$ ostree init --mode archive-z2 --repo=.<br />
$ EDITOR="nano -w" ostree commit -b runtime/org.mydomain.BasePlatform/x86_64/2016-06-26 --tree=dir=''myruntime''<br />
$ EDITOR="nano -w" ostree commit -b runtime/org.mydomain.BaseSdk/x86_64/2016-06-26 --tree=dir=mysdk<br />
$ ostree summary -u<br />
<br />
Install the runtime and SDK.<br />
<br />
$ flatpak remote-add --user --no-gpg-verify myarchos file://$(pwd)<br />
$ flatpak install --user myarchos org.mydomain.BasePlatform 2016-06-26<br />
$ flatpak install --user myarchos org.mydomain.BaseSdk 2016-06-26<br />
<br />
=== Creating apps with pacman ===<br />
<br />
As an alternative to building applications [http://flatpak.org/developer.html the usual way], we can use pacman to create a containerized version of the regular Arch packages. Note that {{ic|/usr}} is read-only when creating apps, so we can not use Arch’s packages when building an app. To create a real app with pacman, we can either<br />
<br />
* use pacman to create a runtime containing all dependencies<br />
* and compile the app ourselves [http://flatpak.org/developer.html as usual] or perhaps using pacman with a custom [[PKGBUILD]] tailored to Flatpak which uses {{ic|1=--prefix=/app}} for the {{ic|configure}} script,<br />
<br />
or we can<br />
* use pacman to create a runtime containing the app installed with pacman<br />
* and create a dummy app to launch it.<br />
<br />
For doing the latter, first create a runtime using pacman such as this one for {{Pkg|gedit}}. The runtime is first initialized and prepared for use with pacman.<br />
<br />
$ flatpak build-init -w geditruntime org.mydomain.geditruntime org.mydomain.BaseSdk org.mydomain.BasePlatform 2016-06-26<br />
$ flatpak build geditruntime sed -i "s/^#Server/Server/g" /etc/pacman.d/mirrorlist<br />
$ flatpak build geditruntime ln -s /usr/var/lib /var/lib<br />
$ flatpak build geditruntime fakeroot pacman-key --init<br />
$ flatpak build geditruntime fakeroot pacman-key --populate archlinux<br />
<br />
Then the package is installed. The host’s network connection must be made available to pacman.<br />
<br />
$ flatpak build --share=network geditruntime fakechroot fakeroot pacman --root /usr -S gedit<br />
<br />
You can test the installation before finishing the runtime (without proper sandboxing).<br />
<br />
$ flatpak build --socket=x11 geditruntime gedit<br />
<br />
Now finish building the runtime and export it to a new local repository. pacman’s GnuPG keys have permissions that may interfere and need to be removed first.<br />
<br />
$ flatpak build geditruntime rm -r /etc/pacman.d/gnupg<br />
$ flatpak build-finish geditruntime<br />
$ sed -i "s/\[Application\]/\[Runtime\]/;s/runtime=org.mydomain.BasePlatform/runtime=org.mydomain.geditruntime/" geditruntime/metadata<br />
$ flatpak build-export -r geditrepo geditruntime<br />
<br />
Then create a dummy app.<br />
<br />
$ flatpak build-init geditapp org.gnome.gedit org.mydomain.BaseSdk org.mydomain.geditruntime<br />
<br />
Now finish the dummy app. You can fine-tune the app’s access permissions when sandboxed by giving additional options when finishing the build. For possible options see the [[#See also|Flatpak documentation]] and the [https://gitlab.gnome.org/GNOME/gnome-apps-nightly/tree/master GNOME manifest files]. Alternatively, adapt {{ic|geditapp/metadata}} to your needs after finishing the build but before exporting. When the metadata file is complete, export the app to the repository.<br />
<br />
$ flatpak build-finish geditapp --socket=x11 ''[possibly other options]'' --command=gedit<br />
$ flatpak build-export geditrepo geditapp<br />
<br />
Install it along with the runtime.<br />
<br />
$ flatpak --user remote-add --no-gpg-verify geditrepo geditrepo<br />
$ flatpak install --user geditrepo org.mydomain.geditruntime<br />
$ flatpak install --user geditrepo org.gnome.gedit<br />
$ flatpak run org.gnome.gedit<br />
<br />
== See also ==<br />
<br />
* [http://flatpak.org Official website]<br />
* [https://github.com/flatpak/flatpak/wiki Official Github wiki]<br />
* [[Wikipedia::Flatpak|Wikipedia page]]<br />
* [https://wiki.gnome.org/Projects/SandboxedApps Gnome SandboxedApps]<br />
* [https://community.kde.org/Guidelines_and_HOWTOs/Flatpak KDE Testing Runtime and Applications]</div>Phireskyhttps://wiki.archlinux.org/index.php?title=Flatpak&diff=639212Flatpak2020-10-19T19:00:55Z<p>Phiresky: path dir is auto added</p>
<hr />
<div>[[Category:Development]]<br />
[[de:Flatpak]]<br />
[[es:Flatpak]]<br />
[[ja:Flatpak]]<br />
[[pt:Flatpak]]<br />
[[zh-hans:Flatpak]]<br />
{{Related articles start}}<br />
{{Related|Snapd}}<br />
{{Related|bubblewrap}}<br />
{{Related articles end}}<br />
From the project [https://github.com/flatpak/flatpak/blob/master/README.md README]: "''[http://flatpak.org Flatpak] is a system for building, distributing and running sandboxed desktop applications on Linux.''"<br />
<br />
From {{man|1|flatpak}}:<br />
<br />
:''Flatpak is a tool for managing applications and the runtimes they use. In the Flatpak model, applications can be built and distributed independently from the host system they are used on, and they are isolated from the host system ('sandboxed') to some degree, at runtime.''<br />
:''Flatpak uses [https://ostree.readthedocs.io/en/latest/ OSTree] to distribute and deploy data. The repositories it uses are OSTree repositories and can be manipulated with the ostree utility. Installed runtimes and applications are OSTree checkouts.''<br />
<br />
== Installation ==<br />
<br />
[[Install]] the {{pkg|flatpak}} package.<br />
<br />
{{Note|If you want to build flatpaks with {{ic|flatpak-builder}} you will need to install the optional dependencies of {{pkg|elfutils}} and {{pkg|patch}}.}}<br />
<br />
== Managing repositories ==<br />
<br />
{{Note|By default, every flatpak command works system-wide, i.e. packages are installed for all users in the computer and flatpak requires the user to supply root's password. To install packages and work with repositories on a single user (with no need of superuser rights) you can add the option {{ic|--user}} to each command. If you want, for example, to add a repository only visible to you, you should run {{ic|flatpak remote-add --user ''name'' ''location''}}. To install a package visible only to you, run {{ic|flatpack install --user ''package-name''}}.}}<br />
<br />
=== Add a repository ===<br />
<br />
To add a remote flatpak repository do:<br />
<br />
$ flatpak remote-add ''name'' ''location''<br />
<br />
where ''name'' is the name for the new remote, and ''location'' is the path or URL for the repository.<br />
<br />
For example to add the official [https://flathub.org/ Flathub repository]:<br />
<br />
$ flatpak remote-add --if-not-exists flathub <nowiki>https://dl.flathub.org/repo/flathub.flatpakrepo</nowiki><br />
<br />
=== Delete a repository ===<br />
<br />
To delete a remote flatpak repository do:<br />
<br />
$ flatpak remote-delete ''name''<br />
<br />
where ''name'' is the name of the remote repository to be deleted.<br />
<br />
=== List repositories ===<br />
<br />
To list all the added repositories do:<br />
<br />
$ flatpak remotes<br />
<br />
== Managing runtimes and applications ==<br />
<br />
=== Search for a remote runtime or application ===<br />
<br />
Before being able to search for a runtime or application in a newly added remote repository, we need to retrieve the appstream data for it:<br />
<br />
{{hc|$ flatpak update|<br />
Looking for updates...<br />
Updating appstream data for remote ''name''<br />
}}<br />
<br />
Then we can proceed to search for a package with {{ic|flatpak search ''packagename''}}, ''e.g.'' to look for the package {{ic|libreoffice}} with the {{ic|flathub}} remote configured:<br />
<br />
{{hc|$ flatpak search libreoffice|<br />
Application ID Version Branch Remotes Description <br />
org.libreoffice.LibreOffice stable flathub The LibreOffice productivity suite<br />
}}<br />
<br />
=== List all available runtimes and applications ===<br />
<br />
To list all available runtimes and applications in a remote repository named ''remote'' do:<br />
<br />
$ flatpak remote-ls ''remote''<br />
<br />
=== Install a runtime or application ===<br />
<br />
To install a runtime or application do:<br />
<br />
$ flatpak install ''remote'' ''name''<br />
<br />
where ''remote'' is the name of the remote repository, and ''name'' is the name of the application or runtime to install.<br />
<br />
{{Tip|You can use partial identifiers {{ic|flatpak install ''partial-name''}} (for example {{ic|flatpak install libreoffice}}).}}<br />
<br />
=== List installed runtimes and applications ===<br />
<br />
To list installed runtimes and applications do:<br />
<br />
$ flatpak list<br />
<br />
=== Run applications ===<br />
<br />
Binaries are available in {{ic|/var/lib/flatpak/exports/bin}}, which is automatically added to $PATH by {{ic|/etc/profile.d/flatpak-bindir.sh}}. You may have to re-login to apply the change.<br />
<br />
Flatpak applications can also be run with the command line:<br />
<br />
$ flatpak run ''name''<br />
<br />
=== Update a runtime or application ===<br />
<br />
To update a runtime or application named ''name'' do:<br />
<br />
$ flatpak update ''name''<br />
<br />
=== Uninstall a runtime or application ===<br />
<br />
To uninstall a runtime or application named ''name'' do:<br />
<br />
$ flatpak uninstall ''name''<br />
<br />
{{Tip|You can uninstall unused flatpak "refs" (aka orphans with no application/runtime) with {{ic|flatpak uninstall --unused}}.}}<br />
<br />
=== Adding Flatpak .desktop files to your menu ===<br />
Flatpak expects window managers to respect the XDG_DATA_DIRS environment variable to discover applications. This may require restarting the session or the launcher may not support this. In such a case where you can edit the list of directories scanned, add these to it:<br />
<br />
~/.local/share/flatpak/exports/share/applications<br />
/var/lib/flatpak/exports/share/applications<br />
<br />
This is known to be necessary in Awesome.<br />
<br />
=== Viewing sandbox permissions of application ===<br />
Flatpak applications come with predefined sandbox rules which defines the resources and file system paths the application is allowed to access.<br />
To view the specific application permissions do:<br />
$ flatpak info --show-permissions ''name''<br />
The reference of the sandbox permission names can be found on [https://docs.flatpak.org/en/latest/sandbox-permissions-reference.html official flatpak documentation].<br />
<br />
=== Overriding sandbox permissions of applications ===<br />
If you find the predefined permissions of the application too lax or too restrictive you can change to anything you want using {{ic|flatpak override}} command.<br />
For example:<br />
flatpak override --nofilesystem=home ''name''<br />
This will prevent the application access to your home folder.<br />
<br />
Every type of permission such as device, filesystem or socket has an command line option that allows that particular permission and a separated option that denies. For example, in case of device access {{ic|1=--device=''device_name''}} allows access, {{ic|1=--nodevice=''device_name''}} denies the permission to access device.<br />
<br />
For all permission types commands consult the manual page: {{man|1|flatpak-override}}<br />
<br />
Permission overrides can be reset to defaults with command:<br />
$ flatpak override --reset ''name''<br />
<br />
[https://flathub.org/apps/details/com.github.tchx84.Flatseal Flatseal] is a GUI permissions manager which offer simple point-and-click permissions operations.<br />
<br />
== Creating a custom base runtime ==<br />
<br />
{{Expansion|This can certainly be improved. It also has problems with D-Bus for GNOME apps.}}<br />
<br />
{{Warning|If you want to release your software to the public as a Flatpak, an Arch-based runtime is unsuitable. In this case, you will want to follow [http://docs.flatpak.org official documentation] to integrate your software into the proper Flatpak ecosystem using the [http://flatpak.org/runtimes.html common runtimes].}}<br />
<br />
{{Note|<br />
* You may want to use an untrusted, unprivileged user account for bundling untrusted software because the software is not sandboxed during app and runtime creation.<br />
* When distributing bundles to others, you may be legally obliged to provide the source code of some of the bundled software upon request. You may want to use [[ABS]] to build these packages from source.<br />
}}<br />
<br />
You can create a custom Arch-based base runtime and base SDK for Flatpak using pacman. You can then use it for building and packaging applications. This is an alternative for personal use to the default {{ic|org.freedesktop.BasePlatform}} and {{ic|org.freedesktop.BaseSdk}} runtimes.<br />
<br />
In addition to {{Pkg|flatpak}}, you need to have installed {{Pkg|fakeroot}} and for pacman hooks support also {{Pkg|fakechroot}}.<br />
<br />
First, start by creating a directory for building the runtime and possibly applications.<br />
<br />
$ mkdir ''myflatpakbuilddir''<br />
$ cd ''myflatpakbuilddir''<br />
<br />
You can then prepare a directory for building the runtime base platform. The files subdirectory will contain what will later be the {{ic|/usr}} directory in the sandbox. Therefore you will need to create symbolic links so the default {{ic|/usr/share}} etc. from Arch can still be accessed at the usual path.<br />
<br />
$ mkdir -p ''myruntime''/files/var/lib/pacman<br />
$ touch ''myruntime''/files/.ref<br />
$ ln -s /usr/usr/share ''myruntime''/files/share<br />
$ ln -s /usr/usr/include ''myruntime''/files/include<br />
$ ln -s /usr/usr/local ''myruntime''/files/local<br />
<br />
Make your host OS fonts available to the Arch runtime:<br />
<br />
$ mkdir -p ''myruntime''/files/usr/share/fonts<br />
$ ln -s /run/host/fonts ''myruntime''/files/usr/share/fonts/flatpakhostfonts<br />
<br />
You need and may want to adapt your {{ic|pacman.conf}} before installing packages to the runtime. Copy {{ic|/etc/pacman.conf}} to your build directory and then make the following changes:<br />
<br />
* Remove the {{ic|CheckSpace}} option so pacman will not complain about errors finding the root filesystem for checking disk space.<br />
* Remove any undesired custom repositories and {{ic|IgnorePkg}}, {{ic|IgnoreGroup}}, {{ic|NoUpgrade}} and {{ic|NoExtract}} settings that are needed only for the host system.<br />
<br />
Now install the packages for the runtime.<br />
<br />
$ fakechroot fakeroot pacman -Syu --root ''myruntime''/files --dbpath ''myruntime''/files/var/lib/pacman --config pacman.conf base<br />
$ mv pacman.conf ''myruntime''/files/etc/pacman.conf<br />
<br />
Set up the [[Locale|locales]] to be used by editing {{ic|''myruntime''/files/etc/locale.gen}}. Then regenerate the runtime’s locales.<br />
<br />
$ fakechroot chroot ''myruntime''/files locale-gen<br />
<br />
The base SDK can be created from the base runtime with added applications needed for building packages and running pacman.<br />
<br />
$ cp -r ''myruntime'' mysdk<br />
$ fakechroot fakeroot pacman -S --root mysdk/files --dbpath mysdk/files/var/lib/pacman --config mysdk/files/etc/pacman.conf base-devel fakeroot fakechroot --needed<br />
<br />
Insert metadata about runtime and SDK.<br />
<br />
{{hc|''myruntime''/metadata|2=<br />
[Runtime]<br />
name=org.mydomain.BasePlatform<br />
runtime=org.mydomain.BasePlatform/x86_64/2016-06-26<br />
sdk=org.mydomain.BaseSdk/x86_64/2016-06-26<br />
}}<br />
<br />
{{hc|mysdk/metadata|2=<br />
[Runtime]<br />
name=org.mydomain.BaseSdk<br />
runtime=org.mydomain.BasePlatform/x86_64/2016-06-26<br />
sdk=org.mydomain.BaseSdk/x86_64/2016-06-26<br />
}}<br />
<br />
Add base runtime and SDK to a local repository in the current directory. You may want to give them appropriate commit messages such as “My Arch base runtime” and “My Arch base SDK”.<br />
<br />
$ ostree init --mode archive-z2 --repo=.<br />
$ EDITOR="nano -w" ostree commit -b runtime/org.mydomain.BasePlatform/x86_64/2016-06-26 --tree=dir=''myruntime''<br />
$ EDITOR="nano -w" ostree commit -b runtime/org.mydomain.BaseSdk/x86_64/2016-06-26 --tree=dir=mysdk<br />
$ ostree summary -u<br />
<br />
Install the runtime and SDK.<br />
<br />
$ flatpak remote-add --user --no-gpg-verify myarchos file://$(pwd)<br />
$ flatpak install --user myarchos org.mydomain.BasePlatform 2016-06-26<br />
$ flatpak install --user myarchos org.mydomain.BaseSdk 2016-06-26<br />
<br />
=== Creating apps with pacman ===<br />
<br />
As an alternative to building applications [http://flatpak.org/developer.html the usual way], we can use pacman to create a containerized version of the regular Arch packages. Note that {{ic|/usr}} is read-only when creating apps, so we can not use Arch’s packages when building an app. To create a real app with pacman, we can either<br />
<br />
* use pacman to create a runtime containing all dependencies<br />
* and compile the app ourselves [http://flatpak.org/developer.html as usual] or perhaps using pacman with a custom [[PKGBUILD]] tailored to Flatpak which uses {{ic|1=--prefix=/app}} for the {{ic|configure}} script,<br />
<br />
or we can<br />
* use pacman to create a runtime containing the app installed with pacman<br />
* and create a dummy app to launch it.<br />
<br />
For doing the latter, first create a runtime using pacman such as this one for {{Pkg|gedit}}. The runtime is first initialized and prepared for use with pacman.<br />
<br />
$ flatpak build-init -w geditruntime org.mydomain.geditruntime org.mydomain.BaseSdk org.mydomain.BasePlatform 2016-06-26<br />
$ flatpak build geditruntime sed -i "s/^#Server/Server/g" /etc/pacman.d/mirrorlist<br />
$ flatpak build geditruntime ln -s /usr/var/lib /var/lib<br />
$ flatpak build geditruntime fakeroot pacman-key --init<br />
$ flatpak build geditruntime fakeroot pacman-key --populate archlinux<br />
<br />
Then the package is installed. The host’s network connection must be made available to pacman.<br />
<br />
$ flatpak build --share=network geditruntime fakechroot fakeroot pacman --root /usr -S gedit<br />
<br />
You can test the installation before finishing the runtime (without proper sandboxing).<br />
<br />
$ flatpak build --socket=x11 geditruntime gedit<br />
<br />
Now finish building the runtime and export it to a new local repository. pacman’s GnuPG keys have permissions that may interfere and need to be removed first.<br />
<br />
$ flatpak build geditruntime rm -r /etc/pacman.d/gnupg<br />
$ flatpak build-finish geditruntime<br />
$ sed -i "s/\[Application\]/\[Runtime\]/;s/runtime=org.mydomain.BasePlatform/runtime=org.mydomain.geditruntime/" geditruntime/metadata<br />
$ flatpak build-export -r geditrepo geditruntime<br />
<br />
Then create a dummy app.<br />
<br />
$ flatpak build-init geditapp org.gnome.gedit org.mydomain.BaseSdk org.mydomain.geditruntime<br />
<br />
Now finish the dummy app. You can fine-tune the app’s access permissions when sandboxed by giving additional options when finishing the build. For possible options see the [[#See also|Flatpak documentation]] and the [https://gitlab.gnome.org/GNOME/gnome-apps-nightly/tree/master GNOME manifest files]. Alternatively, adapt {{ic|geditapp/metadata}} to your needs after finishing the build but before exporting. When the metadata file is complete, export the app to the repository.<br />
<br />
$ flatpak build-finish geditapp --socket=x11 ''[possibly other options]'' --command=gedit<br />
$ flatpak build-export geditrepo geditapp<br />
<br />
Install it along with the runtime.<br />
<br />
$ flatpak --user remote-add --no-gpg-verify geditrepo geditrepo<br />
$ flatpak install --user geditrepo org.mydomain.geditruntime<br />
$ flatpak install --user geditrepo org.gnome.gedit<br />
$ flatpak run org.gnome.gedit<br />
<br />
== See also ==<br />
<br />
* [http://flatpak.org Official website]<br />
* [https://github.com/flatpak/flatpak/wiki Official Github wiki]<br />
* [[Wikipedia::Flatpak|Wikipedia page]]<br />
* [https://wiki.gnome.org/Projects/SandboxedApps Gnome SandboxedApps]<br />
* [https://community.kde.org/Guidelines_and_HOWTOs/Flatpak KDE Testing Runtime and Applications]</div>Phireskyhttps://wiki.archlinux.org/index.php?title=Zsh&diff=635335Zsh2020-09-14T08:15:03Z<p>Phiresky: Add command not found handler example using pacman -F</p>
<hr />
<div>[[Category:Command-line shells]]<br />
[[cs:Zsh]]<br />
[[de:Zsh]]<br />
[[es:Zsh]]<br />
[[fa:Zsh]]<br />
[[fr:Zsh]]<br />
[[ja:Zsh]]<br />
[[ru:Zsh]]<br />
[[zh-hans:Zsh]]<br />
[https://www.zsh.org/ Zsh] is a powerful [[shell]] that operates as both an interactive shell and as a scripting language interpreter. While being compatible with the POSIX sh (not by default, only if issuing {{ic|emulate sh}}), it offers advantages such as improved [http://zsh.sourceforge.net/Guide/zshguide06.html tab completion] and [http://zsh.sourceforge.net/Doc/Release/Expansion.html globbing].<br />
<br />
The [http://zsh.sourceforge.net/FAQ/zshfaq01.html#l4 Zsh FAQ] offers more reasons to use Zsh.<br />
<br />
== Installation ==<br />
<br />
Before starting, users may want to see what shell is currently being used:<br />
<br />
$ echo $SHELL<br />
<br />
[[Install]] the {{Pkg|zsh}} package. For additional completion definitions, install the {{pkg|zsh-completions}} package as well.<br />
<br />
=== Initial configuration ===<br />
<br />
Make sure that Zsh has been installed correctly by running the following in a terminal:<br />
<br />
$ zsh<br />
<br />
You should now see ''zsh-newuser-install'', which will walk you through some basic configuration. If you want to skip this, press {{ic|q}}. If you did not see it, you can invoke it manually with:<br />
<br />
$ autoload -Uz zsh-newuser-install<br />
$ zsh-newuser-install -f<br />
<br />
{{Note|Make sure your terminal's size is at least 72×15 otherwise ''zsh-newuser-install'' will not run.}}<br />
<br />
=== Making Zsh your default shell ===<br />
<br />
Change your shell to {{ic|/usr/bin/zsh}}. See [[Command-line shell#Changing your default shell]].<br />
<br />
{{Tip|If replacing {{Pkg|bash}}, users may want to move some code from {{ic|~/.bashrc}} to {{ic|~/.zshrc}} (e.g. the prompt and the [[Bash#Aliases|aliases]]) and from {{ic|~/.bash_profile}} to {{ic|~/.zprofile}} (e.g. [[xinit#Autostart X at login|the code that starts the X Window System]]).}}<br />
<br />
== Startup/Shutdown files ==<br />
<br />
{{Tip|<br />
* See [http://zsh.sourceforge.net/Guide/zshguide02.html A User's Guide to the Z-Shell] for explanation on interactive and login shells, and what to put in your startup files.<br />
* You could consider [[Command-line shell#Standardisation|implementing a standard path]] for your Zsh configuration files.<br />
}}<br />
<br />
{{Note|<br />
* If {{ic|$ZDOTDIR}} is not set, {{ic|$HOME}} is used instead.<br />
* If option {{ic|RCS}} is unset in any of the files, no configuration files will be read after that file.<br />
* If option {{ic|GLOBAL_RCS}} is unset in any of the files, no global configuration files ({{ic|/etc/zsh/*}}) will be read after that file.<br />
}}<br />
<br />
When starting, Zsh will read commands from the following files in this order by default, provided they exist. <br />
<br />
* {{ic|/etc/zsh/zshenv}} Used for setting [[environment variables]] for all users; it should not contain commands that produce output or assume the shell is attached to a TTY. When this file exists it will '''''always''''' be read, this cannot be overridden.<br />
* {{ic|$ZDOTDIR/.zshenv}} Used for setting user's environment variables; it should not contain commands that produce output or assume the shell is attached to a TTY. When this file exists it will '''''always''''' be read.<br />
* {{ic|/etc/zsh/zprofile}} Used for executing commands at start for all users, will be read when starting as a '''''login shell'''''. Please note that on Arch Linux, by default it contains [https://github.com/archlinux/svntogit-packages/blob/packages/zsh/trunk/zprofile one line] which source the {{ic|/etc/profile}}. See warning below before wanting to remove that!<br />
** {{ic|/etc/profile}} This file should be sourced by all POSIX sh-compatible shells upon login: it sets up {{ic|$PATH}} and other environment variables and application-specific ({{ic|/etc/profile.d/*.sh}}) settings upon login.<br />
* {{ic|$ZDOTDIR/.zprofile}} Used for executing user's commands at start, will be read when starting as a '''''login shell'''''. Typically used to autostart graphical sessions and to set session-wide environment variables.<br />
* {{ic|/etc/zsh/zshrc}} Used for setting interactive shell configuration and executing commands for all users, will be read when starting as an '''''interactive shell'''''.<br />
* {{ic|$ZDOTDIR/.zshrc}} Used for setting user's interactive shell configuration and executing commands, will be read when starting as an '''''interactive shell'''''.<br />
* {{ic|/etc/zsh/zlogin}} Used for executing commands for all users at ending of initial progress, will be read when starting as a '''''login shell'''''.<br />
* {{ic|$ZDOTDIR/.zlogin}} Used for executing user's commands at ending of initial progress, will be read when starting as a '''''login shell'''''. Typically used to autostart command line utilities. Should not be used to autostart graphical sessions, as at this point the session might contain configuration meant only for an interactive shell.<br />
* {{ic|$ZDOTDIR/.zlogout}} Used for executing commands when a '''''login shell''''' '''exits'''.<br />
* {{ic|/etc/zsh/zlogout}} Used for executing commands for all users when a '''''login shell''''' '''exits'''.<br />
<br />
See [https://blog.flowblok.id.au/2013-02/shell-startup-scripts.html#implementation the graphic representation].<br />
<br />
{{Note|<br />
* {{ic|$HOME/.profile}} is not a part of the Zsh startup files and '''is not sourced''' by Zsh unless Zsh is invoked as {{ic|sh}} or {{ic|ksh}} and started as a login shell. For more details about the sh and [[ksh]] compatibility modes refer to {{man|1|zsh|COMPATIBILITY}}.<br />
* The paths used in Arch's {{Pkg|zsh}} package are different from the default ones used in the [[man page]]s ({{Bug|48992}}).<br />
}}<br />
<br />
{{Warning|Do not remove the default [https://github.com/archlinux/svntogit-packages/blob/packages/zsh/trunk/zprofile one line] in {{ic|/etc/zsh/zprofile}}, otherwise it will break the integrity of other packages which provide some scripts in {{ic|/etc/profile.d/}}.}}<br />
<br />
== Configure Zsh ==<br />
<br />
Although Zsh is usable out of the box, it is almost certainly not set up the way most users would like to use it. But due to the sheer amount of customization available in Zsh, configuring Zsh can be a daunting and time-consuming experience.<br />
<br />
=== Simple .zshrc ===<br />
<br />
Included below is a sample configuration file. It provides a decent set of default options as well as giving examples of many ways that Zsh can be customized. In order to use this configuration save it as a file named {{ic|.zshrc}}.<br />
<br />
{{Tip|Apply the changes without needing to logout and then back in by running {{ic|source ~/.zshrc}}.}}<br />
<br />
Here is a simple {{ic|.zshrc}}:<br />
<br />
{{hc|~/.zshrc|<br />
autoload -Uz compinit promptinit<br />
compinit<br />
promptinit<br />
<br />
# This will set the default prompt to the walters theme<br />
prompt walters<br />
}}<br />
<br />
=== Configuring $PATH ===<br />
<br />
Zsh ties the {{ic|PATH}} variable to a {{ic|path}} array. They are automatically synchronized. This allows us to easily manipulate {{ic|PATH}} by simply modifying the array. See [http://zsh.sourceforge.net/Guide/zshguide02.html#l24 A User's Guide to the Z-Shell] for details.<br />
<br />
The line {{ic|typeset -U PATH path}}, where the {{ic|-U}} stands for unique, instructs the shell to discard duplicates from both {{ic|$PATH}} and {{ic|$path}}:<br />
<br />
{{hc|~/.zshenv|2=<br />
typeset -U PATH path<br />
path=("$HOME/.local/bin" ''/other/things/in/path'' "$path[@]")<br />
export PATH<br />
}}<br />
<br />
=== Command completion ===<br />
<br />
Perhaps the most compelling feature of Zsh is its advanced autocompletion abilities. At the very least, enable autocompletion in {{ic|.zshrc}}. To enable autocompletion, add the following to your {{ic|~/.zshrc}}:<br />
<br />
{{hc|~/.zshrc|<br />
autoload -Uz compinit<br />
compinit<br />
}}<br />
<br />
The above configuration includes ssh/scp/sftp hostnames completion but in order for this feature to work, users must not enable ssh's hostname hashing (i.e. option {{ic|HashKnownHosts}} in ssh client configuration).<br />
<br />
For autocompletion with an arrow-key driven interface, add the following to:<br />
<br />
{{hc|~/.zshrc|<br />
zstyle ':completion:*' menu select<br />
}}<br />
<br />
To activate the menu, press {{ic|Tab}} twice.<br />
<br />
For autocompletion of command line switches for aliases, add the following to:<br />
<br />
{{hc|~/.zshrc|<br />
setopt COMPLETE_ALIASES<br />
}}<br />
<br />
For enabling autocompletion of privileged environments in privileged commands (e.g. if you complete a command starting with [[sudo]], completion scripts will also try to determine your completions with sudo), include:<br />
<br />
{{hc|~/.zshrc|<br />
zstyle ':completion::complete:*' gain-privileges 1<br />
}}<br />
<br />
{{Warning|This will let Zsh completion scripts run commands with sudo privileges. You should not enable this if you use untrusted autocompletion scripts.}}<br />
<br />
{{Note|This special kind of context-aware completion is only available for a small number of commands.}}<br />
<br />
=== Key bindings ===<br />
<br />
Zsh does not use [[readline]], instead it uses its own and more powerful Zsh Line Editor (ZLE). It does not read {{ic|/etc/inputrc}} or {{ic|~/.inputrc}}. Read [https://sgeb.io/posts/2014/04/zsh-zle-custom-widgets/ A closer look at the zsh line editor and creating custom widgets] for an introduction to ZLE configuration.<br />
<br />
ZLE has an [[Emacs]] mode and a [[vi]] mode. If one of the {{ic|VISUAL}} or {{ic|EDITOR}} [[environment variables]] contain the string {{ic|vi}} then vi mode will be used; otherwise, it will default to Emacs mode. Set the mode explicitly with {{ic|bindkey -e}} or {{ic|bindkey -v}} respectively for Emacs mode or vi mode.<br />
<br />
Key bindings are assigned by mapping an escape sequence matching a keypress to a ZLE widget. The available widgets, with descriptions of their actions and their default keybindings, are listed in {{man|1|zshzle|STANDARD WIDGETS}} and {{man|1|zshcontrib|ZLE FUNCTIONS}}.<br />
<br />
The recommended way to set key bindings in Zsh is by using string capabilities from {{man|5|terminfo}}. For example[https://web.archive.org/web/20180704181216/http://zshwiki.org/home/zle/bindkeys][https://www.zsh.org/mla/users/2010/msg00065.html]:<br />
<br />
{{hc|~/.zshrc|2=<br />
# create a zkbd compatible hash;<br />
# to add other keys to this hash, see: man 5 terminfo<br />
typeset -g -A key<br />
<br />
key[Home]="${terminfo[khome]}"<br />
key[End]="${terminfo[kend]}"<br />
key[Insert]="${terminfo[kich1]}"<br />
key[Backspace]="${terminfo[kbs]}"<br />
key[Delete]="${terminfo[kdch1]}"<br />
key[Up]="${terminfo[kcuu1]}"<br />
key[Down]="${terminfo[kcud1]}"<br />
key[Left]="${terminfo[kcub1]}"<br />
key[Right]="${terminfo[kcuf1]}"<br />
key[PageUp]="${terminfo[kpp]}"<br />
key[PageDown]="${terminfo[knp]}"<br />
key[Shift-Tab]="${terminfo[kcbt]}"<br />
<br />
# setup key accordingly<br />
[[ -n "${key[Home]}" ]] && bindkey -- "${key[Home]}" beginning-of-line<br />
[[ -n "${key[End]}" ]] && bindkey -- "${key[End]}" end-of-line<br />
[[ -n "${key[Insert]}" ]] && bindkey -- "${key[Insert]}" overwrite-mode<br />
[[ -n "${key[Backspace]}" ]] && bindkey -- "${key[Backspace]}" backward-delete-char<br />
[[ -n "${key[Delete]}" ]] && bindkey -- "${key[Delete]}" delete-char<br />
[[ -n "${key[Up]}" ]] && bindkey -- "${key[Up]}" up-line-or-history<br />
[[ -n "${key[Down]}" ]] && bindkey -- "${key[Down]}" down-line-or-history<br />
[[ -n "${key[Left]}" ]] && bindkey -- "${key[Left]}" backward-char<br />
[[ -n "${key[Right]}" ]] && bindkey -- "${key[Right]}" forward-char<br />
[[ -n "${key[PageUp]}" ]] && bindkey -- "${key[PageUp]}" beginning-of-buffer-or-history<br />
[[ -n "${key[PageDown]}" ]] && bindkey -- "${key[PageDown]}" end-of-buffer-or-history<br />
[[ -n "${key[Shift-Tab]}" ]] && bindkey -- "${key[Shift-Tab]}" reverse-menu-complete<br />
<br />
# Finally, make sure the terminal is in application mode, when zle is<br />
# active. Only then are the values from $terminfo valid.<br />
if (( ${+terminfo[smkx]} && ${+terminfo[rmkx]} )); then<br />
autoload -Uz add-zle-hook-widget<br />
function zle_application_mode_start { echoti smkx }<br />
function zle_application_mode_stop { echoti rmkx }<br />
add-zle-hook-widget -Uz zle-line-init zle_application_mode_start<br />
add-zle-hook-widget -Uz zle-line-finish zle_application_mode_stop<br />
fi<br />
}}<br />
<br />
==== History search ====<br />
<br />
You need to set up the {{ic|key}} array and make sure that ZLE enters application mode to use the following instructions; see [[#Key bindings]].<br />
<br />
To enable history search add these lines to {{ic|.zshrc}} file:<br />
<br />
{{hc|~/.zshrc|<br />
autoload -Uz up-line-or-beginning-search down-line-or-beginning-search<br />
zle -N up-line-or-beginning-search<br />
zle -N down-line-or-beginning-search<br />
<br />
[[ -n "${key[Up]}" ]] && bindkey -- "${key[Up]}" up-line-or-beginning-search<br />
[[ -n "${key[Down]}" ]] && bindkey -- "${key[Down]}" down-line-or-beginning-search<br />
}}<br />
<br />
By doing this, only the past commands matching the current line up to the current cursor position will be shown when {{ic|Up}} or {{ic|Down}} keys are pressed.<br />
<br />
==== Shift, Alt, Ctrl and Meta modifiers ====<br />
<br />
xterm-compatible terminals can use extended key-definitions from {{man|5|user_caps}}. Those are combinations of {{ic|Shift}}, {{ic|Alt}}, {{ic|Ctrl}} and {{ic|Meta}} together with {{ic|Up}}, {{ic|Down}}, {{ic|Left}}, {{ic|Right}}, {{ic|PageUp}}, {{ic|PageDown}}, {{ic|Home}}, {{ic|End}} or {{ic|Del}}. Refer to the [https://sourceforge.net/p/zsh/code/ci/master/tree/Functions/Misc/zkbd zkbd source] for a list of recommended names for the modifier keys and key combinations.<br />
<br />
For example, for {{ic|Ctrl+Left}} to move to the beginning of the previous word and {{ic|Ctrl+Right}} to move to the beginning of the next word:<br />
<br />
{{hc|~/.zshrc|2=<br />
key[Control-Left]="${terminfo[kLFT5]}"<br />
key[Control-Right]="${terminfo[kRIT5]}"<br />
<br />
[[ -n "${key[Control-Left]}" ]] && bindkey -- "${key[Control-Left]}" backward-word<br />
[[ -n "${key[Control-Right]}" ]] && bindkey -- "${key[Control-Right]}" forward-word<br />
}}<br />
<br />
=== Prompts ===<br />
<br />
Zsh offers the options of using a prompt theme or, for users who are dissatisfied with the themes (or want to expand their usefulness), the possibility to build a custom prompt.<br />
<br />
==== Prompt themes ====<br />
<br />
Prompt themes are a quick and easy way to set up a colored prompt in Zsh. See {{man|1|zshcontrib|PROMPT THEMES}} for more information about them.<br />
<br />
To use a theme, make sure that prompt theme system is set to autoload in {{ic|.zshrc}}. This can be done by adding these lines to:<br />
<br />
{{hc|~/.zshrc|<br />
autoload -Uz promptinit<br />
promptinit<br />
}}<br />
<br />
Available prompt themes are listed by running the command:<br />
<br />
$ prompt -l<br />
<br />
For example, to use the {{ic|walters}} theme, enter:<br />
<br />
$ prompt walters<br />
<br />
To preview all available themes, use this command:<br />
<br />
$ prompt -p<br />
<br />
===== Manually installing prompt themes =====<br />
<br />
It is possible to install themes manually, without external configuration manager tools. For a local installation, first create a folder and add it to the {{ic|fpath}} array, eg:<br />
<br />
$ mkdir ~/.zprompts<br />
$ fpath=("$HOME/.zprompts" "$fpath[@]")<br />
<br />
Now create a symbolic link of your theme file in this folder:<br />
<br />
$ ln -s mytheme.zsh ~/.zprompts/prompt_mytheme_setup<br />
<br />
If instead you wish to install a theme globally, do:<br />
<br />
# ln -s mytheme.zsh /usr/share/zsh/functions/Prompts/prompt_mytheme_setup<br />
<br />
Now you should be able to activate it using:<br />
<br />
$ prompt mytheme<br />
<br />
If everything works, you can edit your {{ic|.zshrc}} accordingly.<br />
<br />
==== Customized prompt ====<br />
<br />
{{Expansion|Add a simple colorless {{ic|PROMPT}} example.}}<br />
<br />
Additionally to a primary left-sided prompt {{ic|PS1}} ({{ic|PROMPT}}, {{ic|prompt}}) that is common to all shells, Zsh also supports a right-sided prompt {{ic|RPS1}} ({{ic|RPROMPT}}). These two variables are the ones you will want to set to a custom value.<br />
<br />
Other special purpose prompts, such as {{ic|PS2}} ({{ic|PROMPT2}}), {{ic|PS3}} ({{ic|PROMPT3}}), {{ic|PS4}} ({{ic|PROMPT4}}), {{ic|RPS1}} ({{ic|RPROMPT}}), {{ic|RPS2}} ({{ic|RPROMPT2}}) and {{ic|SPROMPT}}, are explained in {{man|1|zshparam|PARAMETERS USED BY THE SHELL}}.<br />
<br />
All prompts can be customized with prompt escapes. The available prompt escapes are listed in {{man|1|zshmisc|EXPANSION OF PROMPT SEQUENCES}}.<br />
<br />
===== Colors =====<br />
<br />
Zsh sets colors differently than [[Bash/Prompt customization|Bash]], you do not need to use convoluted ANSI escape sequences or terminal capabilities from {{man|5|terminfo}}. Zsh provides convenient prompt escapes to set the foreground color, background color and other visual effects; see {{man|1|zshmisc|Visual effects}} for a list of them and their descriptions.<br />
<br />
[http://zsh.sourceforge.net/FAQ/zshfaq03.html#l42 Colors] can be specified using a decimal integer, the name of one of the eight most widely-supported colors or as a # followed by an RGB triplet in hexadecimal format. See the description of fg=colour in {{man|1|zshzle|CHARACTER HIGHLIGHTING}} for more details.<br />
<br />
Most terminals support the following colors by name:<br />
<br />
{| class="wikitable"<br />
|-<br />
! Name !! Number<br />
|-<br />
| {{ic|black}} || {{ic|0}}<br />
|-<br />
| {{ic|red}} || {{ic|1}}<br />
|-<br />
| {{ic|green}} || {{ic|2}}<br />
|-<br />
| {{ic|yellow}} || {{ic|3}}<br />
|-<br />
| {{ic|blue}} || {{ic|4}}<br />
|-<br />
| {{ic|magenta}} || {{ic|5}}<br />
|-<br />
| {{ic|cyan}} || {{ic|6}}<br />
|-<br />
| {{ic|white}} || {{ic|7}}<br />
|}<br />
<br />
Color numbers 0–255 for terminal emulators compatible with xterm 256 colors can be found in the [https://upload.wikimedia.org/wikipedia/commons/1/15/Xterm_256color_chart.svg xterm-256color chart].<br />
<br />
With a correctly set TERM environment variable, the terminal's supported maximum number of colors can be found from the {{man|5|terminfo}} database using {{ic|echoti colors}}. In the case of [https://gist.github.com/XVilka/8346728 24-bit colors], also check the COLORTERM environment variable with {{ic|print $COLORTERM}}. If it returns {{ic|24bit}} or {{ic|truecolor}} then your terminal supports 16777216 (2<sup>24</sup>) colors even if terminfo shows a smaller number.<br />
<br />
{{Note|<br />
* The colors 0–15 may differ between terminal emulators and their used color schemes.<br />
* Many terminal emulators display bold with a brighter color.<br />
}}<br />
<br />
{{Tip|<br />
* Prompt escapes can be tested with command {{ic|print -P ''"prompt escapes"''}}, for example: {{bc|$ print -P '%B%F{red}co%F{green}lo%F{blue}rs%f%b'}}<br />
* If you use 24-bit colors, you might want to load the {{ic|zsh/nearcolor}} module in terminals that do not support them. E.g.: {{bc|<nowiki>[[ "$COLORTERM" == (24bit|truecolor) || "${terminfo[colors]}" -eq '16777216' ]] || zmodload zsh/nearcolor</nowiki>}} See {{man|1|zshmodules|THE ZSH/NEARCOLOR MODULE}} for details about the {{ic|zsh/nearcolor}} module.<br />
}}<br />
<br />
===== Example =====<br />
<br />
{{Expansion|Add an example using a color from the 16–255 range and one with 24-bit color.}}<br />
<br />
This is an example of a two-sided prompt:<br />
<br />
PROMPT='%F{green}%n%f@%F{magenta}%m%f %F{blue}%B%~%b%f %# '<br />
RPROMPT='[%F{yellow}%?%f]'<br />
<br />
And here is how it will be displayed:<br />
<br />
<div style="font-family: monospace; white-space: pre; padding: 1em; background-color: #000; border: 1px solid #bcd; color: #c0c0c0; overflow: hidden;"><span style="float:left;"><span style="color: #008000;">username</span>@<span style="color: #800080;">host</span> <span style="color: #0000ff;">~</span> % </span><span style="float:right;">[<span style="color: #808000;">0</span>]</span></div><br />
<br />
=== Sample .zshrc files ===<br />
<br />
* To get the same setup as the [https://www.archlinux.org/download/ monthly ISO releases] (which use Zsh by default), install {{Pkg|grml-zsh-config}}. It includes the many tweaks and advanced optimizations from [https://grml.org/zsh/ grml].<br />
* https://github.com/MrElendig/dotfiles-alice/blob/master/.zshrc - basic setup, with dynamic prompt and window title/hardinfo.<br />
* https://github.com/slashbeast/conf-mgmt/blob/master/roles/home_files/files/DOTzshrc - zshrc with multiple features, be sure to check out comments into it. Notable features: confirm function to ensure that user want to run poweroff, reboot or hibernate, support for GIT in prompt (done without vcsinfo), tab completion with menu, printing current executed command into window's title bar and more.<br />
<br />
See [[dotfiles#User repositories]] for more.<br />
<br />
== Tips and tricks ==<br />
<br />
=== Autostart X at login ===<br />
<br />
See [[xinit#Autostart X at login]].<br />
<br />
=== Restore terminal settings after a program exits abnormally ===<br />
<br />
Many programs change the terminal state, and often do not restore terminal settings on exiting abnormally (e.g. when crashing or encountering SIGINT). <br />
<br />
This can typically be solved by executing {{man|1|reset}}:<br />
<br />
$ reset<br />
<br />
The following sections describe ways to avoid the need to manually reset the terminal.<br />
<br />
==== The ttyctl command ====<br />
<br />
The [http://zsh.sourceforge.net/Doc/Release/Shell-Builtin-Commands.html#index-tty_002c-freezing ttyctl] command can be used to "freeze/unfreeze" the terminal. To freeze the interactive shell on launch, use the following:<br />
<br />
{{hc|~/.zshrc|<br />
ttyctl -f<br />
}}<br />
<br />
==== Resetting the terminal with escape sequences ====<br />
<br />
[https://www.in-ulm.de/~mascheck/various/alternate_charset/ Alternate linedrawing character set] can screw up the terminal in a way which ttyctl cannot prevent.<br />
<br />
A simple solution is to output the escape sequences that reset the terminal from the {{ic|precmd}} hook function, so that they are executed every time before the prompt is drawn. For example, using [https://www.in-ulm.de/~mascheck/various/alternate_charset/#solution the escape sequence] {{ic|\e[0m\e(B\e)0\017\e[?5l\e7\e[0;0r\e8}}:<br />
<br />
{{hc|~/.zshrc|2=<br />
<br />
autoload -Uz add-zsh-hook<br />
<br />
function reset_broken_terminal () {<br />
printf '%b' '\e[0m\e(B\e)0\017\e[?5l\e7\e[0;0r\e8'<br />
}<br />
<br />
add-zsh-hook -Uz precmd reset_broken_terminal<br />
}}<br />
<br />
To test if it works, run: <br />
<br />
$ print '\e(0\e)B'<br />
<br />
=== Remembering recent directories ===<br />
<br />
==== Dirstack ====<br />
<br />
Zsh can be configured to remember the DIRSTACKSIZE last visited folders. This can then be used to ''cd'' them very quickly. You need to add some lines to your configuration file:<br />
<br />
{{hc|~/.zshrc|<nowiki><br />
autoload -Uz add-zsh-hook<br />
<br />
DIRSTACKFILE="${XDG_CACHE_HOME:-$HOME/.cache}/zsh/dirs"<br />
if [[ -f "$DIRSTACKFILE" ]] && (( ${#dirstack} == 0 )); then<br />
dirstack=("${(@f)"$(< "$DIRSTACKFILE")"}")<br />
[[ -d "${dirstack[1]}" ]] && cd -- "${dirstack[1]}"<br />
fi<br />
chpwd_dirstack() {<br />
print -l -- "$PWD" "${(u)dirstack[@]}" > "$DIRSTACKFILE"<br />
}<br />
add-zsh-hook -Uz chpwd chpwd_dirstack<br />
<br />
DIRSTACKSIZE='20'<br />
<br />
setopt AUTO_PUSHD PUSHD_SILENT PUSHD_TO_HOME<br />
<br />
## Remove duplicate entries<br />
setopt PUSHD_IGNORE_DUPS<br />
<br />
## This reverts the +/- operators.<br />
setopt PUSHD_MINUS<br />
</nowiki>}}<br />
<br />
Now use<br />
<br />
$ dirs -v<br />
<br />
to print the dirstack. Use {{ic|cd -<NUM>}} to go back to a visited folder. Use autocompletion after the dash. This proves very handy if using the autocompletion menu.<br />
<br />
{{Note|This will not work if you have more than one ''zsh'' session open, and attempt to {{ic|cd}}, due to a conflict in both sessions writing to the same file.}}<br />
<br />
==== cdr ====<br />
<br />
cdr allows you to change the working directory to a previous working directory from a list maintained automatically. It stores all entries in files that are maintained across sessions and (by default) between terminal emulators in the current session.<br />
<br />
See {{man|1|zshcontrib|REMEMBERING RECENT DIRECTORIES}} for setup instructions.<br />
<br />
=== Help command ===<br />
<br />
Unlike [[Bash]], Zsh does not enable a built in {{ic|help}} command, instead it provides {{ic|run-help}}. By default {{ic|run-help}} is an alias to {{ic|man}}, it can be either executed manually by prepending it to a command or it can be invoked for the currently typed command with the keyboard shortcuts {{ic|Alt+h}} or {{ic|Esc}} {{ic|h}}.<br />
<br />
Since by default it is just an alias to [[man]], it will only work on external commands. To improve its functionality, so that it works on shell builtins and other shell features, you need to use the {{ic|run-help}} function. See {{man|1|zshcontrib}} for more information on the {{ic|run-help}} and its assistant functions.<br />
<br />
First load the {{ic|run-help}} function and then remove the existing {{ic|run-help}} alias. For convenience {{ic|help}} can be aliased to {{ic|run-help}}. For example, add following to your {{ic|zshrc}}:<br />
<br />
{{bc|1=<br />
autoload -Uz run-help<br />
unalias run-help<br />
alias help=run-help<br />
}}<br />
<br />
Assistant functions have to be enabled separately:<br />
<br />
autoload -Uz run-help-git run-help-ip run-help-openssl run-help-p4 run-help-sudo run-help-svk run-help-svn<br />
<br />
For example, {{ic|run-help git commit}} command will now open the [[man page]] {{man|1|git-commit}} instead of {{man|1|git}}.<br />
<br />
=== Persistent rehash ===<br />
<br />
Typically, compinit will not automatically find new executables in the {{ic|$PATH}}. For example, after you install a new package, the files in {{ic|/usr/bin/}} would not be immediately or automatically included in the completion. Thus, to have these new executables included, one would run:<br />
<br />
$ rehash<br />
<br />
This 'rehash' can be set to happen automatically.[https://github.com/robbyrussell/oh-my-zsh/issues/3440] Simply include the following in your {{ic|zshrc}}:<br />
<br />
{{hc|~/.zshrc|<br />
zstyle ':completion:*' rehash true<br />
}}<br />
<br />
==== On-demand rehash ====<br />
<br />
As above, however [[pacman]] can be configured with [[Pacman#Hooks|hooks]] to automatically request a {{ic|rehash}}, which does not incur the performance penalty of constant rehashing as above. To enable this, create the {{ic|/etc/pacman.d/hooks}} directory, and a {{ic|/var/cache/zsh}} directory, then create a hook file:<br />
<br />
{{hc|head=/etc/pacman.d/hooks/zsh.hook|output=<br />
[Trigger]<br />
Operation = Install<br />
Operation = Upgrade<br />
Operation = Remove<br />
Type = Path<br />
Target = usr/bin/*<br />
[Action]<br />
Depends = zsh<br />
When = PostTransaction<br />
Exec = /usr/bin/install -Dm644 /dev/null /var/cache/zsh/pacman<br />
}}<br />
<br />
This keeps the modification date of the file {{ic|/var/cache/zsh/pacman}} consistent with the last time a package was installed, upgraded or removed. Then, {{ic|zsh}} must be coaxed into rehashing its own command cache when it goes out of date, by adding to your {{ic|~/.zshrc}}:<br />
<br />
{{hc|~/.zshrc|<nowiki><br />
zshcache_time="$(date +%s%N)"<br />
<br />
autoload -Uz add-zsh-hook<br />
<br />
rehash_precmd() {<br />
if [[ -a /var/cache/zsh/pacman ]]; then<br />
local paccache_time="$(date -r /var/cache/zsh/pacman +%s%N)"<br />
if (( zshcache_time < paccache_time )); then<br />
rehash<br />
zshcache_time="$paccache_time"<br />
fi<br />
fi<br />
}<br />
<br />
add-zsh-hook -Uz precmd rehash_precmd<br />
</nowiki>}}<br />
<br />
If the {{ic|precmd}} hook is triggered before {{ic|/var/cache/zsh/pacman}} is updated, completion may not work until a new prompt is initiated. Running an empty command, e.g. pressing {{ic|enter}}, should be sufficient.<br />
<br />
==== Alternative on-demand rehash using SIGUSR1 ====<br />
<br />
As above, however the hook file looks like this:<br />
<br />
{{hc|/etc/pacman.d/hooks/zsh-rehash.hook|output=<br />
[Trigger]<br />
Operation = Install<br />
Operation = Upgrade<br />
Operation = Remove<br />
Type = Path<br />
Target = usr/bin/*<br />
<br />
[Action]<br />
Depends = zsh<br />
Depends = procps-ng<br />
When = PostTransaction<br />
Exec = /usr/bin/pkill zsh --signal=USR1<br />
}}<br />
<br />
{{Warning|This sends SIGUSR1 to all running {{ic|zsh}} instances. Note that the default behavior for SIGUSR1 is terminate so when you first configure this all running {{ic|zsh}} instances of all users (including login shells) will terminate if they have not sourced the trap below.}}<br />
<br />
{{hc|~/.zshrc|<br />
TRAPUSR1() { rehash }<br />
}}<br />
<br />
The ''function trap'' above can be replaced with a ''list trap'' {{ic|trap 'rehash' USR1}}. See {{man|1|zshmisc|Trap Functions}} for differences between types of traps.<br />
<br />
This method will instantly {{ic|rehash}} all {{ic|zsh}} instances, removing the need to press enter to trigger {{ic|precmd}}.<br />
<br />
=== Bind key to ncurses application ===<br />
<br />
Bind a ncurses application to a keystroke, but it will not accept interaction. Use {{ic|BUFFER}} variable to make it work. The following example lets users open [[ncmpcpp]] using {{ic|Alt+\}}:<br />
<br />
{{hc|~/.zshrc|2=<br />
ncmpcppShow() {<br />
BUFFER="ncmpcpp"<br />
zle accept-line<br />
}<br />
zle -N ncmpcppShow<br />
bindkey '^[\' ncmpcppShow<br />
}}<br />
<br />
An alternate method, that will keep everything you entered in the line before calling application:<br />
<br />
{{hc|~/.zshrc|2=<br />
ncmpcppShow() {<br />
ncmpcpp <$TTY<br />
zle redisplay<br />
}<br />
zle -N ncmpcppShow<br />
bindkey '^[\' ncmpcppShow<br />
}}<br />
<br />
=== File manager key binds ===<br />
<br />
Key binds like those used in graphic file managers may come handy. The first comes back in directory history ({{ic|Alt+Left}}), the second let the user go to the parent directory ({{ic|Alt+Up}}). They also display the directory content.<br />
<br />
{{hc|~/.zshrc|<nowiki><br />
cdUndoKey() {<br />
popd<br />
zle reset-prompt<br />
print<br />
ls<br />
zle reset-prompt<br />
}<br />
<br />
cdParentKey() {<br />
pushd ..<br />
zle reset-prompt<br />
print<br />
ls<br />
zle reset-prompt<br />
}<br />
<br />
zle -N cdParentKey<br />
zle -N cdUndoKey<br />
bindkey '^[[1;3A' cdParentKey<br />
bindkey '^[[1;3D' cdUndoKey<br />
</nowiki>}}<br />
<br />
=== xterm title ===<br />
<br />
If your terminal emulator supports it, you can set its title from Zsh. This allows dynamically changing the title to display relevant information about the shell state, for example showing the user name and current directory or the currently executing command.<br />
<br />
The xterm title is set with the [https://www.tldp.org/HOWTO/Xterm-Title-3.html#ss3.1 xterm escape sequence] {{ic|\e]2;}}{{ic|\a}}. For example:<br />
<br />
$ print -n '\e]2;My xterm title\a'<br />
<br />
will set the title to<br />
<br />
My xterm title<br />
<br />
A simple way to have a dynamic title is to set the title in the {{ic|precmd}} and {{ic|preexec}} hook functions. See {{man|1|zshmisc|Hook Functions}} for a list of available hook functions and their descriptions.<br />
<br />
By using {{ic|print -P}} you can additionally take advantage of Zsh's prompt escapes.<br />
<br />
{{Tip|<br />
* Title printing can be split up in multiple commands as long as they are sequential.<br />
* [[GNU Screen]] sends the xterm title to the hardstatus ({{ic|%h}}). If you want to use Screen's [https://www.gnu.org/software/screen/manual/html_node/String-Escapes.html string escapes] (e.g. for colors) you should set the hardstatus with the {{ic|\e_}}{{ic|\e\\}} escape sequence. Otherwise, if string escapes are used in {{ic|\e]2;}}{{ic|\a}}, the terminal emulator will get a garbled title due to it being incapable of interpreting Screen's string escapes.<br />
}}<br />
<br />
{{Note|<br />
* Do not use the {{ic|-P}} option of {{ic|print}} when printing variables to prevent them from being parsed as prompt escapes.<br />
* Use the {{ic|q}} [http://zsh.sourceforge.net/Doc/Release/Expansion.html#Parameter-Expansion-Flags parameter expansion flag] when printing variables to prevent them from being parsed as escape sequences.<br />
}}<br />
<br />
{{hc|~/.zshrc|<nowiki><br />
autoload -Uz add-zsh-hook<br />
<br />
function xterm_title_precmd () {<br />
print -Pn -- '\e]2;%n@%m %~\a'<br />
[[ "$TERM" == 'screen'* ]] && print -Pn -- '\e_\005{g}%n\005{-}@\005{m}%m\005{-} \005{B}%~\005{-}\e\\'<br />
}<br />
<br />
function xterm_title_preexec () {<br />
print -Pn -- '\e]2;%n@%m %~ %# ' && print -n -- "${(q)1}\a"<br />
[[ "$TERM" == 'screen'* ]] && { print -Pn -- '\e_\005{g}%n\005{-}@\005{m}%m\005{-} \005{B}%~\005{-} %# ' && print -n -- "${(q)1}\e\\"; }<br />
}<br />
<br />
if [[ "$TERM" == (alacritty*|gnome*|konsole*|putty*|rxvt*|screen*|tmux*|xterm*) ]]; then<br />
add-zsh-hook -Uz precmd xterm_title_precmd<br />
add-zsh-hook -Uz preexec xterm_title_preexec<br />
fi<br />
</nowiki>}}<br />
<br />
==== Terminal emulator tab title ====<br />
<br />
Some terminal emulators and multiplexers support setting the title of the tab. The escape sequences depend on the terminal:<br />
<br />
{| class="wikitable sortable"<br />
! Terminal<br />
! Escape sequences<br />
! Description<br />
|-<br />
! [[GNU Screen]]<br />
| {{ic|\ek}}{{ic|\e\\}}<br />
| Screen's window title ({{ic|%t}}).<br />
|-<br />
! Konsole<br />
| {{ic|\e]30;}}{{ic|\a}}<br />
| Konsole's tab title.<br />
|}<br />
<br />
=== Shell environment detection ===<br />
<br />
See [https://gitlab.com/jdorel-documentation/shell-environment-detection a repository about shell environment detection] for tests to detect the shell environment. This includes login/interactive shell, Xorg session, TTY and SSH session.<br />
<br />
=== /dev/tcp equivalent: ztcp ===<br />
<br />
Use the {{ic|zsh/net/tcp}} module:<br />
<br />
$ zmodload zsh/net/tcp<br />
<br />
You can now establish TCP connections:<br />
<br />
$ ztcp example.com 80<br />
<br />
=== Shortcut to exit shell on partial command line ===<br />
<br />
By default, {{ic|Ctrl+d}} will not close your shell if the command line is filled, this fixes it:<br />
<br />
{{hc|.zshrc|<br />
exit_zsh() { exit }<br />
zle -N exit_zsh<br />
bindkey '^D' exit_zsh<br />
}}<br />
<br />
== Third-party extensions ==<br />
<br />
=== Configuration frameworks ===<br />
<br />
* {{App|oh-my-zsh|A popular, community-driven framework for managing your Zsh configuration. It comes bundled with a ton of helpful functions, helpers, plugins, themes.|https://github.com/robbyrussell/oh-my-zsh|{{AUR|oh-my-zsh-git}}}}<br />
* {{App|Prezto|A configuration framework for Zsh. It comes with modules, enriching the command line interface environment with sane defaults, aliases, functions, auto completion, and prompt themes.|https://github.com/sorin-ionescu/prezto|{{AUR|prezto-git}}}}<br />
* {{App|ZIM|A configuration framework with blazing speed and modular extensions. Zim is very easy to customize, and comes with a rich set of modules and features without compromising on speed or functionality.|https://github.com/zimfw/zimfw|{{AUR|zsh-zim-git}}}}<br />
<br />
=== Plugin managers ===<br />
<br />
* {{App|Antibody|A performance-focused plugin manager similar to Antigen.|https://github.com/getantibody/antibody|{{AUR|antibody}}}}<br />
* {{App|zinit (previously "zplugin")|Flexible Zsh plugin manager with clean fpath, reports, completion management, turbo mode|https://github.com/zdharma/zinit|{{AUR|zsh-zplugin-git}}}}<br />
* {{App|Antigen|A plugin manager for Zsh, inspired by oh-my-zsh and vundle. [https://github.com/zsh-users/antigen/issues/673 ABANDONED]|https://github.com/zsh-users/antigen|{{AUR|antigen-git}}}}<br />
* {{App|zgen|A lightweight and simple plugin manager for Zsh. [https://github.com/tarjoilija/zgen/issues/123 ABANDONED]|https://github.com/tarjoilija/zgen|{{AUR|zgen-git}}}}<br />
* {{App|zplug|A next-generation plugin manager for Zsh. [https://github.com/zplug/zplug/issues/403#issuecomment-477520784 ABANDONED]|https://github.com/zplug/zplug|{{AUR|zplug}}}}<br />
<br />
=== Fish-like syntax highlighting and autosuggestions ===<br />
<br />
[[Fish]] provides very powerful shell syntax highlighting and autosuggestions. To use both in Zsh, you can install {{pkg|zsh-syntax-highlighting}}, {{pkg|zsh-autosuggestions}}, and finally [[source]] one or both of the provided scripts from your zshrc:<br />
<br />
source /usr/share/zsh/plugins/zsh-syntax-highlighting/zsh-syntax-highlighting.zsh<br />
source /usr/share/zsh/plugins/zsh-autosuggestions/zsh-autosuggestions.zsh<br />
<br />
=== The "command not found" handler ===<br />
<br />
[[pacman]] includes functionality to search for packages containing a file. The following command-not-found handler will use pacman directly to search for matching packages when an unknown command is executed.<br />
<br />
{{hc|1=~/.zshrc|2=<br />
command_not_found_handler() {<br />
local pkgs cmd="$1"<br />
echo "zsh: command not found: $cmd"<br />
<br />
files=(${(f)"$(pacman --files --machinereadable "/usr/bin/$cmd")"})<br />
if (( $? != 0 )); then<br />
return 1<br />
fi<br />
echo "$cmd may be found in the following packages:"<br />
for file in $files; do<br />
res=(${(0)file})<br />
repo=$res[1]<br />
package=$res[2]<br />
version=$res[3]<br />
file=$res[4]<br />
echo "$repo/$package: /$file"<br />
done<br />
}<br />
}}<br />
<br />
{{Note|The files database of pacman is separate from the normal sync database and it needs to be fetched using {{ic|pacman -Fy}}. See [[Pacman#Search_for_a_package_that_contains_a_specific_file]] for details.}}<br />
<br />
<br />
Alternatively, [[pkgfile]] includes a Zsh script file that provides a {{ic|command_not_found_handler}} function that will automatically search the pkgfile database when entering an unrecognized command.<br />
<br />
You need to [[source]] the script to enable it. For example:<br />
<br />
{{hc|~/.zshrc|<br />
source /usr/share/doc/pkgfile/command-not-found.zsh<br />
}}<br />
<br />
{{Note|The pkgfile database may need to be updated before this will work. See [[pkgfile#Installation]] for details.}}<br />
<br />
== Uninstallation ==<br />
<br />
Change the default shell before removing the {{Pkg|zsh}} package.<br />
<br />
{{Warning|Failure to follow the below procedure may result in users no longer having access to a working shell.}}<br />
<br />
Run following command:<br />
<br />
$ chsh -s /bin/bash ''user''<br />
<br />
Use it for every user with ''zsh'' set as their login shell (including root if needed). When completed, the {{Pkg|zsh}} package can be removed.<br />
<br />
Alternatively, change the default shell back to Bash by editing {{ic|/etc/passwd}} as root.<br />
<br />
{{Warning|It is strongly recommended to use {{man|8|vipw}} when editing {{ic|/etc/passwd}} as it helps prevent invalid entries and/or syntax errors.}}<br />
<br />
For example, change the following:<br />
<br />
''username'':x:1000:1000:''Full Name'',,,:/home/''username'':/bin/zsh<br />
<br />
To this:<br />
<br />
''username'':x:1000:1000:''Full Name'',,,:/home/''username'':/bin/bash<br />
<br />
== See also ==<br />
<br />
* [[Wikipedia:Zsh]]<br />
* [http://zsh.sourceforge.net/Intro/intro_toc.html An Introduction to the Z Shell]<br />
* [http://zsh.sourceforge.net/Guide/zshguide.html A User's Guide to ZSH]<br />
* [http://zsh.sourceforge.net/Doc/Release/index-frame.html The Z Shell Manual] (different format available [http://zsh.sourceforge.net/Doc/ here])<br />
* [http://zsh.sourceforge.net/FAQ/zshfaq01.html Zsh FAQ]<br />
* [http://zshwiki.org/home/ Zsh Wiki]<br />
* {{man|1|zsh-lovers}} (available as {{pkg|zsh-lovers}} package)<br />
* [[Gentoo: Zsh/Guide]]<br />
* [http://www.bash2zsh.com/zsh_refcard/refcard.pdf Bash2Zsh Reference Card]</div>Phireskyhttps://wiki.archlinux.org/index.php?title=NVIDIA/Troubleshooting&diff=542158NVIDIA/Troubleshooting2018-09-19T15:04:56Z<p>Phiresky: /* Multi-monitor */ add command to get CurrentMetaMode</p>
<hr />
<div>[[Category:Graphics]]<br />
[[Category:X server]]<br />
[[ja:NVIDIA/トラブルシューティング]]<br />
[[ru:NVIDIA/Troubleshooting]]<br />
== Corrupted screen: "Six screens" Problem ==<br />
<br />
For some users, using GeForce GT 100M's, the screen gets corrupted after X starts, divided into 6 sections with a resolution limited to 640x480.<br />
The same problem has been recently reported with Quadro 2000 and hi-res displays.<br />
<br />
To solve this problem, enable the Validation Mode {{ic|NoTotalSizeCheck}} in section {{ic|Device}}:<br />
Section "Device"<br />
...<br />
Option "ModeValidation" "NoTotalSizeCheck"<br />
...<br />
EndSection<br />
<br />
== '/dev/nvidia0' input/output error ==<br />
<br />
{{Accuracy|Verify that the BIOS related suggestions work and are not coincidentally set while troubleshooting.|section='/dev/nvidia0' Input/Output error... suggested fixes}}<br />
This error can occur for several different reasons, and the most common solution given for this error is to check for group/file permissions, which in almost every case is ''not'' the problem. The NVIDIA documentation does not talk in detail on what you should<br />
do to correct this problem but there are a few things that have worked for some people. The problem can be a IRQ conflict with another device or bad routing by either the kernel or your BIOS.<br />
<br />
First thing to try is to remove other video devices such as video capture cards and see if the problem goes away. If there are too many video processors on the same system it can lead into the kernel being unable to start them because of memory allocation problems with the video controller. In particular on systems with low video memory this can occur even if there is only one video processor. In such case you should find out the amount of your system's video memory (e.g. with {{ic|lspci -v}}) and pass allocation parameters to the kernel, e.g. for a 32-bit kernel:<br />
vmalloc=384M<br />
<br />
If running a 64bit kernel, a driver defect can cause the NVIDIA module to fail initializing when IOMMU is on. Turning it off in the BIOS has been confirmed to work for some users. [http://www.nvnews.net/vbulletin/showthread.php?s=68bb2fabadcb53b10b286aa42d13c5bc&t=159335][[User:Clickthem#nvidia module]]<br />
<br />
Another thing to try is to change your BIOS IRQ routing from {{ic|Operating system controlled}} to {{ic|BIOS controlled}} or the other way around. The first one can be passed as a kernel parameter:<br />
PCI=biosirq<br />
<br />
The {{ic|noacpi}} kernel parameter has also been suggested as a solution but since it disables ACPI completely it should be used with caution. Some hardware are easily damaged by overheating.<br />
<br />
{{Note|The kernel parameters can be passed either through the kernel command line or the bootloader configuration file. See your bootloader Wiki page for more information.}}<br />
<br />
== Crashing in general ==<br />
<br />
* Try disabling {{ic|RenderAccel}} in xorg.conf.<br />
* If Xorg outputs an error about {{ic|"conflicting memory type"}} or {{ic|"failed to allocate primary buffer: out of memory"}}, or crashes with a "Signal 11" while using nvidia-96xx drivers, add {{ic|nopat}} to your [[kernel parameters]].<br />
* If the NVIDIA compiler complains about different versions of GCC between the current one and the one used for compiling the kernel, add in {{ic|/etc/profile}}:<br />
export IGNORE_CC_MISMATCH=1<br />
* If Xorg is crashing , try disabling PAT. Pass the argument {{ic|nopat}} to [[kernel parameters]].<br />
More information about troubleshooting the driver can be found in the [https://forums.geforce.com/ NVIDIA forums.]<br />
<br />
== Bad performance after installing a new driver version ==<br />
<br />
If FPS have dropped in comparison with older drivers, check if direct rendering is enabled ({{ic|glxinfo}} is included in {{Pkg|mesa-demos}}):<br />
$ glxinfo | grep direct<br />
<br />
If the command prints:<br />
direct rendering: No<br />
<br />
A possible solution could be to regress to the previously installed driver version and rebooting afterwards.<br />
<br />
== Avoid screen tearing ==<br />
{{Note|This has been reported to reduce the performance of some OpenGL applications and may produce issues in WebGL.}}<br />
<br />
Tearing can be avoided by forcing a full composition pipeline, regardless of the compositor you are using. To test whether this option will work, run:<br />
$ nvidia-settings --assign CurrentMetaMode="nvidia-auto-select +0+0 { ForceFullCompositionPipeline = On }"<br />
<br />
Or click on the ''Advanced'' button that is available on the ''X Server Display Configuration'' menu option. Select either ''Force Composition Pipeline'' or ''Force Full Composition Pipeline'' and click on ''Apply''.<br />
<br />
In order to make the change permanent, it must be added to the {{ic|"Screen"}} section of the [[Xorg]] configuration file. When making this change, {{ic|TripleBuffering}} should be enabled and {{ic|AllowIndirectGLXProtocol}} should be disabled in the driver configuration as well. See example configuration below:<br />
<br />
{{hc|/etc/X11/xorg.conf.d/20-nvidia.conf|<nowiki><br />
Section "Device"<br />
Identifier "Nvidia Card"<br />
Driver "nvidia"<br />
VendorName "NVIDIA Corporation"<br />
BoardName "GeForce GTX 1050 Ti"<br />
EndSection<br />
<br />
Section "Screen"<br />
Identifier "Screen0"<br />
Device "Device0"<br />
Monitor "Monitor0"<br />
Option "metamodes" "nvidia-auto-select +0+0 {ForceCompositionPipeline=On, ForceFullCompositionPipeline=On}"<br />
Option "AllowIndirectGLXProtocol" "off"<br />
Option "TripleBuffer" "on"<br />
EndSection<br />
</nowiki>}}<br />
<br />
If you do not have an Xorg configuration file, you can create one for your present hardware using {{ic|nvidia-xconfig}} (see [[NVIDIA#Automatic configuration]]) and move it from {{ic|/etc/X11/xorg.conf}} to the preferred location {{ic|/etc/X11/xorg.conf.d/20-nvidia.conf}}.<br />
<br />
{{Note|Many of the configuration options produced in {{ic|20-nvidia.conf}} by using {{ic|nvidia-xconfig}} are set automatically by the driver and are not needed. To only use this file for enabling composition pipeline, only the section {{ic|"Screen"}} containing lines with values for {{ic|Identifier}} and {{ic|Option}} are necessary. Other sections may be removed from this file.}}<br />
<br />
=== Multi-monitor ===<br />
<br />
For multi-monitor setup you will need to specify {{ic|1=ForceCompositionPipeline=On}} for each display. For example:<br />
<br />
$ nvidia-settings --assign CurrentMetaMode="DP-2: nvidia-auto-select +0+0 {ForceCompositionPipeline=On}, DP-4: nvidia-auto-select +3840+0 {ForceCompositionPipeline=On}"<br />
<br />
Without doing this, the {{ic|nvidia-settings}} command will disable your secondary display.<br />
<br />
You can get the current screen names and offsets using {{ic|--query}}:<br />
<br />
$ nvidia-settings --query CurrentMetaMode<br />
<br />
The above line is for two 3840x2160 monitors connected to DP-2 and DP-4. You will need to read the correct {{ic|CurrentMetaMode}} by exporting {{ic|xorg.conf}} and append {{ic|ForceCompositionPipeline}} to each of your displays. Setting {{ic|ForceCompositionPipeline}} only affects the targeted display. <br />
<br />
{{Tip|Multi monitor setups using different model monitors may have slightly different refresh rates. If vsync is enabled by the driver it will sync to only one of these refresh rates which can cause the appearance of screen tearing on incorrectly synced monitors. Select to sync the display device which is the primarily used monitor as others will not sync properly. This is configurable in {{ic|~/.nvidia-settings-rc}} as {{ic|<nowiki>0/XVideoSyncToDisplayID=</nowiki>}} or by installing {{pkg|nvidia-settings}} and using the graphical configuration options.}}<br />
<br />
=== Avoid screen tearing in KDE (KWin) ===<br />
At the moment there are two workarounds to try. Do '''not''' apply both workarounds because this may lead to high CPU load [https://bugs.kde.org/show_bug.cgi?id=322060].<br />
<br />
==== 1. GL threads ====<br />
Set GL threads to sleep by exporting {{ic|1=__GL_YIELD="USLEEP"}} to just {{ic|kwin_x11}}. Unlike setting up a global environment variable, this affects only KWin. It should also have the advantage over other workarounds, like forcing triple buffering or forcing composition pipeline in the driver, that it doesn't introduce additional stuttering when scrolling in Firefox or moving windows.<br />
<br />
The script can be executed automatically at login with an autostart script:<br />
<br />
{{hc|~/.config/autostart-scripts/kwin.sh|<nowiki><br />
#!/bin/bash<br />
<br />
(sleep 2s && __GL_YIELD="USLEEP" kwin_x11 --replace)<br />
</nowiki>}}<br />
Flag the script as executable.<br />
<br />
The {{ic|sleep}} argument helps to prevent issues when KWin is restarted/hanging after logging in, you might need to increase this time.<br />
<br />
==== 2. Use TripleBuffering ====<br />
<br />
Make sure {{ic|TripleBuffering}} has been enabled for the driver, see [[#Avoid screen tearing]].<br />
<br />
Create the {{ic|/etc/profile.d/kwin.sh}} file:<br />
<br />
{{hc|/etc/profile.d/kwin.sh|2=<br />
export KWIN_TRIPLE_BUFFER=1<br />
}}<br />
<br />
Use '''OpenGL 2.0 or later''' as rendering backend under ''System Settings > Display and Monitor > Compositor''.<br />
<br />
== Modprobe Error: "Could not insert 'nvidia': No such device" on linux >=4.8 ==<br />
<br />
With linux 4.8, one can get the following errors when trying to use the discrete card:<br />
{{hc|$ modprobe nvidia -vv|<nowiki><br />
modprobe: INFO: custom logging function 0x409c10 registered<br />
modprobe: INFO: Failed to insert module '/lib/modules/4.8.6-1-ARCH/extramodules/nvidia.ko.gz': No such device<br />
modprobe: ERROR: could not insert 'nvidia': No such device<br />
modprobe: INFO: context 0x24481e0 released<br />
insmod /lib/modules/4.8.6-1-ARCH/extramodules/nvidia.ko.gz <br />
</nowiki>}}<br />
{{hc|$ dmesg|<nowiki><br />
...<br />
NVRM: The NVIDIA GPU 0000:01:00.0 (PCI ID: 10de:139b)<br />
NVRM: installed in this system is not supported by the 370.28<br />
NVRM: NVIDIA Linux driver release. Please see 'Appendix<br />
NVRM: A - Supported NVIDIA GPU Products' in this release's<br />
NVRM: README, available on the Linux driver download page<br />
NVRM: at www.nvidia.com.<br />
...<br />
</nowiki>}}<br />
<br />
This problem is caused by bad commits pertaining to PCIe power management in the Linux Kernel (as documented in [https://devtalk.nvidia.com/default/topic/971733/-370-28-with-kernel-4-8-on-gt-2015-machines-driver-claims-card-not-supported-if-nvidia-is-not-primary-card/ this NVIDIA DevTalk thread]). <br />
<br />
The workaround is to add {{ic|1=pcie_port_pm=off}} to your [[kernel parameters]]. Note that this disables PCIe power management for all devices.<br />
<br />
== Poor performance after resuming from suspend ==<br />
<br />
If you are getting poor performance after resuming from suspend, you need to register the nvidia kernel module with the ACPI subsystem. This can be done by [[Kernel modules#Setting module options|loading]] the {{ic|nvidia}} module with the {{ic|1=NVreg_RegisterForACPIEvents=1 NVreg_EnableMSI=1}} options.<br />
<br />
== CPU spikes with 400 series cards ==<br />
<br />
If you are experiencing intermittent CPU spikes with a 400 series card, it may be caused by PowerMizer constantly changing the GPU's clock frequency. Switching PowerMizer's setting from Adaptive to Performance, add the following to the {{ic|Device}} section of your Xorg configuration:<br />
<br />
Option "RegistryDwords" "PowerMizerEnable=0x1; PerfLevelSrc=0x3322; PowerMizerDefaultAC=0x1"<br />
<br />
== Full system freeze or crashes when using Flash ==<br />
<br />
If you experience occasional full system freezes using [[Flash]], a possible workaround is to disable Hardware Acceleration:<br />
<br />
{{hc|/etc/adobe/mms.cfg|2=<br />
EnableLinuxHWVideoDecode=0<br />
}}<br />
<br />
Or, if you want to keep Hardware acceleration enabled but allowing a higher chance of screen tearing, you may try to before starting a browser:<br />
export VDPAU_NVIDIA_NO_OVERLAY=1<br />
<br />
== Laptops: X hangs on login/out, worked around with Ctrl+Alt+Backspace ==<br />
<br />
If, while using the legacy NVIDIA drivers, Xorg hangs on login and logout (particularly with an odd screen split into two black and white/gray pieces), but logging in is still possible via {{ic|Ctrl+Alt+Backspace}} (or whatever the new "kill X" key binding is), try adding this in {{ic|/etc/modprobe.d/modprobe.conf}}:<br />
options nvidia NVreg_Mobile=1<br />
<br />
One user had luck with this instead, but it makes performance drop significantly for others:<br />
options nvidia NVreg_DeviceFileUID=0 NVreg_DeviceFileGID=33 NVreg_DeviceFileMode=0660 NVreg_SoftEDIDs=0 NVreg_Mobile=1<br />
<br />
Note that {{ic|NVreg_Mobile}} needs to be changed according to the laptop:<br />
* 1 for Dell laptops.<br />
* 2 for non-Compal Toshiba laptops.<br />
* 3 for other laptops.<br />
* 4 for Compal Toshiba laptops.<br />
* 5 for Gateway laptops.<br />
<br />
See [ftp://download.nvidia.com/XFree86/Linux-x86/355.11/README/README.txt NVIDIA Driver's README: Appendix K] for more information.<br />
<br />
== Screen(s) found, but none have a usable configuration ==<br />
<br />
Sometimes NVIDIA and X have trouble finding the active screen. If your graphics card has multiple outputs try plugging your monitor into the other ones. On a laptop it may be because your graphics card has VGA/TV out. Xorg.0.log will provide more info.<br />
<br />
Another thing to try is adding invalid {{ic|"ConnectedMonitor" Option}} to {{ic|Section "Device"}}<br />
to force Xorg throws error and shows you how correct it.<br />
[ftp://download.nvidia.com/XFree86/Linux-x86/355.11/README/xconfigoptions.html Here]<br />
more about ConnectedMonitor setting.<br />
<br />
After re-run X see Xorg.0.log to get valid CRT-x,DFP-x,TV-x values.<br />
<br />
{{ic|nvidia-xconfig --query-gpu-info}} could be helpful.<br />
<br />
== Blackscreen at X startup / Machine poweroff at X shutdown ==<br />
<br />
If you have installed an update of Nvidia and your screen stays black after launching Xorg, or if shutting down Xorg causes a machine poweroff, try the below workarounds:<br />
<br />
* Prepend "xrandr --auto" to your xinitrc<br />
<br />
* Use the {{ic|<nowiki>rcutree.rcu_idle_gp_delay=1</nowiki>}} [[kernel parameter]].<br />
<br />
* You can also try to add the {{ic|nvidia}} module directly to your [[mkinitcpio]] config file.<br />
<br />
* If the screen still stays black with '''both''' the {{ic|<nowiki>rcutree.rcu_idle_gp_delay=1</nowiki>}} [[kernel parameter]] and the {{ic|nvidia}} module directly in the [[mkinitcpio]] config file, try re-installing {{Pkg|nvidia}} and {{Pkg|nvidia-utils}} in that order, and finally reload the driver:<br />
<br />
# modprobe nvidia<br />
<br />
== Backlight is not turning off in some occasions ==<br />
<br />
By default, DPMS should turn off backlight with the timeouts set or by running xset. However, probably due to a bug in the proprietary Nvidia drivers the result is a blank screen with no powersaving whatsoever. To workaround it, until the bug has been fixed you can use the {{ic|vbetool}} as root.<br />
<br />
Install the {{Pkg|vbetool}} package.<br />
<br />
Turn off your screen on demand and then by pressing a random key backlight turns on again:<br />
<br />
vbetool dpms off && read -n1; vbetool dpms on<br />
<br />
Alternatively, xrandr is able to disable and re-enable monitor outputs without requiring root.<br />
<br />
xrandr --output DP-1 --off; read -n1; xrandr --output DP-1 --auto<br />
<br />
== Xorg fails to load or Red Screen of Death ==<br />
<br />
If you get a red screen and use GRUB, disable the GRUB framebuffer by editing {{ic|/etc/default/grub}} and uncomment {{ic|1=GRUB_TERMINAL_OUTPUT=console}}. For more information see [[GRUB/Tips and tricks#Disable framebuffer]].<br />
<br />
== Black screen on systems with Intel integrated GPU ==<br />
<br />
If you have an Intel CPU with an integrated GPU (e.g. Intel HD 4000) and have installed the {{Pkg|nvidia}} package, you may experience a black screen on boot, when changing virtual terminal, or when exiting an X session. This may be caused by a conflict between the graphics modules. This is solved by blacklisting the Intel GPU modules. Create the file {{ic|/etc/modprobe.d/blacklist.conf}} and prevent the ''i915'' and ''intel_agp'' modules from loading on boot:<br />
<br />
{{hc|/etc/modprobe.d/blacklist.conf|<br />
install i915 /usr/bin/false<br />
install intel_agp /usr/bin/false<br />
}}<br />
<br />
== No audio over HDMI ==<br />
<br />
Sometimes nvidia HDMI audio devices are not shown when you do<br />
<br />
aplay -l<br />
<br />
For whatever reason on some new machines, the audio chip on the nvidia GPU is disabled at boot. Read more [https://devtalk.nvidia.com/default/topic/1024022/linux/gtx-1060-no-audio-over-hdmi-only-hda-intel-detected-azalia/?offset=4 here] and [https://bbs.archlinux.org/viewtopic.php?id=230125 here]<br />
<br />
You need to reload the nvidia device with audio enabled. In order to do that make sure that your GPU is on (in case of laptops/Bumblebee) and that you are not running X on it, because it's going to reset:<br />
<br />
# setpci -s 01:00.0 0x488.l=0x2000000:0x2000000<br />
# rmmod nvidia-drm nvidia-modeset nvidia<br />
# echo 1 > /sys/bus/pci/devices/0000:01:00.0/remove<br />
# echo 1 > /sys/bus/pci/devices/0000:00:01.0/rescan<br />
# modprobe nvidia-drm<br />
# xinit -- -retro<br />
<br />
If you are running your TTY on nvidia, put the lines in a script so you don't end up with no screen.<br />
<br />
== Black screen on systems with VIA integrated GPU ==<br />
<br />
As above, blacklisting the ''viafb'' module may resolve conflicts with NVIDIA drivers:<br />
<br />
{{hc|/etc/modprobe.d/blacklist.conf|<br />
install viafb /usr/bin/false<br />
}}<br />
<br />
== X fails with "no screens found" with Intel iGPU ==<br />
<br />
Like above, if you have an Intel CPU with an integrated GPU and X fails to start with <br />
<br />
[ 76.633] (EE) No devices detected.<br />
[ 76.633] Fatal server error:<br />
[ 76.633] no screens found<br />
<br />
then you need to add your discrete card's BusID to your X configuration. Find it:<br />
<br />
{{hc|<nowiki># lspci | grep VGA</nowiki>|<br />
00:02.0 VGA compatible controller: Intel Corporation Xeon E3-1200 v2/3rd Gen Core processor Graphics Controller (rev 09)<br />
01:00.0 VGA compatible controller: NVIDIA Corporation GK107 [GeForce GTX 650] (rev a1)<br />
}}<br />
<br />
then you fix it by adding it to the card's Device section in your X configuration. In my case:<br />
<br />
{{hc|/etc/X11/xorg.conf.d/10-nvidia.conf|<br />
Section "Device"<br />
Identifier "Device0"<br />
Driver "nvidia"<br />
VendorName "NVIDIA Corporation"<br />
BusID "PCI:1:0:0"<br />
EndSection<br />
}}<br />
<br />
Note how {{ic|01:00.0}} is written as {{ic|1:0:0}}.<br />
<br />
== Xorg fails during boot, but otherwise starts fine ==<br />
<br />
On very fast booting systems, systemd may attempt to start the display manager before the NVIDIA driver has fully initialized. You will see a message like the following in your logs only when Xorg runs during boot.<br />
{{hc|/var/log/Xorg.0.log|output=<br />
[ 1.807] (EE) NVIDIA(0): Failed to initialize the NVIDIA kernel module. Please see the<br />
[ 1.807] (EE) NVIDIA(0): system's kernel log for additional error messages and<br />
[ 1.808] (EE) NVIDIA(0): consult the NVIDIA README for details.<br />
[ 1.808] (EE) NVIDIA(0): *** Aborting ***<br />
}}<br />
In this case you will need to establish an ordering dependency from the display manager to the DRI device. First create device units for DRI devices by creating a new udev rules file.<br />
{{hc|/etc/udev/rules.d/99-systemd-dri-devices.rules|output=<br />
ACTION=="add", KERNEL=="card*", SUBSYSTEM=="drm", TAG+="systemd"<br />
}}<br />
Then create dependencies from the display manager to the device(s).<br />
{{hc|/etc/systemd/system/display-manager.service.d/10-wait-for-dri-devices.conf|output=<br />
[Unit]<br />
Wants=dev-dri-card0.device<br />
After=dev-dri-card0.device<br />
}}<br />
If you have additional cards needed for the desktop then list them in Wants and After seperated by spaces.<br />
<br />
== xrandr BadMatch ==<br />
<br />
If you are trying to configure a WQHD monitor such as DELL U2515H using [[xrandr]] and {{ic|xrandr --addmode}} gives you the error {{ic|X Error of failed request: BadMatch}}, it might be because the proprietary NVIDIA driver clips the pixel clock maximum frequency of HDMI output to 225 MHz or lower. To set the monitor to maximum resolution you have to install [[nouveau]] drivers. You can force nouveau to use a specific pixel clock frequency by setting {{ic|1=nouveau.hdmimhz=297}} (or {{ic|330}}) in your [[Kernel parameters]].<br />
<br />
Alternatively, it may be that your monitor's EDID is incorrect. See [[#Override EDID]].<br />
<br />
Another reason could be that per default current NVidia drivers will only allow modes explicitly reported by EDID; but sometimes refresh rates and/or resolutions are desired which are not reported by the monitor (although the EDID information is correct; it's just that current NVidia drivers are too restrictive).<br />
<br />
If this happens, you may want to add an option to {{ic|/etc/X11/xorg.conf.d/}} to allow non-EDID modes:<br />
<br />
{{bc|<br />
Section "Device"<br />
Identifier "Device0"<br />
Driver "nvidia"<br />
VendorName "NVIDIA Corporation"<br />
...<br />
Option "ModeValidation" "AllowNonEdidModes"<br />
...<br />
EndSection<br />
}}<br />
<br />
This can be set per-output. See NVidia driver readme (Appendix B. X Config Options) for more information.<br />
<br />
== Override EDID ==<br />
<br />
See [[Kernel mode setting#Forcing modes and EDID]], [[Xrandr#Troubleshooting]] and [[Qnix QX2710#Fixing X11 with Nvidia]].<br />
<br />
== Overclocking with nvidia-settings GUI not working ==<br />
<br />
{{Style|Duplication, vague "not working"}}<br />
<br />
Workaround is to use nvidia-settings CLI to query and set certain variables after enabling overclocking (as explained in [[NVIDIA/Tips and tricks#Enabling overclocking]], see {{man|1|nvidia-settings}} for more information).<br />
<br />
Example to query all variables:<br />
<br />
nvidia-settings -q all<br />
<br />
Example to set PowerMizerMode to prefer performance mode:<br />
<br />
nvidia-settings -a [gpu:0]/GPUPowerMizerMode=1<br />
<br />
Example to set fan speed to fixed 21%:<br />
<br />
nvidia-settings -a [gpu:0]/GPUFanControlState=1 -a [fan:0]/GPUTargetFanSpeed=21<br />
<br />
Example to set multiple variables at once (overclock GPU by 50MHz, overclock video memory by 50MHz, increase GPU voltage by 100mV):<br />
<br />
nvidia-settings -a GPUGraphicsClockOffsetAllPerformanceLevels=50 -a GPUMemoryTransferRateOffsetGPUGraphicsClockOffsetAllPerformanceLevels=50 -a GPUOverVoltageOffset=100</div>Phireskyhttps://wiki.archlinux.org/index.php?title=Dm-crypt/System_configuration&diff=532789Dm-crypt/System configuration2018-08-08T12:54:36Z<p>Phiresky: add systemd-cryptsetup information</p>
<hr />
<div>{{Lowercase title}}<br />
[[Category:Disk encryption]]<br />
[[ja:dm-crypt/システム設定]]<br />
{{Expansion|Aggregate here all the generic information on system configuration from the other sub-articles of [[dm-crypt]].}}<br />
<br />
{{Tip|If in need to remotely unlock root or other early-boot filesystems (headless machine, distant servers...), follow the specific instructions from [[dm-crypt/Specialties#Remote unlocking of the root (or other) partition]].}}<br />
<br />
== mkinitcpio ==<br />
<br />
Depending on the particular scenarios, a subset of the following [[mkinitcpio]] hooks will have to be enabled:<br />
<br />
{| class="wikitable"<br />
! busybox !! systemd !! Use case<br />
|-<br />
|style="text-align: center;white-space:nowrap;"| {{ic|encrypt}}<br />
|style="text-align: center;white-space:nowrap;"| {{ic|sd-encrypt}}<br />
| Always needed when encrypting the root partition, or a partition that needs to be mounted ''before'' root. It is not needed in all the other cases, as system initialization scripts like {{ic|/etc/crypttab}} take care of unlocking other encrypted partitions. This hook must be placed ''after'' the {{ic|udev}} or {{ic|systemd}} hook.<br />
|-<br />
|colspan="2" style="text-align: center;white-space:nowrap;"| {{ic|keyboard}}<br />
| Needed to make keyboards work in early userspace.<br />
|-<br />
|style="text-align: center;white-space:nowrap;"| {{ic|keymap}}<br />
|rowspan="2" style="text-align: center;white-space:nowrap;"| {{ic|sd-vconsole}}<br />
| Provides support for non-US keymaps for typing encryption passwords; it must come ''before'' the {{ic|encrypt}} hook. Set your keymap in {{ic|/etc/vconsole.conf}}, see [[Keyboard configuration in console#Persistent configuration]].<br />
|-<br />
|style="text-align: center;white-space:nowrap;"| {{ic|consolefont}}<br />
| Loads an alternative console font in early userspace. Set your font in {{ic|/etc/vconsole.conf}}, see [[Fonts#Persistent configuration]].<br />
|}<br />
<br />
[[mkinitcpio#Common hooks|Other hooks]] needed should be clear from other manual steps followed during the installation of the system.<br />
<br />
Remember to [[regenerate the initramfs]] after saving the changes.<br />
<br />
=== Examples ===<br />
<br />
A typical {{ic|/etc/mkinitcpio.conf}} configuration using {{ic|encrypt}} hook:<br />
<br />
{{hc|/etc/mkinitcpio.conf|2=<br />
...<br />
HOOKS=(base udev autodetect keyboard keymap consolefont modconf block encrypt lvm2 filesystems fsck)<br />
...<br />
}}<br />
<br />
A configuration with systemd-based initramfs using {{ic|sd-encrypt}} hook:<br />
<br />
{{hc|/etc/mkinitcpio.conf|2=<br />
...<br />
HOOKS=(base systemd autodetect keyboard sd-vconsole modconf block sd-encrypt sd-lvm2 filesystems fsck)<br />
...<br />
}}<br />
<br />
== Boot loader ==<br />
<br />
In order to enable booting an encrypted root partition, a subset of the following kernel parameters need to be set. See [[kernel parameters]] for instructions specific to your [[boot loader]]. <br />
<br />
For example, if using [[GRUB#Root partition|GRUB]], the relevant parameters are added to {{ic|/etc/default/grub}} before [[GRUB#Generate_the_main_configuration_file|generating the main configuration file]]. See also [[GRUB#Warning when installing in chroot]] as another point to be aware of when installing the GRUB loader.<br />
<br />
The kernel parameters you need to specify depend on whether or not you are using the {{ic|encrypt}} hook or the {{ic|sd-encrypt}} hook.<br />
<br />
=== Kernel parameters ===<br />
<br />
[[Kernel parameters]] like {{ic|root}} and {{ic|resume}} are specified the same way for both {{ic|encrypt}} and {{ic|sd-encrypt}} hooks.<br />
<br />
==== root ====<br />
<br />
The {{ic|1=root=}} parameter specifies the {{ic|''device''}} of the actual (decrypted) root file system:<br />
<br />
root=''device''<br />
<br />
* If the file system is formatted directly on the decrypted device file this will be {{ic|/dev/mapper/''dmname''}}.<br />
* If a LVM gets activated first and contains an [[dm-crypt/Encrypting_an_entire_system#LUKS_on_LVM|encrypted logical rootvolume]], the above form applies as well. <br />
* If the root file system is contained in a logical volume of a fully [[encrypted LVM]], the device mapper for it will be in the general form of {{ic|1=root=/dev/''volumegroup''/''logicalvolume''}}.<br />
<br />
{{Tip|This parameter is not needed to be specified manually when using [[GRUB]]. Executing ''grub-mkconfig'' is meant to determine the correct UUID of the decrypted root filesystem and specify it in the generated {{ic|grub.cfg}} automatically.}}<br />
<br />
==== resume ====<br />
<br />
resume=''device''<br />
<br />
* {{ic|''device''}} is the device file of the decrypted (swap) filesystem used for [[Power management/Suspend and hibernate#Hibernation|suspend to disk]]. If swap is on a separate partition, it will be in the form of {{ic|/dev/mapper/swap}}. See also [[dm-crypt/Swap encryption]].<br />
<br />
=== Using encrypt hook ===<br />
<br />
{{Note|Compared to the [[#Using sd-encrypt hook|sd-encrypt]] hook, the {{ic|encrypt}} hook has some limitations. It does not support:<br />
* Unlocking [[dm-crypt/Specialties#The encrypt hook and multiple disks|multiple encrypted disks]] ({{Bug|23182}}). Only '''one''' device can be unlocked in the initramfs.<br />
* Using a [[dm-crypt/Specialties#Encrypted system using a detached LUKS header|detached LUKS header]] ({{Bug|42851}}).<br />
}}<br />
<br />
==== cryptdevice ====<br />
<br />
This parameter will make the system prompt for the passphrase to unlock the device containing the encrypted root on a cold boot. It is parsed by the {{ic|encrypt}} hook to identify which device contains the encrypted system:<br />
<br />
cryptdevice=''device'':''dmname''<br />
<br />
* {{ic|''device''}} is the path to the device backing the encrypted device. Usage of [[Persistent block device naming]] is advisable.<br />
* {{ic|''dmname''}} is the '''d'''evice-'''m'''apper name given to the device after decryption, which will be available as {{ic|/dev/mapper/''dmname''}}.<br />
* If a LVM contains the [[dm-crypt/Encrypting_an_entire_system#LUKS_on_LVM|encrypted root]], the LVM gets activated first and the volume group containing the logical volume of the encrypted root serves as ''device''. It is then followed by the respective volume group to be mapped to root. The parameter follows the form of {{ic|1=cryptdevice=''/dev/vgname/lvname'':''dmname''}}.<br />
<br />
==== cryptkey ====<br />
<br />
This parameter specifies the location of a keyfile and is required by the {{ic|''encrypt''}} hook for reading such a keyfile to unlock the {{ic|''cryptdevice''}} (unless a key is in the default location, see below). It can have three parameter sets, depending on whether the keyfile exists as a file in a particular device, a bitstream starting on a specific location, or a file in the initramfs. <br />
<br />
For a file in a device the format is:<br />
<br />
cryptkey=''device'':''fstype'':''path''<br />
<br />
* {{ic|''device''}} is the raw block device where the key exists.<br />
* {{ic|''fstype''}} is the filesystem type of {{ic|''device''}} (or auto).<br />
* {{ic|''path''}} is the absolute path of the keyfile within the device.<br />
<br />
Example: {{ic|1=cryptkey=/dev/usbstick:vfat:/secretkey}}<br />
<br />
For a bitstream on a device the key's location is specified with the following: <br />
<br />
cryptkey=''device'':''offset'':''size'' <br />
<br />
where the offset and size are in bytes. Example: {{ic|1=cryptkey=/dev/sdZ:0:512}} reads a 512 byte keyfile starting at the beginning of the device. <br />
<br />
{{Tip|If the device path you want to access contains the character {{ic|:}}, you have to escape it with a backslash {{ic|\}}. In that case the cryptkey parameter would be as follow: {{ic|1=cryptkey=/dev/disk/by-id/usb-123456-0\:0:0:512}} for a usb key with the id {{ic|usb-123456-0:0}}.}}<br />
<br />
For a file [[mkinitcpio#BINARIES and FILES|included]] in the initramfs the format is[https://git.archlinux.org/svntogit/packages.git/tree/trunk/hooks-encrypt?h=packages/cryptsetup#n14]:<br />
<br />
cryptkey=rootfs:''path''<br />
<br />
Example: {{ic|1=cryptkey=rootfs:/secretkey}}<br />
<br />
Also note that if {{ic|cryptkey}} is not specified, it defaults to {{ic|/crypto_keyfile.bin}} (in the initramfs).[https://git.archlinux.org/svntogit/packages.git/tree/trunk/hooks-encrypt?h=packages/cryptsetup#n8]<br />
<br />
See also [[dm-crypt/Device encryption#Keyfiles]].<br />
<br />
==== crypto ====<br />
<br />
This parameter is specific to pass ''dm-crypt'' plain mode options to the ''encrypt'' hook. <br />
<br />
It takes the form<br />
<br />
crypto=<hash>:<cipher>:<keysize>:<offset>:<skip><br />
<br />
The arguments relate directly to the ''cryptsetup'' options. See [[dm-crypt/Device encryption#Encryption options for plain mode]].<br />
<br />
For a disk encrypted with just ''plain'' default options, the {{ic|crypto}} arguments must be specified, but each entry can be left blank: <br />
<br />
crypto=::::<br />
<br />
A specific example of arguments is <br />
<br />
crypto=sha512:twofish-xts-plain64:512:0:<br />
<br />
=== Using sd-encrypt hook ===<br />
<br />
In all of the following {{ic|rd.luks}} can be replaced with {{ic|luks}}. {{ic|rd.luks}} parameters are only honored by the initrd. {{ic|luks}} parameters are honored by both the main system and initrd. Unless you want to control devices which get unlocked after boot from kernel command line, use {{ic|rd.luks}}. See {{man|8|systemd-cryptsetup-generator}} for more options and more details.<br />
<br />
{{Tip|<br />
* If the file {{ic|/etc/crypttab.initramfs}} exists, [[mkinitcpio]] will add it to the initramfs as {{ic|/etc/crypttab}}, you can specify devices that need to be unlocked at boot there. Syntax is documented in [[#crypttab]] and {{man|5|crypttab}}.<br />
* {{ic|/etc/crypttab.initramfs}} is not limited to using only UUID like {{ic|rd.luks}}. You can use any of the [[Persistent block device naming#Persistent naming methods|persistent block device naming methods]].<br />
}}<br />
<br />
{{Note|<br />
* All of the {{ic|rd.luks}} parameters can be specified multiple times to unlock multiple LUKS encrypted volumes.<br />
* The {{ic|rd.luks}} parameters only support unlocking detectable LUKS devices. To unlock a plain dm-crypt device or a LUKS device with a detached header, you must specify it in {{ic|/etc/crypttab.initramfs}}. See [[#crypttab]] for the syntax.<br />
}}<br />
<br />
{{Warning|If you are using {{ic|/etc/crypttab}} or {{ic|/etc/crypttab.initramfs}} together with {{ic|luks.*}} or {{ic|rd.luks.*}} parameters, only those devices specified on the kernel command line will be activated and you will see {{ic|Not creating device 'devicename' because it was not specified on the kernel command line.}}. To activate all devices in {{ic|/etc/crypttab}} do not specify any {{ic|luks.*}} parameters and use {{ic|rd.luks.*}}. To activate all devices in {{ic|/etc/crypttab.initramfs}} do not specify any {{ic|luks.*}} or {{ic|rd.luks.*}} parameters.}}<br />
<br />
==== rd.luks.uuid ====<br />
<br />
{{Tip|{{ic|rd.luks.uuid}} can be omitted when using {{ic|rd.luks.name}}.}}<br />
<br />
rd.luks.uuid=''UUID''<br />
<br />
Specify the [[UUID]] of the device to be decrypted on boot with this flag. If the UUID is in {{ic|/etc/crypttab.initramfs}}, the options listed there will be used. For {{ic|luks.uuid}} options from {{ic|/etc/crypttab.initramfs}} or {{ic|/etc/crypttab}} will be used.<br />
<br />
By default the mapped device will be located at {{ic|/dev/mapper/luks-''UUID''}} where ''UUID'' is the UUID of the LUKS partition.<br />
<br />
==== rd.luks.name ====<br />
<br />
rd.luks.name=''UUID''=''name''<br />
<br />
Specify the name of the mapped device after the LUKS partition is open. For example, specifying {{ic|1=''UUID''=cryptroot}} causes the unlocked device to be located at {{ic|/dev/mapper/cryptroot}}. If this is not specified the mapped device will be located at {{ic|/dev/mapper/luks-''UUID''}} where ''UUID'' is the UUID of the LUKS partition. When using this parameter you can omit {{ic|rd.luks.uuid}}.<br />
<br />
This is equivalent to the second parameter of {{ic|encrypt}}'s {{ic|cryptdevice}}.<br />
<br />
==== rd.luks.options ====<br />
<br />
rd.luks.options=UUID=''options''<br />
<br />
or<br />
<br />
rd.luks.options=''options''<br />
<br />
Specify options for the device listed after {{ic|UUID}} or, if not specified, for all UUIDs not specified elsewhere (e.g., crypttab).<br />
<br />
This is roughly equivalent to the third parameter of {{ic|encrypt}}'s {{ic|cryptdevice}}.<br />
<br />
Follows a similar format to options in crypttab - options are separated by commas, options with values are specified using {{ic|1=''option''=''value''}}.<br />
<br />
For example:<br />
<br />
rd.luks.options=timeout=10s,swap,cipher=aes-cbc-essiv:sha256,size=256<br />
<br />
==== rd.luks.key ====<br />
<br />
{{Note|{{ic|sd-encrypt}} hook only supports keyfiles that are embedded in the initramfs (i.e. specified in the {{ic|FILES}} array in {{ic|/etc/mkinitcpio.conf}}). See [https://github.com/systemd/systemd/issues/9181 systemd issue 9181].}}<br />
<br />
rd.luks.key=UUID=''mykeyfile''<br />
or<br />
rd.luks.key=''mykeyfile''<br />
<br />
Specify the location of a password file used to decrypt the device specified in {{ic|rd.luks.UUID}}. There is no default location like there is with the {{ic|encrypt}} hook parameter {{ic|cryptkey}}.<br />
<br />
==== Timeout ====<br />
<br />
There are two options that affect the timeout for entering the password during boot:<br />
* {{ic|1=rd.luks.options=timeout=''mytimeout''}} specifies the timeout for querying for a password<br />
* {{ic|1=rootflags=x-systemd.device-timeout=''mytimeout''}} specifies how long systemd should wait for the rootfs device to show up before giving up (defaults to 90 seconds)<br />
<br />
If you want to disable the timeout altogether, then set both timeouts to zero:<br />
<br />
rd.luks.options=timeout=0 rootflags=x-systemd.device-timeout=0<br />
<br />
== crypttab ==<br />
<br />
The {{ic|/etc/crypttab}} (encrypted device table) file is similar to the [[fstab]] file and contains a list of encrypted devices to be unlocked during system boot up. This file can be used for automatically mounting encrypted swap devices or secondary file systems.<br />
<br />
{{ic|crypttab}} is read ''before'' {{ic|fstab}}, so that dm-crypt containers can be unlocked before the file system inside is mounted. Note that {{ic|crypttab}} is read ''after'' the system has booted up, therefore it is not a replacement for unlocking encrypted partitions by using [[#mkinitcpio|mkinitcpio]] hooks and [[#Boot loader|boot loader options]] as in the case of [[dm-crypt/Encrypting an entire system|encrypting the root partition]]. {{ic|crypttab}} processing at boot time is made by the {{ic|systemd-cryptsetup-generator}} automatically.<br />
<br />
See {{man|5|crypttab}} for details, read below for some examples, and the [[#Mounting at boot time]] section for instructions on how to use UUIDs to mount an encrypted device.<br />
<br />
{{Note|1=When using [[systemd-boot]] and the {{ic|sd-encrypt}} hook, if a non-root partition's passphrase is the same as root's, there is no need to put that non-root partition in crypttab due to passphrase caching. See [https://bbs.archlinux.org/viewtopic.php?id=219859 this forum thread] for more information.}}<br />
<br />
{{Warning|<br />
* If the ''nofail'' option is specified, the password entry screen may disappear while typing the password. ''nofail'' should therefore only be used together with keyfiles.<br />
* There are issues with [[systemd]] when processing {{ic|crypttab}} entries for ''dm-crypt'' [[dm-crypt/Device_encryption#Encryption options for plain mode|plain mode]] ({{ic|--type plain}}) devices: <br />
** For {{ic|--type plain}} devices with a keyfile, it is necessary to add the {{ic|1=hash=plain}} option to crypttab due to a [https://bugs.freedesktop.org/show_bug.cgi?id&#61;52630 systemd incompatibility]. '''Do not''' use {{ic|systemd-cryptsetup}} manually for device creation to work around it.<br />
** It may be further required to add the {{ic|plain}} option explicitly to force {{ic|systemd-cryptsetup}} to recognize a {{ic|--type plain}}) device at boot. See [https://github.com/systemd/systemd/issues/442 systemd issue 442].<br />
}}<br />
<br />
{{hc|/etc/crypttab|2=<br />
# Example crypttab file. Fields are: name, underlying device, passphrase, cryptsetup options.<br />
<br />
# Mount /dev/lvm/swap re-encrypting it with a fresh key each reboot<br />
swap /dev/lvm/swap /dev/urandom swap,cipher=aes-xts-plain64,size=256<br />
<br />
# Mount /dev/lvm/tmp as /dev/mapper/tmp using plain dm-crypt with a random passphrase, making its contents unrecoverable after it is dismounted.<br />
tmp /dev/lvm/tmp /dev/urandom tmp,cipher=aes-xts-plain64,size=256 <br />
<br />
# Mount /dev/lvm/home as /dev/mapper/home using LUKS, and prompt for the passphrase at boot time.<br />
home /dev/lvm/home<br />
<br />
# Mount /dev/sdb1 as /dev/mapper/backup using LUKS, with a passphrase stored in a file.<br />
backup /dev/sdb1 /home/alice/backup.key<br />
}}<br />
<br />
=== Mounting at boot time ===<br />
<br />
If you want to mount an encrypted drive at boot time, enter the device's UUID in {{ic|/etc/crypttab}}. You get the UUID (partition) by using the command {{ic|lsblk -f}} and adding it to {{ic|crypttab}} in the form:<br />
<br />
{{hc|/etc/crypttab|2=<br />
externaldrive UUID=2f9a8428-ac69-478a-88a2-4aa458565431 none luks,timeout=180<br />
}}<br />
<br />
The first parameter is your preferred device mapper's name for the encrypted drive. The option {{ic|none}} will trigger a prompt during boot to type the passphrase for unlocking the partition. The {{ic|timeout}} option defines a timeout in seconds for entering the decryption password during boot.<br />
<br />
{{Note|Keep in mind that the {{ic|timeout}} option in {{ic|crypttab}} only determines the amount of time allowed for ''entering the password'' of the encrypted device. In addition, [[systemd]] also has a default timeout which determines the amount of time allowed for ''the device to be available'' (defaulting to 90 seconds), which is independent of the password timer. In consequence, even when the {{ic|timeout}} option in {{ic|crypttab}} is set to a value larger than 90 seconds (or it is at its default value of 0, meaning unlimited time), ''systemd'' will still only wait a maximum of 90 seconds for the device to be unlocked. In order to change the time ''systemd'' will wait for a device to be available, the option {{ic|x-systemd.device-timeout}} (see {{man|5|systemd.mount}}) can be set in [[fstab]] for said device. It is probably desired, then, that the amount of time of the {{ic|timeout}} option in {{ic|crypttab}} is equal to the amount of time of the {{ic|x-systemd.device-timeout}} option in {{ic|fstab}} for each device mounted at boot time.}}<br />
<br />
A [[dm-crypt/Device_encryption#Keyfiles|keyfile]] can also be set up and referenced instead of {{ic|none}}. This results in an automatic unlocking, if the keyfile is accessible during boot. Since LUKS offers the option to have multiple keys, the chosen option can also be changed later.<br />
<br />
Use the device mapper's name you have defined in {{ic|/etc/crypttab}} in {{ic|/etc/fstab}} as follows:<br />
<br />
{{hc|/etc/fstab|2=<br />
/dev/mapper/externaldrive /mnt/backup ext4 defaults,errors=remount-ro 0 2<br />
}}<br />
<br />
Since {{ic|/dev/mapper/externaldrive}} already is the result of a unique partition mapping, there is no need to specify an UUID for it. In any case, the mapper with the filesystem will have a different UUID than the partition it is encrypted in.<br />
<br />
==== Mounting a stacked blockdevice ====<br />
<br />
The systemd generators also automatically process stacked block devices at boot. <br />
<br />
For example, you can create a [[RAID]] setup, use cryptsetup on it and create an [[LVM]] logical volume with respective filesystem inside the encrypted block device. A resulting: <br />
<br />
{{hc|$ lsblk -f|<br />
─sdXX linux_raid_member <br />
│ └─md0 crypto_LUKS <br />
│ └─cryptedbackup LVM2_member <br />
│ └─vgraid-lvraid ext4 /mnt/backup<br />
└─sdYY linux_raid_member <br />
└─md0 crypto_LUKS <br />
└─cryptedbackup LVM2_member <br />
└─vgraid-lvraid ext4 /mnt/backup<br />
}}<br />
<br />
will ask for the passphrase and mount automatically at boot. <br />
<br />
Given you specify the correct corresponding crypttab (e.g. UUID for the {{ic|crypto_LUKS}} device) and fstab ({{ic|/dev/vgraid/lvraid}}) entries, there is no need to add additional mkinitcpio hooks/configuration, because {{ic|/etc/crypttab}} processing applies to non-root mounts only. One exception is when the {{ic|mdadm_udev}} hook is used ''already'' (e.g. for the root device). In this case {{ic|/etc/madadm.conf}} and the initramfs need updating to achieve the correct root raid is picked first.<br />
<br />
=== Mounting on demand ===<br />
<br />
You can use <br />
<br />
# systemctl start systemd-cryptsetup@externaldrive<br />
<br />
instead of<br />
<br />
# cryptsetup luksOpen UUID=... externaldrive<br />
<br />
when you have an entry as follows in your /etc/crypttab:<br />
<br />
{{hc|head=/etc/crypttab|output=externaldrive UUID=... none noauto}}<br />
<br />
That way you do not need to remember the exact crypttab options. It will prompt you for the passphrase if needed.<br />
<br />
The corresponding unit file is generated automatically by [https://www.freedesktop.org/software/systemd/man/systemd-cryptsetup-generator.html systemd-cryptsetup-generator].<br />
<br />
You can list all generated unit files using<br />
<br />
$ systemctl list-unit-files | grep systemd-cryptsetup<br />
<br />
== Troubleshooting ==<br />
<br />
=== System stuck on boot/password prompt does not show ===<br />
<br />
If you are using [[Plymouth]], make sure to use the correct modules (see: [[Plymouth#The plymouth hook]]) or disable it. Otherwise Plymouth will swallow the password prompt, making a system boot impossible.</div>Phireskyhttps://wiki.archlinux.org/index.php?title=Systemd-nspawn&diff=449842Systemd-nspawn2016-09-07T16:50:51Z<p>Phiresky: Make "Configure networking" clearer</p>
<hr />
<div>{{Lowercase title}}<br />
[[Category:Virtualization]]<br />
[[ja:Systemd-nspawn]]<br />
[[ru:Systemd-nspawn]]<br />
{{Related articles start}}<br />
{{Related|systemd}}<br />
{{Related|Linux Containers}}<br />
{{Related|systemd-networkd}}<br />
{{Related|Docker}}<br />
{{Related|Lxc-systemd}}<br />
{{Related articles end}}<br />
<br />
''systemd-nspawn'' is like the [[chroot]] command, but it is a ''chroot on steroids''.<br />
<br />
''systemd-nspawn'' may be used to run a command or OS in a light-weight namespace container. It is more powerful than [[chroot]] since it fully virtualizes the file system hierarchy, as well as the process tree, the various IPC subsystems and the host and domain name.<br />
<br />
''systemd-nspawn'' limits access to various kernel interfaces in the container to read-only, such as {{ic|/sys}}, {{ic|/proc/sys}} or {{ic|/sys/fs/selinux}}. Network interfaces and the system clock may not be changed from within the container. Device nodes may not be created. The host system cannot be rebooted and kernel modules may not be loaded from within the container.<br />
<br />
This mechanism differs from [[Lxc-systemd]] or [[Libvirt]]-lxc, as it is a much simpler tool to configure.<br />
<br />
== Installation ==<br />
<br />
''systemd-nspawn'' is part of and packaged with {{Pkg|systemd}}. <br />
<br />
== Examples ==<br />
<br />
=== Create and boot a minimal Arch Linux distribution in a container ===<br />
<br />
First install {{Pkg|arch-install-scripts}}.<br />
<br />
Next, create a directory to hold the container. In this example we will use {{ic|~/MyContainer}}. <br />
<br />
Next, we use pacstrap to install a basic arch-system into the container. At minimum we need to install the {{Grp|base}} group. <br />
<br />
# pacstrap -i -c -d ~/MyContainer base [additional pkgs/groups]<br />
<br />
{{Tip|The {{ic|-i}} option will '''avoid''' auto-confirmation of package selection. As you do not need to install the Linux kernel in the container, you can remove it from the package list selection to save space. See [[Pacman#Usage]].}}<br />
<br />
{{Note|The package {{Pkg|linux-firmware}} required by {{Pkg|linux}}, which is included in the {{Grp|base}} group and isn't necessary to run the container, causes some issues to {{ic|systemd-tmpfiles-setup.service}} during the booting process with {{ic|systemd-nspawn}}. It's possible to install the {{Grp|base}} group but excluding the {{Pkg|linux}} package and its dependencies when building the container with {{ic|# pacstrap -i -c -d ~/MyContainer base --ignore linux [additional pkgs/groups]}}. The {{ic|--ignore}} flag will be simply passed to {{Pkg|pacman}}. See {{Bug|46591}} for more information.}}<br />
<br />
Once your installation is finished, boot into the container:<br />
<br />
# systemd-nspawn -b -D ~/MyContainer -n<br />
<br />
The {{ic|-b}} option will boot the container (i.e. run {{ic|systemd}} as PID=1), instead of just running a shell. {{ic|-D}} specifies the directory that becomes the container's root directory and {{ic|-n}} will set up a private network between host and container.<br />
<br />
After the container starts, log in as "root" with no password.<br />
<br />
The container can be powered off by running {{ic|poweroff}} from within the container. From the host, containers can be controlled by the [[#machinectl|machinectl]] tool.<br />
<br />
{{Note|To terminate the ''session'' from within the container, hold {{ic|Ctrl}} and rapidly press {{ic|]}} three times. Non US keyboard will use {{ic|%}} instead of {{ic|]}}}}<br />
<br />
==== Bootstrap Arch Linux i686 inside x86_64 host ====<br />
<br />
It is possible to install a minimal i686 Arch Linux inside a subdirectory and use it as systemd-nspawn container instead of [[chroot]] or [[virtualization]]. This is useful for testing {{ic|PKGBUILD}} compilation for i686 and other tasks. Make sure you use a {{ic|pacman.conf}} '''without''' {{ic|multilib}} repository.<br />
<br />
# pacman_conf=/tmp/pacman.conf # this is pacman.conf without multilib<br />
# mkdir /mnt/i686-archlinux<br />
# linux32 pacstrap -C "$pacman_conf" -di /mnt/i686-archlinux base base-devel<br />
<br />
You may deselect {{ic|linux}} from {{ic|base}} group, since the resulting bootstrap directory is not meant to be booted on real or virtualized hardware.<br />
<br />
To start the resulting i686 Arch Linux systemd-nspawn instance, just issue the following command.<br />
<br />
# linux32 systemd-nspawn -D /mnt/i686-archlinux<br />
<br />
=== Create a Debian or Ubuntu environment ===<br />
<br />
Install {{Pkg|debootstrap}}, {{Aur|gnupg1}}, and one or both of {{Aur|debian-archive-keyring}} and {{Aur|ubuntu-keyring}} (obviously install the keyrings for the distros you want).<br />
<br />
{{Note|systemd-nspawn requires that the os in the container has systemd running as PID 1, this means Ubuntu before 15.04 will not work out of the box and requires additional configuration to switch from upstart to systemd.}}<br />
<br />
From there it's rather easy to setup Debian or Ubuntu environments:<br />
<br />
# cd /var/lib/machines<br />
# debootstrap <codename> myContainer <repository><br />
<br />
For Debian valid code names are either the rolling names like "stable" and "testing" or release names like "stretch" and "sid", for Ubuntu the code name like "wily" or "hardy" should be used. A complete list is in /usr/share/debootstrap/scripts<br />
<br />
Unlike Arch, Debian and Ubuntu will not let you login without a password on first login. To set the root password login without the '-b' option and set a password:<br />
<br />
# systemd-nspawn -D myContainer<br />
# passwd<br />
# logout<br />
<br />
=== Enable container on boot ===<br />
<br />
When using a container frequently, you may want to start it on boot.<br />
<br />
First [[enable]] the {{ic|machines.target}} target, then {{ic|systemd-nspawn@''myContainer''.service}}, where {{ic|myContainer}} is an nspawn container in {{ic|/var/lib/machines}}.<br />
<br />
{{Tip|<br />
* Symbolic links to containers in {{ic|/var/lib/machines}} do not work as of {{Pkg|systemd}} v229, see [https://github.com/systemd/systemd/issues/2001].<br />
* To customize the startup of a container, [[edit]] the {{ic|systemd-nspawn@''myContainer''}} unit instance. See {{ic|systemd-nspawn(1)}} for all options.<br />
}}<br />
<br />
=== Build and test packages ===<br />
<br />
See [[Creating packages for other distributions]] for example uses.<br />
<br />
== Management ==<br />
<br />
=== machinectl ===<br />
<br />
{{Note|The ''machinectl'' tool requires [[systemd]] and {{Pkg|dbus}} to be installed in the container. See [https://github.com/systemd/systemd/issues/685] for detailed discussion.}}<br />
<br />
Managing your containers is essentially done with the {{ic|machinectl}} command. See {{ic|machinectl(1)}} for details.<br />
<br />
Examples:<br />
<br />
Spawn a new shell inside a running container: <br />
<br />
$ machinectl login ''MyContainer''<br />
<br />
Show detailed information about a container: <br />
<br />
$ machinectl status ''MyContainer''<br />
<br />
Reboot a container:<br />
<br />
$ machinectl reboot ''MyContainer''<br />
<br />
Poweroff a container:<br />
<br />
$ machinectl poweroff ''MyContainer''<br />
<br />
{{Tip|Poweroff and reboot operations can be performed from within a container session using the ''systemctl'' {{ic|poweroff}} or {{ic|reboot}} commands.}}<br />
<br />
Download an image:<br />
<br />
# machinectl pull-tar ''URL'' ''name''<br />
<br />
=== systemd toolchain ===<br />
<br />
Much of the core systemd toolchain has been updated to work with containers. Tools that do usually provide a {{ic|1=-M, --machine=}} option which will take a container name as argument.<br />
<br />
Examples:<br />
<br />
See journal logs for a particular machine:<br />
<br />
$ journalctl -M ''MyContainer''<br />
<br />
Show control group contents:<br />
<br />
$ systemd-cgls -M ''MyContainer''<br />
<br />
See startup time of container:<br />
<br />
$ systemd-analyze -M ''MyContainer''<br />
<br />
For an overview of resource usage:<br />
<br />
$ systemd-cgtop<br />
<br />
== Tips and tricks ==<br />
<br />
=== Use an X environment ===<br />
<br />
See [[Xhost]] and [[Change root#Run graphical applications from chroot]].<br />
<br />
You will need to set the {{ic|DISPLAY}} environment variable inside your container session to connect to the external X server.<br />
<br />
X stores some required files in the {{ic|/tmp}} directory. In order for your container to display anything, it needs access to those files. To do so, append the {{ic|--bind<nowiki>=</nowiki>/tmp/.X11-unix:/tmp/.X11-unix}} option when starting the container.<br />
<br />
=== Run Firefox ===<br />
<br />
See [[Firefox tweaks#Run Firefox inside an nspawn container|Firefox tweaks]].<br />
<br />
=== Access host filesystem ===<br />
<br />
See {{ic|--bind}} and {{ic|--bind-ro}} in {{ic|man systemd-nspawn}}.<br />
<br />
If both the host and the container are Arch Linux, then one could, for example, share the pacman cache:<br />
<br />
# systemd-nspawn --bind=/var/cache/pacman/pkg<br />
<br />
=== Configure networking ===<br />
<br />
{{Style}}<br />
<br />
<br />
For the most simple setup, allowing outgoing connections to the internet, you can use [[systemd-networkd]] for network management and DHCP and {{ic|systemd-resolved}} for DNS.<br />
<br />
# systemctl enable --now systemd-networkd systemd-resolved<br />
# ln -sf /run/systemd/resolve/resolv.conf /etc/resolv.conf # let systemd-resolved manage /etc/resolv.conf<br />
<br />
This assumes you have started {{ic|systemd-nspawn}} with the {{ic|-n}} switch, creating a virtual Ethernet link to the host.<br />
<br />
Instead of using {{ic|systemd-resolved}} you can also manually [[textedit|edit]] your container's {{ic|/etc/resolv.conf}} by adding your DNS server's IP address.<br />
<br />
Note the canonical [[systemd-networkd]] host and container .network files are from https://github.com/systemd/systemd/tree/master/network .<br />
<br />
See [[systemd-networkd#Usage with containers]] for more complex examples.<br />
<br />
==== nsswitch.conf ====<br />
<br />
{{Merge|systemd-networkd}}<br />
<br />
To make it easier to connect to a container from the host, you can enable local DNS resolution for container names. In {{ic|/etc/nsswitch.conf}}, add {{ic|mymachines}} to the {{ic|hosts:}} section, e.g.<br />
<br />
hosts: files mymachines dns myhostname<br />
<br />
Then, any DNS lookup for hostname {{ic|foo}} on the host will first consult {{ic|/etc/hosts}}, then the names of local containers, then upstream DNS etc.<br />
<br />
==== Use host networking ====<br />
<br />
To disable private networking used by containers started with {{ic|machinectl start MyContainer}}, [[edit]] the configuration of {{ic|systemd-nspawn@.service}} with {{ic|systemctl edit systemd-nspawn@.service}} and set the {{ic|1=ExecStart=}} option without the {{ic|--network-veth}} parameter unlike the original service:<br />
<br />
{{hc|/etc/systemd/system/systemd-nspawn@.service.d/override.conf|<nowiki><br />
[Service]<br />
ExecStart=<br />
ExecStart=/usr/bin/systemd-nspawn --quiet --keep-unit --boot --link-journal=try-guest --machine=%I<br />
</nowiki>}}<br />
<br />
The newly started containers will use the hosts networking.<br />
<br />
==== Virtual Ethernet interfaces ====<br />
<br />
If a container is started with {{ic|systemd-nspawn ... -n}}, systemd will automatically create one virtual Ethernet interface on the host, and one in the container, connected by a virtual Ethernet cable.<br />
<br />
If the name of the container is {{ic|foo}}, the name of the virtual Ethernet interface on the host is {{ic|ve-foo}}. The name of the virtual Ethernet interface in the container is always {{ic|host0}}.<br />
<br />
When examining the interfaces with {{ic|ip link}}, interface names will be shown with a suffix, such as {{ic|ve-foo@if2}} and {{ic|host0@if9}}. The {{ic|@ifN}} is not actually part of the name of the interface; instead, {{ic|ip link}} appends this information to indicate which "slot" the virtual Ethernet cable connects to on the other end.<br />
<br />
For example, a host virtual Ethernet interface shown as {{ic|ve-foo@if2}} will connect to container {{ic|foo}}, and inside the container to the second network interface -- the one shown with index 2 when running {{ic|ip link}} inside the container. Similarly, in the container, the interface named {{ic|host0@if9}} will connect to the 9th slot on the host.<br />
<br />
=== Run on a non-systemd system ===<br />
<br />
See [[Init#systemd-nspawn]].<br />
<br />
=== Specify per-container settings ===<br />
<br />
To specify per-container settings and not overrides for all (e.g. bind a directory to only one container)[https://github.com/systemd/systemd/issues/3442#issuecomment-223837408], the ".nspawn" file definition can be used [https://www.freedesktop.org/software/systemd/man/systemd.nspawn.html#]<br />
<br />
man systemd.nspawn<br />
<br />
=== Use Btrfs subvolume as container root ===<br />
<br />
To use a btrfs subvolume as a template for the container's root, use the {{ic|--template}} flag. This takes a snapshot of the subvolume and populates the root directory for the container with it.<br />
<br />
{{Note|If the template path specified is not the root of a subvolume, the '''entire''' tree is copied. This will be very time consuming.}}<br />
<br />
For example, to use a snapshot located at {{ic|/.snapshots/403/snapshot}}:<br />
<br />
# systemd-nspawn --template=/.snapshots/403/snapshots -b -D ''my-container''<br />
<br />
where {{ic|''my-container''}} is the name of the directory that will be created for the container. After powering off, the newly created subvolume is retained.<br />
<br />
=== Use temporary Btrfs snapshot of container ===<br />
<br />
One can use the {{ic|--ephemeral}} or {{ic|-x}} flag to create a temporary btrfs snapshot of the container and use it as the container root. Any changes made while booted in the container will be lost. For example:<br />
<br />
# systemd-nspawn -D ''my-container'' -xb<br />
<br />
where ''my-container'' is the directory of an '''existing''' container or system. For example, if {{ic|/}} is a btrfs subvolume one could create an ephemeral container of the currently running host system by doing:<br />
<br />
# systemd-nspawn -D / -xb <br />
<br />
After powering off the container, the btrfs subvolume that was created is immediately removed.<br />
<br />
== Troubleshooting ==<br />
<br />
=== root login fails ===<br />
<br />
If you get the following error when you try to login (i.e. using {{ic|machinectl login <name>}}):<br />
<br />
arch-nspawn login: root<br />
Login incorrect<br />
<br />
And {{ic|journalctl}} shows:<br />
<br />
pam_securetty(login:auth): access denied: tty 'pts/0' is not secure !<br />
<br />
Add {{ic|pts/0}} to the list of terminal names in {{ic|/etc/securetty}} on the '''container''' filesystem, see [http://unix.stackexchange.com/questions/41840/effect-of-entries-in-etc-securetty/41939#41939]. You can also opt to delete {{ic|/etc/securetty}} on the '''container''' to allow root to login to any tty, see [https://github.com/systemd/systemd/issues/852].<br />
<br />
=== Unable to upgrade some packages on the container ===<br />
<br />
It can sometimes be impossible to upgrade some packages on the container, {{Pkg|filesystem}} being a perfect example. The issue is due to {{ic|/sys}} being mounted as Read Only. The workaround is to remount the directory in Read Write when running {{ic|mount -o remount,rw -t sysfs sysfs /sys}}, do the upgrade then reboot the container.<br />
<br />
== See also ==<br />
<br />
* [http://www.freedesktop.org/software/systemd/man/machinectl.html machinectl man page]<br />
* [http://www.freedesktop.org/software/systemd/man/systemd-nspawn.html systemd-nspawn man page]<br />
* [http://lwn.net/Articles/572957/ Creating containers with systemd-nspawn]<br />
* [https://www.youtube.com/results?search_query=systemd-nspawn&aq=f Presentation by Lennart Pottering on systemd-nspawn]<br />
* [http://dabase.com/e/12009/ Running Firefox in a systemd-nspawn container]</div>Phireskyhttps://wiki.archlinux.org/index.php?title=Systemd-nspawn&diff=449552Systemd-nspawn2016-09-05T15:24:09Z<p>Phiresky: Simple network for minimal Arch Linux distribution in a container</p>
<hr />
<div>{{Lowercase title}}<br />
[[Category:Virtualization]]<br />
[[ja:Systemd-nspawn]]<br />
[[ru:Systemd-nspawn]]<br />
{{Related articles start}}<br />
{{Related|systemd}}<br />
{{Related|Linux Containers}}<br />
{{Related|systemd-networkd}}<br />
{{Related|Docker}}<br />
{{Related|Lxc-systemd}}<br />
{{Related articles end}}<br />
<br />
''systemd-nspawn'' is like the [[chroot]] command, but it is a ''chroot on steroids''.<br />
<br />
''systemd-nspawn'' may be used to run a command or OS in a light-weight namespace container. It is more powerful than [[chroot]] since it fully virtualizes the file system hierarchy, as well as the process tree, the various IPC subsystems and the host and domain name.<br />
<br />
''systemd-nspawn'' limits access to various kernel interfaces in the container to read-only, such as {{ic|/sys}}, {{ic|/proc/sys}} or {{ic|/sys/fs/selinux}}. Network interfaces and the system clock may not be changed from within the container. Device nodes may not be created. The host system cannot be rebooted and kernel modules may not be loaded from within the container.<br />
<br />
This mechanism differs from [[Lxc-systemd]] or [[Libvirt]]-lxc, as it is a much simpler tool to configure.<br />
<br />
== Installation ==<br />
<br />
''systemd-nspawn'' is part of and packaged with {{Pkg|systemd}}. <br />
<br />
== Examples ==<br />
<br />
=== Create and boot a minimal Arch Linux distribution in a container ===<br />
<br />
First install {{Pkg|arch-install-scripts}}.<br />
<br />
Next, create a directory to hold the container. In this example we will use {{ic|~/MyContainer}}. <br />
<br />
Next, we use pacstrap to install a basic arch-system into the container. At minimum we need to install the {{Grp|base}} group. <br />
<br />
# pacstrap -i -c -d ~/MyContainer base [additional pkgs/groups]<br />
<br />
{{Tip|The {{ic|-i}} option will '''avoid''' auto-confirmation of package selection. As you do not need to install the Linux kernel in the container, you can remove it from the package list selection to save space. See [[Pacman#Usage]].}}<br />
<br />
{{Note|The package {{Pkg|linux-firmware}} required by {{Pkg|linux}}, which is included in the {{Grp|base}} group and isn't necessary to run the container, causes some issues to {{ic|systemd-tmpfiles-setup.service}} during the booting process with {{ic|systemd-nspawn}}. It's possible to install the {{Grp|base}} group but excluding the {{Pkg|linux}} package and its dependencies when building the container with {{ic|# pacstrap -i -c -d ~/MyContainer base --ignore linux [additional pkgs/groups]}}. The {{ic|--ignore}} flag will be simply passed to {{Pkg|pacman}}. See {{Bug|46591}} for more information.}}<br />
<br />
Once your installation is finished, boot into the container:<br />
<br />
# systemd-nspawn -b -D ~/MyContainer -n<br />
<br />
The {{ic|-b}} option will boot the container (i.e. run {{ic|systemd}} as PID=1), instead of just running a shell. {{ic|-D}} specifies the directory that becomes the container's root directory and {{ic|-n}} will set up a private network between host and container.<br />
<br />
After the container starts, log in as "root" with no password.<br />
<br />
To allow simple networking, enable DHCP and DNS using<br />
<br />
# ln -sf /run/systemd/resolve/resolv.conf /etc/resolv.conf<br />
# systemctl enable --now systemd-networkd systemd-resolved<br />
<br />
For more information, see [[#Configure_networking|Configure networking]].<br />
<br />
The container can be powered off by running {{ic|poweroff}} from within the container. From the host, containers can be controlled by the [[#machinectl|machinectl]] tool.<br />
<br />
{{Note|To terminate the ''session'' from within the container, hold {{ic|Ctrl}} and rapidly press {{ic|]}} three times. Non US keyboard will use {{ic|%}} instead of {{ic|]}}}}<br />
<br />
==== Bootstrap Arch Linux i686 inside x86_64 host ====<br />
<br />
It is possible to install a minimal i686 Arch Linux inside a subdirectory and use it as systemd-nspawn container instead of [[chroot]] or [[virtualization]]. This is useful for testing {{ic|PKGBUILD}} compilation for i686 and other tasks. Make sure you use a {{ic|pacman.conf}} '''without''' {{ic|multilib}} repository.<br />
<br />
# pacman_conf=/tmp/pacman.conf # this is pacman.conf without multilib<br />
# mkdir /mnt/i686-archlinux<br />
# linux32 pacstrap -C "$pacman_conf" -di /mnt/i686-archlinux base base-devel<br />
<br />
You may deselect {{ic|linux}} from {{ic|base}} group, since the resulting bootstrap directory is not meant to be booted on real or virtualized hardware.<br />
<br />
To start the resulting i686 Arch Linux systemd-nspawn instance, just issue the following command.<br />
<br />
# linux32 systemd-nspawn -D /mnt/i686-archlinux<br />
<br />
=== Create a Debian or Ubuntu environment ===<br />
<br />
Install {{Pkg|debootstrap}}, {{Aur|gnupg1}}, and one or both of {{Aur|debian-archive-keyring}} and {{Aur|ubuntu-keyring}} (obviously install the keyrings for the distros you want).<br />
<br />
{{Note|systemd-nspawn requires that the os in the container has systemd running as PID 1, this means Ubuntu before 15.04 will not work out of the box and requires additional configuration to switch from upstart to systemd.}}<br />
<br />
From there it's rather easy to setup Debian or Ubuntu environments:<br />
<br />
# cd /var/lib/machines<br />
# debootstrap <codename> myContainer <repository><br />
<br />
For Debian valid code names are either the rolling names like "stable" and "testing" or release names like "stretch" and "sid", for Ubuntu the code name like "wily" or "hardy" should be used. A complete list is in /usr/share/debootstrap/scripts<br />
<br />
Unlike Arch, Debian and Ubuntu will not let you login without a password on first login. To set the root password login without the '-b' option and set a password:<br />
<br />
# systemd-nspawn -D myContainer<br />
# passwd<br />
# logout<br />
<br />
=== Enable container on boot ===<br />
<br />
When using a container frequently, you may want to start it on boot.<br />
<br />
First [[enable]] the {{ic|machines.target}} target, then {{ic|systemd-nspawn@''myContainer''.service}}, where {{ic|myContainer}} is an nspawn container in {{ic|/var/lib/machines}}.<br />
<br />
{{Tip|<br />
* Symbolic links to containers in {{ic|/var/lib/machines}} do not work as of {{Pkg|systemd}} v229, see [https://github.com/systemd/systemd/issues/2001].<br />
* To customize the startup of a container, [[edit]] the {{ic|systemd-nspawn@''myContainer''}} unit instance. See {{ic|systemd-nspawn(1)}} for all options.<br />
}}<br />
<br />
=== Build and test packages ===<br />
<br />
See [[Creating packages for other distributions]] for example uses.<br />
<br />
== Management ==<br />
<br />
=== machinectl ===<br />
<br />
{{Note|The ''machinectl'' tool requires [[systemd]] and {{Pkg|dbus}} to be installed in the container. See [https://github.com/systemd/systemd/issues/685] for detailed discussion.}}<br />
<br />
Managing your containers is essentially done with the {{ic|machinectl}} command. See {{ic|machinectl(1)}} for details.<br />
<br />
Examples:<br />
<br />
Spawn a new shell inside a running container: <br />
<br />
$ machinectl login ''MyContainer''<br />
<br />
Show detailed information about a container: <br />
<br />
$ machinectl status ''MyContainer''<br />
<br />
Reboot a container:<br />
<br />
$ machinectl reboot ''MyContainer''<br />
<br />
Poweroff a container:<br />
<br />
$ machinectl poweroff ''MyContainer''<br />
<br />
{{Tip|Poweroff and reboot operations can be performed from within a container session using the ''systemctl'' {{ic|poweroff}} or {{ic|reboot}} commands.}}<br />
<br />
Download an image:<br />
<br />
# machinectl pull-tar ''URL'' ''name''<br />
<br />
=== systemd toolchain ===<br />
<br />
Much of the core systemd toolchain has been updated to work with containers. Tools that do usually provide a {{ic|1=-M, --machine=}} option which will take a container name as argument.<br />
<br />
Examples:<br />
<br />
See journal logs for a particular machine:<br />
<br />
$ journalctl -M ''MyContainer''<br />
<br />
Show control group contents:<br />
<br />
$ systemd-cgls -M ''MyContainer''<br />
<br />
See startup time of container:<br />
<br />
$ systemd-analyze -M ''MyContainer''<br />
<br />
For an overview of resource usage:<br />
<br />
$ systemd-cgtop<br />
<br />
== Tips and tricks ==<br />
<br />
=== Use an X environment ===<br />
<br />
See [[Xhost]] and [[Change root#Run graphical applications from chroot]].<br />
<br />
You will need to set the {{ic|DISPLAY}} environment variable inside your container session to connect to the external X server.<br />
<br />
X stores some required files in the {{ic|/tmp}} directory. In order for your container to display anything, it needs access to those files. To do so, append the {{ic|--bind<nowiki>=</nowiki>/tmp/.X11-unix:/tmp/.X11-unix}} option when starting the container.<br />
<br />
=== Run Firefox ===<br />
<br />
See [[Firefox tweaks#Run Firefox inside an nspawn container|Firefox tweaks]].<br />
<br />
=== Access host filesystem ===<br />
<br />
See {{ic|--bind}} and {{ic|--bind-ro}} in {{ic|man systemd-nspawn}}.<br />
<br />
If both the host and the container are Arch Linux, then one could, for example, share the pacman cache:<br />
<br />
# systemd-nspawn --bind=/var/cache/pacman/pkg<br />
<br />
=== Configure networking ===<br />
<br />
{{Style}}<br />
<br />
Note the canonical [[systemd-networkd]] host and container .network files are from https://github.com/systemd/systemd/tree/master/network<br />
<br />
You need to set up the container .network manually after pacstrapping and {{ic|# systemctl enable [[systemd-networkd]]}} (your dhcp client) with systemd-nspawn's -n switch to ensure a virtual Ethernet link is setup. Don't forget to set up DNS, e.g. by either 1) [[textedit|edit]] your container's {{ic|/etc/resolv.conf}} by adding your DNS server's IP address, or have 2) [[systemd-resolved]] manage {{ic|/etc/resolv.conf}} for you.<br />
<br />
See [[systemd-networkd#Usage with containers]] for more complex examples.<br />
<br />
==== nsswitch.conf ====<br />
<br />
{{Merge|systemd-networkd}}<br />
<br />
To make it easier to connect to a container from the host, you can enable local DNS resolution for container names. In {{ic|/etc/nsswitch.conf}}, add {{ic|mymachines}} to the {{ic|hosts:}} section, e.g.<br />
<br />
hosts: files mymachines dns myhostname<br />
<br />
Then, any DNS lookup for hostname {{ic|foo}} on the host will first consult {{ic|/etc/hosts}}, then the names of local containers, then upstream DNS etc.<br />
<br />
==== Use host networking ====<br />
<br />
To disable private networking used by containers started with {{ic|machinectl start MyContainer}}, [[edit]] the configuration of {{ic|systemd-nspawn@.service}} with {{ic|systemctl edit systemd-nspawn@.service}} and set the {{ic|1=ExecStart=}} option without the {{ic|--network-veth}} parameter unlike the original service:<br />
<br />
{{hc|/etc/systemd/system/systemd-nspawn@.service.d/override.conf|<nowiki><br />
[Service]<br />
ExecStart=<br />
ExecStart=/usr/bin/systemd-nspawn --quiet --keep-unit --boot --link-journal=try-guest --machine=%I<br />
</nowiki>}}<br />
<br />
The newly started containers will use the hosts networking.<br />
<br />
==== Virtual Ethernet interfaces ====<br />
<br />
If a container is started with {{ic|systemd-nspawn ... -n}}, systemd will automatically create one virtual Ethernet interface on the host, and one in the container, connected by a virtual Ethernet cable.<br />
<br />
If the name of the container is {{ic|foo}}, the name of the virtual Ethernet interface on the host is {{ic|ve-foo}}. The name of the virtual Ethernet interface in the container is always {{ic|host0}}.<br />
<br />
When examining the interfaces with {{ic|ip link}}, interface names will be shown with a suffix, such as {{ic|ve-foo@if2}} and {{ic|host0@if9}}. The {{ic|@ifN}} is not actually part of the name of the interface; instead, {{ic|ip link}} appends this information to indicate which "slot" the virtual Ethernet cable connects to on the other end.<br />
<br />
For example, a host virtual Ethernet interface shown as {{ic|ve-foo@if2}} will connect to container {{ic|foo}}, and inside the container to the second network interface -- the one shown with index 2 when running {{ic|ip link}} inside the container. Similarly, in the container, the interface named {{ic|host0@if9}} will connect to the 9th slot on the host.<br />
<br />
=== Run on a non-systemd system ===<br />
<br />
See [[Init#systemd-nspawn]].<br />
<br />
=== Specify per-container settings ===<br />
<br />
To specify per-container settings and not overrides for all (e.g. bind a directory to only one container)[https://github.com/systemd/systemd/issues/3442#issuecomment-223837408], the ".nspawn" file definition can be used [https://www.freedesktop.org/software/systemd/man/systemd.nspawn.html#]<br />
<br />
man systemd.nspawn<br />
<br />
=== Use Btrfs subvolume as container root ===<br />
<br />
To use a btrfs subvolume as a template for the container's root, use the {{ic|--template}} flag. This takes a snapshot of the subvolume and populates the root directory for the container with it.<br />
<br />
{{Note|If the template path specified is not the root of a subvolume, the '''entire''' tree is copied. This will be very time consuming.}}<br />
<br />
For example, to use a snapshot located at {{ic|/.snapshots/403/snapshot}}:<br />
<br />
# systemd-nspawn --template=/.snapshots/403/snapshots -b -D ''my-container''<br />
<br />
where {{ic|''my-container''}} is the name of the directory that will be created for the container. After powering off, the newly created subvolume is retained.<br />
<br />
=== Use temporary Btrfs snapshot of container ===<br />
<br />
One can use the {{ic|--ephemeral}} or {{ic|-x}} flag to create a temporary btrfs snapshot of the container and use it as the container root. Any changes made while booted in the container will be lost. For example:<br />
<br />
# systemd-nspawn -D ''my-container'' -xb<br />
<br />
where ''my-container'' is the directory of an '''existing''' container or system. For example, if {{ic|/}} is a btrfs subvolume one could create an ephemeral container of the currently running host system by doing:<br />
<br />
# systemd-nspawn -D / -xb <br />
<br />
After powering off the container, the btrfs subvolume that was created is immediately removed.<br />
<br />
== Troubleshooting ==<br />
<br />
=== root login fails ===<br />
<br />
If you get the following error when you try to login (i.e. using {{ic|machinectl login <name>}}):<br />
<br />
arch-nspawn login: root<br />
Login incorrect<br />
<br />
And {{ic|journalctl}} shows:<br />
<br />
pam_securetty(login:auth): access denied: tty 'pts/0' is not secure !<br />
<br />
Add {{ic|pts/0}} to the list of terminal names in {{ic|/etc/securetty}} on the '''container''' filesystem, see [http://unix.stackexchange.com/questions/41840/effect-of-entries-in-etc-securetty/41939#41939]. You can also opt to delete {{ic|/etc/securetty}} on the '''container''' to allow root to login to any tty, see [https://github.com/systemd/systemd/issues/852].<br />
<br />
=== Unable to upgrade some packages on the container ===<br />
<br />
It can sometimes be impossible to upgrade some packages on the container, {{Pkg|filesystem}} being a perfect example. The issue is due to {{ic|/sys}} being mounted as Read Only. The workaround is to remount the directory in Read Write when running {{ic|mount -o remount,rw -t sysfs sysfs /sys}}, do the upgrade then reboot the container.<br />
<br />
== See also ==<br />
<br />
* [http://www.freedesktop.org/software/systemd/man/machinectl.html machinectl man page]<br />
* [http://www.freedesktop.org/software/systemd/man/systemd-nspawn.html systemd-nspawn man page]<br />
* [http://lwn.net/Articles/572957/ Creating containers with systemd-nspawn]<br />
* [https://www.youtube.com/results?search_query=systemd-nspawn&aq=f Presentation by Lennart Pottering on systemd-nspawn]<br />
* [http://dabase.com/e/12009/ Running Firefox in a systemd-nspawn container]</div>Phireskyhttps://wiki.archlinux.org/index.php?title=Talk:PCI_passthrough_via_OVMF&diff=417836Talk:PCI passthrough via OVMF2016-01-29T13:21:00Z<p>Phiresky: </p>
<hr />
<div>== Supported Hardware Table ==<br />
<br />
I think it would be a good idea to add a table of working / not working hardware like this:<br />
<br />
{| class="wikitable"<br />
! Motherboard<br />
! CPU<br />
! Host GPU<br />
! Client GPU<br />
! uname -r<br />
! ACS Override needed<br />
! Notes<br />
|-<br />
| [https://www.asus.com/Motherboards/Z170-A/ ASUS Z170-A]<br />
| Intel i7-6700K<br />
| Integrated<br />
| [https://www.asus.com/Graphics-Cards/STRIXGTX980TIDC3OC6GD5GAMING/ ASUS Strix GTX 980 Ti]<br />
| 4.3.3-3-ARCH<br />
| no<br />
| No problems<br />
|-<br />
| <br />
| <br />
| <br />
| <br />
| <br />
| <br />
| <br />
|-<br />
| <br />
| <br />
| <br />
| <br />
| <br />
| <br />
| <br />
|}<br />
<br />
--[[User:Phiresky|Phiresky]] ([[User talk:Phiresky|talk]]) 12:28, 29 January 2016 (UTC)<br />
<br />
: Well, there's: https://docs.google.com/spreadsheets/d/1LnGpTrXalwGVNy0PWJDURhyxa3sgqkGXmvNCIvIMenk/edit#gid=2 and it has (if I read it properly) 208 unique motherboards. I'm not sure if Wiki can handle such a long list. [[User:Annisar|Annisar]] ([[User talk:Annisar|talk]])<br />
<br />
: Huh, looks like I missed that. Yeah, that list is probably too long. Kind of unclean / hard to read and can't edit it though. [[User:Phiresky|Phiresky]] ([[User talk:Phiresky|talk]]) 13:20, 29 January 2016 (UTC)<br />
<br />
== Missing links ==<br />
<br />
"cp -R ./usr/share /usr/share" creates "/usr/share/share" with the contents of .usr/share in it. I don't think you can do the copy with one command.<br />
Do you have plans to update the missing links? I'm referring to this paragraph: "This will be the BIOS that the VM will use. Non-UEFI users may need to use i440fx without OVMF, and the i915 vga arbiter patch for Intel graphics as host, see this forum thread. For users that do have a UEFI compatible motherboard but a UEFI incompatible graphics card, look at this post." [[User:Wmarler|Wmarler]] ([[User talk:Wmarler|talk]]) 05:42, 3 April 2015 (UTC)<br />
<br />
== Qemu permissions ==<br />
<br />
I didn't need to change the qemu permissions to make passing through vfio devices work. Here's what I have:<br />
<br />
{{hc|<nowiki>/etc/libvirt/qemu.conf</nowiki>|<nowiki>...<br />
#user = "root"<br />
group="78"<br />
#clear_emulator_capabilities = 1<br />
...<br />
#cgroup_device_acl = [<br />
# "/dev/null", "/dev/full", "/dev/zero",<br />
# "/dev/random", "/dev/urandom",<br />
# "/dev/ptmx", "/dev/kvm", "/dev/kqemu",<br />
# "/dev/rtc","/dev/hpet", "/dev/vfio/vfio"<br />
#]<br />
...</nowiki>}}<br />
<br />
[[User:M01|M01]] ([[User talk:M01|talk]])<br />
<br />
: Works for me too. Seems that <code>qemu</code> drops the root permissions only once it has all needed descriptors opened. [[User:Hiliev|Hiliev]] ([[User talk:Hiliev|talk]])<br />
<br />
== Article Specificity ==<br />
<br />
I see that this article's title is focused on OVMF, but it seems to allow an alternate method as well.<br />
<br />
Considering that PCI Passthrough would be a user's goal and the "via OVMF" is only one of several details involved in the "How", it makes sense to me to move the article to "PCI Passthrough". Am I by-chance just not seeing the actual page-title of the main article on this subject?<br />
<br />
Thoughts?<br />
<br />
(Sidenote, adding info about the [[linux-vfio]] packages in AUR to the Installation section.)<br />
<br />
[[User:Physicist1616|Physicist1616]] ([[User talk:Physicist1616|talk]]) 17:09, 5 October 2015 (UTC)</div>Phireskyhttps://wiki.archlinux.org/index.php?title=Talk:PCI_passthrough_via_OVMF&diff=417817Talk:PCI passthrough via OVMF2016-01-29T12:43:35Z<p>Phiresky: add kernel to table</p>
<hr />
<div>== Supported Hardware Table ==<br />
<br />
I think it would be a good idea to add a table of working / not working hardware like this:<br />
<br />
{| class="wikitable"<br />
! Motherboard<br />
! CPU<br />
! Host GPU<br />
! Client GPU<br />
! uname -r<br />
! ACS Override needed<br />
! Notes<br />
|-<br />
| [https://www.asus.com/Motherboards/Z170-A/ ASUS Z170-A]<br />
| Intel i7-6700K<br />
| Integrated<br />
| [https://www.asus.com/Graphics-Cards/STRIXGTX980TIDC3OC6GD5GAMING/ ASUS Strix GTX 980 Ti]<br />
| 4.3.3-3-ARCH<br />
| no<br />
| No problems<br />
|-<br />
| <br />
| <br />
| <br />
| <br />
| <br />
| <br />
| <br />
|-<br />
| <br />
| <br />
| <br />
| <br />
| <br />
| <br />
| <br />
|}<br />
<br />
--[[User:Phiresky|Phiresky]] ([[User talk:Phiresky|talk]]) 12:28, 29 January 2016 (UTC)<br />
<br />
== Missing links ==<br />
<br />
"cp -R ./usr/share /usr/share" creates "/usr/share/share" with the contents of .usr/share in it. I don't think you can do the copy with one command.<br />
Do you have plans to update the missing links? I'm referring to this paragraph: "This will be the BIOS that the VM will use. Non-UEFI users may need to use i440fx without OVMF, and the i915 vga arbiter patch for Intel graphics as host, see this forum thread. For users that do have a UEFI compatible motherboard but a UEFI incompatible graphics card, look at this post." [[User:Wmarler|Wmarler]] ([[User talk:Wmarler|talk]]) 05:42, 3 April 2015 (UTC)<br />
<br />
== Qemu permissions ==<br />
<br />
I didn't need to change the qemu permissions to make passing through vfio devices work. Here's what I have:<br />
<br />
{{hc|<nowiki>/etc/libvirt/qemu.conf</nowiki>|<nowiki>...<br />
#user = "root"<br />
group="78"<br />
#clear_emulator_capabilities = 1<br />
...<br />
#cgroup_device_acl = [<br />
# "/dev/null", "/dev/full", "/dev/zero",<br />
# "/dev/random", "/dev/urandom",<br />
# "/dev/ptmx", "/dev/kvm", "/dev/kqemu",<br />
# "/dev/rtc","/dev/hpet", "/dev/vfio/vfio"<br />
#]<br />
...</nowiki>}}<br />
<br />
[[User:M01|M01]] ([[User talk:M01|talk]])<br />
<br />
: Works for me too. Seems that <code>qemu</code> drops the root permissions only once it has all needed descriptors opened. [[User:Hiliev|Hiliev]] ([[User talk:Hiliev|talk]])<br />
<br />
== Article Specificity ==<br />
<br />
I see that this article's title is focused on OVMF, but it seems to allow an alternate method as well.<br />
<br />
Considering that PCI Passthrough would be a user's goal and the "via OVMF" is only one of several details involved in the "How", it makes sense to me to move the article to "PCI Passthrough". Am I by-chance just not seeing the actual page-title of the main article on this subject?<br />
<br />
Thoughts?<br />
<br />
(Sidenote, adding info about the [[linux-vfio]] packages in AUR to the Installation section.)<br />
<br />
[[User:Physicist1616|Physicist1616]] ([[User talk:Physicist1616|talk]]) 17:09, 5 October 2015 (UTC)</div>Phireskyhttps://wiki.archlinux.org/index.php?title=Talk:PCI_passthrough_via_OVMF&diff=417816Talk:PCI passthrough via OVMF2016-01-29T12:30:12Z<p>Phiresky: Supported Hardware Table suggestion</p>
<hr />
<div>== Supported Hardware Table ==<br />
<br />
I think it would be a good idea to add a table of working / not working hardware like this:<br />
<br />
{| class="wikitable"<br />
! style="font-weight: bold;" | Motherboard<br />
! style="font-weight: bold;" | CPU<br />
! Host GPU<br />
! style="font-weight: bold;" | Client GPU<br />
! style="font-weight: bold;" | ACS Override needed<br />
! style="font-weight: bold;" | Notes<br />
|-<br />
| [https://www.asus.com/Motherboards/Z170-A/ ASUS Z170-A]<br />
| Intel i7-6700K<br />
| Integrated<br />
| [https://www.asus.com/Graphics-Cards/STRIXGTX980TIDC3OC6GD5GAMING/ ASUS Strix GTX 980 Ti]<br />
| no<br />
| No problems<br />
|-<br />
| <br />
| <br />
| <br />
| <br />
| <br />
| <br />
|-<br />
| <br />
| <br />
| <br />
| <br />
| <br />
| <br />
|}<br />
<br />
--[[User:Phiresky|Phiresky]] ([[User talk:Phiresky|talk]]) 12:28, 29 January 2016 (UTC)<br />
<br />
== Missing links ==<br />
<br />
"cp -R ./usr/share /usr/share" creates "/usr/share/share" with the contents of .usr/share in it. I don't think you can do the copy with one command.<br />
Do you have plans to update the missing links? I'm referring to this paragraph: "This will be the BIOS that the VM will use. Non-UEFI users may need to use i440fx without OVMF, and the i915 vga arbiter patch for Intel graphics as host, see this forum thread. For users that do have a UEFI compatible motherboard but a UEFI incompatible graphics card, look at this post." [[User:Wmarler|Wmarler]] ([[User talk:Wmarler|talk]]) 05:42, 3 April 2015 (UTC)<br />
<br />
== Qemu permissions ==<br />
<br />
I didn't need to change the qemu permissions to make passing through vfio devices work. Here's what I have:<br />
<br />
{{hc|<nowiki>/etc/libvirt/qemu.conf</nowiki>|<nowiki>...<br />
#user = "root"<br />
group="78"<br />
#clear_emulator_capabilities = 1<br />
...<br />
#cgroup_device_acl = [<br />
# "/dev/null", "/dev/full", "/dev/zero",<br />
# "/dev/random", "/dev/urandom",<br />
# "/dev/ptmx", "/dev/kvm", "/dev/kqemu",<br />
# "/dev/rtc","/dev/hpet", "/dev/vfio/vfio"<br />
#]<br />
...</nowiki>}}<br />
<br />
[[User:M01|M01]] ([[User talk:M01|talk]])<br />
<br />
: Works for me too. Seems that <code>qemu</code> drops the root permissions only once it has all needed descriptors opened. [[User:Hiliev|Hiliev]] ([[User talk:Hiliev|talk]])<br />
<br />
== Article Specificity ==<br />
<br />
I see that this article's title is focused on OVMF, but it seems to allow an alternate method as well.<br />
<br />
Considering that PCI Passthrough would be a user's goal and the "via OVMF" is only one of several details involved in the "How", it makes sense to me to move the article to "PCI Passthrough". Am I by-chance just not seeing the actual page-title of the main article on this subject?<br />
<br />
Thoughts?<br />
<br />
(Sidenote, adding info about the [[linux-vfio]] packages in AUR to the Installation section.)<br />
<br />
[[User:Physicist1616|Physicist1616]] ([[User talk:Physicist1616|talk]]) 17:09, 5 October 2015 (UTC)</div>Phireskyhttps://wiki.archlinux.org/index.php?title=Wake-on-LAN&diff=398109Wake-on-LAN2015-09-03T15:55:04Z<p>Phiresky: readd wol-systemd aur package</p>
<hr />
<div>[[Category:Networking]]<br />
[[ja:Wake-on-LAN]]<br />
Wake-on-LAN (WOL) is a feature to switch on a computer via a network connection (be it the internet or intranet).<br />
<br />
== Hardware settings ==<br />
<br />
The target computer's motherboard and [[wikipedia:NIC|NIC]] have to support Wake-on-LAN. Wireless cards do not support Wake-on-LAN, so the target computer has to be physically connected (i.e. by cable) to router or the source computer.<br />
<br />
The Wake-on-LAN feature also has to be enabled in the PC's BIOS. Different motherboard manufactures use slightly different language for this feature. Look for terminology such as "PCI Power up", "Allow PCI wake up event" or "Boot from PCI/PCI-E".<br />
<br />
It is known that some motherboards are affected by a nasty bug that causes reboots rather than shutdowns under certain situations (see [https://bbs.archlinux.org/viewtopic.php?id=173648 this] thread for example). To prevent this bug from surfacing, it is recommended to do the following on the target machine:<br />
# Disable all references to "xHCI" as it pertains to USB settings.<br />
# Disable EuP 2013 if it is implicitly an option.<br />
# Optionally enable WOL for keyboard actions.<br />
<br />
{{Note|There are mixed opinions as to the value of #3 above and it may be motherboard dependent.}}<br />
<br />
== Software configuration ==<br />
<br />
Depending on the hardware, the network drivers may have WOL switched off by default. To query this status or to change the settings, install {{Pkg|ethtool}}.<br />
<br />
Query the network device via this command:<br />
<br />
{{hc|<nowiki># ethtool net0 | grep Wake-on</nowiki>|<nowiki><br />
Supports Wake-on: pumbag<br />
Wake-on: d<br />
</nowiki>}}<br />
<br />
The values define what activity to wake on: {{ic|d}} (disabled), {{ic|p}} (PHY activity), {{ic|u}} (unicast activity), {{ic|m}} (multicast activity), {{ic|b}} (broadcast activity), {{ic|a}} (ARP activity), and {{ic|g}} (magic packet activity). The value of {{ic|g}} is required for WOL to work.<br />
<br />
To enable the WOL feature in the driver:<br />
<br />
# ethtool -s net0 wol g<br />
<br />
This command does not last beyond the next reboot, so it must be repeated via some mechanism. Common solutions are listed in the following subsections.<br />
<br />
=== With netctl ===<br />
<br />
If using netctl, one can make this setting persistent by adding the following the netctl profile:<br />
<br />
{{hc|/etc/netctl/''profile''|2=<br />
ExecUpPost='/usr/bin/ethtool -s net0 wol g'<br />
}}<br />
<br />
=== With systemd ===<br />
<br />
If [[systemd-networkd]] is used to setup machine's network then it is easy to enable Wake-On-Lan using {{ic|systemd.link}} configuration. Add {{ic|WakeOnLan}} option to the network link file:<br />
<br />
{{hc|/etc/systemd/network/wired.link|2=<br />
[Link]<br />
WakeOnLan=magic<br />
}}<br />
<br />
See {{ic|man systemd.link}} for more information.<br />
<br />
=== With standalone systemd service ===<br />
<br />
This is an equivalent of previous {{ic|systemd.link}} option, but uses a standalone systemd service.<br />
<br />
{{hc|/etc/systemd/system/wol@.service|2=<br />
[Unit]<br />
Description=Wake-on-LAN for %i<br />
Requires=network.target<br />
After=network.target<br />
<br />
[Service]<br />
ExecStart=/usr/bin/ethtool -s %i wol g<br />
Type=oneshot<br />
<br />
[Install]<br />
WantedBy=multi-user.target<br />
}}<br />
<br />
Alternatively install the {{AUR|wol-systemd}} package.<br />
<br />
Then activate this new service by [[starting]] {{ic|wol@''interface''.service}}.<br />
<br />
=== With udev ===<br />
<br />
udev is capable of running any command as soon as a device is visible. Here, udev will turn on WOL. Put the following in /etc/udev/rules.d/50-wol.rules, replacing N with the number for the target interface:<br />
<br />
ACTION=="add", SUBSYSTEM=="net", KERNEL=="netN", RUN+="/usr/bin/ethtool -s %k wol g"<br />
<br />
This tells udev to run "/usr/bin/ethtool -s netN wol g" as soon as the device netN exists. To turn on wake on lan for all new devices, replace the N with a '*':<br />
<br />
ACTION=="add", SUBSYSTEM=="net", KERNEL=="net*", RUN+="/usr/bin/ethtool -s %k wol g"<br />
<br />
=== With cron ===<br />
<br />
A command can be run each time the computer is (re)booted using "@reboot" in a crontab. First, make sure [[Cron#Installation|cron]] is enabled, and then [[Cron#Basic_commands|edit a crontab]] for the root user that contains the following line:<br />
<br />
@reboot /usr/bin/ethtool -s [net-device] wol g<br />
<br />
== Trigger a wake up ==<br />
<br />
To trigger WOL on a target machine, its MAC address and external or internal IP should be known.<br />
<br />
To obtain the internal IP address and MAC address of the target computer, execute the following command:<br />
<br />
{{hc|$ ip addr|<br />
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default<br />
link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00<br />
inet 127.0.0.1/8 scope host lo<br />
valid_lft forever preferred_lft forever<br />
inet6 ::1/128 scope host<br />
valid_lft forever preferred_lft forever<br />
2: enp1s0: <BROADCAST,MULTICAST,PROMISC,UP,LOWER_UP> mtu 1500 qdisc fq_codel master br0 state UP group default qlen 1000<br />
link/ether '''48:05:ca:09:0e:6a''' brd ff:ff:ff:ff:ff:ff<br />
inet '''192.168.1.20/24''' brd 192.168.1.255 scope global br0<br />
valid_lft forever preferred_lft forever<br />
inet6 fe80::6a05:caff:fe09:e6a/64 scope link<br />
valid_lft forever preferred_lft forever<br />
}}<br />
<br />
Here the internal IP address is {{ic|192.168.1.20}} and the MAC address is {{ic|48:05:ca:09:0e:6a}}.<br />
<br />
One program able to send magic packets for WOL is {{Pkg|wol}}.<br />
<br />
=== On the same LAN ===<br />
<br />
If you are connected directly to another computer through a network cable, or the traffic within a LAN is not firewalled, then using Wake-on-LAN should be very easy since there is no need to worry about port redirects.<br />
<br />
In the simplest case the default broadcast address {{ic|255.255.255.255}} is used:<br />
<br />
$ wol ''target_MAC_address''<br />
<br />
To broadcast the magic packet only to a specific subnet or host, use the {{ic|-i}} switch:<br />
<br />
$ wol -i ''target_IP'' ''target_MAC_address''<br />
<br />
{{Tip|If you intend to continue using Wake-on-LAN, it is recommended to assign a static IP address to the target computer.}}<br />
<br />
=== With port forwarding ===<br />
<br />
When the source and target computers are separated by a router, Wake-on-LAN can be used via [[wikipedia:Port forwarding|port forwarding]]. The router must be instructed to forward any signal heading for a specific port to the internal IP of the target PC. See for example [[Firewalls]] for configuration details.<br />
<br />
To trigger the wakeup:<br />
<br />
$ wol -p ''forwarded_port'' -i ''router_IP'' ''target_MAC_address''<br />
<br />
In case of multiple computers behind the router, it is recommended to assign a different port forward to each target IP.<br />
<br />
=== Across the internet ===<br />
<br />
{{Style|Emphasize the difference from [[#With port forwarding]] (if any).}}<br />
<br />
The syntax needed in this case:<br />
<br />
$ wol -p ''target_port'' -i ''target_IP_or_hostname'' ''target_MAC_address''<br />
<br />
*Assuming that you know the external IP of the target machine, and that the [[#With port forwarding|router ports]] on both sides have been forwarding correctly, then this should be exactly as the syntax states.<br />
<br />
Usually it is necessary to forward your wol port (typically UDP 9) to the broadcast address on your network, not to a particular IP. Most routers do not allow you to forward to broadcast, however if you can get shell access to your router (through telnet, ssh, serial cable, etc) you can implement this workaround:<br />
$ ip neighbor add 192.168.1.254 lladdr FF:FF:FF:FF:FF:FF dev net0<br />
<br />
(The above command assumes your network is 192.168.1.0/24 and use net0 as network interface). Now, forward UDP port 9 to 192.168.1.254. This has worked for me on a Linksys WRT54G running Tomato, and on the Verizon FIOS ActionTec router.<br />
<br />
For notes on how to do it on DD-WRT routers, see [http://www.dd-wrt.com/wiki/index.php/WOL#Remote_Wake_On_LAN_via_Port_Forwarding this tutorial].<br />
<br />
{{Expansion|Request for OpenWRT instructions.}}<br />
<br />
== Miscellaneous ==<br />
<br />
=== Battery draining problem ===<br />
Some laptops have a battery draining problem after shutdown [http://ubuntuforums.org/archive/index.php/t-1729782.html]. This might be caused by enabled WOL. To solve this problem, disable it by using ethtool as mentioned above.<br />
<br />
# ethtool -s net0 wol d<br />
<br />
=== Example WOL script ===<br />
Here is a script to automate WOL to several different machines:<br />
<br />
{{bc|<nowiki><br />
#!/bin/bash<br />
<br />
# definition of MAC addresses<br />
monster=01:12:46:82:ab:4f<br />
chronic=00:3a:53:21:bc:30<br />
powerless=1a:32:41:02:29:92<br />
ghost=01:1a:d2:56:6b:e6<br />
<br />
while [ "$input1" != quit ]; do<br />
echo "Which PC to wake?"<br />
echo "p) powerless"<br />
echo "m) monster"<br />
echo "c) chronic"<br />
echo "g) ghost"<br />
echo "b) wake monster, wait 40sec, then wake chronic"<br />
echo "q) quit and take no action"<br />
read input1<br />
if [ $input1 == p ]; then<br />
/usr/bin/wol $powerless<br />
exit 1<br />
fi<br />
<br />
if [ $input1 == m ]; then<br />
/usr/bin/wol $monster<br />
exit 1<br />
fi<br />
<br />
if [ $input1 == c ]; then<br />
/usr/bin/wol $chronic<br />
exit 1<br />
fi<br />
<br />
# this line requires an IP address in /etc/hosts for ghost<br />
# and should use wol over the internet provided that port 9<br />
# is forwarded to ghost on ghost's router<br />
<br />
if [ $input1 == g ]; then<br />
/usr/bin/wol -v -h -p 9 ghost $ghost<br />
exit 1<br />
fi<br />
<br />
if [ $input1 == b ]; then<br />
/usr/bin/wol $monster<br />
echo "monster sent, now waiting for 40sec then waking chronic"<br />
sleep 40<br />
/usr/bin/wol $chronic<br />
exit 1<br />
fi<br />
<br />
if [ $input1 == Q ] || [ $input1 == q ]; then<br />
echo "later!"<br />
exit 1<br />
fi<br />
<br />
done<br />
echo "this is the (quit) end!! c-ya!"<br />
</nowiki>}}<br />
<br />
== Troubleshooting ==<br />
<br />
=== Realtek ===<br />
<br />
Users with Realtek 8168 8169 8101 8111(C) based NICs (cards / and on-board) may notice a problem where the NIC seems to be disabled on boot and has no Link light. See [[Network configuration#Realtek no link / WOL problem]]<br />
<br />
== See also ==<br />
<br />
* [http://www.depicus.com/wake-on-lan/woli.aspx Wake-On-Lan]</div>Phireskyhttps://wiki.archlinux.org/index.php?title=Advanced_Format&diff=389262Advanced Format2015-07-30T12:16:39Z<p>Phiresky: fix spelling mistake in product number</p>
<hr />
<div>[[Category:Storage]]<br />
{{Poor writing|Partially overlaps with [[Partitioning#Partition alignment]], these pages are not very well interlinked.}}<br />
<br />
==Introduction==<br />
The [[wikipedia:Advanced Format|Advanced Format]] feature reduces overhead by using 4 kilobyte sectors instead of the traditional 512 byte sectors. The old format gave a format efficiency of 87%. Advanced Format results in a format efficiency of 96% which increases space by up to 11%. The 4k sector is slated to become the next standard for HDDs by 2014.<br />
<br />
===More Detailed Explanation===<br />
The main idea behind using 4096-byte sectors is to increase the bit density on each track by reducing the number of gaps which hold Sync/DAM and ECC (Error Correction Code) information between data sectors. For eight 512-byte sectors, the track also holds eight sector gaps.<br />
<br />
By having one single sector of size 4096-byte (8 x 512-byte), the track holds only 1 sector gap for each data sector thus reducing an overhead for a need to support multiple Sync/DAM and ECC blocks and at the same time increasing bit density.<br />
<br />
Linux partitioning tools by default start each partition on sector 63 which leads to a bad performance in HDDs that use this 4K sector size due to misalignment to 4K sector from the beginning of the track.<br />
<br />
===External Links===<br />
*[http://www.anandtech.com/Show/Index/2888?cPage=2&all=False&sort=0&page=1 Western Digital’s Advanced Format: The 4K Sector Transition Begins]<br />
*[http://www.wdc.com/wdproducts/library/WhitePapers/ENG/2579-771430.pdf White paper entitled "Advanced Format Technology."]<br />
*Failure to align one's HDD results in poor read/write performance. See [http://www.linuxconfig.org/linux-wd-ears-advanced-format this article] for specific examples.<br />
<br />
==Current HDD Models that Employ a 4k Sectors==<br />
As of June 2011, there are a limited number of HDDs that support "Advanced Format" or 4k sectors as shown below.<br />
<br />
All drives in this list have a physical sector size of 4096 bytes, but not all drives correctly report this to the OS. The actual value reported (via new fields in the ATA-8 spec) is shown in the table as the physical reported sector size. As this is the value partitioning tools use for alignment, it is important that it should be 4096 to avoid misalignment issues.<br />
<br />
The logical sector size is the sector size used for data transfer. This value multiplied by the number of LBA sectors on the disk gives the disk capacity. Thus a disk with 4096 byte logical sectors will have a lower maximum LBA for the same capacity compared to a drive with 512 byte sectors. Drives with 512 byte logical sectors offer better compatibility with legacy operating systems (roughly those released before 2009) however drives with 4096 byte logical sectors may offer marginally better performance (e.g. more read/write requests may fit into the NCQ buffer.)<br />
<br />
{|class="wikitable"<br />
!rowspan=2| Manufacturer !!rowspan=2| Model !!rowspan=2| Capacity !!colspan=2| Reported sector size (bytes)<br />
|-<br />
! Logical !! Physical<br />
|-<br />
|colspan=5| '''3.5"'''<br />
|-<br />
| Samsung || HD204UI || 2.0 TB || 512 || 512<br />
|-<br />
| Seagate || ST3500413AS || 500.0 GB || 512 || 512<br />
|-<br />
| Seagate || ST500DM002 || 500.0 GB || 512 || 4096<br />
|-<br />
| Seagate || ST1000DL002 || 1.0 TB || 512 || 4096<br />
|-<br />
| Seagate || ST1000DM003 || 1.0 TB || 512 || 4096<br />
|-<br />
| Seagate || ST2000DL003 || 2.0 TB || 512 || 512<br />
|-<br />
| Seagate || ST2000DM001 || 2.0 TB || 512 || 4096<br />
|-<br />
| Seagate || ST3000DM001 || 3.0 TB || 512 || 4096<br />
|-<br />
| Seagate || ST4000VN000 || 4.0 TB || 512 || 4096<br />
|-<br />
| Western Digital || WD3000F9YZ || 2.0 TB || 512 || 4096<br />
|-<br />
| Western Digital || WD30EZRX || 3.0 TB || 512 || 4096<br />
|-<br />
| Western Digital || WD20EZRX || 2.0 TB || 512 || 4096<br />
|-<br />
| Western Digital || WD30EZRSDTL || 3.0 TB<br />
|-<br />
| Western Digital || WD25EZRSDTL || 2.5 TB<br />
|-<br />
| Western Digital || WD20EARX || 2.0 TB || 512 || 4096<br />
|-<br />
| Western Digital || WD20EFRX || 2.0 TB || 512 || 4096<br />
|-<br />
| Western Digital || WD30EFRX || 3.0 TB || 512 || 4096<br />
|-<br />
| Western Digital || WD40EFRX || 4.0 TB || 512 || 4096<br />
|-<br />
| Western Digital || WD60EFRX || 6.0 TB || 512 || 4096<br />
|-<br />
| Western Digital || WD10EARS || 1.0 TB<br />
|-<br />
| Western Digital || WD15EARS || 1.5 TB || 512 || [http://excess.org/article/2010/11/wd-hdd-lying-about-4k-sectors/ 4096]<br />
|-<br />
| Western Digital || WD20EARS || 2.0 TB || 512 || [http://community.wd.com/t5/Desktop-Mobile-Drives/Physical-Sector-Size-different-between-WD20EARS-00MVWB0-and/td-p/218226 4096] or [http://community.wdc.com/t5/Desktop/4k-sector-drive-reporting-512-byte-sectors-to-OS-why/td-p/205060 512]<br />
|-<br />
| Western Digital || WD10EURS || 1.0 TB<br />
|-<br />
| Western Digital || WD8000AARS || 800.0 GB<br />
|-<br />
| Western Digital || WD6400AARS || 640.0 GB<br />
|-<br />
|colspan=5| '''2.5"'''<br />
|-<br />
| Samsung || ST1000LM024|| 1.0 TB || 512 || 4096<br />
|-<br />
| Samsung || ST2000LM003|| 2.0 TB || 512 || 4096<br />
|-<br />
| Seagate || ST320LT007 || 320 GB || 512 || 4096<br />
|-<br />
| Seagate || ST9750420AS || 750 GB || 512 || 4096<br />
|-<br />
| Western Digital || WD10JPVT || 1.0 TB || 512 || 4096<br />
|-<br />
| Western Digital || WD10TPVT || 1.0 TB<br />
|-<br />
| Western Digital || WD7500BPVT || 750.0 GB<br />
|-<br />
| Western Digital || WD7500KPVT || 750.0 GB<br />
|-<br />
| Western Digital || WD6400BPVT || 640.0 GB<br />
|-<br />
| Western Digital || WD5000BPVT || 500.0 GB<br />
|-<br />
| Western Digital || WD3200BPVT || 320.0 GB<br />
|-<br />
| Western Digital || WD2500BPVT || 250.0 GB || 512 || 4096<br />
|-<br />
| Western Digital || WD1600BPVT || 160.0 GB<br />
|-<br />
| Western Digital || WD7500BPKX || 750.0 GB || 512 || 4096<br />
|-<br />
| TOSHIBA || MQ01ABD100 || 1.0 TB || 512 || 4096<br />
|-<br />
| TOSHIBA || MQ01ABC150 || 1.5 TB || 512 || 4096<br />
|}<br />
<br />
{{Note| Readers are encouraged to add to this table.}}<br />
<br />
== How to determine if HDD employ a 4k sector ==<br />
<br />
The physical and logical sector size of hard disk /dev/sd''X'' can be determined by reading the following sysfs entries:<br />
$ cat /sys/class/block/sd''X''/queue/physical_block_size<br />
$ cat /sys/class/block/sd''X''/queue/logical_block_size<br />
<br />
Tools which will report the physical sector of a drive (provided the drive will report it correctly) includes<br />
* smartmontools (since 5.41 ; <tt>smartmontools -a</tt>, in information section)<br />
* hdparm (since 9.12 ; <tt>hdparm -I</tt>, in configuration section)<br />
<br />
Note that both works even for USB-attached discs (if the USB bridge supports SAT aka SCSI/ATA Translation, ANSI INCITS 431-2007).<br />
<br />
==Aligning Partitions==<br />
<br />
{{Note|This should no longer require manual intervention. Any tools using recent libblkid versions are capable of handling Advanced Format automatically.}}<br />
<br />
Versions with this support include:<br />
<br />
* fdisk, since util-linux >= 2.15. You should start with ‘-c -u’ to disable DOS compatibility and use sectors instead of cylinders.<br />
* parted, since parted >= 2.1.<br />
* mdadm, since util-linux >= 2.15<br />
* lvm2, since util-linux >= 2.15<br />
* mkfs.{ext,xfs,gfs2,ocfs2} all support libblkid directly.<br />
<br />
Refer to [https://www.tolaris.com/2011/07/21/libblkid-or-why-you-dont-need-to-worry-about-4k-disk-format/ this page] for further information.<br />
<br />
===Check your partitions alignment===<br />
{{Note|This only works with [[MBR]], not [[GPT]].}}<br />
# fdisk -lu /dev/sda<br />
...<br />
# Device Boot Start End Blocks Id System<br />
# /dev/sda1 2048 46876671 23437312 7 HPFS/NTFS<br />
<br />
2048 (default since fdisk 2.17.2) means that your HDD is aligned correctly.<br />
Any other value divisible by 8 is good as well.<br />
<br />
===GPT (Recommended)===<br />
When using [[GPT]] partition tables, one need only use gdisk to create partitions which are aligned by default. For an example, see [[SSD#Detailed Usage Example]].<br />
<br />
==Special Consideration for WD Green HDDs==<br />
<br />
FYI - this section has nothing to do with Advanced Format technology, but this is an appropriate location to share it with users. The WD20EARS (and other sizes include 1.0 and 1.5 TB driver in the series) will attempt to park the read heads once every 8 seconds FOR THE LIFE OF THE HDD which is just horrible! To see if you are affected use the smartctl command (part of smartmontools). If the last column changes rapidly, this section applies to your drive. <br />
# smartctl /dev/sdb -a | grep Load_Cycle<br />
193 Load_Cycle_Count 0x0032 001 001 000 Old_age Always - 597115<br />
<br />
=== Disable via hdparm ===<br />
Use hdparm in {{ic|/etc/systemd/system/lcc_fix.service}} to disable this 'feature' and likely add life to your hdd:<br />
<br />
{{hc|/etc/systemd/system/lcc_fix.service |<nowiki><br />
[Unit]<br />
Description=WDIDLE3<br />
<br />
[Service]<br />
Type=oneshot<br />
ExecStart=/usr/bin/hdparm -J 300 --please-destroy-my-drive /dev/sdX<br />
TimeoutSec=0<br />
StandardInput=tty<br />
RemainAfterExit=yes<br />
<br />
[Install]<br />
WantedBy=multi-user.target<br />
</nowiki>}}<br />
<br />
Start the service<br />
# systemctl start lcc_fix.service<br />
<br />
Enable the service to autorun at boot.<br />
# systemctl enable lcc_fix.service<br />
<br />
==== Is this safe? ====<br />
Why do we need to pass the "--please-destroy-my-drive" flag? Here is an email from hdparm author, Mark Lord:<br />
<br />
> I have a Western DIgital \"Green\" drive (wd20ears). I noticed you added a -J switch and that <br />
> it is said to adjust the idle3 timeout. What frightens me is the output you gave it:<br />
> <br />
> How safe or not is this to use?<br />
<br />
I use it on my own drives. It works for me.<br />
<br />
If you can run the WDIDLE3.EXE MS-Dos program,<br />
then use it instead -- it was written by WD,<br />
and only they know how things really work there.<br />
<br />
If you cannot use the WDIDLE3.EXE, then you<br />
could consider "hdparm -J". It works for me,<br />
but it may or may not void some kind of warranty.<br />
<br />
Cheers<br />
-- <br />
Mark Lord<br />
Real-Time Remedies Inc.<br />
mlord@pobox.com<br />
<br />
=== Disable via changing firmware value (persistent) ===<br />
<br />
{{Warning|The tool used in this process is experimental, use at your own risk!}}<br />
<br />
{{Note|This method is persistent, you only need to do this once for every drive.}}<br />
<br />
This method will use a utility called idle3ctl to alter the firmware value for the idle3 timer on WD hard drives (similar to wdidle3.exe from WD). The advantage compared to the official utility is you do not need to create a DOS bootdisk first to change the idle3 timer value. Additionally idle3ctl might also work over USB-to-S-ATA bridges (in some cases).<br />
Download [http://idle3-tools.sourceforge.net/ idle3ctl], extract and compile it.<br />
Within the folder that contains the newly compiled binary, execute<br />
<br />
# ./idle3ctl -g /dev/your_wd_hdd<br />
<br />
to get the raw idle3 timer value.<br />
You can disable the IntelliPark feature completely, with:<br />
<br />
# ./idle3ctl -d /dev/your_wd_hdd<br />
<br />
or set it to a different value (''0''-''255'') with (e.g. 10 seconds):<br />
<br />
# ./idle3ctl -s 100 /dev/your_wd_hd<br />
<br />
The range ''0''-''128'' is in 0.1s and ''129-255'' in 30s. For the changes to take effect, the drive needs to go through one powercycle, meaning powering it off and on again (on internal drives, a reboot is not sufficient).<br />
<br />
If your WD hard drive is not recognized, you can use the ''--force'' option. For more options see:<br />
<br />
$ ./idle3ctl -h</div>Phiresky