https://wiki.archlinux.org/api.php?action=feedcontributions&user=Justin8&feedformat=atomArchWiki - User contributions [en]2024-03-28T13:37:17ZUser contributionsMediaWiki 1.41.0https://wiki.archlinux.org/index.php?title=Bluetooth&diff=461367Bluetooth2017-01-04T04:39:12Z<p>Justin8: /* Audio */ Added steps on enabling a2dp sink mode</p>
<hr />
<div>[[Category:Bluetooth]]<br />
[[cs:Bluetooth]]<br />
[[de:Bluetooth]]<br />
[[es:Bluetooth]]<br />
[[fr:Bluetooth]]<br />
[[it:Bluetooth]]<br />
[[ja:Bluetooth]]<br />
[[ru:Bluetooth]]<br />
[[zh-CN:Bluetooth]]<br />
{{Related articles start}}<br />
{{Related|Bluez4}}<br />
{{Related|Bluetooth mouse}}<br />
{{Related|Bluetooth headset}}<br />
{{Related|Blueman}}<br />
{{Related articles end}}<br />
[http://www.bluetooth.org/ Bluetooth] is a standard for the short-range wireless interconnection of cellular phones, computers, and other electronic devices. In Linux, the canonical implementation of the Bluetooth protocol stack is [http://www.bluez.org/ BlueZ].<br />
<br />
== Installation ==<br />
<br />
[[Install]] the {{Pkg|bluez}} and {{Pkg|bluez-utils}} packages. The {{Pkg|bluez}} package provides the Bluetooth protocol stack, and the {{Pkg|bluez-utils}} package provides the {{ic|bluetoothctl}} utility. <br />
<br />
Load the generic bluetooth driver, if not already loaded:<br />
# modprobe btusb<br />
<br />
Then [[start]] the {{ic|bluetooth.service}} systemd unit. You can [[enable]] it to start automatically at boot time.<br />
<br />
{{Note|<br />
* By default the bluetooth daemon will only give out bnep0 devices to users that are a member of the {{ic|lp}} group. Make sure to add your user to that group if you intend to connect to a bluetooth tether. You can change the group that is required in the file {{ic|/etc/dbus-1/system.d/bluetooth.conf}}.<br />
* Some Bluetooth adapters are bundled with a Wi-Fi card (e.g. [http://www.intel.com/content/www/us/en/wireless-products/centrino-advanced-n-6235.html Intel Centrino]). These require that the Wi-Fi card is firstly enabled (typically a keyboard shortcut on a laptop) in order to make the Bluetooth adapter visible to the kernel.<br />
* Some Bluetooth cards (e.g. Broadcom) conflict with the network adapter. Thus, you need to make sure that your Bluetooth device get connected before the network service boot.}}<br />
<br />
== Configuration via the CLI ==<br />
=== Bluetoothctl ===<br />
Pairing a device from the shell is one of the simplest and most reliable options. The exact procedure depends on the devices involved and their input functionality. What follows is a general outline of pairing a device using {{ic|/usr/bin/bluetoothctl}}:<br />
<br />
Start the {{ic|bluetoothctl}} interactive command. There one can input {{ic|help}} to get a list of available commands. <br />
* Turn the power to the controller on by entering {{ic|power on}}. It is off by default.<br />
* Enter {{ic|devices}} to get the MAC Address of the device with which to pair.<br />
* Enter device discovery mode with {{ic|scan on}} command if device is not yet on the list.<br />
* Turn the agent on with {{ic|agent on}}.<br />
* Enter {{ic|pair ''MAC Address''}} to do the pairing (tab completion works).<br />
* If using a device without a PIN, one may need to manually trust the device before it can reconnect successfully. Enter {{ic|trust ''MAC Address''}} to do so.<br />
* Finally, use {{ic|connect ''MAC_address''}} to establish a connection.<br />
<br />
An example session may look this way:<br />
# bluetoothctl <br />
[NEW] Controller 00:10:20:30:40:50 pi [default]<br />
[bluetooth]# agent KeyboardOnly <br />
Agent registered<br />
[bluetooth]# default-agent <br />
Default agent request successful<br />
[bluetooth]# scan on<br />
Discovery started<br />
[CHG] Controller 00:10:20:30:40:50 Discovering: yes<br />
[NEW] Device 00:12:34:56:78:90 myLino<br />
[CHG] Device 00:12:34:56:78:90 LegacyPairing: yes<br />
[bluetooth]# pair 00:12:34:56:78:90<br />
Attempting to pair with 00:12:34:56:78:90<br />
[CHG] Device 00:12:34:56:78:90 Connected: yes<br />
[CHG] Device 00:12:34:56:78:90 Connected: no<br />
[CHG] Device 00:12:34:56:78:90 Connected: yes<br />
Request PIN code<br />
[agent] Enter PIN code: 1234<br />
[CHG] Device 00:12:34:56:78:90 Paired: yes<br />
Pairing successful<br />
[CHG] Device 00:12:34:56:78:90 Connected: no<br />
[bluetooth]# connect 00:12:34:56:78:90<br />
Attempting to connect to 00:12:34:56:78:90<br />
[CHG] Device 00:12:34:56:78:90 Connected: yes<br />
Connection successful<br />
<br />
In order to have the device active after a reboot, a udev rule is needed:<br />
<br />
{{hc|/etc/udev/rules.d/10-local.rules|2=<br />
# Set bluetooth power up<br />
ACTION=="add", KERNEL=="hci0", RUN+="/usr/bin/hciconfig %k up"}}<br />
<br />
{{Tip|Replace {{ic|1=KERNEL=="hci0"}} with {{ic|1=KERNEL=="hci[0-9]*"}} to match all interfaces.}}<br />
<br />
After a suspend/resume-cycle, the device can be powered on automatically using a custom ''systemd'' service:<br />
<br />
{{hc|/etc/systemd/system/bluetooth-auto-power@.service|<nowiki><br />
[Unit]<br />
Description=Bluetooth auto power on<br />
After=bluetooth.service sys-subsystem-bluetooth-devices-%i.device suspend.target<br />
<br />
[Service]<br />
Type=oneshot<br />
ExecStart=/usr/bin/hciconfig %i up<br />
<br />
[Install]<br />
WantedBy=suspend.target<br />
</nowiki>}}<br />
<br />
[[Enable]] an instance of the unit using your bluetooth device name, for example {{ic|bluetooth-auto-power@hci0.service}}.<br />
<br />
Alternatively and instead of the custom service and udev rule, you can simply use the new [http://www.bluez.org/release-of-bluez-5-35/ AutoEnable feature introduced in BlueZ 5.35] by uncommenting {{ic|[Policy]}} and {{ic|AutoEnable}} lines in {{ic|/etc/bluetooth/main.conf}}.<br />
<br />
== Configuration with a graphical front-end ==<br />
<br />
The following packages allow for a graphical interface to customize Bluetooth.<br />
<br />
=== GNOME Bluetooth ===<br />
<br />
[https://wiki.gnome.org/Projects/GnomeBluetooth GNOME Bluetooth] is [[GNOME]]'s Bluetooth tool. The {{Pkg|gnome-bluetooth}} package provides the back-end, {{Pkg|gnome-shell}} provides the status monitor applet, and {{Pkg|gnome-control-center}} provides the configuration front-end GUI that can be accessed by typing Bluetooth on the Activities overview, or with the {{ic|gnome-control-center bluetooth}} command. You can also launch the {{ic|bluetooth-sendto}} command directly to send files to a remote device.<br />
<br />
To receive files, you must install the {{Pkg|gnome-user-share}} package. You can then go to ''Settings -> Sharing'' to authorize receiving files from paired devices over Bluetooth.<br />
<br />
{{Tip|To add a Bluetooth entry to the ''Send To'' menu in Thunar's file properties menu, see instructions [http://docs.xfce.org/xfce/thunar/send-to here]. (The command that needs to be configured is {{ic|bluetooth-sendto %F}}).}}<br />
<br />
=== Bluedevil ===<br />
<br />
[https://projects.kde.org/projects/kde/workspace/bluedevil Bluedevil] is [[KDE]]'s Bluetooth tool. It can be [[installed]] with the package {{Pkg|bluedevil}} (KDE Plasma 5).<br />
<br />
If there is no Bluetooth icon visible in Dolphin and in the system tray, enable it in the system tray options or add a widget. You can configure Bluedevil and detect Bluetooth devices by clicking the icon. An interface is also available from the KDE System Settings.<br />
<br />
=== Blueberry ===<br />
<br />
''Blueberry'' is an alternative front-end for GNOME Bluetooth, which works in all desktop environments. It can be installed with the {{Pkg|blueberry}} package. It provides a configuration tool (''blueberry'') and a system tray applet (''blueberry-tray'').<br />
<br />
=== Blueman ===<br />
<br />
See [[Blueman]].<br />
<br />
== Using Obex for sending and receiving files ==<br />
=== ObexFS ===<br />
Another option, rather than using KDE or Gnome Bluetooth packages, is ObexFS which allows for the mounting of phones which are treated like any other filesystem.<br />
{{Note|To use ObexFS, one needs a device that provides an ObexFTP service.}}<br />
<br />
Install {{Pkg|obexfs}} and mount supported phones by running:<br />
$ obexfs -b ''MAC_address_of_device'' /mountpoint<br />
<br />
Once you have finished, to unmount the device use the command:<br />
$ fusermount -u /mountpoint<br />
<br />
For more mounting options see http://dev.zuckschwerdt.org/openobex/wiki/ObexFs<br />
<br />
{{Note|Ensure that the bluetooth device you are mounting is '''not''' set to mount ''read-only''. You should be able to do this from the device's settings. If the device is mounted ''read-only'' you may encounter a permissions error when trying to transfer files to the device.}}<br />
<br />
=== ObexFTP transfers ===<br />
<br />
If your device supports the Obex FTP service but you do not wish to mount the device you can transfer files to and from the device using the obexftp command.<br />
<br />
To send a file to a device run the command:<br />
<br />
$ obexftp -b ''MAC_address_of_device'' -p /path/to/file<br />
<br />
To retrieve a file from a device run the command:<br />
<br />
$ obexftp -b ''MAC_address_of_device'' -g filename<br />
<br />
{{Note|Ensure that the file you are retrieving is in the device's ''exchange folder''. If the file is in a subfolder of the exchange folder then provide the correct path in the command.}}<br />
<br />
=== Obex Object Push ===<br />
For devices that do not support Obex FTP service, check if Obex Object Push is supported.<br />
<br />
# sdptool browse ''XX:XX:XX:XX:XX:XX''<br />
<br />
Read the output, look for Obex Object Push, remember the channel for this service. If supported, one can use {{pkg|ussp-push}} to send files to this device:<br />
<br />
# ussp-push ''XX:XX:XX:XX:XX:XX''@''CHANNEL'' ''file'' ''wanted_file_name_on_phone''<br />
<br />
=== Using your computer's speakers as a bluetooth headset ===<br />
<br />
This can allow you to do things such as playing what is on your phone through your computer speakers.<br />
<br />
Add the following to the file {{ic|/etc/bluetooth/audio.conf}} (create it if not present):<br />
<br />
[General]<br />
Enable=Source<br />
<br />
More info in:<br />
* https://gist.github.com/joergschiller/1673341<br />
* http://www.lightofdawn.org/blog/?viewDetailed=00031<br />
<br />
== Audio ==<br />
<br />
In order to be able to use audio equipment like bluetooth headphones, you need to install the additional {{Pkg|pulseaudio-bluetooth}} package.<br />
<br />
Please have a look at the [[Bluetooth headset]] page for more information about bluetooth audio and bluetooth headsets.<br />
<br />
In order to enable your system to be detected as an A2DP sink (e.g. to play music from your phone via your computer speakers) the following must be added under <code>[General]</code> in <code>/etc/bluetooth/main.conf</code>:<br />
<br />
<code>Enable=Source,Sink,Media,Socket</code><br />
<br />
== Troubleshooting ==<br />
<br />
=== bluetoothctl ===<br />
If bluetoothctl cannot find any controller, the bluetooth device may be blocked. Try to unblock it using {{Pkg|rfkill}}:<br />
<br />
# rfkill unblock bluetooth<br />
<br />
=== gnome-bluetooth ===<br />
<br />
If you see this when trying to enable receiving files in bluetooth-properties:<br />
Bluetooth OBEX start failed: Invalid path<br />
Bluetooth FTP start failed: Invalid path<br />
Then install {{Pkg|xdg-user-dirs}} and issue:<br />
$ xdg-user-dirs-update<br />
You can edit the paths using:<br />
$ vi ~/.config/user-dirs.dirs<br />
<br />
=== Bluetooth USB Dongle ===<br />
<br />
If you are using a USB dongle, you should check that your Bluetooth dongle is recognized. You can do that by running {{ic|journalctl -f}} when you have plugged in the USB dongle (or inspecting {{ic|/var/log/messages.log}}). It should look something like the following (look out for hci):<br />
<br />
{{bc|<br />
Feb 20 15:00:24 hostname kernel: [ 2661.349823] usb 4-1: new full-speed USB device number 3 using uhci_hcd<br />
Feb 20 15:00:24 hostname bluetoothd[4568]: HCI dev 0 registered<br />
Feb 20 15:00:24 hostname bluetoothd[4568]: Listening for HCI events on hci0<br />
Feb 20 15:00:25 hostname bluetoothd[4568]: HCI dev 0 up<br />
Feb 20 15:00:25 hostname bluetoothd[4568]: Adapter /org/bluez/4568/hci0 has been enabled<br />
}}<br />
<br />
If you only get the first two lines, you may see that it found the device but you need to bring it up.<br />
Example:<br />
<br />
{{hc|hciconfig -a hci0|<br />
hci0: Type: USB<br />
BD Address: 00:00:00:00:00:00 ACL MTU: 0:0 SCO MTU: 0:0<br />
DOWN <br />
RX bytes:0 acl:0 sco:0 events:0 errors:0<br />
TX bytes:0 acl:0 sco:0 commands:0 errors:<br />
}}<br />
<br />
# hciconfig hci0 up<br />
<br />
{{hc|hciconfig -a hci0|<br />
hci0: Type: USB<br />
BD Address: 00:02:72:C4:7C:06 ACL MTU: 377:10 SCO MTU: 64:8<br />
UP RUNNING <br />
RX bytes:348 acl:0 sco:0 events:11 errors:0<br />
TX bytes:38 acl:0 sco:0 commands:11 errors:0<br />
}}<br />
<br />
If this fails with an error like:<br />
Operation not possible due to RF-kill<br />
it could be due either to the {{ic|rfkill}} utility, in which case it should be resolved with<br />
# rfkill unblock all<br />
or, it could simply be the hardware switch of the computer. The hardware bluetooth switch (at least sometimes) controls access to USB bluetooth dongles also. Flip/press this switch and try bringing the device up again.<br />
<br />
To verify that the device was detected you can use {{ic|hcitool}} which is part of the {{ic|bluez-utils}}. You can get a list of available devices and their identifiers and their MAC address by issuing:<br />
<br />
{{hc|$ hcitool dev|<br />
Devices:<br />
hci0 00:1B:DC:0F:DB:40<br />
}}<br />
<br />
More detailed information about the device can be retrieved by using {{ic|hciconfig}}.<br />
<br />
{{hc|$ hciconfig -a hci0|<br />
hci0: Type: USB<br />
BD Address: 00:1B:DC:0F:DB:40 ACL MTU: 310:10 SCO MTU: 64:8<br />
UP RUNNING PSCAN ISCAN<br />
RX bytes:1226 acl:0 sco:0 events:27 errors:0<br />
TX bytes:351 acl:0 sco:0 commands:26 errors:0<br />
Features: 0xff 0xff 0x8f 0xfe 0x9b 0xf9 0x00 0x80<br />
Packet type: DM1 DM3 DM5 DH1 DH3 DH5 HV1 HV2 HV3<br />
Link policy: RSWITCH HOLD SNIFF PARK<br />
Link mode: SLAVE ACCEPT <br />
Name: 'BlueZ (0)'<br />
Class: 0x000100<br />
Service Classes: Unspecified<br />
Device Class: Computer, Uncategorized<br />
HCI Ver: 2.0 (0x3) HCI Rev: 0xc5c LMP Ver: 2.0 (0x3) LMP Subver: 0xc5c<br />
Manufacturer: Cambridge Silicon Radio (10)<br />
}}<br />
<br />
==== Audio devices start to skip at short distance from dongle ====<br />
<br />
If other devices share the same USB host, they can [https://bbs.archlinux.org/viewtopic.php?pid=1440161#p1440161 interrupt communication with audio devices]. Make sure it is the only device attached to its bus. For example:<br />
<br />
{{hc|$ lsusb|<br />
Bus 002 Device 002: ID 0a12:0001 Cambridge Silicon Radio, Ltd Bluetooth Dongle (HCI mode)<br />
Bus 002 Device 001: ID 1d6b:0002 Linux Foundation 2.0 root hub<br />
Bus 001 Device 004: ID 048d:1345 Integrated Technology Express, Inc. Multi Cardreader<br />
Bus 001 Device 003: ID 0424:a700 Standard Microsystems Corp. 2 Port Hub<br />
Bus 001 Device 002: ID 8087:0024 Intel Corp. Integrated Rate Matching Hub<br />
Bus 001 Device 001: ID 1d6b:0002 Linux Foundation 2.0 root hub<br />
}}<br />
<br />
=== Logitech Bluetooth USB Dongle ===<br />
<br />
There are Logitech dongles (ex. Logitech MX5000) that can work in two modes: Embedded and HCI. In embedded mode dongle emulates a USB device so it seems to your PC that you are using a normal USB mouse/keyoard.<br />
<br />
If you hold the little red Button on the USB BT mini-receiver it will enable the other mode. Hold the red button on the BT dongle and plug it into the computer, and after 3-5 seconds of holding the button, the Bluetooth icon will appear in the system tray. [http://ubuntuforums.org/showthread.php?t=1332197 Discussion]<br />
<br />
Alternatively, you can install the {{Pkg|bluez-hid2hci}} package. When you connect your Logitech dongle it will automatically switch.<br />
<br />
=== hcitool scan: Device not found ===<br />
<br />
* On some Dell laptops (e.g. Studio 15) you have to switch the Bluetooth mode from HID to HCI. Install the {{Pkg|bluez-hid2hci}} package, then [[udev]] should do this automatically. Alternatively, you can run this command to switch to HCI manually:<br />
# /usr/lib/udev/hid2hci<br />
<br />
* If the device will not show up and you have a Windows operating system on your machine, try booting it and enable the bluetooth adapter from windows.<br />
<br />
* Sometimes also this simple command helps:<br />
# hciconfig hci0 up<br />
<br />
=== rfkill unblock: Do not unblock ===<br />
<br />
If your device still soft blocked and you run connman, try this:<br />
<br />
$ connmanctl enable bluetooth<br />
<br />
=== My computer is not visible ===<br />
<br />
Cannot discover computer from your phone? Enable PSCAN and ISCAN:<br />
# enable PSCAN and ISCAN<br />
$ hciconfig hci0 piscan <br />
# check it worked<br />
{{hc|$ hciconfig|<br />
hci0: Type: USB<br />
BD Address: 00:12:34:56:78:9A ACL MTU: 192:8 SCO MTU: 64:8<br />
'''UP RUNNING PSCAN ISCAN'''<br />
RX bytes:20425 acl:115 sco:0 events:526 errors:0<br />
TX bytes:5543 acl:84 sco:0 commands:340 errors:0<br />
}}<br />
<br />
{{Note|Check DiscoverableTimeout and PairableTimeout in {{ic|/etc/bluetooth/main.conf}}}}<br />
<br />
Try changing device class in {{ic|/etc/bluetooth/main.conf}} as following:<br />
# Default device class. Only the major and minor device class bits are<br />
# considered.<br />
#Class = 0x000100 (from default config)<br />
Class = 0x100100<br />
<br />
This was the only solution to make my computer visible for my phone.<br />
<br />
=== Logitech keyboard does not pair ===<br />
<br />
If you do not get the passkey when you try to pair your Logitech keyboard, type the following command:<br />
# hciconfig hci0 sspmode 0<br />
<br />
If after pairing, the keyboard still does not connect, check the output of {{ic|hcidump -at}}. If the latter indicates repeatedly connections-disconnections like the following message:<br />
<br />
status 0x00 handle 11 reason 0x13<br />
Reason: Remote User Terminated Connection<br />
<br />
then, the only solution for now is to install [[bluez4|the old Bluetooth stack]].<br />
<br />
=== HSP/HFP profiles ===<br />
<br />
bluez5 removed support for the HSP/HFP profiles (telephony headset for [[TeamSpeak]], [[Skype]], etc.). You need to install [[PulseAudio]] (>= version 6) or another application that implements HSP/HFP itself.<br />
<br />
=== Thinkpad Bluetooth Laser Mouse problems ===<br />
<br />
If you are experiencing that your Thinkpad Bluetooth Laser Mouse rapidly connects and then (after a few milliseconds) disconnects again every few seconds (when you move the mouse or press a button), try pairing it with the code {{ic|0000}} instead pairing without a code.<br />
<br />
=== Foxconn / Hon Hai / Lite-On Broadcom device ===<br />
<br />
Some of these devices require the firmware to be flashed into the device at boot. The firmware is not provided but can converted from a Microsoft Windows ''.hex'' file into a ''.hcd'' using [https://github.com/jessesung/hex2hcd hex2hcd] (which is installed with {{Pkg|bluez-utils}}).<br />
<br />
In order to get the right ''.hex'' file, try searching the device vendor:product code obtained with ''lsusb'', for example:<br />
<br />
...<br />
Bus 002 Device 004: ID '''04ca:2006''' Lite-On Technology Corp. Broadcom BCM43142A0 Bluetooth Device<br />
...<br />
<br />
or<br />
<br />
Bus 004 Device 004: Id '''0489:e031''' Foxconn / Hon Hai<br />
<br />
Alternatively, boot into Windows (a virtual machine installation will suffice) and get the firmware name from the Device Manager utility. If you want to know the model of your device but cannot see it in ''lsusb'', you might see it in ''lsusb -v'' as {{ic|iProduct}}.<br />
<br />
The ''.hex'' file can be extracted from the downloaded Windows driver without having to run Windows for it. Download the right driver, for example [http://www.fujitsupc.com/downloads/mobile/BLUETOOTH_WIDCOMM_V6.5.0.3100_WIN7-32_FPC46-1771-01.EXE Bluetooth Widcomm] (listed among the drivers for [http://support.fujitsupc.com/CS/Portal/supportsearch.do?srch=DOWNLOADS&Series=P%20Series&Model=P771&ProductType=Notebook%20PC Lifebook P771]), which contains the drivers for many Broadcomm devices. In case of Bluetooth Widcomm, the driver is a self-extracting RAR archive, so it can be extracted using ''{{Pkg|unrar}} x''. To find out which of the many ''.hex'' files is the right one for you, look in the file {{ic|Win32/bcbtums-win7x86-brcm.inf}} and search for {{ic|[RAMUSB'''E031'''.CopyList]}}, where {{ic|E031}} should be replaced with the product code (the second hex number in ''lsusb'') of your device in upper-case. Underneath you should see the file name of the right ''.hex'' file.<br />
<br />
Once you have the ''.hcd'' file, copy it into {{ic|/lib/firmware/brcm/BCM.hcd}} - this filename is suggested by {{ic|dmesg}} and it may change in your case so check your ''dmesg'' output in order to verify. Then reload the ''btusb'' module:<br />
<br />
# rmmod btusb<br />
# modprobe btusb<br />
<br />
In some cases (with older kernels?), you have to flash the ''.hcd'' file with the ''brcm_patchram_plus'' utility, provided by {{AUR|brcm_patchram_plus-git}}{{Broken package link|{{aur-mirror|brcm_patchram_plus-git}}}}. First, make sure in ''dmesg'' that the device is recognized by ''btusb'' as a bluetooth device. Then, run the following (replace ''04ca 2006'' with your vendor product pair):<br />
<br />
# echo '04ca 2006' > /sys/bus/usb/drivers/btusb/new_id<br />
<br />
Turn on the device:<br />
<br />
# hciconfig hci0 up<br />
<br />
Flash the firmware: <br />
<br />
# brcm_patchram_plus_usb --patchram fw-04ca_2006.hcd hci0<br />
<br />
The device should now be available. See [https://bbs.archlinux.org/viewtopic.php?id=162688 BBS#162688] for information on making these changes persistent.</div>Justin8https://wiki.archlinux.org/index.php?title=ZFS&diff=456640ZFS2016-11-13T01:28:48Z<p>Justin8: Removed information that isn't relevant outside of initial creation from the swap section</p>
<hr />
<div>[[Category:File systems]]<br />
[[ja:ZFS]]<br />
{{Related articles start}}<br />
{{Related|File systems}}<br />
{{Related|Experimenting with ZFS}}<br />
{{Related|Installing Arch Linux on ZFS}}<br />
{{Related|ZFS on FUSE}}<br />
{{Related articles end}}<br />
<br />
[[Wikipedia:ZFS|ZFS]] is an advanced filesystem created by [[Wikipedia:Sun Microsystems|Sun Microsystems]] (now owned by Oracle) and released for OpenSolaris in November 2005. <br />
<br />
Features of ZFS include: pooled storage (integrated volume management – zpool), [[Wikipedia:Copy-on-write|Copy-on-write]], [[Wikipedia:Snapshot (computer storage)|snapshots]], data integrity verification and automatic repair (scrubbing), [[Wikipedia:RAID-Z|RAID-Z]], a maximum [[Wikipedia:Exabyte|16 Exabyte]] file size, and a maximum 256 Quadrillion [[Wikipedia:Zettabyte|Zettabytes]] storage with no limit on number of filesystems (datasets) or files[http://docs.oracle.com/cd/E19253-01/819-5461/zfsover-2/index.html]. ZFS is licensed under the [[Wikipedia:CDDL|Common Development and Distribution License]] (CDDL).<br />
<br />
Described as [http://web.archive.org/web/20060428092023/http://www.sun.com/2004-0914/feature/ "The last word in filesystems"] ZFS is stable, fast, secure, and future-proof. Being licensed under the CDDL, and thus incompatible with GPL, it is not possible for ZFS to be distributed along with the Linux Kernel. This requirement, however, does not prevent a native Linux kernel module from being developed and distributed by a third party, as is the case with [http://zfsonlinux.org/ zfsonlinux.org] (ZOL).<br />
<br />
ZOL is a project funded by the [https://www.llnl.gov/ Lawrence Livermore National Laboratory] to develop a native Linux kernel module for its massive storage requirements and super computers.<br />
<br />
{{Note|Due to potential legal incompatibilities between CDDL license of ZFS code, and GPL of Linux kernel ([https://sfconservancy.org/blog/2016/feb/25/zfs-and-linux/ ],[[wikipedia:Common_Development_and_Distribution_License#GPL_compatibility|CDDL-GPL]],[[wikipedia:ZFS#Linux|ZFS in Linux]]) - ZFS development not upstreamed nor supported by kernel. While ZFS, as filesystem, being heavily in need of kernel functionality and work stability.<br />
<br />
This results in:<br />
* ZFS sill resides in [[Arch User Repository]] and unofficial [[Unofficial user repositories#archzfs|archzfs]] repository.<br />
* ZFSonLinux project keeping up to Linux kernel versions and making stable releases and Arch ZFS maintainers releasing them while Arch Linux already get next newer version of kernel, perpetually.<br />
* This situation locks-down normal rolling update process by unsatisfied dependencies, because new kernel version, proposed by update, most of the time unsupported by ZFS.}}<br />
<br />
==Installation==<br />
=== General ===<br />
Install from the [[Arch User Repository]] or the [[Unofficial user repositories#archzfs|archzfs]] repository:<br />
* {{AUR|zfs-linux}} for [http://zfsonlinux.org/ stable] releases.<br />
* {{AUR|zfs-linux-git}} for [https://github.com/zfsonlinux/zfs/releases development] releases (with support of newer kernel versions).<br />
* {{AUR|zfs-linux-lts}} for stable releases for LTS kernels.<br />
<br />
These branches have according for them {{ic|zfs-utils}}, {{ic|spl-linux}}, {{ic|spl-utils-linux}} dependencies. SPL (Solaris Porting Layer) is a Linux Kernel module implementing Solaris APIs for ZFS compatibility.<br />
<br />
{{warning|The ZFS and SPL kernel modules are tied to a specific kernel version. It would not be possible to apply any kernel updates until updated packages are uploaded to AUR or the [[Unofficial user repositories#archzfs|archzfs]] repository.}}<br />
<br />
Test the installation by issuing {{ic|zpool status}} on the command line. If an "insmod" error is produced, try {{ic|depmod -a}}.<br />
<br />
{{Tip| You can [[downgrade]] your linux version to the one from [[Unofficial user repositories#archzfs|archzfs]] repo if your current kenel is newer.}}<br />
<br />
=== Root on ZFS ===<br />
<br />
When performing an Arch install on ZFS, {{AUR|zfs-git}}{{Broken package link|{{aur-mirror|zfs-git}}}} and its dependencies can be installed in the archiso environment as outlined in the previous section.<br />
<br />
It may be useful to prepare a [[#Embed the archzfs packages into an archiso|customized archiso]] with ZFS support builtin. For a much more detailed guide on installing Arch with ZFS as its root file system, see [[Installing Arch Linux on ZFS]].<br />
<br />
=== DKMS ===<br />
<br />
Users can make use of DKMS [[Dynamic Kernel Module Support]] to rebuild the ZFS modules automatically with every kernel upgrade. <br />
<br />
Install {{AUR|zfs-dkms}} or {{AUR|zfs-dkms-git}} and apply the post-install instructions given by these packages.<br />
<br />
{{Tip|Add an {{ic|IgnorePkg}} entry to [[pacman.conf]] to prevent these packages from upgrading when doing a regular update.}}<br />
<br />
==Experimenting with ZFS ==<br />
<br />
Users wishing to experiment with ZFS on ''virtual block devices'' (known in ZFS terms as VDEVs) which can be simple files like {{ic|~/zfs0.img}} {{ic|~/zfs1.img}} {{ic|~/zfs2.img}} etc. with no possibility of real data loss are encouraged to see the [[Experimenting with ZFS]] article. Common tasks like building a RAIDZ array, purposefully corrupting data and recovering it, snapshotting datasets, etc. are covered.<br />
<br />
==Configuration==<br />
<br />
ZFS is considered a "zero administration" filesystem by its creators; therefore, configuring ZFS is very straight forward. Configuration is done primarily with two commands: {{ic|zfs}} and {{ic|zpool}}.<br />
<br />
===Automatic Start===<br />
<br />
For ZFS to live by its "zero administration" namesake, the zfs daemon must be loaded at startup. A benefit to this is that it is not necessary to mount the zpool in {{ic|/etc/fstab}}; the zfs daemon can import and mount zfs pools automatically. The daemon mounts the zfs pools reading the file {{ic|/etc/zfs/zpool.cache}}.<br />
<br />
For each pool you want automatically mounted by the zfs daemon execute:<br />
# zpool set cachefile=/etc/zfs/zpool.cache <pool><br />
<br />
Enable the service so it is automatically started at boot time:<br />
<br />
# systemctl enable zfs.target<br />
<br />
To manually start the daemon:<br />
<br />
# systemctl start zfs.target<br />
<br />
{{Note|If {{ic|zfs-mount.service}} fails on boot due to running before the kernel module is loaded, you may have to manually enable the {{ic|zfs-import-cache.service}}. <br />
<br />
# systemctl enable zfs-import-cache.service<br />
<br />
}}<br />
<br />
==Create a storage pool==<br />
<br />
Use {{ic| # parted --list}} to see a list of all available drives. It is not necessary nor recommended to partition the drives before creating the zfs filesystem.<br />
<br />
{{Note|If some or all device have been used in a software RAID set it is paramount to erase any old RAID configuration information. ([[Mdadm#Prepare_the_Devices]]) }}<br />
<br />
{{Warning|For Advanced Format Disks with 4KB sector size, an ashift of 12 is recommended for best performance. Advanced Format disks emulate a sector size of 512 bytes for compatibility with legacy systems, this causes ZFS to sometimes use an ashift option number that is not ideal. Once the pool has been created, the only way to change the ashift option is to recreate the pool. Using an ashift of 12 would also decrease available capacity. See [https://github.com/zfsonlinux/zfs/wiki/faq#performance-considerations 1.10 What’s going on with performance?], [https://github.com/zfsonlinux/zfs/wiki/faq#advanced-format-disks 1.15 How does ZFS on Linux handle Advanced Format disks?], and [http://wiki.illumos.org/display/illumos/ZFS+and+Advanced+Format+disks ZFS and Advanced Format disks].}}<br />
<br />
Having identified the list of drives, it is now time to get the id's of the drives to add to the zpool. The [https://github.com/zfsonlinux/zfs/wiki/faq#selecting-dev-names-when-creating-a-pool zfs on Linux developers recommend] using device ids when creating ZFS storage pools of less than 10 devices. To find the id's, simply:<br />
<br />
# ls -lh /dev/disk/by-id/<br />
<br />
The ids should look similar to the following:<br />
<br />
lrwxrwxrwx 1 root root 9 Aug 12 16:26 ata-ST3000DM001-9YN166_S1F0JKRR -> ../../sdc<br />
lrwxrwxrwx 1 root root 9 Aug 12 16:26 ata-ST3000DM001-9YN166_S1F0JTM1 -> ../../sde<br />
lrwxrwxrwx 1 root root 9 Aug 12 16:26 ata-ST3000DM001-9YN166_S1F0KBP8 -> ../../sdd<br />
lrwxrwxrwx 1 root root 9 Aug 12 16:26 ata-ST3000DM001-9YN166_S1F0KDGY -> ../../sdb<br />
<br />
{{Style|Missing references to [[Persistent block device naming]], it is useless to explain the differences (or even what they are) here.}}<br />
<br />
Disk labels and UUID can also be used for ZFS mounts by using [[GUID Partition Table|GPT]] partitions. ZFS drives have labels but Linux is unable to read them at boot. Unlike [[Master Boot Record|MBR]] partitions, GPT partitions directly support both UUID and labels independent of the format inside the partition. Partitioning rather than using the whole disk for ZFS offers two additional advantages. The OS does not generate bogus partition numbers from whatever unpredictable data ZFS has written to the partition sector, and if desired, you can easily over provision SSD drives, and slightly over provision spindle drives to ensure that different models with slightly different sector counts can zpool replace into your mirrors. This is a lot of organization and control over ZFS using readily available tools and techniques at almost zero cost.<br />
<br />
Use [[GUID Partition Table|gdisk]] to partition the all or part of the drive as a single partition. gdisk does not automatically name partitions so if partition labels are desired use gdisk command "c" to label the partitions. Some reasons you might prefer labels over UUID are: labels are easy to control, labels can be titled to make the purpose of each disk in your arrangement readily apparent, and labels are shorter and easier to type. These are all advantages when the server is down and the heat is on. GPT partition labels have plenty of space and can store most international characters [[wikipedia:GUID_Partition_Table#Partition_entries]] allowing large data pools to be labeled in an organized fashion.<br />
<br />
Drives partitioned with GPT have labels and UUID that look like this. <br />
<br />
# ls -l /dev/disk/by-partlabel<br />
# lrwxrwxrwx 1 root root 10 Apr 30 01:44 zfsdata1 -> ../../sdd1<br />
# lrwxrwxrwx 1 root root 10 Apr 30 01:44 zfsdata2 -> ../../sdc1<br />
# lrwxrwxrwx 1 root root 10 Apr 30 01:59 zfsl2arc -> ../../sda1<br />
<br />
# ls -l /dev/disk/by-partuuid<br />
# lrwxrwxrwx 1 root root 10 Apr 30 01:44 148c462c-7819-431a-9aba-5bf42bb5a34e -> ../../sdd1<br />
# lrwxrwxrwx 1 root root 10 Apr 30 01:59 4f95da30-b2fb-412b-9090-fc349993df56 -> ../../sda1<br />
# lrwxrwxrwx 1 root root 10 Apr 30 01:44 e5ccef58-5adf-4094-81a7-3bac846a885f -> ../../sdc1<br />
<br />
Now, finally, create the ZFS pool:<br />
<br />
# zpool create -f -m <mount> <pool> raidz <ids><br />
<br />
* '''create''': subcommand to create the pool.<br />
<br />
* '''-f''': Force creating the pool. This is to overcome the "EFI label error". See [[#Does not contain an EFI label]].<br />
<br />
* '''-m''': The mount point of the pool. If this is not specified, then the pool will be mounted to {{ic|/<pool>}}.<br />
<br />
* '''pool''': This is the name of the pool.<br />
<br />
* '''raidz''': This is the type of virtual device that will be created from the pool of devices. Raidz is a special implementation of raid5. See [https://blogs.oracle.com/bonwick/entry/raid_z Jeff Bonwick's Blog -- RAID-Z] for more information about raidz.<br />
<br />
* '''ids''': The names of the drives or partitions that to include into the pool. Get it from {{ic|/dev/disk/by-id}}.<br />
<br />
Here is an example for the full command:<br />
<br />
# zpool create -f -m /mnt/data bigdata \<br />
raidz \<br />
ata-ST3000DM001-9YN166_S1F0KDGY \<br />
ata-ST3000DM001-9YN166_S1F0JKRR \<br />
ata-ST3000DM001-9YN166_S1F0KBP8 \<br />
ata-ST3000DM001-9YN166_S1F0JTM1<br />
<br />
=== Advanced format disks ===<br />
<br />
In case Advanced Format disks are used which have a native sector size of 4096 bytes instead of 512 bytes, the automated sector size detection algorithm of ZFS might detect 512 bytes because the backwards compatibility with legacy systems. This would result in degraded performance. To make sure a correct sector size is used, the {{ic|<nowiki>ashift=12</nowiki>}} option should be used (See the [https://github.com/zfsonlinux/zfs/wiki/faq#advanced-format-disks ZFS on Linux FAQ]). The full command would in this case be:<br />
<br />
# zpool create -f -o ashift=12 -m /mnt/data bigdata \<br />
raidz \<br />
ata-ST3000DM001-9YN166_S1F0KDGY \<br />
ata-ST3000DM001-9YN166_S1F0JKRR \<br />
ata-ST3000DM001-9YN166_S1F0KBP8 \<br />
ata-ST3000DM001-9YN166_S1F0JTM1<br />
<br />
=== Verifying pool creation ===<br />
<br />
If the command is successful, there will be no output. Using the {{ic|$ mount}} command will show that the pool is mounted. Using {{ic|# zpool status}} will show that the pool has been created.<br />
<br />
{{hc|# zpool status|<br />
pool: bigdata<br />
state: ONLINE<br />
scan: none requested<br />
config:<br />
<br />
NAME STATE READ WRITE CKSUM<br />
bigdata ONLINE 0 0 0<br />
-0 ONLINE 0 0 0<br />
ata-ST3000DM001-9YN166_S1F0KDGY-part1 ONLINE 0 0 0<br />
ata-ST3000DM001-9YN166_S1F0JKRR-part1 ONLINE 0 0 0<br />
ata-ST3000DM001-9YN166_S1F0KBP8-part1 ONLINE 0 0 0<br />
ata-ST3000DM001-9YN166_S1F0JTM1-part1 ONLINE 0 0 0<br />
<br />
errors: No known data errors<br />
}}<br />
<br />
At this point it would be good to reboot the machine to ensure that the ZFS pool is mounted at boot. It is best to deal with all errors before transferring data.<br />
<br />
=== GRUB-compatible pool creation ===<br />
<br />
By default, ''zpool'' will enable all features on a pool. If {{ic|/boot}} resides on ZFS and when using [[GRUB]], you must only enable read-only, or non-read-only features supported by GRUB, otherwise GRUB will not be able to read the pool. As of GRUB 2.02.beta3, GRUB supports all features in ZFS-on-Linux 0.6.5.7. However, the Git master branch of ZoL contains one extra feature, {{ic|large_dnodes}} that is not yet supported by GRUB.<br />
<br />
This example line is only necessary if you are using the Git branch of ZoL:<br />
<br />
# zpool create -f -d \<br />
-o feature@async_destroy=enabled \<br />
-o feature@empty_bpobj=enabled \<br />
-o feature@lz4_compress=enabled \<br />
-o feature@spacemap_histogram=enabled \<br />
-o feature@enabled_txg=enabled \<br />
-o feature@hole_birth=enabled \<br />
-o feature@bookmarks=enabled \<br />
-o feature@filesystem_limits \<br />
-o feature@embedded_data=enabled \<br />
-o feature@large_blocks=enabled \<br />
<pool_name> <vdevs><br />
<br />
=== Importing a pool created by id ===<br />
<br />
Eventually a pool may fail to auto mount and you need to import to bring your pool back. Take care to avoid the most obvious solution.<br />
<br />
# ###zpool import zfsdata # Do not do this! Always use -d<br />
<br />
This will import your pools using {{ic|/dev/sd?}} which will lead to problems the next time you rearrange your drives. This may be as simple as rebooting with a USB drive left in the machine, which harkens back to a time when PC's would not boot when a floppy disk was left in a machine. Adapt one of the following commands to import your pool so that pool imports retain the persistence they were created with.<br />
<br />
# zpool import -d /dev/disk/by-id zfsdata<br />
# zpool import -d /dev/disk/by-partlabel zfsdata<br />
# zpool import -d /dev/disk/by-partuuid zfsdata<br />
<br />
== Tuning ==<br />
<br />
=== General ===<br />
Many parameters are available for zfs file systems, you can view a full list with {{ic|zfs get all <pool>}}. Two common ones to adjust are '''atime''' and '''compression'''.<br />
<br />
Atime is enabled by default but for most users, it represents superfluous writes to the zpool and it can be disabled using the zfs command:<br />
# zfs set atime=off <pool><br />
<br />
As an alternative to turning off atime completely, '''relatime''' is available. This brings the default ext4/xfs atime semantics to ZFS, where access time is only updated if the modified time or changed time changes, or if the existing access time has not been updated within the past 24 hours. It is a compromise between atime=off and atime=on. This property ''only'' takes effect if '''atime''' is '''on''':<br />
# zfs set relatime=on <pool><br />
<br />
Compression is just that, transparent compression of data. ZFS supports a few different algorithms, presently lz4 is the default. '''gzip''' is also available for seldom-written yet highly-compressable data; consult the man page for more details. Enable compression using the zfs command:<br />
# zfs set compression=on <pool><br />
<br />
Other options for zfs can be displayed again, using the zfs command:<br />
# zfs get all <pool><br />
<br />
=== SSD Caching ===<br />
You can also add SSD devices as a write intent log (external ZIL or SLOG) and also as a layer 2 adaptive replacement cache (l2arc). The process to add them is very similar to creating a new VDEV.<br />
<br />
All of the below references to device-id are the IDs from /dev/disk/by-id/*<br />
<br />
To add a ZIL:<br />
zpool add <pool> log <device-id><br />
<br />
or to add a mirrored ZIL:<br />
zpool add <pool> log mirror <device-id-1> <device-id-2><br />
<br />
<br />
To add an l2arc:<br />
zpool add <pool> cache <device-id><br />
<br />
or to add a mirrored l2arc:<br />
zpool add <pool> cache mirror <device-id-1> <device-id-2><br />
<br />
=== Database ===<br />
ZFS, unlike most other file systems, has a variable record size, or what is commonly referred to as a block size. By default, the recordsize on ZFS is 128KiB, which means it will dynamically allocate blocks of any size from 512B to 128KiB depending on the size of file being written. This can often help fragmentation and file access, at the cost that ZFS would have to allocate new 128KiB blocks each time only a few bytes are written to.<br />
<br />
Most RDBMSes work in 8KiB-sized blocks by default. Although the block size is tunable for [[MySQL|MySQL/MariaDB]], [[PostgreSQL]], and [[Oracle]], all three of them use an 8KiB block size ''by default''. For both performance concerns and keeping snapshot differences to a minimum (for backup purposes, this is helpful), it is usually desirable to tune ZFS instead to accommodate the databases, using a command such as:<br />
# zfs set recordsize=8K <pool>/postgres<br />
<br />
These RDBMSes also tend to implement their own caching algorithm, often similar to ZFS's own ARC. In the interest of saving memory, it is best to simply disable ZFS's caching of the database's file data and let the database do its own job:<br />
# zfs set primarycache=metadata <pool>/postgres<br />
<br />
If your pool has no configured log devices, ZFS reserves space on the pool's data disks for its intent log (the ZIL). ZFS uses this for crash recovery, but databases are often syncing their data files to the file system on their own transaction commits anyway. The end result of this is that ZFS will be committing data '''twice''' to the data disks, and it can severely impact performance. You can tell ZFS to prefer to not use the ZIL, and in which case, data is only committed to the file system once. Setting this for non-database file systems, or for pools with configured log devices, can actually ''negatively'' impact the performance, so beware:<br />
# zfs set logbias=throughput <pool>/postgres<br />
<br />
These can also be done at file system creation time, for example:<br />
# zfs create -o recordsize=8K \<br />
-o primarycache=metadata \<br />
-o mountpoint=/var/lib/postgres \<br />
-o logbias=throughput \<br />
<pool>/postgres<br />
<br />
Please note: these kinds of tuning parameters are ideal for specialized applications like RDBMSes. You can easily ''hurt'' ZFS's performance by setting these on a general-purpose file system such as your /home directory.<br />
<br />
=== /tmp ===<br />
If you would like to use ZFS to store your /tmp directory, which may be useful for storing arbitrarily-large sets of files or simply keeping your RAM free of idle data, you can generally improve performance of certain applications writing to /tmp by disabling file system sync. This causes ZFS to ignore an application's sync requests (eg, with {{ic|fsync}} or {{ic|O_SYNC}}) and return immediately. While this has severe application-side data consistency consequences (never disable sync for a database!), files in /tmp are less likely to be important and affected. Please note this does ''not'' affect the integrity of ZFS itself, only the possibility that data an application expects on-disk may not have actually been written out following a crash.<br />
# zfs set sync=disabled <pool>/tmp<br />
<br />
Additionally, for security purposes, you may want to disable '''setuid''' and '''devices''' on the /tmp file system, which prevents some kinds of privilege-escalation attacks or the use of device nodes:<br />
# zfs set setuid=off <pool>/tmp<br />
# zfs set devices=off <pool>/tmp<br />
<br />
Combining all of these for a create command would be as follows:<br />
# zfs create -o setuid=off -o devices=off -o sync=disabled -o mountpoint=/tmp <pool>/tmp<br />
<br />
Please note, also, that if you want /tmp on ZFS, you will need to mask (disable) [[systemd]]'s automatic tmpfs-backed /tmp, else ZFS will be unable to mount your dataset at boot-time or import-time:<br />
# systemctl mask tmp.mount<br />
<br />
=== ZVOLs ===<br />
<br />
ZFS volumes (ZVOLs) can suffer from the same block size-related issues as RDBMSes, but it is worth noting that the default recordsize for ZVOLs is 8KiB already. If possible, it is best to align any partitions contained in a ZVOL to your recordsize (current versions of fdisk and gdisk by default automatically align at 1MiB segments, which works), and file system block sizes to the same size. Other than this, you might tweak the '''recordsize''' to accommodate the data inside the ZVOL as necessary (though 8KiB tends to be a good value for most file systems, even when using 4KiB blocks on that level).<br />
<br />
==== RAIDZ and Advanced Format physical disks ====<br />
<br />
Each block of a ZVOL gets its own parity disks, and if you have physical media with logical block sizes of 4096B, 8192B, or so on, the parity needs to be stored in whole physical blocks, and this can drastically increase the space requirements of a ZVOL, requiring 2× or more physical storage capacity than the ZVOL's logical capacity. Setting the '''recordsize''' to 16k or 32k can help reduce this footprint drastically.<br />
<br />
See [https://github.com/zfsonlinux/zfs/issues/1807 ZFS on Linux issue #1807 for details]<br />
<br />
== Usage ==<br />
<br />
Users can optionally create a dataset under the zpool as opposed to manually creating directories under the zpool. Datasets allow for an increased level of control (quotas for example) in addition to snapshots. To be able to create and mount a dataset, a directory of the same name must not pre-exist in the zpool. To create a dataset, use:<br />
<br />
# zfs create <nameofzpool>/<nameofdataset><br />
<br />
It is then possible to apply ZFS specific attributes to the dataset. For example, one could assign a quota limit to a specific directory within a dataset:<br />
<br />
# zfs set quota=20G <nameofzpool>/<nameofdataset>/<directory><br />
<br />
To see all the commands available in ZFS, use :<br />
<br />
$ man zfs<br />
<br />
or:<br />
<br />
$ man zpool<br />
<br />
=== Scrub ===<br />
<br />
ZFS pools should be scrubbed at least once a week. To scrub the pool:<br />
<br />
# zpool scrub <pool><br />
<br />
To do automatic scrubbing once a week, set the following line in the root crontab:<br />
<br />
{{hc|# crontab -e|<br />
...<br />
30 19 * * 5 zpool scrub <pool><br />
...<br />
}}<br />
<br />
Replace {{ic|<pool>}} with the name of the ZFS pool.<br />
<br />
=== Check zfs pool status ===<br />
<br />
To print a nice table with statistics about the ZFS pool, including and read/write errors, use<br />
<br />
# zpool status -v<br />
<br />
=== Destroy a storage pool ===<br />
<br />
ZFS makes it easy to destroy a mounted storage pool, removing all metadata about the ZFS device. This command destroys any data contained in the pool:<br />
<br />
# zpool destroy <pool><br />
<br />
And now when checking the status:<br />
<br />
{{hc|# zpool status|no pools available}}<br />
<br />
To find the name of the pool, see [[#Check zfs pool status]].<br />
<br />
=== Export a storage pool ===<br />
<br />
If a storage pool is to be used on another system, it will first need to be exported. It is also necessary to export a pool if it has been imported from the archiso as the hostid is different in the archiso as it is in the booted system. The zpool command will refuse to import any storage pools that have not been exported. It is possible to force the import with the {{ic|-f}} argument, but this is considered bad form.<br />
<br />
Any attempts made to import an un-exported storage pool will result in an error stating the storage pool is in use by another system. This error can be produced at boot time abruptly abandoning the system in the busybox console and requiring an archiso to do an emergency repair by either exporting the pool, or adding the {{ic|zfs_force&#61;1}} to the kernel boot parameters (which is not ideal). See [[#On boot the zfs pool does not mount stating: "pool may be in use from other system"]]<br />
<br />
To export a pool,<br />
<br />
# zpool export bigdata<br />
<br />
=== Rename a Zpool ===<br />
Renaming a zpool that is already created is accomplished in 2 steps:<br />
<br />
# zpool export oldname<br />
# zpool import oldname newname<br />
<br />
=== Setting a Different Mount Point ===<br />
The mount point for a given zpool can be moved at will with one command:<br />
# zfs set mountpoint=/foo/bar poolname<br />
<br />
=== Swap volume ===<br />
<br />
ZFS does not allow to use swapfiles, but users can use a ZFS volume (ZVOL) as swap. It is importart to set the ZVOL block size to match the system page size, which can be obtained by the {{ic|getconf PAGESIZE}} command (default on x86_64 is 4KiB). Another option useful for keeping the system running well in low-memory situations is not caching the ZVOL data.<br />
<br />
Create a 8GiB zfs volume:<br />
<br />
# zfs create -V 8G -b $(getconf PAGESIZE) \<br />
-o primarycache=metadata \<br />
-o com.sun:auto-snapshot=false <pool>/swap<br />
<br />
Prepare it as swap partition:<br />
<br />
# mkswap -f /dev/zvol/<pool>/swap<br />
# swapon /dev/zvol/<pool>/swap<br />
<br />
To make it permanent, edit {{ic|/etc/fstab}}. ZVOLs support discard, which can potentially help ZFS's block allocator and reduce fragmentation for all other datasets when/if swap is not full.<br />
<br />
Add a line to {{ic|/etc/fstab}}:<br />
<br />
/dev/zvol/<pool>/swap none swap discard 0 0<br />
<br />
{{Out of date|The hibernate hook is deprecated. Does the limitation still apply?}}<br />
Keep in mind the Hibernate hook must be loaded before filesystems, so using ZVOL as swap will not allow to use hibernate function. If you need hibernate, keep a partition for it.<br />
<br />
=== Automatic snapshots ===<br />
<br />
==== ZFS Automatic Snapshot Service for Linux ====<br />
<br />
The {{AUR|zfs-auto-snapshot-git}} package from [[Arch User Repository|AUR]] provides a shell script to automate the management of snapshots, with each named by date and label (hourly, daily, etc), giving quick and convenient snapshotting of all ZFS datasets. The package also installs cron tasks for quarter-hourly, hourly, daily, weekly, and monthly snapshots. Optionally adjust the --keep parameter from the defaults depending on how far back the snapshots are to go (the monthly script by default keeps data for up to a year).<br />
<br />
To prevent a dataset from being snapshotted at all, set {{ic|1=com.sun:auto-snapshot=false}} on it. Likewise, set more fine-grained control as well by label, if, for example, no monthlies are to be kept on a snapshot, for example, set {{ic|1=com.sun:auto-snapshot:monthly=false}}.<br />
<br />
{{Note|zfs-auto-snapshot-git will not create snapshots during scrubbing ([[ZFS#Scrub|scrub]]). It is possible to override this by editing provided systemd unit ([[Systemd#Editing_provided_units]]) and removing `--skip-scrub` from `ExecStart` line. Consequences not known, someone please edit.<br />
}}<br />
<br />
==== ZFS Snapshot Manager ====<br />
<br />
The {{AUR|zfs-snap-manager}} package from [[Arch User Repository|AUR]] provides a python service that takes daily snapshots from a configurable set of ZFS datasets and cleans them out in a [[wikipedia:Backup_rotation_scheme#Grandfather-father-son|"Grandfather-father-son"]] scheme. It can be configured to e.g. keep 7 daily, 5 weekly, 3 monthly and 2 yearly snapshots. <br />
<br />
The package also supports configurable replication to other machines running ZFS by means of {{ic|zfs send}} and {{ic|zfs receive}}. If the destination machine runs this package as well, it could be configured to keep these replicated snapshots for a longer time. This allows a setup where a source machine has only a few daily snapshots locally stored, while on a remote storage server a much longer retention is available.<br />
<br />
==Troubleshooting==<br />
=== ZPool creation fails ===<br />
If the following error occurs then it can be fixed.<br />
<br />
# the kernel failed to rescan the partition table: 16<br />
# cannot label 'sdc': try using parted(8) and then provide a specific slice: -1<br />
<br />
One reason this can occur is because [https://github.com/zfsonlinux/zfs/issues/2582 ZFS expects pool creation to take less than 1 second]. This is a reasonable assumption under ordinary conditions, but in many situations it may take longer. Each drive will need to be cleared again before another attempt can be made.<br />
<br />
# parted /dev/sda rm 1<br />
# parted /dev/sda rm 1<br />
# dd if=/dev/zero of=/dev/sdb bs=512 count=1<br />
# zpool labelclear /dev/sda<br />
<br />
A brute force creation can be attempted over and over again, and with some luck the ZPool creation will take less than 1 second.<br />
Once cause for creation slowdown can be slow burst read writes on a drive. By reading from the disk in parallell to ZPool creation, it may be possible to increase burst speeds.<br />
<br />
# dd if=/dev/sda of=/dev/null<br />
<br />
This can be done with multiple drives by saving the above command for each drive to a file on separate lines and running <br />
<br />
# cat $FILE | parallel<br />
<br />
Then run ZPool creation at the same time.<br />
<br />
=== ZFS is using too much RAM ===<br />
<br />
By default, ZFS caches file operations ([[wikipedia:Adaptive replacement cache|ARC]]) using up to two-thirds of available system memory on the host. To adjust the ARC size, add the following to the [[Kernel parameters]] list:<br />
<br />
zfs.zfs_arc_max=536870912 # (for 512MB)<br />
<br />
For a more detailed description, as well as other configuration options, see [http://wiki.gentoo.org/wiki/ZFS#ARC gentoo-wiki:zfs#arc].<br />
<br />
=== Does not contain an EFI label ===<br />
<br />
The following error will occur when attempting to create a zfs filesystem,<br />
<br />
/dev/disk/by-id/<id> does not contain an EFI label but it may contain partition<br />
<br />
The way to overcome this is to use {{ic|-f}} with the zfs create command.<br />
<br />
=== No hostid found ===<br />
<br />
An error that occurs at boot with the following lines appearing before initscript output:<br />
<br />
ZFS: No hostid found on kernel command line or /etc/hostid.<br />
<br />
This warning occurs because the ZFS module does not have access to the spl hosted. There are two solutions, for this. Either place the spl hostid in the [[kernel parameters]] in the boot loader. For example, adding {{ic|1=spl.spl_hostid=0x00bab10c}}.<br />
<br />
The other solution is to make sure that there is a hostid in {{ic|/etc/hostid}}, and then regenerate the initramfs image. Which will copy the hostid into the initramfs image.<br />
<br />
# mkinitcpio -p linux<br />
<br />
=== On boot the zfs pool does not mount stating: "pool may be in use from other system" ===<br />
<br />
==== Unexported pool ====<br />
<br />
If the new installation does not boot because the zpool cannot be imported, chroot into the installation and properly export the zpool. See [[#Emergency chroot repair with archzfs]].<br />
<br />
Once inside the chroot environment, load the ZFS module and force import the zpool,<br />
<br />
# zpool import -a -f<br />
<br />
now export the pool:<br />
<br />
# zpool export <pool><br />
<br />
To see the available pools, use,<br />
<br />
# zpool status<br />
<br />
It is necessary to export a pool because of the way ZFS uses the hostid to track the system the zpool was created on. The hostid is generated partly based on the network setup. During the installation in the archiso the network configuration could be different generating a different hostid than the one contained in the new installation. Once the zfs filesystem is exported and then re-imported in the new installation, the hostid is reset. See [http://osdir.com/ml/zfs-discuss/2011-06/msg00227.html Re: Howto zpool import/export automatically? - msg#00227].<br />
<br />
If ZFS complains about "pool may be in use" after every reboot, properly export pool as described above, and then rebuild ramdisk in normally booted system:<br />
<br />
# mkinitcpio -p linux<br />
<br />
==== Incorrect hostid ====<br />
<br />
Double check that the pool is properly exported. Exporting the zpool clears the hostid marking the ownership. So during the first boot the zpool should mount correctly. If it does not there is some other problem.<br />
<br />
Reboot again, if the zfs pool refuses to mount it means the hostid is not yet correctly set in the early boot phase and it confuses zfs. Manually tell zfs the correct number, once the hostid is coherent across the reboots the zpool will mount correctly.<br />
<br />
Boot using zfs_force and write down the hostid. This one is just an example.<br />
% hostid<br />
0a0af0f8<br />
<br />
This number have to be added to the [[kernel parameters]] as {{ic|spl.spl_hostid&#61;0x0a0af0f8}}. Another solution is writing the hostid inside the initram image, see the [[Installing Arch Linux on ZFS#After the first boot|installation guide]] explanation about this.<br />
<br />
Users can always ignore the check adding {{ic|zfs_force&#61;1}} in the [[kernel parameters]], but it is not advisable as a permanent solution.<br />
<br />
=== Devices have different sector alignment ===<br />
<br />
Once a drive has become faulted it should be replaced A.S.A.P. with an identical drive.<br />
<br />
# zpool replace bigdata ata-ST3000DM001-9YN166_S1F0KDGY ata-ST3000DM001-1CH166_W1F478BD -f<br />
<br />
but in this instance, the following error is produced:<br />
<br />
cannot replace ata-ST3000DM001-9YN166_S1F0KDGY with ata-ST3000DM001-1CH166_W1F478BD: devices have different sector alignment<br />
<br />
ZFS uses the ashift option to adjust for physical block size. When replacing the faulted disk, ZFS is attempting to use {{ic|<nowiki>ashift=12</nowiki>}}, but the faulted disk is using a different ashift (probably {{ic|<nowiki>ashift=9</nowiki>}}) and this causes the resulting error. <br />
<br />
For Advanced Format Disks with 4KB blocksize, an ashift of 12 is recommended for best performance. See [https://github.com/zfsonlinux/zfs/wiki/faq#performance-considerations 1.10 What’s going on with performance?] and [http://wiki.illumos.org/display/illumos/ZFS+and+Advanced+Format+disks ZFS and Advanced Format disks].<br />
<br />
Use zdb to find the ashift of the zpool: {{ic|zdb | grep ashift}}, then use the {{ic|-o}} argument to set the ashift of the replacement drive:<br />
<br />
# zpool replace bigdata ata-ST3000DM001-9YN166_S1F0KDGY ata-ST3000DM001-1CH166_W1F478BD -o ashift=9 -f<br />
<br />
Check the zpool status for confirmation:<br />
<br />
{{hc|# zpool status -v|<br />
pool: bigdata<br />
state: DEGRADED<br />
status: One or more devices is currently being resilvered. The pool will<br />
continue to function, possibly in a degraded state.<br />
action: Wait for the resilver to complete.<br />
scan: resilver in progress since Mon Jun 16 11:16:28 2014<br />
10.3G scanned out of 5.90T at 81.7M/s, 20h59m to go<br />
2.57G resilvered, 0.17% done<br />
config:<br />
<br />
NAME STATE READ WRITE CKSUM<br />
bigdata DEGRADED 0 0 0<br />
raidz1-0 DEGRADED 0 0 0<br />
replacing-0 OFFLINE 0 0 0<br />
ata-ST3000DM001-9YN166_S1F0KDGY OFFLINE 0 0 0<br />
ata-ST3000DM001-1CH166_W1F478BD ONLINE 0 0 0 (resilvering)<br />
ata-ST3000DM001-9YN166_S1F0JKRR ONLINE 0 0 0<br />
ata-ST3000DM001-9YN166_S1F0KBP8 ONLINE 0 0 0<br />
ata-ST3000DM001-9YN166_S1F0JTM1 ONLINE 0 0 0<br />
<br />
errors: No known data errors}}<br />
<br />
== Tips and tricks ==<br />
<br />
===Embed the archzfs packages into an archiso===<br />
<br />
Follow the [[Archiso]] steps for creating a fully functional Arch Linux live CD/DVD/USB image.<br />
<br />
Enable the [[Unofficial user repositories#archzfs|archzfs]] repository:<br />
<br />
{{hc|~/archlive/pacman.conf|<nowiki><br />
...<br />
[archzfs]<br />
Server = http://archzfs.com/$repo/x86_64<br />
</nowiki>}}<br />
<br />
Add the {{ic|archzfs-linux}} group to the list of packages to be installed (the {{ic|archzfs}} repository provides packages for the x86_64 architecture only).<br />
<br />
{{hc|~/archlive/packages.x86_64|<br />
...<br />
archzfs-linux<br />
}}<br />
<br />
Complete [[Archiso#Build_the_ISO|Build the ISO]] to finally build the iso.<br />
<br />
=== Encryption in ZFS on linux ===<br />
<br />
ZFS on linux does not support encryption directly, but zpools can be created in dm-crypt block devices. Since the zpool is created on the plain-text<br />
abstraction it is possible to have the data encrypted while having all the advantages of ZFS like deduplication, compression, and data robustness.<br />
<br />
dm-crypt, possibly via LUKS, creates devices in {{ic|/dev/mapper}} and their name is fixed. So you just need to change {{ic|zpool create}} commands to<br />
point to that names. The idea is configuring the system to create the {{ic|/dev/mapper}} block devices and import the zpools from there.<br />
Since zpools can be created in multiple devices (raid, mirroring, striping, ...), it is important all the devices are encrypted otherwise the protection<br />
might be partially lost.<br />
<br />
For example, an encrypted zpool can be created using plain dm-crypt (without LUKS) with:<br />
<br />
# cryptsetup --hash=sha512 --cipher=twofish-xts-plain64 --offset=0 \<br />
--key-file=/dev/sdZ --key-size=512 open --type=plain /dev/sdX enc<br />
# zpool create zroot /dev/mapper/enc<br />
<br />
In the case of a root filesystem pool, the {{ic|mkinicpio.conf}} HOOKS line will enable the keyboard for the password, create the devices, and load the pools. It will contain something like:<br />
<br />
HOOKS="... keyboard encrypt zfs ..."<br />
<br />
Since the {{ic|/dev/mapper/enc}} name is fixed no import errors will occur.<br />
<br />
Creating encrypted zpools works fine. But if you need encrypted directories, for example to protect your users' homes, ZFS loses some functionality.<br />
<br />
ZFS will see the encrypted data, not the plain-text abstraction, so compression and deduplication will not work. The reason is that encrypted data has always high entropy making compression ineffective and even from the same input you get different output (thanks to salting) making deduplication impossible.<br />
To reduce the unnecessary overhead it is possible to create a sub-filesystem for each encrypted directory and use [[eCryptfs]] on it.<br />
<br />
For example to have an encrypted home: (the two passwords, encryption and login, must be the same)<br />
# zfs create -o compression=off \<br />
-o dedup=off \<br />
-o mountpoint=/home/<username> \<br />
<zpool>/<username><br />
# useradd -m <username><br />
# passwd <username><br />
# ecryptfs-migrate-home -u <username><br />
<log in user and complete the procedure with ecryptfs-unwrap-passphrase><br />
<br />
=== Emergency chroot repair with archzfs ===<br />
<br />
To get into the ZFS filesystem from live system for maintenance, there are two options:<br />
<br />
# Build custom archiso with ZFS as described in [[#Embed the archzfs packages into an archiso]].<br />
# Boot the latest official archiso and bring up the network. Then enable [[Unofficial_user_repositories#archzfs|archzfs]] repository inside the live system as usual, sync the pacman package database and install the ''archzfs-archiso-linux'' package.<br />
<br />
To start the recovery, load the ZFS kernel modules:<br />
<br />
# modprobe zfs<br />
<br />
Import the pool:<br />
<br />
# zpool import -a -R /mnt<br />
<br />
Mount the boot partitions (if any):<br />
<br />
# mount /dev/sda2 /mnt/boot<br />
# mount /dev/sda1 /mnt/boot/efi<br />
<br />
Chroot into the ZFS filesystem:<br />
<br />
# arch-chroot /mnt /bin/bash<br />
<br />
Check the kernel version:<br />
<br />
# pacman -Qi linux<br />
# uname -r<br />
<br />
uname will show the kernel version of the archiso. If they are different, run depmod (in the chroot) with the correct kernel version of the chroot installation:<br />
<br />
# depmod -a 3.6.9-1-ARCH (version gathered from pacman -Qi linux but using the matching kernel modules directory name under the chroot's /lib/modules)<br />
<br />
This will load the correct kernel modules for the kernel version installed in the chroot installation.<br />
<br />
Regenerate the ramdisk:<br />
<br />
# mkinitcpio -p linux<br />
<br />
There should be no errors.<br />
<br />
=== Bindmount ===<br />
Here a bind mount from /mnt/zfspool to /srv/nfs4/music is created. The configuration ensures that the zfs pool is ready before the bind mount is created.<br />
<br />
==== fstab ====<br />
See [http://www.freedesktop.org/software/systemd/man/systemd.mount.html systemd.mount] for more information on how systemd converts fstab into mount unit files with [http://www.freedesktop.org/software/systemd/man/systemd-fstab-generator.html systemd-fstab-generator].<br />
<br />
{{hc|/etc/fstab|<nowiki><br />
/mnt/zfspool /srv/nfs4/music none bind,defaults,nofail,x-systemd.requires=zfs-mount.service 0 0<br />
</nowiki>}}<br />
<br />
==== systemd mount unit ====<br />
<br />
If it is not possible to bindmount a directory residing on zfs onto another directory using fstab, because the fstab is read before the zfs pool is ready, you can overcome this limitation with a systemd mount unit can be used for the bind mount. The name of the mount unit must be equal to the directory mentioned after "Where", replace slashes with minuses. See [[http://utcc.utoronto.ca/~cks/space/blog/linux/SystemdAndBindMounts]] and [[http://utcc.utoronto.ca/~cks/space/blog/linux/SystemdBindMountUnits]] for more details.<br />
{{hc|srv-nfs4-music.mount|<nowiki><br />
[Mount]<br />
What=/mnt/zfspool<br />
Where=/srv/nfs4/music<br />
Type=none<br />
Options=bind<br />
<br />
[Unit]<br />
DefaultDependencies=no<br />
Conflicts=umount.target<br />
Before=local-fs.target umount.target<br />
After=zfs-mount.service<br />
Requires=zfs-mount.service<br />
ConditionPathIsDirectory=/mnt/zfspool<br />
<br />
[Install]<br />
WantedBy=local-fs.target<br />
</nowiki>}}<br />
<br />
== See also ==<br />
<br />
* [[Installing Arch Linux on ZFS]]<br />
* [http://zfsonlinux.org/ ZFS on Linux]<br />
* [https://github.com/zfsonlinux/zfs/wiki/faq ZFS on Linux FAQ]<br />
* [https://www.freebsd.org/doc/en_US.ISO8859-1/books/handbook/zfs.html FreeBSD Handbook -- The Z File System]<br />
* [http://docs.oracle.com/cd/E19253-01/819-5461/index.html Oracle Solaris ZFS Administration Guide]<br />
* [http://www.solarisinternals.com/wiki/index.php/ZFS_Troubleshooting_Guide Solaris Internals -- ZFS Troubleshooting Guide]<br />
* [http://royal.pingdom.com/2013/06/04/zfs-backup/ Pingdom details how it backs up 5TB of MySQL data every day with ZFS]<br />
* [https://www.linuxquestions.org/questions/linux-from-scratch-13/%5Bhow-to%5D-add-zfs-to-the-linux-kernel-4175514510/ Tutorial on adding the modules to a custom kernel]<br />
<br />
; Aaron Toponce has authored a 17-part blog on ZFS which is an excellent read.<br />
# [https://pthree.org/2012/12/04/zfs-administration-part-i-vdevs/ VDEVs]<br />
# [https://pthree.org/2012/12/05/zfs-administration-part-ii-raidz/ RAIDZ Levels]<br />
# [https://pthree.org/2012/12/06/zfs-administration-part-iii-the-zfs-intent-log/ The ZFS Intent Log]<br />
# [https://pthree.org/2012/12/07/zfs-administration-part-iv-the-adjustable-replacement-cache/ The ARC]<br />
# [https://pthree.org/2012/12/10/zfs-administration-part-v-exporting-and-importing-zpools/ Import/export zpools]<br />
# [https://pthree.org/2012/12/11/zfs-administration-part-vi-scrub-and-resilver/ Scrub and Resilver]<br />
# [https://pthree.org/2012/12/12/zfs-administration-part-vii-zpool-properties/ Zpool Properties]<br />
# [https://pthree.org/2012/12/13/zfs-administration-part-viii-zpool-best-practices-and-caveats/ Zpool Best Practices]<br />
# [https://pthree.org/2012/12/14/zfs-administration-part-ix-copy-on-write/ Copy on Write]<br />
# [https://pthree.org/2012/12/17/zfs-administration-part-x-creating-filesystems/ Creating Filesystems]<br />
# [https://pthree.org/2012/12/18/zfs-administration-part-xi-compression-and-deduplication/ Compression and Deduplication]<br />
# [https://pthree.org/2012/12/19/zfs-administration-part-xii-snapshots-and-clones/ Snapshots and Clones]<br />
# [https://pthree.org/2012/12/20/zfs-administration-part-xiii-sending-and-receiving-filesystems/ Send/receive Filesystems]<br />
# [https://pthree.org/2012/12/21/zfs-administration-part-xiv-zvols/ ZVOLs]<br />
# [https://pthree.org/2012/12/31/zfs-administration-part-xv-iscsi-nfs-and-samba/ iSCSI, NFS, and Samba]<br />
# [https://pthree.org/2013/01/02/zfs-administration-part-xvi-getting-and-setting-properties/ Get/Set Properties]<br />
# [https://pthree.org/2013/01/03/zfs-administration-part-xvii-best-practices-and-caveats/ ZFS Best Practices]</div>Justin8https://wiki.archlinux.org/index.php?title=Talk:ZFS&diff=455305Talk:ZFS2016-10-27T22:05:08Z<p>Justin8: /* Configuration */ Response</p>
<hr />
<div>== Bindmount ==<br />
Where does this file go and what other steps are required?<br />
<br />
I would expect: /etc/systemd/system/<br />
<br />
Then: systemctl enable srv-nfs4-media.mount<br />
<br />
[[User:Msalerno|Msalerno]] ([[User talk:Msalerno|talk]]) 02:36, 22 October 2015 (UTC)<br />
<br />
== resume hook ==<br />
In think in the page is a typo, the page should state ''resume hook'' instead of hibernate, but the limitation still applies. Can anyone confirm that the resume hook must appear before filesystems? [[User:Ezzetabi|Ezzetabi]] ([[User talk:Ezzetabi|talk]]) 09:49, 18 August 2015 (UTC)<br />
<br />
== Automatic build script ==<br />
<br />
I'm fine with deleting the scripts. I only posted it because graysky's script never worked for me. Long stuff like this would be useful if the ArchWiki featured roll-up text. [[User:Severach|Severach]] ([[User talk:Severach|talk]]) 10:07, 9 August 2015 (UTC)<br />
<br />
:I'd suggest to maintain it in a github repo. You get better versioning, syntax highlighting, cloning, etc. -- [[User:Alad|Alad]] ([[User talk:Alad|talk]]) 12:46, 9 August 2015 (UTC)<br />
<br />
::...or an [https://help.github.com/articles/about-gists/#anonymous-gists anonymous gist] if you don't have nor want to create a GitHub account. — [[User:Kynikos|Kynikos]] ([[User talk:Kynikos|talk]]) 08:40, 10 August 2015 (UTC)<br />
<br />
:Isn't that exactly what DKMS is doing? There DKMS packages in the AUR. [[User:Das j|Das j]] ([[User talk:Das j|talk]]) 20:01, 10 January 2016 (UTC)<br />
<br />
== Automatic snapshots ==<br />
{{AUR|zfs-auto-snapshot-git}} seems to have disappeared from the AUR. I haven't been able to find any information on why it was deleted; does anyone know? In any case, it should probably be removed from this page.<br />
[[User:Warai otoko|warai otoko]] ([[User talk:Warai otoko|talk]]) 03:21, 2 September 2015 (UTC)<br />
<br />
:On further inspection, looks like it may have gotten lost in the transition to AUR4. It should be resubmitted if we want to continue recommending it here; I've found it useful, at any rate. [[User:Warai otoko|Warai otoko]] ([[User talk:Warai otoko|talk]]) 04:43, 2 September 2015 (UTC)<br />
<br />
:: I've recreated it. I use this script as well. --[[User:Chungy|Chungy]] ([[User talk:Chungy|talk]]) 02:49, 3 September 2015 (UTC)<br />
<br />
== Configuration ==<br />
<br />
The configuration section has WAY to few infos about what systemd unit(s) to enable. Thanks to @kerberizer I finally managed to get the mounts working with the command <br />
<br />
# systemctl preset $(tail -n +2 /usr/lib/systemd/system-preset/50-zfs.preset | cut -d ' ' -f 2)<br />
<br />
[[User:Z3ntu|Z3ntu]] ([[User talk:Z3ntu|talk]]) 15:21, 27 October 2016 (UTC)<br />
<br />
<br />
@Z3ntul I have ZFS running on a few systems and never had to enable any services, it should work by default, if not then file a bug on the package<br />
<br />
[[User:Justin8|Justin8]] ([[User talk:Justin8|talk]]) 22:04, 27 October 2016 (UTC)</div>Justin8https://wiki.archlinux.org/index.php?title=ZFS&diff=436775ZFS2016-05-31T00:50:16Z<p>Justin8: </p>
<hr />
<div>[[Category:File systems]]<br />
[[ja:ZFS]]<br />
{{Related articles start}}<br />
{{Related|File systems}}<br />
{{Related|Experimenting with ZFS}}<br />
{{Related|Installing Arch Linux on ZFS}}<br />
{{Related|ZFS on FUSE}}<br />
{{Related articles end}}<br />
<br />
[[Wikipedia:ZFS|ZFS]] is an advanced filesystem created by [[Wikipedia:Sun Microsystems|Sun Microsystems]] (now owned by Oracle) and released for OpenSolaris in November 2005. <br />
<br />
Features of ZFS include: pooled storage (integrated volume management – zpool), [[Wikipedia:Copy-on-write|Copy-on-write]], [[Wikipedia:Snapshot (computer storage)|snapshots]], data integrity verification and automatic repair (scrubbing), [[Wikipedia:RAID-Z|RAID-Z]], a maximum [[Wikipedia:Exabyte|16 Exabyte]] file size, and a maximum 256 Quadrillion [[Wikipedia:Zettabyte|Zettabytes]] storage with no limit on number of filesystems (datasets) or files[http://docs.oracle.com/cd/E19253-01/819-5461/zfsover-2/index.html]. ZFS is licensed under the [[Wikipedia:CDDL|Common Development and Distribution License]] (CDDL).<br />
<br />
Described as [http://web.archive.org/web/20060428092023/http://www.sun.com/2004-0914/feature/ "The last word in filesystems"] ZFS is stable, fast, secure, and future-proof. Being licensed under the CDDL, and thus incompatible with GPL, it is not possible for ZFS to be distributed along with the Linux Kernel. This requirement, however, does not prevent a native Linux kernel module from being developed and distributed by a third party, as is the case with [http://zfsonlinux.org/ zfsonlinux.org] (ZOL).<br />
<br />
ZOL is a project funded by the [https://www.llnl.gov/ Lawrence Livermore National Laboratory] to develop a native Linux kernel module for its massive storage requirements and super computers.<br />
<br />
==Installation==<br />
=== General ===<br />
Install {{AUR|zfs-linux-git}} from the [[Arch User Repository]] or the [[Unofficial user repositories#archzfs|archzfs]] repository. This package has {{AUR|zfs-utils-linux-git}} and {{AUR|spl-linux-git}} as a dependency, which in turn has {{AUR|spl-utils-linux-git}} as dependency. SPL (Solaris Porting Layer) is a Linux Kernel module implementing Solaris APIs for ZFS compatibility.<br />
<br />
For users that desire ZFS builds from stable releases, {{AUR|zfs-linux-lts}} is available from the [[Arch User Repository]] or the [[Unofficial user repositories#archzfs|archzfs]] repository. A script to build ZFS and its dependencies automatically can be found [[#Automated build script|here]].<br />
<br />
{{warning|The ZFS and SPL kernel modules are tied to a specific kernel version. It would not be possible to apply any kernel updates until updated packages are uploaded to AUR or the [[Unofficial user repositories#archzfs|archzfs]] repository.}}<br />
<br />
Test the installation by issuing {{ic|zpool status}} on the command line. If an "insmod" error is produced, try {{ic|depmod -a}}.<br />
<br />
{{Tip| You can [[downgrade]] your linux version to the one from [[Unofficial user repositories#archzfs|archzfs]] repo if your current kenel is newer.}}<br />
<br />
=== Root on ZFS ===<br />
<br />
When performing an Arch install on ZFS, {{AUR|zfs-git}}{{Broken package link|{{aur-mirror|zfs-git}}}} and its dependencies can be installed in the archiso environment as outlined in the previous section.<br />
<br />
It may be useful to prepare a [[#Embed the archzfs packages into an archiso|customized archiso]] with ZFS support builtin. For a much more detailed guide on installing Arch with ZFS as its root file system, see [[Installing Arch Linux on ZFS]].<br />
<br />
=== DKMS ===<br />
<br />
Users can make use of DKMS [[Dynamic Kernel Module Support]] to rebuild the ZFS modules automatically with every kernel upgrade. <br />
<br />
Install {{AUR|zfs-dkms}} or {{AUR|zfs-dkms-git}} and apply the post-install instructions given by these packages.<br />
<br />
{{Tip|Add an {{ic|IgnorePkg}} entry to [[pacman.conf]] to prevent these packages from upgrading when doing a regular update.}}<br />
<br />
==Experimenting with ZFS ==<br />
<br />
Users wishing to experiment with ZFS on ''virtual block devices'' (known in ZFS terms as VDEVs) which can be simple files like {{ic|~/zfs0.img}} {{ic|~/zfs1.img}} {{ic|~/zfs2.img}} etc. with no possibility of real data loss are encouraged to see the [[Experimenting with ZFS]] article. Common tasks like building a RAIDZ array, purposefully corrupting data and recovering it, snapshotting datasets, etc. are covered.<br />
<br />
==Configuration==<br />
<br />
ZFS is considered a "zero administration" filesystem by its creators; therefore, configuring ZFS is very straight forward. Configuration is done primarily with two commands: {{ic|zfs}} and {{ic|zpool}}.<br />
<br />
===Automatic Start===<br />
<br />
For ZFS to live by its "zero administration" namesake, the zfs daemon must be loaded at startup. A benefit to this is that it is not necessary to mount the zpool in {{ic|/etc/fstab}}; the zfs daemon can import and mount zfs pools automatically. The daemon mounts the zfs pools reading the file {{ic|/etc/zfs/zpool.cache}}.<br />
<br />
For each pool you want automatically mounted by the zfs daemon execute:<br />
# zpool set cachefile=/etc/zfs/zpool.cache <pool><br />
<br />
Enable the service so it is automatically started at boot time:<br />
<br />
# systemctl enable zfs.target<br />
<br />
To manually start the daemon:<br />
<br />
# systemctl start zfs.target<br />
<br />
==Create a storage pool==<br />
<br />
Use {{ic| # parted --list}} to see a list of all available drives. It is not necessary nor recommended to partition the drives before creating the zfs filesystem.<br />
<br />
{{Note|If some or all device have been used in a software RAID set it is paramount to erase any old RAID configuration information. ([[Mdadm#Prepare_the_Devices]]) }}<br />
<br />
{{Warning|For Advanced Format Disks with 4KB sector size, an ashift of 12 is recommended for best performance. Advanced Format disks emulate a sector size of 512 bytes for compatibility with legacy systems, this causes ZFS to sometimes use an ashift option number that is not ideal. Once the pool has been created, the only way to change the ashift option is to recreate the pool. Using an ashift of 12 would also decrease available capacity. See [http://zfsonlinux.org/faq.html#PerformanceConsideration 1.10 What’s going on with performance?], [http://zfsonlinux.org/faq.html#HowDoesZFSonLinuxHandleAdvacedFormatDrives 1.15 How does ZFS on Linux handle Advanced Format disks?], and [http://wiki.illumos.org/display/illumos/ZFS+and+Advanced+Format+disks ZFS and Advanced Format disks].}}<br />
<br />
Having identified the list of drives, it is now time to get the id's of the drives to add to the zpool. The [http://zfsonlinux.org/faq.html#WhatDevNamesShouldIUseWhenCreatingMyPool zfs on Linux developers recommend] using device ids when creating ZFS storage pools of less than 10 devices. To find the id's, simply:<br />
<br />
# ls -lh /dev/disk/by-id/<br />
<br />
The ids should look similar to the following:<br />
<br />
lrwxrwxrwx 1 root root 9 Aug 12 16:26 ata-ST3000DM001-9YN166_S1F0JKRR -> ../../sdc<br />
lrwxrwxrwx 1 root root 9 Aug 12 16:26 ata-ST3000DM001-9YN166_S1F0JTM1 -> ../../sde<br />
lrwxrwxrwx 1 root root 9 Aug 12 16:26 ata-ST3000DM001-9YN166_S1F0KBP8 -> ../../sdd<br />
lrwxrwxrwx 1 root root 9 Aug 12 16:26 ata-ST3000DM001-9YN166_S1F0KDGY -> ../../sdb<br />
<br />
{{Style|Missing references to [[Persistent block device naming]], it is useless to explain the differences (or even what they are) here.}}<br />
<br />
Disk labels and UUID can also be used for ZFS mounts by using [[GUID Partition Table|GPT]] partitions. ZFS drives have labels but Linux is unable to read them at boot. Unlike [[Master Boot Record|MBR]] partitions, GPT partitions directly support both UUID and labels independent of the format inside the partition. Partitioning rather than using the whole disk for ZFS offers two additional advantages. The OS does not generate bogus partition numbers from whatever unpredictable data ZFS has written to the partition sector, and if desired, you can easily over provision SSD drives, and slightly over provision spindle drives to ensure that different models with slightly different sector counts can zpool replace into your mirrors. This is a lot of organization and control over ZFS using readily available tools and techniques at almost zero cost.<br />
<br />
Use [[GUID Partition Table|gdisk]] to partition the all or part of the drive as a single partition. gdisk does not automatically name partitions so if partition labels are desired use gdisk command "c" to label the partitions. Some reasons you might prefer labels over UUID are: labels are easy to control, labels can be titled to make the purpose of each disk in your arrangement readily apparent, and labels are shorter and easier to type. These are all advantages when the server is down and the heat is on. GPT partition labels have plenty of space and can store most international characters [[wikipedia:GUID_Partition_Table#Partition_entries]] allowing large data pools to be labeled in an organized fashion.<br />
<br />
Drives partitioned with GPT have labels and UUID that look like this. <br />
<br />
# ls -l /dev/disk/by-partlabel<br />
# lrwxrwxrwx 1 root root 10 Apr 30 01:44 zfsdata1 -> ../../sdd1<br />
# lrwxrwxrwx 1 root root 10 Apr 30 01:44 zfsdata2 -> ../../sdc1<br />
# lrwxrwxrwx 1 root root 10 Apr 30 01:59 zfsl2arc -> ../../sda1<br />
<br />
# ls -l /dev/disk/by-partuuid<br />
# lrwxrwxrwx 1 root root 10 Apr 30 01:44 148c462c-7819-431a-9aba-5bf42bb5a34e -> ../../sdd1<br />
# lrwxrwxrwx 1 root root 10 Apr 30 01:59 4f95da30-b2fb-412b-9090-fc349993df56 -> ../../sda1<br />
# lrwxrwxrwx 1 root root 10 Apr 30 01:44 e5ccef58-5adf-4094-81a7-3bac846a885f -> ../../sdc1<br />
<br />
Now, finally, create the ZFS pool:<br />
<br />
# zpool create -f -m <mount> <pool> raidz <ids><br />
<br />
* '''create''': subcommand to create the pool.<br />
<br />
* '''-f''': Force creating the pool. This is to overcome the "EFI label error". See [[#Does not contain an EFI label]].<br />
<br />
* '''-m''': The mount point of the pool. If this is not specified, then the pool will be mounted to {{ic|/<pool>}}.<br />
<br />
* '''pool''': This is the name of the pool.<br />
<br />
* '''raidz''': This is the type of virtual device that will be created from the pool of devices. Raidz is a special implementation of raid5. See [https://blogs.oracle.com/bonwick/entry/raid_z Jeff Bonwick's Blog -- RAID-Z] for more information about raidz.<br />
<br />
* '''ids''': The names of the drives or partitions that to include into the pool. Get it from {{ic|/dev/disk/by-id}}.<br />
<br />
Here is an example for the full command:<br />
<br />
# zpool create -f -m /mnt/data bigdata \<br />
raidz \<br />
ata-ST3000DM001-9YN166_S1F0KDGY \<br />
ata-ST3000DM001-9YN166_S1F0JKRR \<br />
ata-ST3000DM001-9YN166_S1F0KBP8 \<br />
ata-ST3000DM001-9YN166_S1F0JTM1<br />
<br />
=== Advanced format disks ===<br />
<br />
In case Advanced Format disks are used which have a native sector size of 4096 bytes instead of 512 bytes, the automated sector size detection algorithm of ZFS might detect 512 bytes because the backwards compatibility with legacy systems. This would result in degraded performance. To make sure a correct sector size is used, the {{ic|<nowiki>ashift=12</nowiki>}} option should be used (See the [http://zfsonlinux.org/faq.html#HowDoesZFSonLinuxHandleAdvacedFormatDrives ZFS on Linux FAQ]). The full command would in this case be:<br />
<br />
# zpool create -f -o ashift=12 -m /mnt/data bigdata \<br />
raidz \<br />
ata-ST3000DM001-9YN166_S1F0KDGY \<br />
ata-ST3000DM001-9YN166_S1F0JKRR \<br />
ata-ST3000DM001-9YN166_S1F0KBP8 \<br />
ata-ST3000DM001-9YN166_S1F0JTM1<br />
<br />
=== Verifying pool creation ===<br />
<br />
If the command is successful, there will be no output. Using the {{ic|$ mount}} command will show that the pool is mounted. Using {{ic|# zpool status}} will show that the pool has been created.<br />
<br />
{{hc|# zpool status|<br />
pool: bigdata<br />
state: ONLINE<br />
scan: none requested<br />
config:<br />
<br />
NAME STATE READ WRITE CKSUM<br />
bigdata ONLINE 0 0 0<br />
-0 ONLINE 0 0 0<br />
ata-ST3000DM001-9YN166_S1F0KDGY-part1 ONLINE 0 0 0<br />
ata-ST3000DM001-9YN166_S1F0JKRR-part1 ONLINE 0 0 0<br />
ata-ST3000DM001-9YN166_S1F0KBP8-part1 ONLINE 0 0 0<br />
ata-ST3000DM001-9YN166_S1F0JTM1-part1 ONLINE 0 0 0<br />
<br />
errors: No known data errors<br />
}}<br />
<br />
At this point it would be good to reboot the machine to ensure that the ZFS pool is mounted at boot. It is best to deal with all errors before transferring data.<br />
<br />
=== GRUB-compatible pool creation ===<br />
<br />
By default, ''zpool'' will enable all features on a pool. If {{ic|/boot}} resides on ZFS and when using [[GRUB]], you must only enable read-only, or non-read-only features supported by GRUB ({{ic|lz4_compress}} as of version 2.02.beta2). Otherwise GRUB will not be able to read the pool.<br />
<br />
# zpool create -f -d \<br />
-o feature@async_destroy=enabled \<br />
-o feature@empty_bpobj=enabled \<br />
-o feature@lz4_compress=enabled \<br />
-o feature@spacemap_histogram=enabled \<br />
-o feature@enabled_txg=enabled \<br />
<pool_name> <vdevs><br />
<br />
To create a mirrored vdev for GRUB you would use a command like this:<br />
<br />
# zpool create <pool_name> mirror -f -d \<br />
-o feature@async_destroy=enabled \<br />
-o feature@empty_bpobj=enabled \<br />
-o feature@lz4_compress=enabled \<br />
-o feature@spacemap_histogram=enabled \<br />
-o feature@enabled_txg=enabled \<br />
/dev/disk/by-id/ata-disk-part2 /dev/disk/by-id/ata-disk-part2<br />
<br />
{{Tip|As of September 2015, GRUB's development tree supports {{ic|extensible_dataset}}, {{ic|hole_birth}}, {{ic|embedded_data}}, and {{ic|large_blocks}}, making it viable to use a pool with all features enabled, either at create time or by using {{ic|zpool upgrade <pool_name>}}, if {{AUR|grub-git}} is installed.}}<br />
<br />
=== Importing a pool created by id ===<br />
<br />
Eventually a pool may fail to auto mount and you need to import to bring your pool back. Take care to avoid the most obvious solution.<br />
<br />
# ###zpool import zfsdata # Do not do this! Always use -d<br />
<br />
This will import your pools using {{ic|/dev/sd?}} which will lead to problems the next time you rearrange your drives. This may be as simple as rebooting with a USB drive left in the machine, which harkens back to a time when PC's would not boot when a floppy disk was left in a machine. Adapt one of the following commands to import your pool so that pool imports retain the persistence they were created with.<br />
<br />
# zpool import -d /dev/disk/by-id zfsdata<br />
# zpool import -d /dev/disk/by-partlabel zfsdata<br />
# zpool import -d /dev/disk/by-partuuid zfsdata<br />
<br />
== Tuning ==<br />
<br />
=== General ===<br />
Many parameters are available for zfs file systems, you can view a full list with {{ic|zfs get all <pool>}}. Two common ones to adjust are '''atime''' and '''compression'''.<br />
<br />
Atime is enabled by default but for most users, it represents superfluous writes to the zpool and it can be disabled using the zfs command:<br />
# zfs set atime=off <pool><br />
<br />
As an alternative to turning off atime completely, '''relatime''' is available. This brings the default ext4/xfs atime semantics to ZFS, where access time is only updated if the modified time or changed time changes, or if the existing access time has not been updated within the past 24 hours. It is a compromise between atime=off and atime=on. This property ''only'' takes effect if '''atime''' is '''on''':<br />
# zfs set relatime=on <pool><br />
<br />
Compression is just that, transparent compression of data. ZFS supports a few different algorithms, presently lz4 is the default. '''gzip''' is also available for seldom-written yet highly-compressable data; consult the man page for more details. Enable compression using the zfs command:<br />
# zfs set compression=on <pool><br />
<br />
Other options for zfs can be displayed again, using the zfs command:<br />
# zfs get all <pool><br />
<br />
=== SSD Caching ===<br />
You can also add SSD devices as a write intent log (external ZIL or SLOG) and also as a layer 2 adaptive replacement cache (l2arc). The process to add them is very similar to creating a new VDEV.<br />
<br />
All of the below references to device-id are the IDs from /dev/disk/by-id/*<br />
<br />
To add a ZIL:<br />
zpool add <pool> log <device-id><br />
<br />
or to add a mirrored ZIL:<br />
zpool add <pool> log mirror <device-id-1> <device-id-2><br />
<br />
<br />
To add an l2arc:<br />
zpool add <pool> cache <device-id><br />
<br />
or to add a mirrored l2arc:<br />
zpool add <pool> cache mirror <device-id-1> <device-id-2><br />
<br />
=== Database ===<br />
ZFS, unlike most other file systems, has a variable record size, or what is commonly referred to as a block size. By default, the recordsize on ZFS is 128KiB, which means it will dynamically allocate blocks of any size from 512B to 128KiB depending on the size of file being written. This can often help fragmentation and file access, at the cost that ZFS would have to allocate new 128KiB blocks each time only a few bytes are written to.<br />
<br />
Most RDBMSes work in 8KiB-sized blocks by default. Although the block size is tunable for [[MySQL|MySQL/MariaDB]], [[PostgreSQL]], and [[Oracle]], all three of them use an 8KiB block size ''by default''. For both performance concerns and keeping snapshot differences to a minimum (for backup purposes, this is helpful), it is usually desirable to tune ZFS instead to accommodate the databases, using a command such as:<br />
# zfs set recordsize=8K <pool>/postgres<br />
<br />
These RDBMSes also tend to implement their own caching algorithm, often similar to ZFS's own ARC. In the interest of saving memory, it is best to simply disable ZFS's caching of the database's file data and let the database do its own job:<br />
# zfs set primarycache=metadata <pool>/postgres<br />
<br />
If your pool has no configured log devices, ZFS reserves space on the pool's data disks for its intent log (the ZIL). ZFS uses this for crash recovery, but databases are often syncing their data files to the file system on their own transaction commits anyway. The end result of this is that ZFS will be committing data '''twice''' to the data disks, and it can severely impact performance. You can tell ZFS to prefer to not use the ZIL, and in which case, data is only committed to the file system once. Setting this for non-database file systems, or for pools with configured log devices, can actually ''negatively'' impact the performance, so beware:<br />
# zfs set logbias=throughput <pool>/postgres<br />
<br />
These can also be done at file system creation time, for example:<br />
# zfs create -o recordsize=8K \<br />
-o primarycache=metadata \<br />
-o mountpoint=/var/lib/postgres \<br />
-o logbias=throughput \<br />
<pool>/postgres<br />
<br />
Please note: these kinds of tuning parameters are ideal for specialized applications like RDBMSes. You can easily ''hurt'' ZFS's performance by setting these on a general-purpose file system such as your /home directory.<br />
<br />
=== /tmp ===<br />
If you would like to use ZFS to store your /tmp directory, which may be useful for storing arbitrarily-large sets of files or simply keeping your RAM free of idle data, you can generally improve performance of certain applications writing to /tmp by disabling file system sync. This causes ZFS to ignore an application's sync requests (eg, with {{ic|fsync}} or {{ic|O_SYNC}}) and return immediately. While this has severe application-side data consistency consequences (never disable sync for a database!), files in /tmp are less likely to be important and affected. Please note this does ''not'' affect the integrity of ZFS itself, only the possibility that data an application expects on-disk may not have actually been written out following a crash.<br />
# zfs set sync=disabled <pool>/tmp<br />
<br />
Additionally, for security purposes, you may want to disable '''setuid''' and '''devices''' on the /tmp file system, which prevents some kinds of privilege-escalation attacks or the use of device nodes:<br />
# zfs set setuid=off <pool>/tmp<br />
# zfs set devices=off <pool>/tmp<br />
<br />
Combining all of these for a create command would be as follows:<br />
# zfs create -o setuid=off -o devices=off -o sync=disabled -o mountpoint=/tmp <pool>/tmp<br />
<br />
Please note, also, that if you want /tmp on ZFS, you will need to mask (disable) [[systemd]]'s automatic tmpfs-backed /tmp, else ZFS will be unable to mount your dataset at boot-time or import-time:<br />
# systemctl mask tmp.mount<br />
<br />
=== ZVOLs ===<br />
<br />
ZFS volumes (ZVOLs) can suffer from the same block size-related issues as RDBMSes, but it is worth noting that the default recordsize for ZVOLs is 8KiB already. If possible, it is best to align any partitions contained in a ZVOL to your recordsize (current versions of fdisk and gdisk by default automatically align at 1MiB segments, which works), and file system block sizes to the same size. Other than this, you might tweak the '''recordsize''' to accommodate the data inside the ZVOL as necessary (though 8KiB tends to be a good value for most file systems, even when using 4KiB blocks on that level).<br />
<br />
==== RAIDZ and Advanced Format physical disks ====<br />
<br />
Each block of a ZVOL gets its own parity disks, and if you have physical media with logical block sizes of 4096B, 8192B, or so on, the parity needs to be stored in whole physical blocks, and this can drastically increase the space requirements of a ZVOL, requiring 2× or more physical storage capacity than the ZVOL's logical capacity. Setting the '''recordsize''' to 16k or 32k can help reduce this footprint drastically.<br />
<br />
See [https://github.com/zfsonlinux/zfs/issues/1807 ZFS on Linux issue #1807 for details]<br />
<br />
== Usage ==<br />
<br />
Users can optionally create a dataset under the zpool as opposed to manually creating directories under the zpool. Datasets allow for an increased level of control (quotas for example) in addition to snapshots. To be able to create and mount a dataset, a directory of the same name must not pre-exist in the zpool. To create a dataset, use:<br />
<br />
# zfs create <nameofzpool>/<nameofdataset><br />
<br />
It is then possible to apply ZFS specific attributes to the dataset. For example, one could assign a quota limit to a specific directory within a dataset:<br />
<br />
# zfs set quota=20G <nameofzpool>/<nameofdataset>/<directory><br />
<br />
To see all the commands available in ZFS, use :<br />
<br />
$ man zfs<br />
<br />
or:<br />
<br />
$ man zpool<br />
<br />
=== Scrub ===<br />
<br />
ZFS pools should be scrubbed at least once a week. To scrub the pool:<br />
<br />
# zpool scrub <pool><br />
<br />
To do automatic scrubbing once a week, set the following line in the root crontab:<br />
<br />
{{hc|# crontab -e|<br />
...<br />
30 19 * * 5 zpool scrub <pool><br />
...<br />
}}<br />
<br />
Replace {{ic|<pool>}} with the name of the ZFS pool.<br />
<br />
=== Check zfs pool status ===<br />
<br />
To print a nice table with statistics about the ZFS pool, including and read/write errors, use<br />
<br />
# zpool status -v<br />
<br />
=== Destroy a storage pool ===<br />
<br />
ZFS makes it easy to destroy a mounted storage pool, removing all metadata about the ZFS device. This command destroys any data contained in the pool:<br />
<br />
# zpool destroy <pool><br />
<br />
And now when checking the status:<br />
<br />
{{hc|# zpool status|no pools available}}<br />
<br />
To find the name of the pool, see [[#Check zfs pool status]].<br />
<br />
=== Export a storage pool ===<br />
<br />
If a storage pool is to be used on another system, it will first need to be exported. It is also necessary to export a pool if it has been imported from the archiso as the hostid is different in the archiso as it is in the booted system. The zpool command will refuse to import any storage pools that have not been exported. It is possible to force the import with the {{ic|-f}} argument, but this is considered bad form.<br />
<br />
Any attempts made to import an un-exported storage pool will result in an error stating the storage pool is in use by another system. This error can be produced at boot time abruptly abandoning the system in the busybox console and requiring an archiso to do an emergency repair by either exporting the pool, or adding the {{ic|zfs_force&#61;1}} to the kernel boot parameters (which is not ideal). See [[#On boot the zfs pool does not mount stating: "pool may be in use from other system"]]<br />
<br />
To export a pool,<br />
<br />
# zpool export bigdata<br />
<br />
=== Rename a Zpool ===<br />
Renaming a zpool that is already created is accomplished in 2 steps:<br />
<br />
# zpool export oldname<br />
# zpool import oldname newname<br />
<br />
=== Setting a Different Mount Point ===<br />
The mount point for a given zpool can be moved at will with one command:<br />
# zfs set mountpoint=/foo/bar poolname<br />
<br />
=== Swap volume ===<br />
<br />
ZFS does not allow to use swapfiles, but users can use a ZFS volume (ZVOL) as swap. It is importart to set the ZVOL block size to match the system page size, which can be obtained by the {{ic|getconf PAGESIZE}} command (default on x86_64 is 4KiB). Another option useful for keeping the system running well in low-memory situations is not caching the ZVOL data.<br />
<br />
Create a 8GiB zfs volume:<br />
<br />
# zfs create -V 8G -b $(getconf PAGESIZE) \<br />
-o primarycache=metadata \<br />
-o com.sun:auto-snapshot=false <pool>/swap<br />
<br />
Prepare it as swap partition:<br />
<br />
# mkswap -f /dev/zvol/<pool>/swap<br />
# swapon /dev/zvol/<pool>/swap<br />
<br />
To make it permanent, edit {{ic|/etc/fstab}}. ZVOLs support discard, which can potentially help ZFS's block allocator and reduce fragmentation for all other datasets when/if swap is not full.<br />
<br />
Add a line to {{ic|/etc/fstab}}:<br />
<br />
/dev/zvol/<pool>/swap none swap discard 0 0<br />
<br />
{{Out of date|The hibernate hook is deprecated. Does the limitation still apply?}}<br />
Keep in mind the Hibernate hook must be loaded before filesystems, so using ZVOL as swap will not allow to use hibernate function. If you need hibernate, keep a partition for it.<br />
<br />
Make sure to unmount all ZFS filesystems before rebooting the machine, otherwise any ZFS pools will refuse to be imported:<br />
# zfs umount -a<br />
<br />
=== Automatic snapshots ===<br />
<br />
==== ZFS Automatic Snapshot Service for Linux ====<br />
<br />
The {{AUR|zfs-auto-snapshot-git}} package from [[Arch User Repository|AUR]] provides a shell script to automate the management of snapshots, with each named by date and label (hourly, daily, etc), giving quick and convenient snapshotting of all ZFS datasets. The package also installs cron tasks for quarter-hourly, hourly, daily, weekly, and monthly snapshots. Optionally adjust the --keep parameter from the defaults depending on how far back the snapshots are to go (the monthly script by default keeps data for up to a year).<br />
<br />
To prevent a dataset from being snapshotted at all, set {{ic|1=com.sun:auto-snapshot=false}} on it. Likewise, set more fine-grained control as well by label, if, for example, no monthlies are to be kept on a snapshot, for example, set {{ic|1=com.sun:auto-snapshot:monthly=false}}.<br />
<br />
==== ZFS Snapshot Manager ====<br />
<br />
The {{AUR|zfs-snap-manager}} package from [[Arch User Repository|AUR]] provides a python service that takes daily snapshots from a configurable set of ZFS datasets and cleans them out in a [[wikipedia:Backup_rotation_scheme#Grandfather-father-son|"Grandfather-father-son"]] scheme. It can be configured to e.g. keep 7 daily, 5 weekly, 3 monthly and 2 yearly snapshots. <br />
<br />
The package also supports configurable replication to other machines running ZFS by means of {{ic|zfs send}} and {{ic|zfs receive}}. If the destination machine runs this package as well, it could be configured to keep these replicated snapshots for a longer time. This allows a setup where a source machine has only a few daily snapshots locally stored, while on a remote storage server a much longer retention is available.<br />
<br />
==Troubleshooting==<br />
=== ZPool creation fails ===<br />
If the following error occurs then it can be fixed.<br />
<br />
# the kernel failed to rescan the partition table: 16<br />
# cannot label 'sdc': try using parted(8) and then provide a specific slice: -1<br />
<br />
One reason this can occur is because [https://github.com/zfsonlinux/zfs/issues/2582 ZFS expects pool creation to take less than 1 second]. This is a reasonable assumption under ordinary conditions, but in many situations it may take longer. Each drive will need to be cleared again before another attempt can be made.<br />
<br />
# parted /dev/sda rm 1<br />
# parted /dev/sda rm 1<br />
# dd if=/dev/zero of=/dev/sdb bs=512 count=1<br />
# zpool labelclear /dev/sda<br />
<br />
A brute force creation can be attempted over and over again, and with some luck the ZPool creation will take less than 1 second.<br />
Once cause for creation slowdown can be slow burst read writes on a drive. By reading from the disk in parallell to ZPool creation, it may be possible to increase burst speeds.<br />
<br />
# dd if=/dev/sda of=/dev/null<br />
<br />
This can be done with multiple drives by saving the above command for each drive to a file on separate lines and running <br />
<br />
# cat $FILE | parallel<br />
<br />
Then run ZPool creation at the same time.<br />
<br />
=== ZFS is using too much RAM ===<br />
<br />
By default, ZFS caches file operations ([[wikipedia:Adaptive replacement cache|ARC]]) using up to two-thirds of available system memory on the host. To adjust the ARC size, add the following to the [[Kernel parameters]] list:<br />
<br />
zfs.zfs_arc_max=536870912 # (for 512MB)<br />
<br />
For a more detailed description, as well as other configuration options, see [http://wiki.gentoo.org/wiki/ZFS#ARC gentoo-wiki:zfs#arc].<br />
<br />
=== Does not contain an EFI label ===<br />
<br />
The following error will occur when attempting to create a zfs filesystem,<br />
<br />
/dev/disk/by-id/<id> does not contain an EFI label but it may contain partition<br />
<br />
The way to overcome this is to use {{ic|-f}} with the zfs create command.<br />
<br />
=== No hostid found ===<br />
<br />
An error that occurs at boot with the following lines appearing before initscript output:<br />
<br />
ZFS: No hostid found on kernel command line or /etc/hostid.<br />
<br />
This warning occurs because the ZFS module does not have access to the spl hosted. There are two solutions, for this. Either place the spl hostid in the [[kernel parameters]] in the boot loader. For example, adding {{ic|1=spl.spl_hostid=0x00bab10c}}.<br />
<br />
The other solution is to make sure that there is a hostid in {{ic|/etc/hostid}}, and then regenerate the initramfs image. Which will copy the hostid into the initramfs image.<br />
<br />
# mkinitcpio -p linux<br />
<br />
=== On boot the zfs pool does not mount stating: "pool may be in use from other system" ===<br />
<br />
==== Unexported pool ====<br />
<br />
If the new installation does not boot because the zpool cannot be imported, chroot into the installation and properly export the zpool. See [[#Emergency chroot repair with archzfs]].<br />
<br />
Once inside the chroot environment, load the ZFS module and force import the zpool,<br />
<br />
# zpool import -a -f<br />
<br />
now export the pool:<br />
<br />
# zpool export <pool><br />
<br />
To see the available pools, use,<br />
<br />
# zpool status<br />
<br />
It is necessary to export a pool because of the way ZFS uses the hostid to track the system the zpool was created on. The hostid is generated partly based on the network setup. During the installation in the archiso the network configuration could be different generating a different hostid than the one contained in the new installation. Once the zfs filesystem is exported and then re-imported in the new installation, the hostid is reset. See [http://osdir.com/ml/zfs-discuss/2011-06/msg00227.html Re: Howto zpool import/export automatically? - msg#00227].<br />
<br />
If ZFS complains about "pool may be in use" after every reboot, properly export pool as described above, and then rebuild ramdisk in normally booted system:<br />
<br />
# mkinitcpio -p linux<br />
<br />
==== Incorrect hostid ====<br />
<br />
Double check that the pool is properly exported. Exporting the zpool clears the hostid marking the ownership. So during the first boot the zpool should mount correctly. If it does not there is some other problem.<br />
<br />
Reboot again, if the zfs pool refuses to mount it means the hostid is not yet correctly set in the early boot phase and it confuses zfs. Manually tell zfs the correct number, once the hostid is coherent across the reboots the zpool will mount correctly.<br />
<br />
Boot using zfs_force and write down the hostid. This one is just an example.<br />
% hostid<br />
0a0af0f8<br />
<br />
This number have to be added to the [[kernel parameters]] as {{ic|spl.spl_hostid&#61;0x0a0af0f8}}. Another solution is writing the hostid inside the initram image, see the [[Installing Arch Linux on ZFS#After the first boot|installation guide]] explanation about this.<br />
<br />
Users can always ignore the check adding {{ic|zfs_force&#61;1}} in the [[kernel parameters]], but it is not advisable as a permanent solution.<br />
<br />
=== Devices have different sector alignment ===<br />
<br />
Once a drive has become faulted it should be replaced A.S.A.P. with an identical drive.<br />
<br />
# zpool replace bigdata ata-ST3000DM001-9YN166_S1F0KDGY ata-ST3000DM001-1CH166_W1F478BD -f<br />
<br />
but in this instance, the following error is produced:<br />
<br />
cannot replace ata-ST3000DM001-9YN166_S1F0KDGY with ata-ST3000DM001-1CH166_W1F478BD: devices have different sector alignment<br />
<br />
ZFS uses the ashift option to adjust for physical block size. When replacing the faulted disk, ZFS is attempting to use {{ic|<nowiki>ashift=12</nowiki>}}, but the faulted disk is using a different ashift (probably {{ic|<nowiki>ashift=9</nowiki>}}) and this causes the resulting error. <br />
<br />
For Advanced Format Disks with 4KB blocksize, an ashift of 12 is recommended for best performance. See [http://zfsonlinux.org/faq.html#PerformanceConsideration 1.10 What’s going on with performance?] and [http://wiki.illumos.org/display/illumos/ZFS+and+Advanced+Format+disks ZFS and Advanced Format disks].<br />
<br />
Use zdb to find the ashift of the zpool: {{ic|zdb | grep ashift}}, then use the {{ic|-o}} argument to set the ashift of the replacement drive:<br />
<br />
# zpool replace bigdata ata-ST3000DM001-9YN166_S1F0KDGY ata-ST3000DM001-1CH166_W1F478BD -o ashift=9 -f<br />
<br />
Check the zpool status for confirmation:<br />
<br />
{{hc|# zpool status -v|<br />
pool: bigdata<br />
state: DEGRADED<br />
status: One or more devices is currently being resilvered. The pool will<br />
continue to function, possibly in a degraded state.<br />
action: Wait for the resilver to complete.<br />
scan: resilver in progress since Mon Jun 16 11:16:28 2014<br />
10.3G scanned out of 5.90T at 81.7M/s, 20h59m to go<br />
2.57G resilvered, 0.17% done<br />
config:<br />
<br />
NAME STATE READ WRITE CKSUM<br />
bigdata DEGRADED 0 0 0<br />
raidz1-0 DEGRADED 0 0 0<br />
replacing-0 OFFLINE 0 0 0<br />
ata-ST3000DM001-9YN166_S1F0KDGY OFFLINE 0 0 0<br />
ata-ST3000DM001-1CH166_W1F478BD ONLINE 0 0 0 (resilvering)<br />
ata-ST3000DM001-9YN166_S1F0JKRR ONLINE 0 0 0<br />
ata-ST3000DM001-9YN166_S1F0KBP8 ONLINE 0 0 0<br />
ata-ST3000DM001-9YN166_S1F0JTM1 ONLINE 0 0 0<br />
<br />
errors: No known data errors}}<br />
<br />
== Tips and tricks ==<br />
<br />
===Embed the archzfs packages into an archiso===<br />
<br />
Follow the [[Archiso]] steps for creating a fully functional Arch Linux live CD/DVD/USB image.<br />
<br />
Enable the [[Unofficial user repositories#archzfs|archzfs]] repository:<br />
<br />
{{hc|~/archlive/pacman.conf|<nowiki><br />
...<br />
[archzfs]<br />
Server = http://archzfs.com/$repo/x86_64<br />
</nowiki>}}<br />
<br />
Add the {{ic|archzfs-linux}} group to the list of packages to be installed (the {{ic|archzfs}} repository provides packages for the x86_64 architecture only).<br />
<br />
{{hc|~/archlive/packages.x86_64|<br />
...<br />
archzfs-linux<br />
}}<br />
<br />
Complete [[Archiso#Build_the_ISO|Build the ISO]] to finally build the iso.<br />
<br />
=== Encryption in ZFS on linux ===<br />
<br />
ZFS on linux does not support encryption directly, but zpools can be created in dm-crypt block devices. Since the zpool is created on the plain-text<br />
abstraction it is possible to have the data encrypted while having all the advantages of ZFS like deduplication, compression, and data robustness.<br />
<br />
dm-crypt, possibly via LUKS, creates devices in {{ic|/dev/mapper}} and their name is fixed. So you just need to change {{ic|zpool create}} commands to<br />
point to that names. The idea is configuring the system to create the {{ic|/dev/mapper}} block devices and import the zpools from there.<br />
Since zpools can be created in multiple devices (raid, mirroring, striping, ...), it is important all the devices are encrypted otherwise the protection<br />
might be partially lost.<br />
<br />
For example, an encrypted zpool can be created using plain dm-crypt (without LUKS) with:<br />
<br />
# cryptsetup --hash=sha512 --cipher=twofish-xts-plain64 --offset=0 \<br />
--key-file=/dev/sdZ --key-size=512 open --type=plain /dev/sdX enc<br />
# zpool create zroot /dev/mapper/enc<br />
<br />
In the case of a root filesystem pool, the {{ic|mkinicpio.conf}} HOOKS line will enable the keyboard for the password, create the devices, and load the pools. It will contain something like:<br />
<br />
HOOKS="... keyboard encrypt zfs ..."<br />
<br />
Since the {{ic|/dev/mapper/enc}} name is fixed no import errors will occur.<br />
<br />
Creating encrypted zpools works fine. But if you need encrypted directories, for example to protect your users' homes, ZFS loses some functionality.<br />
<br />
ZFS will see the encrypted data, not the plain-text abstraction, so compression and deduplication will not work. The reason is that encrypted data has always high entropy making compression ineffective and even from the same input you get different output (thanks to salting) making deduplication impossible.<br />
To reduce the unnecessary overhead it is possible to create a sub-filesystem for each encrypted directory and use [[eCryptfs]] on it.<br />
<br />
For example to have an encrypted home: (the two passwords, encryption and login, must be the same)<br />
# zfs create -o compression=off \<br />
-o dedup=off \<br />
-o mountpoint=/home/<username> \<br />
<zpool>/<username><br />
# useradd -m <username><br />
# passwd <username><br />
# ecryptfs-migrate-home -u <username><br />
<log in user and complete the procedure with ecryptfs-unwrap-passphrase><br />
<br />
=== Emergency chroot repair with archzfs ===<br />
<br />
To get into the ZFS filesystem from live system for maintenance, there are two options:<br />
<br />
# Build custom archiso with ZFS as described in [[#Embed the archzfs packages into an archiso]].<br />
# Boot the latest official archiso and bring up the network. Then enable [[Unofficial_user_repositories#archzfs|archzfs]] repository inside the live system as usual, sync the pacman package database and install the ''archzfs-archiso-linux'' package.<br />
<br />
To start the recovery, load the ZFS kernel modules:<br />
<br />
# modprobe zfs<br />
<br />
Import the pool:<br />
<br />
# zpool import -a -R /mnt<br />
<br />
Mount the boot partitions (if any):<br />
<br />
# mount /dev/sda2 /mnt/boot<br />
# mount /dev/sda1 /mnt/boot/efi<br />
<br />
Chroot into the ZFS filesystem:<br />
<br />
# arch-chroot /mnt /bin/bash<br />
<br />
Check the kernel version:<br />
<br />
# pacman -Qi linux<br />
# uname -r<br />
<br />
uname will show the kernel version of the archiso. If they are different, run depmod (in the chroot) with the correct kernel version of the chroot installation:<br />
<br />
# depmod -a 3.6.9-1-ARCH (version gathered from pacman -Qi linux but using the matching kernel modules directory name under the chroot's /lib/modules)<br />
<br />
This will load the correct kernel modules for the kernel version installed in the chroot installation.<br />
<br />
Regenerate the ramdisk:<br />
<br />
# mkinitcpio -p linux<br />
<br />
There should be no errors.<br />
<br />
=== Bindmount ===<br />
Here a bind mount from /mnt/zfspool to /srv/nfs4/music is created. The configuration ensures that the zfs pool is ready before the bind mount is created.<br />
<br />
==== fstab ====<br />
See [http://www.freedesktop.org/software/systemd/man/systemd.mount.html systemd.mount] for more information on how systemd converts fstab into mount unit files with [http://www.freedesktop.org/software/systemd/man/systemd-fstab-generator.html systemd-fstab-generator].<br />
<br />
{{hc|/etc/fstab|<nowiki><br />
/mnt/zfspool /srv/nfs4/music none bind,defaults,nofail,x-systemd.requires=zfs-mount.service 0 0<br />
</nowiki>}}<br />
<br />
==== systemd mount unit ====<br />
<br />
If it is not possible to bindmount a directory residing on zfs onto another directory using fstab, because the fstab is read before the zfs pool is ready, you can overcome this limitation with a systemd mount unit can be used for the bind mount. The name of the mount unit must be equal to the directory mentioned after "Where", replace slashes with minuses. See [[http://utcc.utoronto.ca/~cks/space/blog/linux/SystemdAndBindMounts]] and [[http://utcc.utoronto.ca/~cks/space/blog/linux/SystemdBindMountUnits]] for more details.<br />
{{hc|srv-nfs4-music.mount|<nowiki><br />
[Mount]<br />
What=/mnt/zfspool<br />
Where=/srv/nfs4/music<br />
Type=none<br />
Options=bind<br />
<br />
[Unit]<br />
DefaultDependencies=no<br />
Conflicts=umount.target<br />
Before=local-fs.target umount.target<br />
After=zfs-mount.service<br />
Requires=zfs-mount.service<br />
ConditionPathIsDirectory=/mnt/zfspool<br />
<br />
[Install]<br />
WantedBy=local-fs.target<br />
</nowiki>}}<br />
<br />
== See also ==<br />
<br />
* [[Installing Arch Linux on ZFS]]<br />
* [http://zfsonlinux.org/ ZFS on Linux]<br />
* [http://zfsonlinux.org/faq.html ZFS on Linux FAQ]<br />
* [http://www.freebsd.org/doc/en_US.ISO8859-1/books/handbook/filesystems-zfs.html FreeBSD Handbook -- The Z File System]<br />
* [http://docs.oracle.com/cd/E19253-01/819-5461/index.html Oracle Solaris ZFS Administration Guide]<br />
* [http://www.solarisinternals.com/wiki/index.php/ZFS_Troubleshooting_Guide Solaris Internals -- ZFS Troubleshooting Guide]<br />
* [http://royal.pingdom.com/2013/06/04/zfs-backup/ Pingdom details how it backs up 5TB of MySQL data every day with ZFS]<br />
<br />
; Aaron Toponce has authored a 17-part blog on ZFS which is an excellent read.<br />
# [https://pthree.org/2012/12/04/zfs-administration-part-i-vdevs/ VDEVs]<br />
# [https://pthree.org/2012/12/05/zfs-administration-part-ii-raidz/ RAIDZ Levels]<br />
# [https://pthree.org/2012/12/06/zfs-administration-part-iii-the-zfs-intent-log/ The ZFS Intent Log]<br />
# [https://pthree.org/2012/12/07/zfs-administration-part-iv-the-adjustable-replacement-cache/ The ARC]<br />
# [https://pthree.org/2012/12/10/zfs-administration-part-v-exporting-and-importing-zpools/ Import/export zpools]<br />
# [https://pthree.org/2012/12/11/zfs-administration-part-vi-scrub-and-resilver/ Scrub and Resilver]<br />
# [https://pthree.org/2012/12/12/zfs-administration-part-vii-zpool-properties/ Zpool Properties]<br />
# [https://pthree.org/2012/12/13/zfs-administration-part-viii-zpool-best-practices-and-caveats/ Zpool Best Practices]<br />
# [https://pthree.org/2012/12/14/zfs-administration-part-ix-copy-on-write/ Copy on Write]<br />
# [https://pthree.org/2012/12/17/zfs-administration-part-x-creating-filesystems/ Creating Filesystems]<br />
# [https://pthree.org/2012/12/18/zfs-administration-part-xi-compression-and-deduplication/ Compression and Deduplication]<br />
# [https://pthree.org/2012/12/19/zfs-administration-part-xii-snapshots-and-clones/ Snapshots and Clones]<br />
# [https://pthree.org/2012/12/20/zfs-administration-part-xiii-sending-and-receiving-filesystems/ Send/receive Filesystems]<br />
# [https://pthree.org/2012/12/21/zfs-administration-part-xiv-zvols/ ZVOLs]<br />
# [https://pthree.org/2012/12/31/zfs-administration-part-xv-iscsi-nfs-and-samba/ iSCSI, NFS, and Samba]<br />
# [https://pthree.org/2013/01/02/zfs-administration-part-xvi-getting-and-setting-properties/ Get/Set Properties]<br />
# [https://pthree.org/2013/01/03/zfs-administration-part-xvii-best-practices-and-caveats/ ZFS Best Practices]</div>Justin8https://wiki.archlinux.org/index.php?title=ZFS&diff=436773ZFS2016-05-31T00:34:10Z<p>Justin8: Add commands for adding ZIL/l2arc</p>
<hr />
<div>[[Category:File systems]]<br />
[[ja:ZFS]]<br />
{{Related articles start}}<br />
{{Related|File systems}}<br />
{{Related|Experimenting with ZFS}}<br />
{{Related|Installing Arch Linux on ZFS}}<br />
{{Related|ZFS on FUSE}}<br />
{{Related articles end}}<br />
<br />
[[Wikipedia:ZFS|ZFS]] is an advanced filesystem created by [[Wikipedia:Sun Microsystems|Sun Microsystems]] (now owned by Oracle) and released for OpenSolaris in November 2005. <br />
<br />
Features of ZFS include: pooled storage (integrated volume management – zpool), [[Wikipedia:Copy-on-write|Copy-on-write]], [[Wikipedia:Snapshot (computer storage)|snapshots]], data integrity verification and automatic repair (scrubbing), [[Wikipedia:RAID-Z|RAID-Z]], a maximum [[Wikipedia:Exabyte|16 Exabyte]] file size, and a maximum 256 Quadrillion [[Wikipedia:Zettabyte|Zettabytes]] storage with no limit on number of filesystems (datasets) or files[http://docs.oracle.com/cd/E19253-01/819-5461/zfsover-2/index.html]. ZFS is licensed under the [[Wikipedia:CDDL|Common Development and Distribution License]] (CDDL).<br />
<br />
Described as [http://web.archive.org/web/20060428092023/http://www.sun.com/2004-0914/feature/ "The last word in filesystems"] ZFS is stable, fast, secure, and future-proof. Being licensed under the CDDL, and thus incompatible with GPL, it is not possible for ZFS to be distributed along with the Linux Kernel. This requirement, however, does not prevent a native Linux kernel module from being developed and distributed by a third party, as is the case with [http://zfsonlinux.org/ zfsonlinux.org] (ZOL).<br />
<br />
ZOL is a project funded by the [https://www.llnl.gov/ Lawrence Livermore National Laboratory] to develop a native Linux kernel module for its massive storage requirements and super computers.<br />
<br />
==Installation==<br />
=== General ===<br />
Install {{AUR|zfs-linux-git}} from the [[Arch User Repository]] or the [[Unofficial user repositories#archzfs|archzfs]] repository. This package has {{AUR|zfs-utils-linux-git}} and {{AUR|spl-linux-git}} as a dependency, which in turn has {{AUR|spl-utils-linux-git}} as dependency. SPL (Solaris Porting Layer) is a Linux Kernel module implementing Solaris APIs for ZFS compatibility.<br />
<br />
For users that desire ZFS builds from stable releases, {{AUR|zfs-linux-lts}} is available from the [[Arch User Repository]] or the [[Unofficial user repositories#archzfs|archzfs]] repository. A script to build ZFS and its dependencies automatically can be found [[#Automated build script|here]].<br />
<br />
{{warning|The ZFS and SPL kernel modules are tied to a specific kernel version. It would not be possible to apply any kernel updates until updated packages are uploaded to AUR or the [[Unofficial user repositories#archzfs|archzfs]] repository.}}<br />
<br />
Test the installation by issuing {{ic|zpool status}} on the command line. If an "insmod" error is produced, try {{ic|depmod -a}}.<br />
<br />
{{Tip| You can [[downgrade]] your linux version to the one from [[Unofficial user repositories#archzfs|archzfs]] repo if your current kenel is newer.}}<br />
<br />
=== Root on ZFS ===<br />
<br />
When performing an Arch install on ZFS, {{AUR|zfs-git}}{{Broken package link|{{aur-mirror|zfs-git}}}} and its dependencies can be installed in the archiso environment as outlined in the previous section.<br />
<br />
It may be useful to prepare a [[#Embed the archzfs packages into an archiso|customized archiso]] with ZFS support builtin. For a much more detailed guide on installing Arch with ZFS as its root file system, see [[Installing Arch Linux on ZFS]].<br />
<br />
=== DKMS ===<br />
<br />
Users can make use of DKMS [[Dynamic Kernel Module Support]] to rebuild the ZFS modules automatically with every kernel upgrade. <br />
<br />
Install {{AUR|zfs-dkms}} or {{AUR|zfs-dkms-git}} and apply the post-install instructions given by these packages.<br />
<br />
{{Tip|Add an {{ic|IgnorePkg}} entry to [[pacman.conf]] to prevent these packages from upgrading when doing a regular update.}}<br />
<br />
==Experimenting with ZFS ==<br />
<br />
Users wishing to experiment with ZFS on ''virtual block devices'' (known in ZFS terms as VDEVs) which can be simple files like {{ic|~/zfs0.img}} {{ic|~/zfs1.img}} {{ic|~/zfs2.img}} etc. with no possibility of real data loss are encouraged to see the [[Experimenting with ZFS]] article. Common tasks like building a RAIDZ array, purposefully corrupting data and recovering it, snapshotting datasets, etc. are covered.<br />
<br />
==Configuration==<br />
<br />
ZFS is considered a "zero administration" filesystem by its creators; therefore, configuring ZFS is very straight forward. Configuration is done primarily with two commands: {{ic|zfs}} and {{ic|zpool}}.<br />
<br />
===Automatic Start===<br />
<br />
For ZFS to live by its "zero administration" namesake, the zfs daemon must be loaded at startup. A benefit to this is that it is not necessary to mount the zpool in {{ic|/etc/fstab}}; the zfs daemon can import and mount zfs pools automatically. The daemon mounts the zfs pools reading the file {{ic|/etc/zfs/zpool.cache}}.<br />
<br />
For each pool you want automatically mounted by the zfs daemon execute:<br />
# zpool set cachefile=/etc/zfs/zpool.cache <pool><br />
<br />
Enable the service so it is automatically started at boot time:<br />
<br />
# systemctl enable zfs.target<br />
<br />
To manually start the daemon:<br />
<br />
# systemctl start zfs.target<br />
<br />
==Create a storage pool==<br />
<br />
Use {{ic| # parted --list}} to see a list of all available drives. It is not necessary nor recommended to partition the drives before creating the zfs filesystem.<br />
<br />
{{Note|If some or all device have been used in a software RAID set it is paramount to erase any old RAID configuration information. ([[Mdadm#Prepare_the_Devices]]) }}<br />
<br />
{{Warning|For Advanced Format Disks with 4KB sector size, an ashift of 12 is recommended for best performance. Advanced Format disks emulate a sector size of 512 bytes for compatibility with legacy systems, this causes ZFS to sometimes use an ashift option number that is not ideal. Once the pool has been created, the only way to change the ashift option is to recreate the pool. Using an ashift of 12 would also decrease available capacity. See [http://zfsonlinux.org/faq.html#PerformanceConsideration 1.10 What’s going on with performance?], [http://zfsonlinux.org/faq.html#HowDoesZFSonLinuxHandleAdvacedFormatDrives 1.15 How does ZFS on Linux handle Advanced Format disks?], and [http://wiki.illumos.org/display/illumos/ZFS+and+Advanced+Format+disks ZFS and Advanced Format disks].}}<br />
<br />
Having identified the list of drives, it is now time to get the id's of the drives to add to the zpool. The [http://zfsonlinux.org/faq.html#WhatDevNamesShouldIUseWhenCreatingMyPool zfs on Linux developers recommend] using device ids when creating ZFS storage pools of less than 10 devices. To find the id's, simply:<br />
<br />
# ls -lh /dev/disk/by-id/<br />
<br />
The ids should look similar to the following:<br />
<br />
lrwxrwxrwx 1 root root 9 Aug 12 16:26 ata-ST3000DM001-9YN166_S1F0JKRR -> ../../sdc<br />
lrwxrwxrwx 1 root root 9 Aug 12 16:26 ata-ST3000DM001-9YN166_S1F0JTM1 -> ../../sde<br />
lrwxrwxrwx 1 root root 9 Aug 12 16:26 ata-ST3000DM001-9YN166_S1F0KBP8 -> ../../sdd<br />
lrwxrwxrwx 1 root root 9 Aug 12 16:26 ata-ST3000DM001-9YN166_S1F0KDGY -> ../../sdb<br />
<br />
{{Style|Missing references to [[Persistent block device naming]], it is useless to explain the differences (or even what they are) here.}}<br />
<br />
Disk labels and UUID can also be used for ZFS mounts by using [[GUID Partition Table|GPT]] partitions. ZFS drives have labels but Linux is unable to read them at boot. Unlike [[Master Boot Record|MBR]] partitions, GPT partitions directly support both UUID and labels independent of the format inside the partition. Partitioning rather than using the whole disk for ZFS offers two additional advantages. The OS does not generate bogus partition numbers from whatever unpredictable data ZFS has written to the partition sector, and if desired, you can easily over provision SSD drives, and slightly over provision spindle drives to ensure that different models with slightly different sector counts can zpool replace into your mirrors. This is a lot of organization and control over ZFS using readily available tools and techniques at almost zero cost.<br />
<br />
Use [[GUID Partition Table|gdisk]] to partition the all or part of the drive as a single partition. gdisk does not automatically name partitions so if partition labels are desired use gdisk command "c" to label the partitions. Some reasons you might prefer labels over UUID are: labels are easy to control, labels can be titled to make the purpose of each disk in your arrangement readily apparent, and labels are shorter and easier to type. These are all advantages when the server is down and the heat is on. GPT partition labels have plenty of space and can store most international characters [[wikipedia:GUID_Partition_Table#Partition_entries]] allowing large data pools to be labeled in an organized fashion.<br />
<br />
Drives partitioned with GPT have labels and UUID that look like this. <br />
<br />
# ls -l /dev/disk/by-partlabel<br />
# lrwxrwxrwx 1 root root 10 Apr 30 01:44 zfsdata1 -> ../../sdd1<br />
# lrwxrwxrwx 1 root root 10 Apr 30 01:44 zfsdata2 -> ../../sdc1<br />
# lrwxrwxrwx 1 root root 10 Apr 30 01:59 zfsl2arc -> ../../sda1<br />
<br />
# ls -l /dev/disk/by-partuuid<br />
# lrwxrwxrwx 1 root root 10 Apr 30 01:44 148c462c-7819-431a-9aba-5bf42bb5a34e -> ../../sdd1<br />
# lrwxrwxrwx 1 root root 10 Apr 30 01:59 4f95da30-b2fb-412b-9090-fc349993df56 -> ../../sda1<br />
# lrwxrwxrwx 1 root root 10 Apr 30 01:44 e5ccef58-5adf-4094-81a7-3bac846a885f -> ../../sdc1<br />
<br />
Now, finally, create the ZFS pool:<br />
<br />
# zpool create -f -m <mount> <pool> raidz <ids><br />
<br />
* '''create''': subcommand to create the pool.<br />
<br />
* '''-f''': Force creating the pool. This is to overcome the "EFI label error". See [[#Does not contain an EFI label]].<br />
<br />
* '''-m''': The mount point of the pool. If this is not specified, then the pool will be mounted to {{ic|/<pool>}}.<br />
<br />
* '''pool''': This is the name of the pool.<br />
<br />
* '''raidz''': This is the type of virtual device that will be created from the pool of devices. Raidz is a special implementation of raid5. See [https://blogs.oracle.com/bonwick/entry/raid_z Jeff Bonwick's Blog -- RAID-Z] for more information about raidz.<br />
<br />
* '''ids''': The names of the drives or partitions that to include into the pool. Get it from {{ic|/dev/disk/by-id}}.<br />
<br />
Here is an example for the full command:<br />
<br />
# zpool create -f -m /mnt/data bigdata \<br />
raidz \<br />
ata-ST3000DM001-9YN166_S1F0KDGY \<br />
ata-ST3000DM001-9YN166_S1F0JKRR \<br />
ata-ST3000DM001-9YN166_S1F0KBP8 \<br />
ata-ST3000DM001-9YN166_S1F0JTM1<br />
<br />
=== Advanced format disks ===<br />
<br />
In case Advanced Format disks are used which have a native sector size of 4096 bytes instead of 512 bytes, the automated sector size detection algorithm of ZFS might detect 512 bytes because the backwards compatibility with legacy systems. This would result in degraded performance. To make sure a correct sector size is used, the {{ic|<nowiki>ashift=12</nowiki>}} option should be used (See the [http://zfsonlinux.org/faq.html#HowDoesZFSonLinuxHandleAdvacedFormatDrives ZFS on Linux FAQ]). The full command would in this case be:<br />
<br />
# zpool create -f -o ashift=12 -m /mnt/data bigdata \<br />
raidz \<br />
ata-ST3000DM001-9YN166_S1F0KDGY \<br />
ata-ST3000DM001-9YN166_S1F0JKRR \<br />
ata-ST3000DM001-9YN166_S1F0KBP8 \<br />
ata-ST3000DM001-9YN166_S1F0JTM1<br />
<br />
=== Verifying pool creation ===<br />
<br />
If the command is successful, there will be no output. Using the {{ic|$ mount}} command will show that the pool is mounted. Using {{ic|# zpool status}} will show that the pool has been created.<br />
<br />
{{hc|# zpool status|<br />
pool: bigdata<br />
state: ONLINE<br />
scan: none requested<br />
config:<br />
<br />
NAME STATE READ WRITE CKSUM<br />
bigdata ONLINE 0 0 0<br />
-0 ONLINE 0 0 0<br />
ata-ST3000DM001-9YN166_S1F0KDGY-part1 ONLINE 0 0 0<br />
ata-ST3000DM001-9YN166_S1F0JKRR-part1 ONLINE 0 0 0<br />
ata-ST3000DM001-9YN166_S1F0KBP8-part1 ONLINE 0 0 0<br />
ata-ST3000DM001-9YN166_S1F0JTM1-part1 ONLINE 0 0 0<br />
<br />
errors: No known data errors<br />
}}<br />
<br />
At this point it would be good to reboot the machine to ensure that the ZFS pool is mounted at boot. It is best to deal with all errors before transferring data.<br />
<br />
=== GRUB-compatible pool creation ===<br />
<br />
By default, ''zpool'' will enable all features on a pool. If {{ic|/boot}} resides on ZFS and when using [[GRUB]], you must only enable read-only, or non-read-only features supported by GRUB ({{ic|lz4_compress}} as of version 2.02.beta2). Otherwise GRUB will not be able to read the pool.<br />
<br />
# zpool create -f -d \<br />
-o feature@async_destroy=enabled \<br />
-o feature@empty_bpobj=enabled \<br />
-o feature@lz4_compress=enabled \<br />
-o feature@spacemap_histogram=enabled \<br />
-o feature@enabled_txg=enabled \<br />
<pool_name> <vdevs><br />
<br />
To create a mirrored vdev for GRUB you would use a command like this:<br />
<br />
# zpool create <pool_name> mirror -f -d \<br />
-o feature@async_destroy=enabled \<br />
-o feature@empty_bpobj=enabled \<br />
-o feature@lz4_compress=enabled \<br />
-o feature@spacemap_histogram=enabled \<br />
-o feature@enabled_txg=enabled \<br />
/dev/disk/by-id/ata-disk-part2 /dev/disk/by-id/ata-disk-part2<br />
<br />
{{Tip|As of September 2015, GRUB's development tree supports {{ic|extensible_dataset}}, {{ic|hole_birth}}, {{ic|embedded_data}}, and {{ic|large_blocks}}, making it viable to use a pool with all features enabled, either at create time or by using {{ic|zpool upgrade <pool_name>}}, if {{AUR|grub-git}} is installed.}}<br />
<br />
=== Importing a pool created by id ===<br />
<br />
Eventually a pool may fail to auto mount and you need to import to bring your pool back. Take care to avoid the most obvious solution.<br />
<br />
# ###zpool import zfsdata # Do not do this! Always use -d<br />
<br />
This will import your pools using {{ic|/dev/sd?}} which will lead to problems the next time you rearrange your drives. This may be as simple as rebooting with a USB drive left in the machine, which harkens back to a time when PC's would not boot when a floppy disk was left in a machine. Adapt one of the following commands to import your pool so that pool imports retain the persistence they were created with.<br />
<br />
# zpool import -d /dev/disk/by-id zfsdata<br />
# zpool import -d /dev/disk/by-partlabel zfsdata<br />
# zpool import -d /dev/disk/by-partuuid zfsdata<br />
<br />
== Tuning ==<br />
<br />
=== General ===<br />
Many parameters are available for zfs file systems, you can view a full list with {{ic|zfs get all <pool>}}. Two common ones to adjust are '''atime''' and '''compression'''.<br />
<br />
Atime is enabled by default but for most users, it represents superfluous writes to the zpool and it can be disabled using the zfs command:<br />
# zfs set atime=off <pool><br />
<br />
As an alternative to turning off atime completely, '''relatime''' is available. This brings the default ext4/xfs atime semantics to ZFS, where access time is only updated if the modified time or changed time changes, or if the existing access time has not been updated within the past 24 hours. It is a compromise between atime=off and atime=on. This property ''only'' takes effect if '''atime''' is '''on''':<br />
# zfs set relatime=on <pool><br />
<br />
Compression is just that, transparent compression of data. ZFS supports a few different algorithms, presently lz4 is the default. '''gzip''' is also available for seldom-written yet highly-compressable data; consult the man page for more details. Enable compression using the zfs command:<br />
# zfs set compression=on <pool><br />
<br />
Other options for zfs can be displayed again, using the zfs command:<br />
# zfs get all <pool><br />
<br />
=== SSD Caching ===<br />
You can also add SSD devices as a write intent log (ZIL or SLOG) and also as a layer 2 adaptive replacement cache (l2arc). The process to add them is very similar to creating a new VDEV.<br />
<br />
All of the below references to device-id are the IDs from /dev/disk/by-id/*<br />
<br />
To add a ZIL:<br />
zpool add <pool> log <device-id><br />
<br />
or to add a mirrored ZIL:<br />
zpool add <pool> log mirror <device-id-1> <device-id-2><br />
<br />
<br />
To add an l2arc:<br />
zpool add <pool> cache <device-id><br />
<br />
or to add a mirrored l2arc:<br />
zpool add <pool> cache mirror <device-id-1> <device-id-2><br />
<br />
=== Database ===<br />
ZFS, unlike most other file systems, has a variable record size, or what is commonly referred to as a block size. By default, the recordsize on ZFS is 128KiB, which means it will dynamically allocate blocks of any size from 512B to 128KiB depending on the size of file being written. This can often help fragmentation and file access, at the cost that ZFS would have to allocate new 128KiB blocks each time only a few bytes are written to.<br />
<br />
Most RDBMSes work in 8KiB-sized blocks by default. Although the block size is tunable for [[MySQL|MySQL/MariaDB]], [[PostgreSQL]], and [[Oracle]], all three of them use an 8KiB block size ''by default''. For both performance concerns and keeping snapshot differences to a minimum (for backup purposes, this is helpful), it is usually desirable to tune ZFS instead to accommodate the databases, using a command such as:<br />
# zfs set recordsize=8K <pool>/postgres<br />
<br />
These RDBMSes also tend to implement their own caching algorithm, often similar to ZFS's own ARC. In the interest of saving memory, it is best to simply disable ZFS's caching of the database's file data and let the database do its own job:<br />
# zfs set primarycache=metadata <pool>/postgres<br />
<br />
If your pool has no configured log devices, ZFS reserves space on the pool's data disks for its intent log (the ZIL). ZFS uses this for crash recovery, but databases are often syncing their data files to the file system on their own transaction commits anyway. The end result of this is that ZFS will be committing data '''twice''' to the data disks, and it can severely impact performance. You can tell ZFS to prefer to not use the ZIL, and in which case, data is only committed to the file system once. Setting this for non-database file systems, or for pools with configured log devices, can actually ''negatively'' impact the performance, so beware:<br />
# zfs set logbias=throughput <pool>/postgres<br />
<br />
These can also be done at file system creation time, for example:<br />
# zfs create -o recordsize=8K \<br />
-o primarycache=metadata \<br />
-o mountpoint=/var/lib/postgres \<br />
-o logbias=throughput \<br />
<pool>/postgres<br />
<br />
Please note: these kinds of tuning parameters are ideal for specialized applications like RDBMSes. You can easily ''hurt'' ZFS's performance by setting these on a general-purpose file system such as your /home directory.<br />
<br />
=== /tmp ===<br />
If you would like to use ZFS to store your /tmp directory, which may be useful for storing arbitrarily-large sets of files or simply keeping your RAM free of idle data, you can generally improve performance of certain applications writing to /tmp by disabling file system sync. This causes ZFS to ignore an application's sync requests (eg, with {{ic|fsync}} or {{ic|O_SYNC}}) and return immediately. While this has severe application-side data consistency consequences (never disable sync for a database!), files in /tmp are less likely to be important and affected. Please note this does ''not'' affect the integrity of ZFS itself, only the possibility that data an application expects on-disk may not have actually been written out following a crash.<br />
# zfs set sync=disabled <pool>/tmp<br />
<br />
Additionally, for security purposes, you may want to disable '''setuid''' and '''devices''' on the /tmp file system, which prevents some kinds of privilege-escalation attacks or the use of device nodes:<br />
# zfs set setuid=off <pool>/tmp<br />
# zfs set devices=off <pool>/tmp<br />
<br />
Combining all of these for a create command would be as follows:<br />
# zfs create -o setuid=off -o devices=off -o sync=disabled -o mountpoint=/tmp <pool>/tmp<br />
<br />
Please note, also, that if you want /tmp on ZFS, you will need to mask (disable) [[systemd]]'s automatic tmpfs-backed /tmp, else ZFS will be unable to mount your dataset at boot-time or import-time:<br />
# systemctl mask tmp.mount<br />
<br />
=== ZVOLs ===<br />
<br />
ZFS volumes (ZVOLs) can suffer from the same block size-related issues as RDBMSes, but it is worth noting that the default recordsize for ZVOLs is 8KiB already. If possible, it is best to align any partitions contained in a ZVOL to your recordsize (current versions of fdisk and gdisk by default automatically align at 1MiB segments, which works), and file system block sizes to the same size. Other than this, you might tweak the '''recordsize''' to accommodate the data inside the ZVOL as necessary (though 8KiB tends to be a good value for most file systems, even when using 4KiB blocks on that level).<br />
<br />
==== RAIDZ and Advanced Format physical disks ====<br />
<br />
Each block of a ZVOL gets its own parity disks, and if you have physical media with logical block sizes of 4096B, 8192B, or so on, the parity needs to be stored in whole physical blocks, and this can drastically increase the space requirements of a ZVOL, requiring 2× or more physical storage capacity than the ZVOL's logical capacity. Setting the '''recordsize''' to 16k or 32k can help reduce this footprint drastically.<br />
<br />
See [https://github.com/zfsonlinux/zfs/issues/1807 ZFS on Linux issue #1807 for details]<br />
<br />
== Usage ==<br />
<br />
Users can optionally create a dataset under the zpool as opposed to manually creating directories under the zpool. Datasets allow for an increased level of control (quotas for example) in addition to snapshots. To be able to create and mount a dataset, a directory of the same name must not pre-exist in the zpool. To create a dataset, use:<br />
<br />
# zfs create <nameofzpool>/<nameofdataset><br />
<br />
It is then possible to apply ZFS specific attributes to the dataset. For example, one could assign a quota limit to a specific directory within a dataset:<br />
<br />
# zfs set quota=20G <nameofzpool>/<nameofdataset>/<directory><br />
<br />
To see all the commands available in ZFS, use :<br />
<br />
$ man zfs<br />
<br />
or:<br />
<br />
$ man zpool<br />
<br />
=== Scrub ===<br />
<br />
ZFS pools should be scrubbed at least once a week. To scrub the pool:<br />
<br />
# zpool scrub <pool><br />
<br />
To do automatic scrubbing once a week, set the following line in the root crontab:<br />
<br />
{{hc|# crontab -e|<br />
...<br />
30 19 * * 5 zpool scrub <pool><br />
...<br />
}}<br />
<br />
Replace {{ic|<pool>}} with the name of the ZFS pool.<br />
<br />
=== Check zfs pool status ===<br />
<br />
To print a nice table with statistics about the ZFS pool, including and read/write errors, use<br />
<br />
# zpool status -v<br />
<br />
=== Destroy a storage pool ===<br />
<br />
ZFS makes it easy to destroy a mounted storage pool, removing all metadata about the ZFS device. This command destroys any data contained in the pool:<br />
<br />
# zpool destroy <pool><br />
<br />
And now when checking the status:<br />
<br />
{{hc|# zpool status|no pools available}}<br />
<br />
To find the name of the pool, see [[#Check zfs pool status]].<br />
<br />
=== Export a storage pool ===<br />
<br />
If a storage pool is to be used on another system, it will first need to be exported. It is also necessary to export a pool if it has been imported from the archiso as the hostid is different in the archiso as it is in the booted system. The zpool command will refuse to import any storage pools that have not been exported. It is possible to force the import with the {{ic|-f}} argument, but this is considered bad form.<br />
<br />
Any attempts made to import an un-exported storage pool will result in an error stating the storage pool is in use by another system. This error can be produced at boot time abruptly abandoning the system in the busybox console and requiring an archiso to do an emergency repair by either exporting the pool, or adding the {{ic|zfs_force&#61;1}} to the kernel boot parameters (which is not ideal). See [[#On boot the zfs pool does not mount stating: "pool may be in use from other system"]]<br />
<br />
To export a pool,<br />
<br />
# zpool export bigdata<br />
<br />
=== Rename a Zpool ===<br />
Renaming a zpool that is already created is accomplished in 2 steps:<br />
<br />
# zpool export oldname<br />
# zpool import oldname newname<br />
<br />
=== Setting a Different Mount Point ===<br />
The mount point for a given zpool can be moved at will with one command:<br />
# zfs set mountpoint=/foo/bar poolname<br />
<br />
=== Swap volume ===<br />
<br />
ZFS does not allow to use swapfiles, but users can use a ZFS volume (ZVOL) as swap. It is importart to set the ZVOL block size to match the system page size, which can be obtained by the {{ic|getconf PAGESIZE}} command (default on x86_64 is 4KiB). Another option useful for keeping the system running well in low-memory situations is not caching the ZVOL data.<br />
<br />
Create a 8GiB zfs volume:<br />
<br />
# zfs create -V 8G -b $(getconf PAGESIZE) \<br />
-o primarycache=metadata \<br />
-o com.sun:auto-snapshot=false <pool>/swap<br />
<br />
Prepare it as swap partition:<br />
<br />
# mkswap -f /dev/zvol/<pool>/swap<br />
# swapon /dev/zvol/<pool>/swap<br />
<br />
To make it permanent, edit {{ic|/etc/fstab}}. ZVOLs support discard, which can potentially help ZFS's block allocator and reduce fragmentation for all other datasets when/if swap is not full.<br />
<br />
Add a line to {{ic|/etc/fstab}}:<br />
<br />
/dev/zvol/<pool>/swap none swap discard 0 0<br />
<br />
{{Out of date|The hibernate hook is deprecated. Does the limitation still apply?}}<br />
Keep in mind the Hibernate hook must be loaded before filesystems, so using ZVOL as swap will not allow to use hibernate function. If you need hibernate, keep a partition for it.<br />
<br />
Make sure to unmount all ZFS filesystems before rebooting the machine, otherwise any ZFS pools will refuse to be imported:<br />
# zfs umount -a<br />
<br />
=== Automatic snapshots ===<br />
<br />
==== ZFS Automatic Snapshot Service for Linux ====<br />
<br />
The {{AUR|zfs-auto-snapshot-git}} package from [[Arch User Repository|AUR]] provides a shell script to automate the management of snapshots, with each named by date and label (hourly, daily, etc), giving quick and convenient snapshotting of all ZFS datasets. The package also installs cron tasks for quarter-hourly, hourly, daily, weekly, and monthly snapshots. Optionally adjust the --keep parameter from the defaults depending on how far back the snapshots are to go (the monthly script by default keeps data for up to a year).<br />
<br />
To prevent a dataset from being snapshotted at all, set {{ic|1=com.sun:auto-snapshot=false}} on it. Likewise, set more fine-grained control as well by label, if, for example, no monthlies are to be kept on a snapshot, for example, set {{ic|1=com.sun:auto-snapshot:monthly=false}}.<br />
<br />
==== ZFS Snapshot Manager ====<br />
<br />
The {{AUR|zfs-snap-manager}} package from [[Arch User Repository|AUR]] provides a python service that takes daily snapshots from a configurable set of ZFS datasets and cleans them out in a [[wikipedia:Backup_rotation_scheme#Grandfather-father-son|"Grandfather-father-son"]] scheme. It can be configured to e.g. keep 7 daily, 5 weekly, 3 monthly and 2 yearly snapshots. <br />
<br />
The package also supports configurable replication to other machines running ZFS by means of {{ic|zfs send}} and {{ic|zfs receive}}. If the destination machine runs this package as well, it could be configured to keep these replicated snapshots for a longer time. This allows a setup where a source machine has only a few daily snapshots locally stored, while on a remote storage server a much longer retention is available.<br />
<br />
==Troubleshooting==<br />
=== ZPool creation fails ===<br />
If the following error occurs then it can be fixed.<br />
<br />
# the kernel failed to rescan the partition table: 16<br />
# cannot label 'sdc': try using parted(8) and then provide a specific slice: -1<br />
<br />
One reason this can occur is because [https://github.com/zfsonlinux/zfs/issues/2582 ZFS expects pool creation to take less than 1 second]. This is a reasonable assumption under ordinary conditions, but in many situations it may take longer. Each drive will need to be cleared again before another attempt can be made.<br />
<br />
# parted /dev/sda rm 1<br />
# parted /dev/sda rm 1<br />
# dd if=/dev/zero of=/dev/sdb bs=512 count=1<br />
# zpool labelclear /dev/sda<br />
<br />
A brute force creation can be attempted over and over again, and with some luck the ZPool creation will take less than 1 second.<br />
Once cause for creation slowdown can be slow burst read writes on a drive. By reading from the disk in parallell to ZPool creation, it may be possible to increase burst speeds.<br />
<br />
# dd if=/dev/sda of=/dev/null<br />
<br />
This can be done with multiple drives by saving the above command for each drive to a file on separate lines and running <br />
<br />
# cat $FILE | parallel<br />
<br />
Then run ZPool creation at the same time.<br />
<br />
=== ZFS is using too much RAM ===<br />
<br />
By default, ZFS caches file operations ([[wikipedia:Adaptive replacement cache|ARC]]) using up to two-thirds of available system memory on the host. To adjust the ARC size, add the following to the [[Kernel parameters]] list:<br />
<br />
zfs.zfs_arc_max=536870912 # (for 512MB)<br />
<br />
For a more detailed description, as well as other configuration options, see [http://wiki.gentoo.org/wiki/ZFS#ARC gentoo-wiki:zfs#arc].<br />
<br />
=== Does not contain an EFI label ===<br />
<br />
The following error will occur when attempting to create a zfs filesystem,<br />
<br />
/dev/disk/by-id/<id> does not contain an EFI label but it may contain partition<br />
<br />
The way to overcome this is to use {{ic|-f}} with the zfs create command.<br />
<br />
=== No hostid found ===<br />
<br />
An error that occurs at boot with the following lines appearing before initscript output:<br />
<br />
ZFS: No hostid found on kernel command line or /etc/hostid.<br />
<br />
This warning occurs because the ZFS module does not have access to the spl hosted. There are two solutions, for this. Either place the spl hostid in the [[kernel parameters]] in the boot loader. For example, adding {{ic|1=spl.spl_hostid=0x00bab10c}}.<br />
<br />
The other solution is to make sure that there is a hostid in {{ic|/etc/hostid}}, and then regenerate the initramfs image. Which will copy the hostid into the initramfs image.<br />
<br />
# mkinitcpio -p linux<br />
<br />
=== On boot the zfs pool does not mount stating: "pool may be in use from other system" ===<br />
<br />
==== Unexported pool ====<br />
<br />
If the new installation does not boot because the zpool cannot be imported, chroot into the installation and properly export the zpool. See [[#Emergency chroot repair with archzfs]].<br />
<br />
Once inside the chroot environment, load the ZFS module and force import the zpool,<br />
<br />
# zpool import -a -f<br />
<br />
now export the pool:<br />
<br />
# zpool export <pool><br />
<br />
To see the available pools, use,<br />
<br />
# zpool status<br />
<br />
It is necessary to export a pool because of the way ZFS uses the hostid to track the system the zpool was created on. The hostid is generated partly based on the network setup. During the installation in the archiso the network configuration could be different generating a different hostid than the one contained in the new installation. Once the zfs filesystem is exported and then re-imported in the new installation, the hostid is reset. See [http://osdir.com/ml/zfs-discuss/2011-06/msg00227.html Re: Howto zpool import/export automatically? - msg#00227].<br />
<br />
If ZFS complains about "pool may be in use" after every reboot, properly export pool as described above, and then rebuild ramdisk in normally booted system:<br />
<br />
# mkinitcpio -p linux<br />
<br />
==== Incorrect hostid ====<br />
<br />
Double check that the pool is properly exported. Exporting the zpool clears the hostid marking the ownership. So during the first boot the zpool should mount correctly. If it does not there is some other problem.<br />
<br />
Reboot again, if the zfs pool refuses to mount it means the hostid is not yet correctly set in the early boot phase and it confuses zfs. Manually tell zfs the correct number, once the hostid is coherent across the reboots the zpool will mount correctly.<br />
<br />
Boot using zfs_force and write down the hostid. This one is just an example.<br />
% hostid<br />
0a0af0f8<br />
<br />
This number have to be added to the [[kernel parameters]] as {{ic|spl.spl_hostid&#61;0x0a0af0f8}}. Another solution is writing the hostid inside the initram image, see the [[Installing Arch Linux on ZFS#After the first boot|installation guide]] explanation about this.<br />
<br />
Users can always ignore the check adding {{ic|zfs_force&#61;1}} in the [[kernel parameters]], but it is not advisable as a permanent solution.<br />
<br />
=== Devices have different sector alignment ===<br />
<br />
Once a drive has become faulted it should be replaced A.S.A.P. with an identical drive.<br />
<br />
# zpool replace bigdata ata-ST3000DM001-9YN166_S1F0KDGY ata-ST3000DM001-1CH166_W1F478BD -f<br />
<br />
but in this instance, the following error is produced:<br />
<br />
cannot replace ata-ST3000DM001-9YN166_S1F0KDGY with ata-ST3000DM001-1CH166_W1F478BD: devices have different sector alignment<br />
<br />
ZFS uses the ashift option to adjust for physical block size. When replacing the faulted disk, ZFS is attempting to use {{ic|<nowiki>ashift=12</nowiki>}}, but the faulted disk is using a different ashift (probably {{ic|<nowiki>ashift=9</nowiki>}}) and this causes the resulting error. <br />
<br />
For Advanced Format Disks with 4KB blocksize, an ashift of 12 is recommended for best performance. See [http://zfsonlinux.org/faq.html#PerformanceConsideration 1.10 What’s going on with performance?] and [http://wiki.illumos.org/display/illumos/ZFS+and+Advanced+Format+disks ZFS and Advanced Format disks].<br />
<br />
Use zdb to find the ashift of the zpool: {{ic|zdb | grep ashift}}, then use the {{ic|-o}} argument to set the ashift of the replacement drive:<br />
<br />
# zpool replace bigdata ata-ST3000DM001-9YN166_S1F0KDGY ata-ST3000DM001-1CH166_W1F478BD -o ashift=9 -f<br />
<br />
Check the zpool status for confirmation:<br />
<br />
{{hc|# zpool status -v|<br />
pool: bigdata<br />
state: DEGRADED<br />
status: One or more devices is currently being resilvered. The pool will<br />
continue to function, possibly in a degraded state.<br />
action: Wait for the resilver to complete.<br />
scan: resilver in progress since Mon Jun 16 11:16:28 2014<br />
10.3G scanned out of 5.90T at 81.7M/s, 20h59m to go<br />
2.57G resilvered, 0.17% done<br />
config:<br />
<br />
NAME STATE READ WRITE CKSUM<br />
bigdata DEGRADED 0 0 0<br />
raidz1-0 DEGRADED 0 0 0<br />
replacing-0 OFFLINE 0 0 0<br />
ata-ST3000DM001-9YN166_S1F0KDGY OFFLINE 0 0 0<br />
ata-ST3000DM001-1CH166_W1F478BD ONLINE 0 0 0 (resilvering)<br />
ata-ST3000DM001-9YN166_S1F0JKRR ONLINE 0 0 0<br />
ata-ST3000DM001-9YN166_S1F0KBP8 ONLINE 0 0 0<br />
ata-ST3000DM001-9YN166_S1F0JTM1 ONLINE 0 0 0<br />
<br />
errors: No known data errors}}<br />
<br />
== Tips and tricks ==<br />
<br />
===Embed the archzfs packages into an archiso===<br />
<br />
Follow the [[Archiso]] steps for creating a fully functional Arch Linux live CD/DVD/USB image.<br />
<br />
Enable the [[Unofficial user repositories#archzfs|archzfs]] repository:<br />
<br />
{{hc|~/archlive/pacman.conf|<nowiki><br />
...<br />
[archzfs]<br />
Server = http://archzfs.com/$repo/x86_64<br />
</nowiki>}}<br />
<br />
Add the {{ic|archzfs-linux}} group to the list of packages to be installed (the {{ic|archzfs}} repository provides packages for the x86_64 architecture only).<br />
<br />
{{hc|~/archlive/packages.x86_64|<br />
...<br />
archzfs-linux<br />
}}<br />
<br />
Complete [[Archiso#Build_the_ISO|Build the ISO]] to finally build the iso.<br />
<br />
=== Encryption in ZFS on linux ===<br />
<br />
ZFS on linux does not support encryption directly, but zpools can be created in dm-crypt block devices. Since the zpool is created on the plain-text<br />
abstraction it is possible to have the data encrypted while having all the advantages of ZFS like deduplication, compression, and data robustness.<br />
<br />
dm-crypt, possibly via LUKS, creates devices in {{ic|/dev/mapper}} and their name is fixed. So you just need to change {{ic|zpool create}} commands to<br />
point to that names. The idea is configuring the system to create the {{ic|/dev/mapper}} block devices and import the zpools from there.<br />
Since zpools can be created in multiple devices (raid, mirroring, striping, ...), it is important all the devices are encrypted otherwise the protection<br />
might be partially lost.<br />
<br />
For example, an encrypted zpool can be created using plain dm-crypt (without LUKS) with:<br />
<br />
# cryptsetup --hash=sha512 --cipher=twofish-xts-plain64 --offset=0 \<br />
--key-file=/dev/sdZ --key-size=512 open --type=plain /dev/sdX enc<br />
# zpool create zroot /dev/mapper/enc<br />
<br />
In the case of a root filesystem pool, the {{ic|mkinicpio.conf}} HOOKS line will enable the keyboard for the password, create the devices, and load the pools. It will contain something like:<br />
<br />
HOOKS="... keyboard encrypt zfs ..."<br />
<br />
Since the {{ic|/dev/mapper/enc}} name is fixed no import errors will occur.<br />
<br />
Creating encrypted zpools works fine. But if you need encrypted directories, for example to protect your users' homes, ZFS loses some functionality.<br />
<br />
ZFS will see the encrypted data, not the plain-text abstraction, so compression and deduplication will not work. The reason is that encrypted data has always high entropy making compression ineffective and even from the same input you get different output (thanks to salting) making deduplication impossible.<br />
To reduce the unnecessary overhead it is possible to create a sub-filesystem for each encrypted directory and use [[eCryptfs]] on it.<br />
<br />
For example to have an encrypted home: (the two passwords, encryption and login, must be the same)<br />
# zfs create -o compression=off \<br />
-o dedup=off \<br />
-o mountpoint=/home/<username> \<br />
<zpool>/<username><br />
# useradd -m <username><br />
# passwd <username><br />
# ecryptfs-migrate-home -u <username><br />
<log in user and complete the procedure with ecryptfs-unwrap-passphrase><br />
<br />
=== Emergency chroot repair with archzfs ===<br />
<br />
To get into the ZFS filesystem from live system for maintenance, there are two options:<br />
<br />
# Build custom archiso with ZFS as described in [[#Embed the archzfs packages into an archiso]].<br />
# Boot the latest official archiso and bring up the network. Then enable [[Unofficial_user_repositories#archzfs|archzfs]] repository inside the live system as usual, sync the pacman package database and install the ''archzfs-archiso-linux'' package.<br />
<br />
To start the recovery, load the ZFS kernel modules:<br />
<br />
# modprobe zfs<br />
<br />
Import the pool:<br />
<br />
# zpool import -a -R /mnt<br />
<br />
Mount the boot partitions (if any):<br />
<br />
# mount /dev/sda2 /mnt/boot<br />
# mount /dev/sda1 /mnt/boot/efi<br />
<br />
Chroot into the ZFS filesystem:<br />
<br />
# arch-chroot /mnt /bin/bash<br />
<br />
Check the kernel version:<br />
<br />
# pacman -Qi linux<br />
# uname -r<br />
<br />
uname will show the kernel version of the archiso. If they are different, run depmod (in the chroot) with the correct kernel version of the chroot installation:<br />
<br />
# depmod -a 3.6.9-1-ARCH (version gathered from pacman -Qi linux but using the matching kernel modules directory name under the chroot's /lib/modules)<br />
<br />
This will load the correct kernel modules for the kernel version installed in the chroot installation.<br />
<br />
Regenerate the ramdisk:<br />
<br />
# mkinitcpio -p linux<br />
<br />
There should be no errors.<br />
<br />
=== Bindmount ===<br />
Here a bind mount from /mnt/zfspool to /srv/nfs4/music is created. The configuration ensures that the zfs pool is ready before the bind mount is created.<br />
<br />
==== fstab ====<br />
See [http://www.freedesktop.org/software/systemd/man/systemd.mount.html systemd.mount] for more information on how systemd converts fstab into mount unit files with [http://www.freedesktop.org/software/systemd/man/systemd-fstab-generator.html systemd-fstab-generator].<br />
<br />
{{hc|/etc/fstab|<nowiki><br />
/mnt/zfspool /srv/nfs4/music none bind,defaults,nofail,x-systemd.requires=zfs-mount.service 0 0<br />
</nowiki>}}<br />
<br />
==== systemd mount unit ====<br />
<br />
If it is not possible to bindmount a directory residing on zfs onto another directory using fstab, because the fstab is read before the zfs pool is ready, you can overcome this limitation with a systemd mount unit can be used for the bind mount. The name of the mount unit must be equal to the directory mentioned after "Where", replace slashes with minuses. See [[http://utcc.utoronto.ca/~cks/space/blog/linux/SystemdAndBindMounts]] and [[http://utcc.utoronto.ca/~cks/space/blog/linux/SystemdBindMountUnits]] for more details.<br />
{{hc|srv-nfs4-music.mount|<nowiki><br />
[Mount]<br />
What=/mnt/zfspool<br />
Where=/srv/nfs4/music<br />
Type=none<br />
Options=bind<br />
<br />
[Unit]<br />
DefaultDependencies=no<br />
Conflicts=umount.target<br />
Before=local-fs.target umount.target<br />
After=zfs-mount.service<br />
Requires=zfs-mount.service<br />
ConditionPathIsDirectory=/mnt/zfspool<br />
<br />
[Install]<br />
WantedBy=local-fs.target<br />
</nowiki>}}<br />
<br />
== See also ==<br />
<br />
* [[Installing Arch Linux on ZFS]]<br />
* [http://zfsonlinux.org/ ZFS on Linux]<br />
* [http://zfsonlinux.org/faq.html ZFS on Linux FAQ]<br />
* [http://www.freebsd.org/doc/en_US.ISO8859-1/books/handbook/filesystems-zfs.html FreeBSD Handbook -- The Z File System]<br />
* [http://docs.oracle.com/cd/E19253-01/819-5461/index.html Oracle Solaris ZFS Administration Guide]<br />
* [http://www.solarisinternals.com/wiki/index.php/ZFS_Troubleshooting_Guide Solaris Internals -- ZFS Troubleshooting Guide]<br />
* [http://royal.pingdom.com/2013/06/04/zfs-backup/ Pingdom details how it backs up 5TB of MySQL data every day with ZFS]<br />
<br />
; Aaron Toponce has authored a 17-part blog on ZFS which is an excellent read.<br />
# [https://pthree.org/2012/12/04/zfs-administration-part-i-vdevs/ VDEVs]<br />
# [https://pthree.org/2012/12/05/zfs-administration-part-ii-raidz/ RAIDZ Levels]<br />
# [https://pthree.org/2012/12/06/zfs-administration-part-iii-the-zfs-intent-log/ The ZFS Intent Log]<br />
# [https://pthree.org/2012/12/07/zfs-administration-part-iv-the-adjustable-replacement-cache/ The ARC]<br />
# [https://pthree.org/2012/12/10/zfs-administration-part-v-exporting-and-importing-zpools/ Import/export zpools]<br />
# [https://pthree.org/2012/12/11/zfs-administration-part-vi-scrub-and-resilver/ Scrub and Resilver]<br />
# [https://pthree.org/2012/12/12/zfs-administration-part-vii-zpool-properties/ Zpool Properties]<br />
# [https://pthree.org/2012/12/13/zfs-administration-part-viii-zpool-best-practices-and-caveats/ Zpool Best Practices]<br />
# [https://pthree.org/2012/12/14/zfs-administration-part-ix-copy-on-write/ Copy on Write]<br />
# [https://pthree.org/2012/12/17/zfs-administration-part-x-creating-filesystems/ Creating Filesystems]<br />
# [https://pthree.org/2012/12/18/zfs-administration-part-xi-compression-and-deduplication/ Compression and Deduplication]<br />
# [https://pthree.org/2012/12/19/zfs-administration-part-xii-snapshots-and-clones/ Snapshots and Clones]<br />
# [https://pthree.org/2012/12/20/zfs-administration-part-xiii-sending-and-receiving-filesystems/ Send/receive Filesystems]<br />
# [https://pthree.org/2012/12/21/zfs-administration-part-xiv-zvols/ ZVOLs]<br />
# [https://pthree.org/2012/12/31/zfs-administration-part-xv-iscsi-nfs-and-samba/ iSCSI, NFS, and Samba]<br />
# [https://pthree.org/2013/01/02/zfs-administration-part-xvi-getting-and-setting-properties/ Get/Set Properties]<br />
# [https://pthree.org/2013/01/03/zfs-administration-part-xvii-best-practices-and-caveats/ ZFS Best Practices]</div>Justin8https://wiki.archlinux.org/index.php?title=ZFS&diff=436566ZFS2016-05-28T12:46:10Z<p>Justin8: /* DKMS */ understanding initcpio is irrelevant to DKMS hooks now.</p>
<hr />
<div>[[Category:File systems]]<br />
[[ja:ZFS]]<br />
{{Related articles start}}<br />
{{Related|File systems}}<br />
{{Related|Experimenting with ZFS}}<br />
{{Related|Installing Arch Linux on ZFS}}<br />
{{Related|ZFS on FUSE}}<br />
{{Related articles end}}<br />
<br />
[[Wikipedia:ZFS|ZFS]] is an advanced filesystem created by [[Wikipedia:Sun Microsystems|Sun Microsystems]] (now owned by Oracle) and released for OpenSolaris in November 2005. <br />
<br />
Features of ZFS include: pooled storage (integrated volume management – zpool), [[Wikipedia:Copy-on-write|Copy-on-write]], [[Wikipedia:Snapshot (computer storage)|snapshots]], data integrity verification and automatic repair (scrubbing), [[Wikipedia:RAID-Z|RAID-Z]], a maximum [[Wikipedia:Exabyte|16 Exabyte]] file size, and a maximum 256 Quadrillion [[Wikipedia:Zettabyte|Zettabytes]] storage with no limit on number of filesystems (datasets) or files[http://docs.oracle.com/cd/E19253-01/819-5461/zfsover-2/index.html]. ZFS is licensed under the [[Wikipedia:CDDL|Common Development and Distribution License]] (CDDL).<br />
<br />
Described as [http://web.archive.org/web/20060428092023/http://www.sun.com/2004-0914/feature/ "The last word in filesystems"] ZFS is stable, fast, secure, and future-proof. Being licensed under the CDDL, and thus incompatible with GPL, it is not possible for ZFS to be distributed along with the Linux Kernel. This requirement, however, does not prevent a native Linux kernel module from being developed and distributed by a third party, as is the case with [http://zfsonlinux.org/ zfsonlinux.org] (ZOL).<br />
<br />
ZOL is a project funded by the [https://www.llnl.gov/ Lawrence Livermore National Laboratory] to develop a native Linux kernel module for its massive storage requirements and super computers.<br />
<br />
==Installation==<br />
=== General ===<br />
Install {{AUR|zfs-linux-git}} from the [[Arch User Repository]] or the [[Unofficial user repositories#archzfs|archzfs]] repository. This package has {{AUR|zfs-utils-linux-git}} and {{AUR|spl-linux-git}} as a dependency, which in turn has {{AUR|spl-utils-linux-git}} as dependency. SPL (Solaris Porting Layer) is a Linux Kernel module implementing Solaris APIs for ZFS compatibility.<br />
<br />
For users that desire ZFS builds from stable releases, {{AUR|zfs-linux-lts}} is available from the [[Arch User Repository]] or the [[Unofficial user repositories#archzfs|archzfs]] repository. A script to build ZFS and its dependencies automatically can be found [[#Automated build script|here]].<br />
<br />
{{warning|The ZFS and SPL kernel modules are tied to a specific kernel version. It would not be possible to apply any kernel updates until updated packages are uploaded to AUR or the [[Unofficial user repositories#archzfs|archzfs]] repository.}}<br />
<br />
Test the installation by issuing {{ic|zpool status}} on the command line. If an "insmod" error is produced, try {{ic|depmod -a}}.<br />
<br />
{{Tip| You can [[downgrade]] your linux version to the one from [[Unofficial user repositories#archzfs|archzfs]] repo if your current kenel is newer.}}<br />
<br />
=== Root on ZFS ===<br />
<br />
When performing an Arch install on ZFS, {{AUR|zfs-git}}{{Broken package link|{{aur-mirror|zfs-git}}}} and its dependencies can be installed in the archiso environment as outlined in the previous section.<br />
<br />
It may be useful to prepare a [[#Embed the archzfs packages into an archiso|customized archiso]] with ZFS support builtin. For a much more detailed guide on installing Arch with ZFS as its root file system, see [[Installing Arch Linux on ZFS]].<br />
<br />
=== DKMS ===<br />
<br />
Users can make use of DKMS [[Dynamic Kernel Module Support]] to rebuild the ZFS modules automatically with every kernel upgrade. <br />
<br />
Install {{AUR|zfs-dkms}} or {{AUR|zfs-dkms-git}} and apply the post-install instructions given by these packages.<br />
<br />
{{Tip|Add an {{ic|IgnorePkg}} entry to [[pacman.conf]] to prevent these packages from upgrading when doing a regular update.}}<br />
<br />
==Experimenting with ZFS ==<br />
<br />
Users wishing to experiment with ZFS on ''virtual block devices'' (known in ZFS terms as VDEVs) which can be simple files like {{ic|~/zfs0.img}} {{ic|~/zfs1.img}} {{ic|~/zfs2.img}} etc. with no possibility of real data loss are encouraged to see the [[Experimenting with ZFS]] article. Common tasks like building a RAIDZ array, purposefully corrupting data and recovering it, snapshotting datasets, etc. are covered.<br />
<br />
==Configuration==<br />
<br />
ZFS is considered a "zero administration" filesystem by its creators; therefore, configuring ZFS is very straight forward. Configuration is done primarily with two commands: {{ic|zfs}} and {{ic|zpool}}.<br />
<br />
===Automatic Start===<br />
<br />
For ZFS to live by its "zero administration" namesake, the zfs daemon must be loaded at startup. A benefit to this is that it is not necessary to mount the zpool in {{ic|/etc/fstab}}; the zfs daemon can import and mount zfs pools automatically. The daemon mounts the zfs pools reading the file {{ic|/etc/zfs/zpool.cache}}.<br />
<br />
For each pool you want automatically mounted by the zfs daemon execute:<br />
# zpool set cachefile=/etc/zfs/zpool.cache <pool><br />
<br />
Enable the service so it is automatically started at boot time:<br />
<br />
# systemctl enable zfs.target<br />
<br />
To manually start the daemon:<br />
<br />
# systemctl start zfs.target<br />
<br />
==Create a storage pool==<br />
<br />
Use {{ic| # parted --list}} to see a list of all available drives. It is not necessary nor recommended to partition the drives before creating the zfs filesystem.<br />
<br />
{{Note|If some or all device have been used in a software RAID set it is paramount to erase any old RAID configuration information. ([[Mdadm#Prepare_the_Devices]]) }}<br />
<br />
{{Warning|For Advanced Format Disks with 4KB sector size, an ashift of 12 is recommended for best performance. Advanced Format disks emulate a sector size of 512 bytes for compatibility with legacy systems, this causes ZFS to sometimes use an ashift option number that is not ideal. Once the pool has been created, the only way to change the ashift option is to recreate the pool. Using an ashift of 12 would also decrease available capacity. See [http://zfsonlinux.org/faq.html#PerformanceConsideration 1.10 What’s going on with performance?], [http://zfsonlinux.org/faq.html#HowDoesZFSonLinuxHandleAdvacedFormatDrives 1.15 How does ZFS on Linux handle Advanced Format disks?], and [http://wiki.illumos.org/display/illumos/ZFS+and+Advanced+Format+disks ZFS and Advanced Format disks].}}<br />
<br />
Having identified the list of drives, it is now time to get the id's of the drives to add to the zpool. The [http://zfsonlinux.org/faq.html#WhatDevNamesShouldIUseWhenCreatingMyPool zfs on Linux developers recommend] using device ids when creating ZFS storage pools of less than 10 devices. To find the id's, simply:<br />
<br />
# ls -lh /dev/disk/by-id/<br />
<br />
The ids should look similar to the following:<br />
<br />
lrwxrwxrwx 1 root root 9 Aug 12 16:26 ata-ST3000DM001-9YN166_S1F0JKRR -> ../../sdc<br />
lrwxrwxrwx 1 root root 9 Aug 12 16:26 ata-ST3000DM001-9YN166_S1F0JTM1 -> ../../sde<br />
lrwxrwxrwx 1 root root 9 Aug 12 16:26 ata-ST3000DM001-9YN166_S1F0KBP8 -> ../../sdd<br />
lrwxrwxrwx 1 root root 9 Aug 12 16:26 ata-ST3000DM001-9YN166_S1F0KDGY -> ../../sdb<br />
<br />
{{Style|Missing references to [[Persistent block device naming]], it is useless to explain the differences (or even what they are) here.}}<br />
<br />
Disk labels and UUID can also be used for ZFS mounts by using [[GUID Partition Table|GPT]] partitions. ZFS drives have labels but Linux is unable to read them at boot. Unlike [[Master Boot Record|MBR]] partitions, GPT partitions directly support both UUID and labels independent of the format inside the partition. Partitioning rather than using the whole disk for ZFS offers two additional advantages. The OS does not generate bogus partition numbers from whatever unpredictable data ZFS has written to the partition sector, and if desired, you can easily over provision SSD drives, and slightly over provision spindle drives to ensure that different models with slightly different sector counts can zpool replace into your mirrors. This is a lot of organization and control over ZFS using readily available tools and techniques at almost zero cost.<br />
<br />
Use [[GUID Partition Table|gdisk]] to partition the all or part of the drive as a single partition. gdisk does not automatically name partitions so if partition labels are desired use gdisk command "c" to label the partitions. Some reasons you might prefer labels over UUID are: labels are easy to control, labels can be titled to make the purpose of each disk in your arrangement readily apparent, and labels are shorter and easier to type. These are all advantages when the server is down and the heat is on. GPT partition labels have plenty of space and can store most international characters [[wikipedia:GUID_Partition_Table#Partition_entries]] allowing large data pools to be labeled in an organized fashion.<br />
<br />
Drives partitioned with GPT have labels and UUID that look like this. <br />
<br />
# ls -l /dev/disk/by-partlabel<br />
# lrwxrwxrwx 1 root root 10 Apr 30 01:44 zfsdata1 -> ../../sdd1<br />
# lrwxrwxrwx 1 root root 10 Apr 30 01:44 zfsdata2 -> ../../sdc1<br />
# lrwxrwxrwx 1 root root 10 Apr 30 01:59 zfsl2arc -> ../../sda1<br />
<br />
# ls -l /dev/disk/by-partuuid<br />
# lrwxrwxrwx 1 root root 10 Apr 30 01:44 148c462c-7819-431a-9aba-5bf42bb5a34e -> ../../sdd1<br />
# lrwxrwxrwx 1 root root 10 Apr 30 01:59 4f95da30-b2fb-412b-9090-fc349993df56 -> ../../sda1<br />
# lrwxrwxrwx 1 root root 10 Apr 30 01:44 e5ccef58-5adf-4094-81a7-3bac846a885f -> ../../sdc1<br />
<br />
Now, finally, create the ZFS pool:<br />
<br />
# zpool create -f -m <mount> <pool> raidz <ids><br />
<br />
* '''create''': subcommand to create the pool.<br />
<br />
* '''-f''': Force creating the pool. This is to overcome the "EFI label error". See [[#Does not contain an EFI label]].<br />
<br />
* '''-m''': The mount point of the pool. If this is not specified, then the pool will be mounted to {{ic|/<pool>}}.<br />
<br />
* '''pool''': This is the name of the pool.<br />
<br />
* '''raidz''': This is the type of virtual device that will be created from the pool of devices. Raidz is a special implementation of raid5. See [https://blogs.oracle.com/bonwick/entry/raid_z Jeff Bonwick's Blog -- RAID-Z] for more information about raidz.<br />
<br />
* '''ids''': The names of the drives or partitions that to include into the pool. Get it from {{ic|/dev/disk/by-id}}.<br />
<br />
Here is an example for the full command:<br />
<br />
# zpool create -f -m /mnt/data bigdata \<br />
raidz \<br />
ata-ST3000DM001-9YN166_S1F0KDGY \<br />
ata-ST3000DM001-9YN166_S1F0JKRR \<br />
ata-ST3000DM001-9YN166_S1F0KBP8 \<br />
ata-ST3000DM001-9YN166_S1F0JTM1<br />
<br />
=== Advanced format disks ===<br />
<br />
In case Advanced Format disks are used which have a native sector size of 4096 bytes instead of 512 bytes, the automated sector size detection algorithm of ZFS might detect 512 bytes because the backwards compatibility with legacy systems. This would result in degraded performance. To make sure a correct sector size is used, the {{ic|<nowiki>ashift=12</nowiki>}} option should be used (See the [http://zfsonlinux.org/faq.html#HowDoesZFSonLinuxHandleAdvacedFormatDrives ZFS on Linux FAQ]). The full command would in this case be:<br />
<br />
# zpool create -f -o ashift=12 -m /mnt/data bigdata \<br />
raidz \<br />
ata-ST3000DM001-9YN166_S1F0KDGY \<br />
ata-ST3000DM001-9YN166_S1F0JKRR \<br />
ata-ST3000DM001-9YN166_S1F0KBP8 \<br />
ata-ST3000DM001-9YN166_S1F0JTM1<br />
<br />
=== Verifying pool creation ===<br />
<br />
If the command is successful, there will be no output. Using the {{ic|$ mount}} command will show that the pool is mounted. Using {{ic|# zpool status}} will show that the pool has been created.<br />
<br />
{{hc|# zpool status|<br />
pool: bigdata<br />
state: ONLINE<br />
scan: none requested<br />
config:<br />
<br />
NAME STATE READ WRITE CKSUM<br />
bigdata ONLINE 0 0 0<br />
-0 ONLINE 0 0 0<br />
ata-ST3000DM001-9YN166_S1F0KDGY-part1 ONLINE 0 0 0<br />
ata-ST3000DM001-9YN166_S1F0JKRR-part1 ONLINE 0 0 0<br />
ata-ST3000DM001-9YN166_S1F0KBP8-part1 ONLINE 0 0 0<br />
ata-ST3000DM001-9YN166_S1F0JTM1-part1 ONLINE 0 0 0<br />
<br />
errors: No known data errors<br />
}}<br />
<br />
At this point it would be good to reboot the machine to ensure that the ZFS pool is mounted at boot. It is best to deal with all errors before transferring data.<br />
<br />
=== GRUB-compatible pool creation ===<br />
<br />
By default, ''zpool'' will enable all features on a pool. If {{ic|/boot}} resides on ZFS and when using [[GRUB]], you must only enable read-only, or non-read-only features supported by GRUB ({{ic|lz4_compress}} as of version 2.02.beta2). Otherwise GRUB will not be able to read the pool.<br />
<br />
# zpool create -f -d \<br />
-o feature@async_destroy=enabled \<br />
-o feature@empty_bpobj=enabled \<br />
-o feature@lz4_compress=enabled \<br />
-o feature@spacemap_histogram=enabled \<br />
-o feature@enabled_txg=enabled \<br />
<pool_name> <vdevs><br />
<br />
To create a mirrored vdev for GRUB you would use a command like this:<br />
<br />
# zpool create <pool_name> mirror -f -d \<br />
-o feature@async_destroy=enabled \<br />
-o feature@empty_bpobj=enabled \<br />
-o feature@lz4_compress=enabled \<br />
-o feature@spacemap_histogram=enabled \<br />
-o feature@enabled_txg=enabled \<br />
/dev/disk/by-id/ata-disk-part2 /dev/disk/by-id/ata-disk-part2<br />
<br />
{{Tip|As of September 2015, GRUB's development tree supports {{ic|extensible_dataset}}, {{ic|hole_birth}}, {{ic|embedded_data}}, and {{ic|large_blocks}}, making it viable to use a pool with all features enabled, either at create time or by using {{ic|zpool upgrade <pool_name>}}, if {{AUR|grub-git}} is installed.}}<br />
<br />
=== Importing a pool created by id ===<br />
<br />
Eventually a pool may fail to auto mount and you need to import to bring your pool back. Take care to avoid the most obvious solution.<br />
<br />
# ###zpool import zfsdata # Do not do this! Always use -d<br />
<br />
This will import your pools using {{ic|/dev/sd?}} which will lead to problems the next time you rearrange your drives. This may be as simple as rebooting with a USB drive left in the machine, which harkens back to a time when PC's would not boot when a floppy disk was left in a machine. Adapt one of the following commands to import your pool so that pool imports retain the persistence they were created with.<br />
<br />
# zpool import -d /dev/disk/by-id zfsdata<br />
# zpool import -d /dev/disk/by-partlabel zfsdata<br />
# zpool import -d /dev/disk/by-partuuid zfsdata<br />
<br />
== Tuning ==<br />
<br />
=== General ===<br />
Many parameters are available for zfs file systems, you can view a full list with {{ic|zfs get all <pool>}}. Two common ones to adjust are '''atime''' and '''compression'''.<br />
<br />
Atime is enabled by default but for most users, it represents superfluous writes to the zpool and it can be disabled using the zfs command:<br />
# zfs set atime=off <pool><br />
<br />
As an alternative to turning off atime completely, '''relatime''' is available. This brings the default ext4/xfs atime semantics to ZFS, where access time is only updated if the modified time or changed time changes, or if the existing access time has not been updated within the past 24 hours. It is a compromise between atime=off and atime=on. This property ''only'' takes effect if '''atime''' is '''on''':<br />
# zfs set relatime=on <pool><br />
<br />
Compression is just that, transparent compression of data. ZFS supports a few different algorithms, presently lz4 is the default. '''gzip''' is also available for seldom-written yet highly-compressable data; consult the man page for more details. Enable compression using the zfs command:<br />
# zfs set compression=on <pool><br />
<br />
Other options for zfs can be displayed again, using the zfs command:<br />
# zfs get all <pool><br />
<br />
=== Database ===<br />
ZFS, unlike most other file systems, has a variable record size, or what is commonly referred to as a block size. By default, the recordsize on ZFS is 128KiB, which means it will dynamically allocate blocks of any size from 512B to 128KiB depending on the size of file being written. This can often help fragmentation and file access, at the cost that ZFS would have to allocate new 128KiB blocks each time only a few bytes are written to.<br />
<br />
Most RDBMSes work in 8KiB-sized blocks by default. Although the block size is tunable for [[MySQL|MySQL/MariaDB]], [[PostgreSQL]], and [[Oracle]], all three of them use an 8KiB block size ''by default''. For both performance concerns and keeping snapshot differences to a minimum (for backup purposes, this is helpful), it is usually desirable to tune ZFS instead to accommodate the databases, using a command such as:<br />
# zfs set recordsize=8K <pool>/postgres<br />
<br />
These RDBMSes also tend to implement their own caching algorithm, often similar to ZFS's own ARC. In the interest of saving memory, it is best to simply disable ZFS's caching of the database's file data and let the database do its own job:<br />
# zfs set primarycache=metadata <pool>/postgres<br />
<br />
If your pool has no configured log devices, ZFS reserves space on the pool's data disks for its intent log (the ZIL). ZFS uses this for crash recovery, but databases are often syncing their data files to the file system on their own transaction commits anyway. The end result of this is that ZFS will be committing data '''twice''' to the data disks, and it can severely impact performance. You can tell ZFS to prefer to not use the ZIL, and in which case, data is only committed to the file system once. Setting this for non-database file systems, or for pools with configured log devices, can actually ''negatively'' impact the performance, so beware:<br />
# zfs set logbias=throughput <pool>/postgres<br />
<br />
These can also be done at file system creation time, for example:<br />
# zfs create -o recordsize=8K \<br />
-o primarycache=metadata \<br />
-o mountpoint=/var/lib/postgres \<br />
-o logbias=throughput \<br />
<pool>/postgres<br />
<br />
Please note: these kinds of tuning parameters are ideal for specialized applications like RDBMSes. You can easily ''hurt'' ZFS's performance by setting these on a general-purpose file system such as your /home directory.<br />
<br />
=== /tmp ===<br />
If you would like to use ZFS to store your /tmp directory, which may be useful for storing arbitrarily-large sets of files or simply keeping your RAM free of idle data, you can generally improve performance of certain applications writing to /tmp by disabling file system sync. This causes ZFS to ignore an application's sync requests (eg, with {{ic|fsync}} or {{ic|O_SYNC}}) and return immediately. While this has severe application-side data consistency consequences (never disable sync for a database!), files in /tmp are less likely to be important and affected. Please note this does ''not'' affect the integrity of ZFS itself, only the possibility that data an application expects on-disk may not have actually been written out following a crash.<br />
# zfs set sync=disabled <pool>/tmp<br />
<br />
Additionally, for security purposes, you may want to disable '''setuid''' and '''devices''' on the /tmp file system, which prevents some kinds of privilege-escalation attacks or the use of device nodes:<br />
# zfs set setuid=off <pool>/tmp<br />
# zfs set devices=off <pool>/tmp<br />
<br />
Combining all of these for a create command would be as follows:<br />
# zfs create -o setuid=off -o devices=off -o sync=disabled -o mountpoint=/tmp <pool>/tmp<br />
<br />
Please note, also, that if you want /tmp on ZFS, you will need to mask (disable) [[systemd]]'s automatic tmpfs-backed /tmp, else ZFS will be unable to mount your dataset at boot-time or import-time:<br />
# systemctl mask tmp.mount<br />
<br />
=== ZVOLs ===<br />
<br />
ZFS volumes (ZVOLs) can suffer from the same block size-related issues as RDBMSes, but it is worth noting that the default recordsize for ZVOLs is 8KiB already. If possible, it is best to align any partitions contained in a ZVOL to your recordsize (current versions of fdisk and gdisk by default automatically align at 1MiB segments, which works), and file system block sizes to the same size. Other than this, you might tweak the '''recordsize''' to accommodate the data inside the ZVOL as necessary (though 8KiB tends to be a good value for most file systems, even when using 4KiB blocks on that level).<br />
<br />
==== RAIDZ and Advanced Format physical disks ====<br />
<br />
Each block of a ZVOL gets its own parity disks, and if you have physical media with logical block sizes of 4096B, 8192B, or so on, the parity needs to be stored in whole physical blocks, and this can drastically increase the space requirements of a ZVOL, requiring 2× or more physical storage capacity than the ZVOL's logical capacity. Setting the '''recordsize''' to 16k or 32k can help reduce this footprint drastically.<br />
<br />
See [https://github.com/zfsonlinux/zfs/issues/1807 ZFS on Linux issue #1807 for details]<br />
<br />
== Usage ==<br />
<br />
Users can optionally create a dataset under the zpool as opposed to manually creating directories under the zpool. Datasets allow for an increased level of control (quotas for example) in addition to snapshots. To be able to create and mount a dataset, a directory of the same name must not pre-exist in the zpool. To create a dataset, use:<br />
<br />
# zfs create <nameofzpool>/<nameofdataset><br />
<br />
It is then possible to apply ZFS specific attributes to the dataset. For example, one could assign a quota limit to a specific directory within a dataset:<br />
<br />
# zfs set quota=20G <nameofzpool>/<nameofdataset>/<directory><br />
<br />
To see all the commands available in ZFS, use :<br />
<br />
$ man zfs<br />
<br />
or:<br />
<br />
$ man zpool<br />
<br />
=== Scrub ===<br />
<br />
ZFS pools should be scrubbed at least once a week. To scrub the pool:<br />
<br />
# zpool scrub <pool><br />
<br />
To do automatic scrubbing once a week, set the following line in the root crontab:<br />
<br />
{{hc|# crontab -e|<br />
...<br />
30 19 * * 5 zpool scrub <pool><br />
...<br />
}}<br />
<br />
Replace {{ic|<pool>}} with the name of the ZFS pool.<br />
<br />
=== Check zfs pool status ===<br />
<br />
To print a nice table with statistics about the ZFS pool, including and read/write errors, use<br />
<br />
# zpool status -v<br />
<br />
=== Destroy a storage pool ===<br />
<br />
ZFS makes it easy to destroy a mounted storage pool, removing all metadata about the ZFS device. This command destroys any data contained in the pool:<br />
<br />
# zpool destroy <pool><br />
<br />
And now when checking the status:<br />
<br />
{{hc|# zpool status|no pools available}}<br />
<br />
To find the name of the pool, see [[#Check zfs pool status]].<br />
<br />
=== Export a storage pool ===<br />
<br />
If a storage pool is to be used on another system, it will first need to be exported. It is also necessary to export a pool if it has been imported from the archiso as the hostid is different in the archiso as it is in the booted system. The zpool command will refuse to import any storage pools that have not been exported. It is possible to force the import with the {{ic|-f}} argument, but this is considered bad form.<br />
<br />
Any attempts made to import an un-exported storage pool will result in an error stating the storage pool is in use by another system. This error can be produced at boot time abruptly abandoning the system in the busybox console and requiring an archiso to do an emergency repair by either exporting the pool, or adding the {{ic|zfs_force&#61;1}} to the kernel boot parameters (which is not ideal). See [[#On boot the zfs pool does not mount stating: "pool may be in use from other system"]]<br />
<br />
To export a pool,<br />
<br />
# zpool export bigdata<br />
<br />
=== Rename a Zpool ===<br />
Renaming a zpool that is already created is accomplished in 2 steps:<br />
<br />
# zpool export oldname<br />
# zpool import oldname newname<br />
<br />
=== Setting a Different Mount Point ===<br />
The mount point for a given zpool can be moved at will with one command:<br />
# zfs set mountpoint=/foo/bar poolname<br />
<br />
=== Swap volume ===<br />
<br />
ZFS does not allow to use swapfiles, but users can use a ZFS volume (ZVOL) as swap. It is importart to set the ZVOL block size to match the system page size, which can be obtained by the {{ic|getconf PAGESIZE}} command (default on x86_64 is 4KiB). Another option useful for keeping the system running well in low-memory situations is not caching the ZVOL data.<br />
<br />
Create a 8GiB zfs volume:<br />
<br />
# zfs create -V 8G -b $(getconf PAGESIZE) \<br />
-o primarycache=metadata \<br />
-o com.sun:auto-snapshot=false <pool>/swap<br />
<br />
Prepare it as swap partition:<br />
<br />
# mkswap -f /dev/zvol/<pool>/swap<br />
# swapon /dev/zvol/<pool>/swap<br />
<br />
To make it permanent, edit {{ic|/etc/fstab}}. ZVOLs support discard, which can potentially help ZFS's block allocator and reduce fragmentation for all other datasets when/if swap is not full.<br />
<br />
Add a line to {{ic|/etc/fstab}}:<br />
<br />
/dev/zvol/<pool>/swap none swap discard 0 0<br />
<br />
{{Out of date|The hibernate hook is deprecated. Does the limitation still apply?}}<br />
Keep in mind the Hibernate hook must be loaded before filesystems, so using ZVOL as swap will not allow to use hibernate function. If you need hibernate, keep a partition for it.<br />
<br />
Make sure to unmount all ZFS filesystems before rebooting the machine, otherwise any ZFS pools will refuse to be imported:<br />
# zfs umount -a<br />
<br />
=== Automatic snapshots ===<br />
<br />
==== ZFS Automatic Snapshot Service for Linux ====<br />
<br />
The {{AUR|zfs-auto-snapshot-git}} package from [[Arch User Repository|AUR]] provides a shell script to automate the management of snapshots, with each named by date and label (hourly, daily, etc), giving quick and convenient snapshotting of all ZFS datasets. The package also installs cron tasks for quarter-hourly, hourly, daily, weekly, and monthly snapshots. Optionally adjust the --keep parameter from the defaults depending on how far back the snapshots are to go (the monthly script by default keeps data for up to a year).<br />
<br />
To prevent a dataset from being snapshotted at all, set {{ic|1=com.sun:auto-snapshot=false}} on it. Likewise, set more fine-grained control as well by label, if, for example, no monthlies are to be kept on a snapshot, for example, set {{ic|1=com.sun:auto-snapshot:monthly=false}}.<br />
<br />
==== ZFS Snapshot Manager ====<br />
<br />
The {{AUR|zfs-snap-manager}} package from [[Arch User Repository|AUR]] provides a python service that takes daily snapshots from a configurable set of ZFS datasets and cleans them out in a [[wikipedia:Backup_rotation_scheme#Grandfather-father-son|"Grandfather-father-son"]] scheme. It can be configured to e.g. keep 7 daily, 5 weekly, 3 monthly and 2 yearly snapshots. <br />
<br />
The package also supports configurable replication to other machines running ZFS by means of {{ic|zfs send}} and {{ic|zfs receive}}. If the destination machine runs this package as well, it could be configured to keep these replicated snapshots for a longer time. This allows a setup where a source machine has only a few daily snapshots locally stored, while on a remote storage server a much longer retention is available.<br />
<br />
==Troubleshooting==<br />
=== ZPool creation fails ===<br />
If the following error occurs then it can be fixed.<br />
<br />
# the kernel failed to rescan the partition table: 16<br />
# cannot label 'sdc': try using parted(8) and then provide a specific slice: -1<br />
<br />
One reason this can occur is because [https://github.com/zfsonlinux/zfs/issues/2582 ZFS expects pool creation to take less than 1 second]. This is a reasonable assumption under ordinary conditions, but in many situations it may take longer. Each drive will need to be cleared again before another attempt can be made.<br />
<br />
# parted /dev/sda rm 1<br />
# parted /dev/sda rm 1<br />
# dd if=/dev/zero of=/dev/sdb bs=512 count=1<br />
# zpool labelclear /dev/sda<br />
<br />
A brute force creation can be attempted over and over again, and with some luck the ZPool creation will take less than 1 second.<br />
Once cause for creation slowdown can be slow burst read writes on a drive. By reading from the disk in parallell to ZPool creation, it may be possible to increase burst speeds.<br />
<br />
# dd if=/dev/sda of=/dev/null<br />
<br />
This can be done with multiple drives by saving the above command for each drive to a file on separate lines and running <br />
<br />
# cat $FILE | parallel<br />
<br />
Then run ZPool creation at the same time.<br />
<br />
=== ZFS is using too much RAM ===<br />
<br />
By default, ZFS caches file operations ([[wikipedia:Adaptive replacement cache|ARC]]) using up to two-thirds of available system memory on the host. To adjust the ARC size, add the following to the [[Kernel parameters]] list:<br />
<br />
zfs.zfs_arc_max=536870912 # (for 512MB)<br />
<br />
For a more detailed description, as well as other configuration options, see [http://wiki.gentoo.org/wiki/ZFS#ARC gentoo-wiki:zfs#arc].<br />
<br />
=== Does not contain an EFI label ===<br />
<br />
The following error will occur when attempting to create a zfs filesystem,<br />
<br />
/dev/disk/by-id/<id> does not contain an EFI label but it may contain partition<br />
<br />
The way to overcome this is to use {{ic|-f}} with the zfs create command.<br />
<br />
=== No hostid found ===<br />
<br />
An error that occurs at boot with the following lines appearing before initscript output:<br />
<br />
ZFS: No hostid found on kernel command line or /etc/hostid.<br />
<br />
This warning occurs because the ZFS module does not have access to the spl hosted. There are two solutions, for this. Either place the spl hostid in the [[kernel parameters]] in the boot loader. For example, adding {{ic|1=spl.spl_hostid=0x00bab10c}}.<br />
<br />
The other solution is to make sure that there is a hostid in {{ic|/etc/hostid}}, and then regenerate the initramfs image. Which will copy the hostid into the initramfs image.<br />
<br />
# mkinitcpio -p linux<br />
<br />
=== On boot the zfs pool does not mount stating: "pool may be in use from other system" ===<br />
<br />
==== Unexported pool ====<br />
<br />
If the new installation does not boot because the zpool cannot be imported, chroot into the installation and properly export the zpool. See [[#Emergency chroot repair with archzfs]].<br />
<br />
Once inside the chroot environment, load the ZFS module and force import the zpool,<br />
<br />
# zpool import -a -f<br />
<br />
now export the pool:<br />
<br />
# zpool export <pool><br />
<br />
To see the available pools, use,<br />
<br />
# zpool status<br />
<br />
It is necessary to export a pool because of the way ZFS uses the hostid to track the system the zpool was created on. The hostid is generated partly based on the network setup. During the installation in the archiso the network configuration could be different generating a different hostid than the one contained in the new installation. Once the zfs filesystem is exported and then re-imported in the new installation, the hostid is reset. See [http://osdir.com/ml/zfs-discuss/2011-06/msg00227.html Re: Howto zpool import/export automatically? - msg#00227].<br />
<br />
If ZFS complains about "pool may be in use" after every reboot, properly export pool as described above, and then rebuild ramdisk in normally booted system:<br />
<br />
# mkinitcpio -p linux<br />
<br />
==== Incorrect hostid ====<br />
<br />
Double check that the pool is properly exported. Exporting the zpool clears the hostid marking the ownership. So during the first boot the zpool should mount correctly. If it does not there is some other problem.<br />
<br />
Reboot again, if the zfs pool refuses to mount it means the hostid is not yet correctly set in the early boot phase and it confuses zfs. Manually tell zfs the correct number, once the hostid is coherent across the reboots the zpool will mount correctly.<br />
<br />
Boot using zfs_force and write down the hostid. This one is just an example.<br />
% hostid<br />
0a0af0f8<br />
<br />
This number have to be added to the [[kernel parameters]] as {{ic|spl.spl_hostid&#61;0x0a0af0f8}}. Another solution is writing the hostid inside the initram image, see the [[Installing Arch Linux on ZFS#After the first boot|installation guide]] explanation about this.<br />
<br />
Users can always ignore the check adding {{ic|zfs_force&#61;1}} in the [[kernel parameters]], but it is not advisable as a permanent solution.<br />
<br />
=== Devices have different sector alignment ===<br />
<br />
Once a drive has become faulted it should be replaced A.S.A.P. with an identical drive.<br />
<br />
# zpool replace bigdata ata-ST3000DM001-9YN166_S1F0KDGY ata-ST3000DM001-1CH166_W1F478BD -f<br />
<br />
but in this instance, the following error is produced:<br />
<br />
cannot replace ata-ST3000DM001-9YN166_S1F0KDGY with ata-ST3000DM001-1CH166_W1F478BD: devices have different sector alignment<br />
<br />
ZFS uses the ashift option to adjust for physical block size. When replacing the faulted disk, ZFS is attempting to use {{ic|<nowiki>ashift=12</nowiki>}}, but the faulted disk is using a different ashift (probably {{ic|<nowiki>ashift=9</nowiki>}}) and this causes the resulting error. <br />
<br />
For Advanced Format Disks with 4KB blocksize, an ashift of 12 is recommended for best performance. See [http://zfsonlinux.org/faq.html#PerformanceConsideration 1.10 What’s going on with performance?] and [http://wiki.illumos.org/display/illumos/ZFS+and+Advanced+Format+disks ZFS and Advanced Format disks].<br />
<br />
Use zdb to find the ashift of the zpool: {{ic|zdb | grep ashift}}, then use the {{ic|-o}} argument to set the ashift of the replacement drive:<br />
<br />
# zpool replace bigdata ata-ST3000DM001-9YN166_S1F0KDGY ata-ST3000DM001-1CH166_W1F478BD -o ashift=9 -f<br />
<br />
Check the zpool status for confirmation:<br />
<br />
{{hc|# zpool status -v|<br />
pool: bigdata<br />
state: DEGRADED<br />
status: One or more devices is currently being resilvered. The pool will<br />
continue to function, possibly in a degraded state.<br />
action: Wait for the resilver to complete.<br />
scan: resilver in progress since Mon Jun 16 11:16:28 2014<br />
10.3G scanned out of 5.90T at 81.7M/s, 20h59m to go<br />
2.57G resilvered, 0.17% done<br />
config:<br />
<br />
NAME STATE READ WRITE CKSUM<br />
bigdata DEGRADED 0 0 0<br />
raidz1-0 DEGRADED 0 0 0<br />
replacing-0 OFFLINE 0 0 0<br />
ata-ST3000DM001-9YN166_S1F0KDGY OFFLINE 0 0 0<br />
ata-ST3000DM001-1CH166_W1F478BD ONLINE 0 0 0 (resilvering)<br />
ata-ST3000DM001-9YN166_S1F0JKRR ONLINE 0 0 0<br />
ata-ST3000DM001-9YN166_S1F0KBP8 ONLINE 0 0 0<br />
ata-ST3000DM001-9YN166_S1F0JTM1 ONLINE 0 0 0<br />
<br />
errors: No known data errors}}<br />
<br />
== Tips and tricks ==<br />
<br />
===Embed the archzfs packages into an archiso===<br />
<br />
Follow the [[Archiso]] steps for creating a fully functional Arch Linux live CD/DVD/USB image.<br />
<br />
Enable the [[Unofficial user repositories#archzfs|archzfs]] repository:<br />
<br />
{{hc|~/archlive/pacman.conf|<nowiki><br />
...<br />
[archzfs]<br />
Server = http://archzfs.com/$repo/x86_64<br />
</nowiki>}}<br />
<br />
Add the {{ic|archzfs-linux}} group to the list of packages to be installed (the {{ic|archzfs}} repository provides packages for the x86_64 architecture only).<br />
<br />
{{hc|~/archlive/packages.x86_64|<br />
...<br />
archzfs-linux<br />
}}<br />
<br />
Complete [[Archiso#Build_the_ISO|Build the ISO]] to finally build the iso.<br />
<br />
=== Encryption in ZFS on linux ===<br />
<br />
ZFS on linux does not support encryption directly, but zpools can be created in dm-crypt block devices. Since the zpool is created on the plain-text<br />
abstraction it is possible to have the data encrypted while having all the advantages of ZFS like deduplication, compression, and data robustness.<br />
<br />
dm-crypt, possibly via LUKS, creates devices in {{ic|/dev/mapper}} and their name is fixed. So you just need to change {{ic|zpool create}} commands to<br />
point to that names. The idea is configuring the system to create the {{ic|/dev/mapper}} block devices and import the zpools from there.<br />
Since zpools can be created in multiple devices (raid, mirroring, striping, ...), it is important all the devices are encrypted otherwise the protection<br />
might be partially lost.<br />
<br />
For example, an encrypted zpool can be created using plain dm-crypt (without LUKS) with:<br />
<br />
# cryptsetup --hash=sha512 --cipher=twofish-xts-plain64 --offset=0 \<br />
--key-file=/dev/sdZ --key-size=512 open --type=plain /dev/sdX enc<br />
# zpool create zroot /dev/mapper/enc<br />
<br />
In the case of a root filesystem pool, the {{ic|mkinicpio.conf}} HOOKS line will enable the keyboard for the password, create the devices, and load the pools. It will contain something like:<br />
<br />
HOOKS="... keyboard encrypt zfs ..."<br />
<br />
Since the {{ic|/dev/mapper/enc}} name is fixed no import errors will occur.<br />
<br />
Creating encrypted zpools works fine. But if you need encrypted directories, for example to protect your users' homes, ZFS loses some functionality.<br />
<br />
ZFS will see the encrypted data, not the plain-text abstraction, so compression and deduplication will not work. The reason is that encrypted data has always high entropy making compression ineffective and even from the same input you get different output (thanks to salting) making deduplication impossible.<br />
To reduce the unnecessary overhead it is possible to create a sub-filesystem for each encrypted directory and use [[eCryptfs]] on it.<br />
<br />
For example to have an encrypted home: (the two passwords, encryption and login, must be the same)<br />
# zfs create -o compression=off \<br />
-o dedup=off \<br />
-o mountpoint=/home/<username> \<br />
<zpool>/<username><br />
# useradd -m <username><br />
# passwd <username><br />
# ecryptfs-migrate-home -u <username><br />
<log in user and complete the procedure with ecryptfs-unwrap-passphrase><br />
<br />
=== Emergency chroot repair with archzfs ===<br />
<br />
To get into the ZFS filesystem from live system for maintenance, there are two options:<br />
<br />
# Build custom archiso with ZFS as described in [[#Embed the archzfs packages into an archiso]].<br />
# Boot the latest official archiso and bring up the network. Then enable [[Unofficial_user_repositories#archzfs|archzfs]] repository inside the live system as usual, sync the pacman package database and install the ''archzfs-archiso-linux'' package.<br />
<br />
To start the recovery, load the ZFS kernel modules:<br />
<br />
# modprobe zfs<br />
<br />
Import the pool:<br />
<br />
# zpool import -a -R /mnt<br />
<br />
Mount the boot partitions (if any):<br />
<br />
# mount /dev/sda2 /mnt/boot<br />
# mount /dev/sda1 /mnt/boot/efi<br />
<br />
Chroot into the ZFS filesystem:<br />
<br />
# arch-chroot /mnt /bin/bash<br />
<br />
Check the kernel version:<br />
<br />
# pacman -Qi linux<br />
# uname -r<br />
<br />
uname will show the kernel version of the archiso. If they are different, run depmod (in the chroot) with the correct kernel version of the chroot installation:<br />
<br />
# depmod -a 3.6.9-1-ARCH (version gathered from pacman -Qi linux but using the matching kernel modules directory name under the chroot's /lib/modules)<br />
<br />
This will load the correct kernel modules for the kernel version installed in the chroot installation.<br />
<br />
Regenerate the ramdisk:<br />
<br />
# mkinitcpio -p linux<br />
<br />
There should be no errors.<br />
<br />
=== Bindmount ===<br />
Here a bind mount from /mnt/zfspool to /srv/nfs4/music is created. The configuration ensures that the zfs pool is ready before the bind mount is created.<br />
<br />
==== fstab ====<br />
See [http://www.freedesktop.org/software/systemd/man/systemd.mount.html systemd.mount] for more information on how systemd converts fstab into mount unit files with [http://www.freedesktop.org/software/systemd/man/systemd-fstab-generator.html systemd-fstab-generator].<br />
<br />
{{hc|/etc/fstab|<nowiki><br />
/mnt/zfspool /srv/nfs4/music none bind,defaults,nofail,x-systemd.requires=zfs-mount.service 0 0<br />
</nowiki>}}<br />
<br />
==== systemd mount unit ====<br />
<br />
If it is not possible to bindmount a directory residing on zfs onto another directory using fstab, because the fstab is read before the zfs pool is ready, you can overcome this limitation with a systemd mount unit can be used for the bind mount. The name of the mount unit must be equal to the directory mentioned after "Where", replace slashes with minuses. See [[http://utcc.utoronto.ca/~cks/space/blog/linux/SystemdAndBindMounts]] and [[http://utcc.utoronto.ca/~cks/space/blog/linux/SystemdBindMountUnits]] for more details.<br />
{{hc|srv-nfs4-music.mount|<nowiki><br />
[Mount]<br />
What=/mnt/zfspool<br />
Where=/srv/nfs4/music<br />
Type=none<br />
Options=bind<br />
<br />
[Unit]<br />
DefaultDependencies=no<br />
Conflicts=umount.target<br />
Before=local-fs.target umount.target<br />
After=zfs-mount.service<br />
Requires=zfs-mount.service<br />
ConditionPathIsDirectory=/mnt/zfspool<br />
<br />
[Install]<br />
WantedBy=local-fs.target<br />
</nowiki>}}<br />
<br />
== See also ==<br />
<br />
* [[Installing Arch Linux on ZFS]]<br />
* [http://zfsonlinux.org/ ZFS on Linux]<br />
* [http://zfsonlinux.org/faq.html ZFS on Linux FAQ]<br />
* [http://www.freebsd.org/doc/en_US.ISO8859-1/books/handbook/filesystems-zfs.html FreeBSD Handbook -- The Z File System]<br />
* [http://docs.oracle.com/cd/E19253-01/819-5461/index.html Oracle Solaris ZFS Administration Guide]<br />
* [http://www.solarisinternals.com/wiki/index.php/ZFS_Troubleshooting_Guide Solaris Internals -- ZFS Troubleshooting Guide]<br />
* [http://royal.pingdom.com/2013/06/04/zfs-backup/ Pingdom details how it backs up 5TB of MySQL data every day with ZFS]<br />
<br />
; Aaron Toponce has authored a 17-part blog on ZFS which is an excellent read.<br />
# [https://pthree.org/2012/12/04/zfs-administration-part-i-vdevs/ VDEVs]<br />
# [https://pthree.org/2012/12/05/zfs-administration-part-ii-raidz/ RAIDZ Levels]<br />
# [https://pthree.org/2012/12/06/zfs-administration-part-iii-the-zfs-intent-log/ The ZFS Intent Log]<br />
# [https://pthree.org/2012/12/07/zfs-administration-part-iv-the-adjustable-replacement-cache/ The ARC]<br />
# [https://pthree.org/2012/12/10/zfs-administration-part-v-exporting-and-importing-zpools/ Import/export zpools]<br />
# [https://pthree.org/2012/12/11/zfs-administration-part-vi-scrub-and-resilver/ Scrub and Resilver]<br />
# [https://pthree.org/2012/12/12/zfs-administration-part-vii-zpool-properties/ Zpool Properties]<br />
# [https://pthree.org/2012/12/13/zfs-administration-part-viii-zpool-best-practices-and-caveats/ Zpool Best Practices]<br />
# [https://pthree.org/2012/12/14/zfs-administration-part-ix-copy-on-write/ Copy on Write]<br />
# [https://pthree.org/2012/12/17/zfs-administration-part-x-creating-filesystems/ Creating Filesystems]<br />
# [https://pthree.org/2012/12/18/zfs-administration-part-xi-compression-and-deduplication/ Compression and Deduplication]<br />
# [https://pthree.org/2012/12/19/zfs-administration-part-xii-snapshots-and-clones/ Snapshots and Clones]<br />
# [https://pthree.org/2012/12/20/zfs-administration-part-xiii-sending-and-receiving-filesystems/ Send/receive Filesystems]<br />
# [https://pthree.org/2012/12/21/zfs-administration-part-xiv-zvols/ ZVOLs]<br />
# [https://pthree.org/2012/12/31/zfs-administration-part-xv-iscsi-nfs-and-samba/ iSCSI, NFS, and Samba]<br />
# [https://pthree.org/2013/01/02/zfs-administration-part-xvi-getting-and-setting-properties/ Get/Set Properties]<br />
# [https://pthree.org/2013/01/03/zfs-administration-part-xvii-best-practices-and-caveats/ ZFS Best Practices]</div>Justin8https://wiki.archlinux.org/index.php?title=ZFS&diff=436519ZFS2016-05-28T05:48:44Z<p>Justin8: This works fine now with the new pacman hooks for dkms</p>
<hr />
<div>[[Category:File systems]]<br />
[[ja:ZFS]]<br />
{{Related articles start}}<br />
{{Related|File systems}}<br />
{{Related|Experimenting with ZFS}}<br />
{{Related|Installing Arch Linux on ZFS}}<br />
{{Related|ZFS on FUSE}}<br />
{{Related articles end}}<br />
<br />
[[Wikipedia:ZFS|ZFS]] is an advanced filesystem created by [[Wikipedia:Sun Microsystems|Sun Microsystems]] (now owned by Oracle) and released for OpenSolaris in November 2005. <br />
<br />
Features of ZFS include: pooled storage (integrated volume management – zpool), [[Wikipedia:Copy-on-write|Copy-on-write]], [[Wikipedia:Snapshot (computer storage)|snapshots]], data integrity verification and automatic repair (scrubbing), [[Wikipedia:RAID-Z|RAID-Z]], a maximum [[Wikipedia:Exabyte|16 Exabyte]] file size, and a maximum 256 Quadrillion [[Wikipedia:Zettabyte|Zettabytes]] storage with no limit on number of filesystems (datasets) or files[http://docs.oracle.com/cd/E19253-01/819-5461/zfsover-2/index.html]. ZFS is licensed under the [[Wikipedia:CDDL|Common Development and Distribution License]] (CDDL).<br />
<br />
Described as [http://web.archive.org/web/20060428092023/http://www.sun.com/2004-0914/feature/ "The last word in filesystems"] ZFS is stable, fast, secure, and future-proof. Being licensed under the CDDL, and thus incompatible with GPL, it is not possible for ZFS to be distributed along with the Linux Kernel. This requirement, however, does not prevent a native Linux kernel module from being developed and distributed by a third party, as is the case with [http://zfsonlinux.org/ zfsonlinux.org] (ZOL).<br />
<br />
ZOL is a project funded by the [https://www.llnl.gov/ Lawrence Livermore National Laboratory] to develop a native Linux kernel module for its massive storage requirements and super computers.<br />
<br />
==Installation==<br />
=== General ===<br />
Install {{AUR|zfs-linux-git}} from the [[Arch User Repository]] or the [[Unofficial user repositories#archzfs|archzfs]] repository. This package has {{AUR|zfs-utils-linux-git}} and {{AUR|spl-linux-git}} as a dependency, which in turn has {{AUR|spl-utils-linux-git}} as dependency. SPL (Solaris Porting Layer) is a Linux Kernel module implementing Solaris APIs for ZFS compatibility.<br />
<br />
For users that desire ZFS builds from stable releases, {{AUR|zfs-linux-lts}} is available from the [[Arch User Repository]] or the [[Unofficial user repositories#archzfs|archzfs]] repository. A script to build ZFS and its dependencies automatically can be found [[#Automated build script|here]].<br />
<br />
{{warning|The ZFS and SPL kernel modules are tied to a specific kernel version. It would not be possible to apply any kernel updates until updated packages are uploaded to AUR or the [[Unofficial user repositories#archzfs|archzfs]] repository.}}<br />
<br />
Test the installation by issuing {{ic|zpool status}} on the command line. If an "insmod" error is produced, try {{ic|depmod -a}}.<br />
<br />
{{Tip| You can [[downgrade]] your linux version to the one from [[Unofficial user repositories#archzfs|archzfs]] repo if your current kenel is newer.}}<br />
<br />
=== Root on ZFS ===<br />
<br />
When performing an Arch install on ZFS, {{AUR|zfs-git}}{{Broken package link|{{aur-mirror|zfs-git}}}} and its dependencies can be installed in the archiso environment as outlined in the previous section.<br />
<br />
It may be useful to prepare a [[#Embed the archzfs packages into an archiso|customized archiso]] with ZFS support builtin. For a much more detailed guide on installing Arch with ZFS as its root file system, see [[Installing Arch Linux on ZFS]].<br />
<br />
=== DKMS ===<br />
<br />
Users can make use of DKMS [[Dynamic Kernel Module Support]] to rebuild the ZFS modules automatically with every kernel upgrade. <br />
<br />
Read the [[Mkinitcpio]] wiki entry for a general understanding of the initial ramdisk environment, and adding the dkms hook [[Mkinitcpio#HOOKS]].<br />
<br />
Install {{AUR|zfs-dkms}} or {{AUR|zfs-dkms-git}} and apply the post-install instructions given by these packages.<br />
<br />
{{Tip|Add an {{ic|IgnorePkg}} entry to [[pacman.conf]] to prevent these packages from upgrading when doing a regular update.}}<br />
<br />
==Experimenting with ZFS ==<br />
<br />
Users wishing to experiment with ZFS on ''virtual block devices'' (known in ZFS terms as VDEVs) which can be simple files like {{ic|~/zfs0.img}} {{ic|~/zfs1.img}} {{ic|~/zfs2.img}} etc. with no possibility of real data loss are encouraged to see the [[Experimenting with ZFS]] article. Common tasks like building a RAIDZ array, purposefully corrupting data and recovering it, snapshotting datasets, etc. are covered.<br />
<br />
==Configuration==<br />
<br />
ZFS is considered a "zero administration" filesystem by its creators; therefore, configuring ZFS is very straight forward. Configuration is done primarily with two commands: {{ic|zfs}} and {{ic|zpool}}.<br />
<br />
===Automatic Start===<br />
<br />
For ZFS to live by its "zero administration" namesake, the zfs daemon must be loaded at startup. A benefit to this is that it is not necessary to mount the zpool in {{ic|/etc/fstab}}; the zfs daemon can import and mount zfs pools automatically. The daemon mounts the zfs pools reading the file {{ic|/etc/zfs/zpool.cache}}.<br />
<br />
For each pool you want automatically mounted by the zfs daemon execute:<br />
# zpool set cachefile=/etc/zfs/zpool.cache <pool><br />
<br />
Enable the service so it is automatically started at boot time:<br />
<br />
# systemctl enable zfs.target<br />
<br />
To manually start the daemon:<br />
<br />
# systemctl start zfs.target<br />
<br />
==Create a storage pool==<br />
<br />
Use {{ic| # parted --list}} to see a list of all available drives. It is not necessary nor recommended to partition the drives before creating the zfs filesystem.<br />
<br />
{{Note|If some or all device have been used in a software RAID set it is paramount to erase any old RAID configuration information. ([[Mdadm#Prepare_the_Devices]]) }}<br />
<br />
{{Warning|For Advanced Format Disks with 4KB sector size, an ashift of 12 is recommended for best performance. Advanced Format disks emulate a sector size of 512 bytes for compatibility with legacy systems, this causes ZFS to sometimes use an ashift option number that is not ideal. Once the pool has been created, the only way to change the ashift option is to recreate the pool. Using an ashift of 12 would also decrease available capacity. See [http://zfsonlinux.org/faq.html#PerformanceConsideration 1.10 What’s going on with performance?], [http://zfsonlinux.org/faq.html#HowDoesZFSonLinuxHandleAdvacedFormatDrives 1.15 How does ZFS on Linux handle Advanced Format disks?], and [http://wiki.illumos.org/display/illumos/ZFS+and+Advanced+Format+disks ZFS and Advanced Format disks].}}<br />
<br />
Having identified the list of drives, it is now time to get the id's of the drives to add to the zpool. The [http://zfsonlinux.org/faq.html#WhatDevNamesShouldIUseWhenCreatingMyPool zfs on Linux developers recommend] using device ids when creating ZFS storage pools of less than 10 devices. To find the id's, simply:<br />
<br />
# ls -lh /dev/disk/by-id/<br />
<br />
The ids should look similar to the following:<br />
<br />
lrwxrwxrwx 1 root root 9 Aug 12 16:26 ata-ST3000DM001-9YN166_S1F0JKRR -> ../../sdc<br />
lrwxrwxrwx 1 root root 9 Aug 12 16:26 ata-ST3000DM001-9YN166_S1F0JTM1 -> ../../sde<br />
lrwxrwxrwx 1 root root 9 Aug 12 16:26 ata-ST3000DM001-9YN166_S1F0KBP8 -> ../../sdd<br />
lrwxrwxrwx 1 root root 9 Aug 12 16:26 ata-ST3000DM001-9YN166_S1F0KDGY -> ../../sdb<br />
<br />
{{Style|Missing references to [[Persistent block device naming]], it is useless to explain the differences (or even what they are) here.}}<br />
<br />
Disk labels and UUID can also be used for ZFS mounts by using [[GUID Partition Table|GPT]] partitions. ZFS drives have labels but Linux is unable to read them at boot. Unlike [[Master Boot Record|MBR]] partitions, GPT partitions directly support both UUID and labels independent of the format inside the partition. Partitioning rather than using the whole disk for ZFS offers two additional advantages. The OS does not generate bogus partition numbers from whatever unpredictable data ZFS has written to the partition sector, and if desired, you can easily over provision SSD drives, and slightly over provision spindle drives to ensure that different models with slightly different sector counts can zpool replace into your mirrors. This is a lot of organization and control over ZFS using readily available tools and techniques at almost zero cost.<br />
<br />
Use [[GUID Partition Table|gdisk]] to partition the all or part of the drive as a single partition. gdisk does not automatically name partitions so if partition labels are desired use gdisk command "c" to label the partitions. Some reasons you might prefer labels over UUID are: labels are easy to control, labels can be titled to make the purpose of each disk in your arrangement readily apparent, and labels are shorter and easier to type. These are all advantages when the server is down and the heat is on. GPT partition labels have plenty of space and can store most international characters [[wikipedia:GUID_Partition_Table#Partition_entries]] allowing large data pools to be labeled in an organized fashion.<br />
<br />
Drives partitioned with GPT have labels and UUID that look like this. <br />
<br />
# ls -l /dev/disk/by-partlabel<br />
# lrwxrwxrwx 1 root root 10 Apr 30 01:44 zfsdata1 -> ../../sdd1<br />
# lrwxrwxrwx 1 root root 10 Apr 30 01:44 zfsdata2 -> ../../sdc1<br />
# lrwxrwxrwx 1 root root 10 Apr 30 01:59 zfsl2arc -> ../../sda1<br />
<br />
# ls -l /dev/disk/by-partuuid<br />
# lrwxrwxrwx 1 root root 10 Apr 30 01:44 148c462c-7819-431a-9aba-5bf42bb5a34e -> ../../sdd1<br />
# lrwxrwxrwx 1 root root 10 Apr 30 01:59 4f95da30-b2fb-412b-9090-fc349993df56 -> ../../sda1<br />
# lrwxrwxrwx 1 root root 10 Apr 30 01:44 e5ccef58-5adf-4094-81a7-3bac846a885f -> ../../sdc1<br />
<br />
Now, finally, create the ZFS pool:<br />
<br />
# zpool create -f -m <mount> <pool> raidz <ids><br />
<br />
* '''create''': subcommand to create the pool.<br />
<br />
* '''-f''': Force creating the pool. This is to overcome the "EFI label error". See [[#Does not contain an EFI label]].<br />
<br />
* '''-m''': The mount point of the pool. If this is not specified, then the pool will be mounted to {{ic|/<pool>}}.<br />
<br />
* '''pool''': This is the name of the pool.<br />
<br />
* '''raidz''': This is the type of virtual device that will be created from the pool of devices. Raidz is a special implementation of raid5. See [https://blogs.oracle.com/bonwick/entry/raid_z Jeff Bonwick's Blog -- RAID-Z] for more information about raidz.<br />
<br />
* '''ids''': The names of the drives or partitions that to include into the pool. Get it from {{ic|/dev/disk/by-id}}.<br />
<br />
Here is an example for the full command:<br />
<br />
# zpool create -f -m /mnt/data bigdata \<br />
raidz \<br />
ata-ST3000DM001-9YN166_S1F0KDGY \<br />
ata-ST3000DM001-9YN166_S1F0JKRR \<br />
ata-ST3000DM001-9YN166_S1F0KBP8 \<br />
ata-ST3000DM001-9YN166_S1F0JTM1<br />
<br />
=== Advanced format disks ===<br />
<br />
In case Advanced Format disks are used which have a native sector size of 4096 bytes instead of 512 bytes, the automated sector size detection algorithm of ZFS might detect 512 bytes because the backwards compatibility with legacy systems. This would result in degraded performance. To make sure a correct sector size is used, the {{ic|<nowiki>ashift=12</nowiki>}} option should be used (See the [http://zfsonlinux.org/faq.html#HowDoesZFSonLinuxHandleAdvacedFormatDrives ZFS on Linux FAQ]). The full command would in this case be:<br />
<br />
# zpool create -f -o ashift=12 -m /mnt/data bigdata \<br />
raidz \<br />
ata-ST3000DM001-9YN166_S1F0KDGY \<br />
ata-ST3000DM001-9YN166_S1F0JKRR \<br />
ata-ST3000DM001-9YN166_S1F0KBP8 \<br />
ata-ST3000DM001-9YN166_S1F0JTM1<br />
<br />
=== Verifying pool creation ===<br />
<br />
If the command is successful, there will be no output. Using the {{ic|$ mount}} command will show that the pool is mounted. Using {{ic|# zpool status}} will show that the pool has been created.<br />
<br />
{{hc|# zpool status|<br />
pool: bigdata<br />
state: ONLINE<br />
scan: none requested<br />
config:<br />
<br />
NAME STATE READ WRITE CKSUM<br />
bigdata ONLINE 0 0 0<br />
-0 ONLINE 0 0 0<br />
ata-ST3000DM001-9YN166_S1F0KDGY-part1 ONLINE 0 0 0<br />
ata-ST3000DM001-9YN166_S1F0JKRR-part1 ONLINE 0 0 0<br />
ata-ST3000DM001-9YN166_S1F0KBP8-part1 ONLINE 0 0 0<br />
ata-ST3000DM001-9YN166_S1F0JTM1-part1 ONLINE 0 0 0<br />
<br />
errors: No known data errors<br />
}}<br />
<br />
At this point it would be good to reboot the machine to ensure that the ZFS pool is mounted at boot. It is best to deal with all errors before transferring data.<br />
<br />
=== GRUB-compatible pool creation ===<br />
<br />
By default, ''zpool'' will enable all features on a pool. If {{ic|/boot}} resides on ZFS and when using [[GRUB]], you must only enable read-only, or non-read-only features supported by GRUB ({{ic|lz4_compress}} as of version 2.02.beta2). Otherwise GRUB will not be able to read the pool.<br />
<br />
# zpool create -f -d \<br />
-o feature@async_destroy=enabled \<br />
-o feature@empty_bpobj=enabled \<br />
-o feature@lz4_compress=enabled \<br />
-o feature@spacemap_histogram=enabled \<br />
-o feature@enabled_txg=enabled \<br />
<pool_name> <vdevs><br />
<br />
To create a mirrored vdev for GRUB you would use a command like this:<br />
<br />
# zpool create <pool_name> mirror -f -d \<br />
-o feature@async_destroy=enabled \<br />
-o feature@empty_bpobj=enabled \<br />
-o feature@lz4_compress=enabled \<br />
-o feature@spacemap_histogram=enabled \<br />
-o feature@enabled_txg=enabled \<br />
/dev/disk/by-id/ata-disk-part2 /dev/disk/by-id/ata-disk-part2<br />
<br />
{{Tip|As of September 2015, GRUB's development tree supports {{ic|extensible_dataset}}, {{ic|hole_birth}}, {{ic|embedded_data}}, and {{ic|large_blocks}}, making it viable to use a pool with all features enabled, either at create time or by using {{ic|zpool upgrade <pool_name>}}, if {{AUR|grub-git}} is installed.}}<br />
<br />
=== Importing a pool created by id ===<br />
<br />
Eventually a pool may fail to auto mount and you need to import to bring your pool back. Take care to avoid the most obvious solution.<br />
<br />
# ###zpool import zfsdata # Do not do this! Always use -d<br />
<br />
This will import your pools using {{ic|/dev/sd?}} which will lead to problems the next time you rearrange your drives. This may be as simple as rebooting with a USB drive left in the machine, which harkens back to a time when PC's would not boot when a floppy disk was left in a machine. Adapt one of the following commands to import your pool so that pool imports retain the persistence they were created with.<br />
<br />
# zpool import -d /dev/disk/by-id zfsdata<br />
# zpool import -d /dev/disk/by-partlabel zfsdata<br />
# zpool import -d /dev/disk/by-partuuid zfsdata<br />
<br />
== Tuning ==<br />
<br />
=== General ===<br />
Many parameters are available for zfs file systems, you can view a full list with {{ic|zfs get all <pool>}}. Two common ones to adjust are '''atime''' and '''compression'''.<br />
<br />
Atime is enabled by default but for most users, it represents superfluous writes to the zpool and it can be disabled using the zfs command:<br />
# zfs set atime=off <pool><br />
<br />
As an alternative to turning off atime completely, '''relatime''' is available. This brings the default ext4/xfs atime semantics to ZFS, where access time is only updated if the modified time or changed time changes, or if the existing access time has not been updated within the past 24 hours. It is a compromise between atime=off and atime=on. This property ''only'' takes effect if '''atime''' is '''on''':<br />
# zfs set relatime=on <pool><br />
<br />
Compression is just that, transparent compression of data. ZFS supports a few different algorithms, presently lz4 is the default. '''gzip''' is also available for seldom-written yet highly-compressable data; consult the man page for more details. Enable compression using the zfs command:<br />
# zfs set compression=on <pool><br />
<br />
Other options for zfs can be displayed again, using the zfs command:<br />
# zfs get all <pool><br />
<br />
=== Database ===<br />
ZFS, unlike most other file systems, has a variable record size, or what is commonly referred to as a block size. By default, the recordsize on ZFS is 128KiB, which means it will dynamically allocate blocks of any size from 512B to 128KiB depending on the size of file being written. This can often help fragmentation and file access, at the cost that ZFS would have to allocate new 128KiB blocks each time only a few bytes are written to.<br />
<br />
Most RDBMSes work in 8KiB-sized blocks by default. Although the block size is tunable for [[MySQL|MySQL/MariaDB]], [[PostgreSQL]], and [[Oracle]], all three of them use an 8KiB block size ''by default''. For both performance concerns and keeping snapshot differences to a minimum (for backup purposes, this is helpful), it is usually desirable to tune ZFS instead to accommodate the databases, using a command such as:<br />
# zfs set recordsize=8K <pool>/postgres<br />
<br />
These RDBMSes also tend to implement their own caching algorithm, often similar to ZFS's own ARC. In the interest of saving memory, it is best to simply disable ZFS's caching of the database's file data and let the database do its own job:<br />
# zfs set primarycache=metadata <pool>/postgres<br />
<br />
If your pool has no configured log devices, ZFS reserves space on the pool's data disks for its intent log (the ZIL). ZFS uses this for crash recovery, but databases are often syncing their data files to the file system on their own transaction commits anyway. The end result of this is that ZFS will be committing data '''twice''' to the data disks, and it can severely impact performance. You can tell ZFS to prefer to not use the ZIL, and in which case, data is only committed to the file system once. Setting this for non-database file systems, or for pools with configured log devices, can actually ''negatively'' impact the performance, so beware:<br />
# zfs set logbias=throughput <pool>/postgres<br />
<br />
These can also be done at file system creation time, for example:<br />
# zfs create -o recordsize=8K \<br />
-o primarycache=metadata \<br />
-o mountpoint=/var/lib/postgres \<br />
-o logbias=throughput \<br />
<pool>/postgres<br />
<br />
Please note: these kinds of tuning parameters are ideal for specialized applications like RDBMSes. You can easily ''hurt'' ZFS's performance by setting these on a general-purpose file system such as your /home directory.<br />
<br />
=== /tmp ===<br />
If you would like to use ZFS to store your /tmp directory, which may be useful for storing arbitrarily-large sets of files or simply keeping your RAM free of idle data, you can generally improve performance of certain applications writing to /tmp by disabling file system sync. This causes ZFS to ignore an application's sync requests (eg, with {{ic|fsync}} or {{ic|O_SYNC}}) and return immediately. While this has severe application-side data consistency consequences (never disable sync for a database!), files in /tmp are less likely to be important and affected. Please note this does ''not'' affect the integrity of ZFS itself, only the possibility that data an application expects on-disk may not have actually been written out following a crash.<br />
# zfs set sync=disabled <pool>/tmp<br />
<br />
Additionally, for security purposes, you may want to disable '''setuid''' and '''devices''' on the /tmp file system, which prevents some kinds of privilege-escalation attacks or the use of device nodes:<br />
# zfs set setuid=off <pool>/tmp<br />
# zfs set devices=off <pool>/tmp<br />
<br />
Combining all of these for a create command would be as follows:<br />
# zfs create -o setuid=off -o devices=off -o sync=disabled -o mountpoint=/tmp <pool>/tmp<br />
<br />
Please note, also, that if you want /tmp on ZFS, you will need to mask (disable) [[systemd]]'s automatic tmpfs-backed /tmp, else ZFS will be unable to mount your dataset at boot-time or import-time:<br />
# systemctl mask tmp.mount<br />
<br />
=== ZVOLs ===<br />
<br />
ZFS volumes (ZVOLs) can suffer from the same block size-related issues as RDBMSes, but it is worth noting that the default recordsize for ZVOLs is 8KiB already. If possible, it is best to align any partitions contained in a ZVOL to your recordsize (current versions of fdisk and gdisk by default automatically align at 1MiB segments, which works), and file system block sizes to the same size. Other than this, you might tweak the '''recordsize''' to accommodate the data inside the ZVOL as necessary (though 8KiB tends to be a good value for most file systems, even when using 4KiB blocks on that level).<br />
<br />
==== RAIDZ and Advanced Format physical disks ====<br />
<br />
Each block of a ZVOL gets its own parity disks, and if you have physical media with logical block sizes of 4096B, 8192B, or so on, the parity needs to be stored in whole physical blocks, and this can drastically increase the space requirements of a ZVOL, requiring 2× or more physical storage capacity than the ZVOL's logical capacity. Setting the '''recordsize''' to 16k or 32k can help reduce this footprint drastically.<br />
<br />
See [https://github.com/zfsonlinux/zfs/issues/1807 ZFS on Linux issue #1807 for details]<br />
<br />
== Usage ==<br />
<br />
Users can optionally create a dataset under the zpool as opposed to manually creating directories under the zpool. Datasets allow for an increased level of control (quotas for example) in addition to snapshots. To be able to create and mount a dataset, a directory of the same name must not pre-exist in the zpool. To create a dataset, use:<br />
<br />
# zfs create <nameofzpool>/<nameofdataset><br />
<br />
It is then possible to apply ZFS specific attributes to the dataset. For example, one could assign a quota limit to a specific directory within a dataset:<br />
<br />
# zfs set quota=20G <nameofzpool>/<nameofdataset>/<directory><br />
<br />
To see all the commands available in ZFS, use :<br />
<br />
$ man zfs<br />
<br />
or:<br />
<br />
$ man zpool<br />
<br />
=== Scrub ===<br />
<br />
ZFS pools should be scrubbed at least once a week. To scrub the pool:<br />
<br />
# zpool scrub <pool><br />
<br />
To do automatic scrubbing once a week, set the following line in the root crontab:<br />
<br />
{{hc|# crontab -e|<br />
...<br />
30 19 * * 5 zpool scrub <pool><br />
...<br />
}}<br />
<br />
Replace {{ic|<pool>}} with the name of the ZFS pool.<br />
<br />
=== Check zfs pool status ===<br />
<br />
To print a nice table with statistics about the ZFS pool, including and read/write errors, use<br />
<br />
# zpool status -v<br />
<br />
=== Destroy a storage pool ===<br />
<br />
ZFS makes it easy to destroy a mounted storage pool, removing all metadata about the ZFS device. This command destroys any data contained in the pool:<br />
<br />
# zpool destroy <pool><br />
<br />
And now when checking the status:<br />
<br />
{{hc|# zpool status|no pools available}}<br />
<br />
To find the name of the pool, see [[#Check zfs pool status]].<br />
<br />
=== Export a storage pool ===<br />
<br />
If a storage pool is to be used on another system, it will first need to be exported. It is also necessary to export a pool if it has been imported from the archiso as the hostid is different in the archiso as it is in the booted system. The zpool command will refuse to import any storage pools that have not been exported. It is possible to force the import with the {{ic|-f}} argument, but this is considered bad form.<br />
<br />
Any attempts made to import an un-exported storage pool will result in an error stating the storage pool is in use by another system. This error can be produced at boot time abruptly abandoning the system in the busybox console and requiring an archiso to do an emergency repair by either exporting the pool, or adding the {{ic|zfs_force&#61;1}} to the kernel boot parameters (which is not ideal). See [[#On boot the zfs pool does not mount stating: "pool may be in use from other system"]]<br />
<br />
To export a pool,<br />
<br />
# zpool export bigdata<br />
<br />
=== Rename a Zpool ===<br />
Renaming a zpool that is already created is accomplished in 2 steps:<br />
<br />
# zpool export oldname<br />
# zpool import oldname newname<br />
<br />
=== Setting a Different Mount Point ===<br />
The mount point for a given zpool can be moved at will with one command:<br />
# zfs set mountpoint=/foo/bar poolname<br />
<br />
=== Swap volume ===<br />
<br />
ZFS does not allow to use swapfiles, but users can use a ZFS volume (ZVOL) as swap. It is importart to set the ZVOL block size to match the system page size, which can be obtained by the {{ic|getconf PAGESIZE}} command (default on x86_64 is 4KiB). Another option useful for keeping the system running well in low-memory situations is not caching the ZVOL data.<br />
<br />
Create a 8GiB zfs volume:<br />
<br />
# zfs create -V 8G -b $(getconf PAGESIZE) \<br />
-o primarycache=metadata \<br />
-o com.sun:auto-snapshot=false <pool>/swap<br />
<br />
Prepare it as swap partition:<br />
<br />
# mkswap -f /dev/zvol/<pool>/swap<br />
# swapon /dev/zvol/<pool>/swap<br />
<br />
To make it permanent, edit {{ic|/etc/fstab}}. ZVOLs support discard, which can potentially help ZFS's block allocator and reduce fragmentation for all other datasets when/if swap is not full.<br />
<br />
Add a line to {{ic|/etc/fstab}}:<br />
<br />
/dev/zvol/<pool>/swap none swap discard 0 0<br />
<br />
{{Out of date|The hibernate hook is deprecated. Does the limitation still apply?}}<br />
Keep in mind the Hibernate hook must be loaded before filesystems, so using ZVOL as swap will not allow to use hibernate function. If you need hibernate, keep a partition for it.<br />
<br />
Make sure to unmount all ZFS filesystems before rebooting the machine, otherwise any ZFS pools will refuse to be imported:<br />
# zfs umount -a<br />
<br />
=== Automatic snapshots ===<br />
<br />
==== ZFS Automatic Snapshot Service for Linux ====<br />
<br />
The {{AUR|zfs-auto-snapshot-git}} package from [[Arch User Repository|AUR]] provides a shell script to automate the management of snapshots, with each named by date and label (hourly, daily, etc), giving quick and convenient snapshotting of all ZFS datasets. The package also installs cron tasks for quarter-hourly, hourly, daily, weekly, and monthly snapshots. Optionally adjust the --keep parameter from the defaults depending on how far back the snapshots are to go (the monthly script by default keeps data for up to a year).<br />
<br />
To prevent a dataset from being snapshotted at all, set {{ic|1=com.sun:auto-snapshot=false}} on it. Likewise, set more fine-grained control as well by label, if, for example, no monthlies are to be kept on a snapshot, for example, set {{ic|1=com.sun:auto-snapshot:monthly=false}}.<br />
<br />
==== ZFS Snapshot Manager ====<br />
<br />
The {{AUR|zfs-snap-manager}} package from [[Arch User Repository|AUR]] provides a python service that takes daily snapshots from a configurable set of ZFS datasets and cleans them out in a [[wikipedia:Backup_rotation_scheme#Grandfather-father-son|"Grandfather-father-son"]] scheme. It can be configured to e.g. keep 7 daily, 5 weekly, 3 monthly and 2 yearly snapshots. <br />
<br />
The package also supports configurable replication to other machines running ZFS by means of {{ic|zfs send}} and {{ic|zfs receive}}. If the destination machine runs this package as well, it could be configured to keep these replicated snapshots for a longer time. This allows a setup where a source machine has only a few daily snapshots locally stored, while on a remote storage server a much longer retention is available.<br />
<br />
==Troubleshooting==<br />
=== ZPool creation fails ===<br />
If the following error occurs then it can be fixed.<br />
<br />
# the kernel failed to rescan the partition table: 16<br />
# cannot label 'sdc': try using parted(8) and then provide a specific slice: -1<br />
<br />
One reason this can occur is because [https://github.com/zfsonlinux/zfs/issues/2582 ZFS expects pool creation to take less than 1 second]. This is a reasonable assumption under ordinary conditions, but in many situations it may take longer. Each drive will need to be cleared again before another attempt can be made.<br />
<br />
# parted /dev/sda rm 1<br />
# parted /dev/sda rm 1<br />
# dd if=/dev/zero of=/dev/sdb bs=512 count=1<br />
# zpool labelclear /dev/sda<br />
<br />
A brute force creation can be attempted over and over again, and with some luck the ZPool creation will take less than 1 second.<br />
Once cause for creation slowdown can be slow burst read writes on a drive. By reading from the disk in parallell to ZPool creation, it may be possible to increase burst speeds.<br />
<br />
# dd if=/dev/sda of=/dev/null<br />
<br />
This can be done with multiple drives by saving the above command for each drive to a file on separate lines and running <br />
<br />
# cat $FILE | parallel<br />
<br />
Then run ZPool creation at the same time.<br />
<br />
=== ZFS is using too much RAM ===<br />
<br />
By default, ZFS caches file operations ([[wikipedia:Adaptive replacement cache|ARC]]) using up to two-thirds of available system memory on the host. To adjust the ARC size, add the following to the [[Kernel parameters]] list:<br />
<br />
zfs.zfs_arc_max=536870912 # (for 512MB)<br />
<br />
For a more detailed description, as well as other configuration options, see [http://wiki.gentoo.org/wiki/ZFS#ARC gentoo-wiki:zfs#arc].<br />
<br />
=== Does not contain an EFI label ===<br />
<br />
The following error will occur when attempting to create a zfs filesystem,<br />
<br />
/dev/disk/by-id/<id> does not contain an EFI label but it may contain partition<br />
<br />
The way to overcome this is to use {{ic|-f}} with the zfs create command.<br />
<br />
=== No hostid found ===<br />
<br />
An error that occurs at boot with the following lines appearing before initscript output:<br />
<br />
ZFS: No hostid found on kernel command line or /etc/hostid.<br />
<br />
This warning occurs because the ZFS module does not have access to the spl hosted. There are two solutions, for this. Either place the spl hostid in the [[kernel parameters]] in the boot loader. For example, adding {{ic|1=spl.spl_hostid=0x00bab10c}}.<br />
<br />
The other solution is to make sure that there is a hostid in {{ic|/etc/hostid}}, and then regenerate the initramfs image. Which will copy the hostid into the initramfs image.<br />
<br />
# mkinitcpio -p linux<br />
<br />
=== On boot the zfs pool does not mount stating: "pool may be in use from other system" ===<br />
<br />
==== Unexported pool ====<br />
<br />
If the new installation does not boot because the zpool cannot be imported, chroot into the installation and properly export the zpool. See [[#Emergency chroot repair with archzfs]].<br />
<br />
Once inside the chroot environment, load the ZFS module and force import the zpool,<br />
<br />
# zpool import -a -f<br />
<br />
now export the pool:<br />
<br />
# zpool export <pool><br />
<br />
To see the available pools, use,<br />
<br />
# zpool status<br />
<br />
It is necessary to export a pool because of the way ZFS uses the hostid to track the system the zpool was created on. The hostid is generated partly based on the network setup. During the installation in the archiso the network configuration could be different generating a different hostid than the one contained in the new installation. Once the zfs filesystem is exported and then re-imported in the new installation, the hostid is reset. See [http://osdir.com/ml/zfs-discuss/2011-06/msg00227.html Re: Howto zpool import/export automatically? - msg#00227].<br />
<br />
If ZFS complains about "pool may be in use" after every reboot, properly export pool as described above, and then rebuild ramdisk in normally booted system:<br />
<br />
# mkinitcpio -p linux<br />
<br />
==== Incorrect hostid ====<br />
<br />
Double check that the pool is properly exported. Exporting the zpool clears the hostid marking the ownership. So during the first boot the zpool should mount correctly. If it does not there is some other problem.<br />
<br />
Reboot again, if the zfs pool refuses to mount it means the hostid is not yet correctly set in the early boot phase and it confuses zfs. Manually tell zfs the correct number, once the hostid is coherent across the reboots the zpool will mount correctly.<br />
<br />
Boot using zfs_force and write down the hostid. This one is just an example.<br />
% hostid<br />
0a0af0f8<br />
<br />
This number have to be added to the [[kernel parameters]] as {{ic|spl.spl_hostid&#61;0x0a0af0f8}}. Another solution is writing the hostid inside the initram image, see the [[Installing Arch Linux on ZFS#After the first boot|installation guide]] explanation about this.<br />
<br />
Users can always ignore the check adding {{ic|zfs_force&#61;1}} in the [[kernel parameters]], but it is not advisable as a permanent solution.<br />
<br />
=== Devices have different sector alignment ===<br />
<br />
Once a drive has become faulted it should be replaced A.S.A.P. with an identical drive.<br />
<br />
# zpool replace bigdata ata-ST3000DM001-9YN166_S1F0KDGY ata-ST3000DM001-1CH166_W1F478BD -f<br />
<br />
but in this instance, the following error is produced:<br />
<br />
cannot replace ata-ST3000DM001-9YN166_S1F0KDGY with ata-ST3000DM001-1CH166_W1F478BD: devices have different sector alignment<br />
<br />
ZFS uses the ashift option to adjust for physical block size. When replacing the faulted disk, ZFS is attempting to use {{ic|<nowiki>ashift=12</nowiki>}}, but the faulted disk is using a different ashift (probably {{ic|<nowiki>ashift=9</nowiki>}}) and this causes the resulting error. <br />
<br />
For Advanced Format Disks with 4KB blocksize, an ashift of 12 is recommended for best performance. See [http://zfsonlinux.org/faq.html#PerformanceConsideration 1.10 What’s going on with performance?] and [http://wiki.illumos.org/display/illumos/ZFS+and+Advanced+Format+disks ZFS and Advanced Format disks].<br />
<br />
Use zdb to find the ashift of the zpool: {{ic|zdb | grep ashift}}, then use the {{ic|-o}} argument to set the ashift of the replacement drive:<br />
<br />
# zpool replace bigdata ata-ST3000DM001-9YN166_S1F0KDGY ata-ST3000DM001-1CH166_W1F478BD -o ashift=9 -f<br />
<br />
Check the zpool status for confirmation:<br />
<br />
{{hc|# zpool status -v|<br />
pool: bigdata<br />
state: DEGRADED<br />
status: One or more devices is currently being resilvered. The pool will<br />
continue to function, possibly in a degraded state.<br />
action: Wait for the resilver to complete.<br />
scan: resilver in progress since Mon Jun 16 11:16:28 2014<br />
10.3G scanned out of 5.90T at 81.7M/s, 20h59m to go<br />
2.57G resilvered, 0.17% done<br />
config:<br />
<br />
NAME STATE READ WRITE CKSUM<br />
bigdata DEGRADED 0 0 0<br />
raidz1-0 DEGRADED 0 0 0<br />
replacing-0 OFFLINE 0 0 0<br />
ata-ST3000DM001-9YN166_S1F0KDGY OFFLINE 0 0 0<br />
ata-ST3000DM001-1CH166_W1F478BD ONLINE 0 0 0 (resilvering)<br />
ata-ST3000DM001-9YN166_S1F0JKRR ONLINE 0 0 0<br />
ata-ST3000DM001-9YN166_S1F0KBP8 ONLINE 0 0 0<br />
ata-ST3000DM001-9YN166_S1F0JTM1 ONLINE 0 0 0<br />
<br />
errors: No known data errors}}<br />
<br />
== Tips and tricks ==<br />
<br />
===Embed the archzfs packages into an archiso===<br />
<br />
Follow the [[Archiso]] steps for creating a fully functional Arch Linux live CD/DVD/USB image.<br />
<br />
Enable the [[Unofficial user repositories#archzfs|archzfs]] repository:<br />
<br />
{{hc|~/archlive/pacman.conf|<nowiki><br />
...<br />
[archzfs]<br />
Server = http://archzfs.com/$repo/x86_64<br />
</nowiki>}}<br />
<br />
Add the {{ic|archzfs-linux}} group to the list of packages to be installed (the {{ic|archzfs}} repository provides packages for the x86_64 architecture only).<br />
<br />
{{hc|~/archlive/packages.x86_64|<br />
...<br />
archzfs-linux<br />
}}<br />
<br />
Complete [[Archiso#Build_the_ISO|Build the ISO]] to finally build the iso.<br />
<br />
=== Encryption in ZFS on linux ===<br />
<br />
ZFS on linux does not support encryption directly, but zpools can be created in dm-crypt block devices. Since the zpool is created on the plain-text<br />
abstraction it is possible to have the data encrypted while having all the advantages of ZFS like deduplication, compression, and data robustness.<br />
<br />
dm-crypt, possibly via LUKS, creates devices in {{ic|/dev/mapper}} and their name is fixed. So you just need to change {{ic|zpool create}} commands to<br />
point to that names. The idea is configuring the system to create the {{ic|/dev/mapper}} block devices and import the zpools from there.<br />
Since zpools can be created in multiple devices (raid, mirroring, striping, ...), it is important all the devices are encrypted otherwise the protection<br />
might be partially lost.<br />
<br />
For example, an encrypted zpool can be created using plain dm-crypt (without LUKS) with:<br />
<br />
# cryptsetup --hash=sha512 --cipher=twofish-xts-plain64 --offset=0 \<br />
--key-file=/dev/sdZ --key-size=512 open --type=plain /dev/sdX enc<br />
# zpool create zroot /dev/mapper/enc<br />
<br />
In the case of a root filesystem pool, the {{ic|mkinicpio.conf}} HOOKS line will enable the keyboard for the password, create the devices, and load the pools. It will contain something like:<br />
<br />
HOOKS="... keyboard encrypt zfs ..."<br />
<br />
Since the {{ic|/dev/mapper/enc}} name is fixed no import errors will occur.<br />
<br />
Creating encrypted zpools works fine. But if you need encrypted directories, for example to protect your users' homes, ZFS loses some functionality.<br />
<br />
ZFS will see the encrypted data, not the plain-text abstraction, so compression and deduplication will not work. The reason is that encrypted data has always high entropy making compression ineffective and even from the same input you get different output (thanks to salting) making deduplication impossible.<br />
To reduce the unnecessary overhead it is possible to create a sub-filesystem for each encrypted directory and use [[eCryptfs]] on it.<br />
<br />
For example to have an encrypted home: (the two passwords, encryption and login, must be the same)<br />
# zfs create -o compression=off \<br />
-o dedup=off \<br />
-o mountpoint=/home/<username> \<br />
<zpool>/<username><br />
# useradd -m <username><br />
# passwd <username><br />
# ecryptfs-migrate-home -u <username><br />
<log in user and complete the procedure with ecryptfs-unwrap-passphrase><br />
<br />
=== Emergency chroot repair with archzfs ===<br />
<br />
To get into the ZFS filesystem from live system for maintenance, there are two options:<br />
<br />
# Build custom archiso with ZFS as described in [[#Embed the archzfs packages into an archiso]].<br />
# Boot the latest official archiso and bring up the network. Then enable [[Unofficial_user_repositories#archzfs|archzfs]] repository inside the live system as usual, sync the pacman package database and install the ''archzfs-archiso-linux'' package.<br />
<br />
To start the recovery, load the ZFS kernel modules:<br />
<br />
# modprobe zfs<br />
<br />
Import the pool:<br />
<br />
# zpool import -a -R /mnt<br />
<br />
Mount the boot partitions (if any):<br />
<br />
# mount /dev/sda2 /mnt/boot<br />
# mount /dev/sda1 /mnt/boot/efi<br />
<br />
Chroot into the ZFS filesystem:<br />
<br />
# arch-chroot /mnt /bin/bash<br />
<br />
Check the kernel version:<br />
<br />
# pacman -Qi linux<br />
# uname -r<br />
<br />
uname will show the kernel version of the archiso. If they are different, run depmod (in the chroot) with the correct kernel version of the chroot installation:<br />
<br />
# depmod -a 3.6.9-1-ARCH (version gathered from pacman -Qi linux but using the matching kernel modules directory name under the chroot's /lib/modules)<br />
<br />
This will load the correct kernel modules for the kernel version installed in the chroot installation.<br />
<br />
Regenerate the ramdisk:<br />
<br />
# mkinitcpio -p linux<br />
<br />
There should be no errors.<br />
<br />
=== Bindmount ===<br />
Here a bind mount from /mnt/zfspool to /srv/nfs4/music is created. The configuration ensures that the zfs pool is ready before the bind mount is created.<br />
<br />
==== fstab ====<br />
See [http://www.freedesktop.org/software/systemd/man/systemd.mount.html systemd.mount] for more information on how systemd converts fstab into mount unit files with [http://www.freedesktop.org/software/systemd/man/systemd-fstab-generator.html systemd-fstab-generator].<br />
<br />
{{hc|/etc/fstab|<nowiki><br />
/mnt/zfspool /srv/nfs4/music none bind,defaults,nofail,x-systemd.requires=zfs-mount.service 0 0<br />
</nowiki>}}<br />
<br />
==== systemd mount unit ====<br />
<br />
If it is not possible to bindmount a directory residing on zfs onto another directory using fstab, because the fstab is read before the zfs pool is ready, you can overcome this limitation with a systemd mount unit can be used for the bind mount. The name of the mount unit must be equal to the directory mentioned after "Where", replace slashes with minuses. See [[http://utcc.utoronto.ca/~cks/space/blog/linux/SystemdAndBindMounts]] and [[http://utcc.utoronto.ca/~cks/space/blog/linux/SystemdBindMountUnits]] for more details.<br />
{{hc|srv-nfs4-music.mount|<nowiki><br />
[Mount]<br />
What=/mnt/zfspool<br />
Where=/srv/nfs4/music<br />
Type=none<br />
Options=bind<br />
<br />
[Unit]<br />
DefaultDependencies=no<br />
Conflicts=umount.target<br />
Before=local-fs.target umount.target<br />
After=zfs-mount.service<br />
Requires=zfs-mount.service<br />
ConditionPathIsDirectory=/mnt/zfspool<br />
<br />
[Install]<br />
WantedBy=local-fs.target<br />
</nowiki>}}<br />
<br />
== See also ==<br />
<br />
* [[Installing Arch Linux on ZFS]]<br />
* [http://zfsonlinux.org/ ZFS on Linux]<br />
* [http://zfsonlinux.org/faq.html ZFS on Linux FAQ]<br />
* [http://www.freebsd.org/doc/en_US.ISO8859-1/books/handbook/filesystems-zfs.html FreeBSD Handbook -- The Z File System]<br />
* [http://docs.oracle.com/cd/E19253-01/819-5461/index.html Oracle Solaris ZFS Administration Guide]<br />
* [http://www.solarisinternals.com/wiki/index.php/ZFS_Troubleshooting_Guide Solaris Internals -- ZFS Troubleshooting Guide]<br />
* [http://royal.pingdom.com/2013/06/04/zfs-backup/ Pingdom details how it backs up 5TB of MySQL data every day with ZFS]<br />
<br />
; Aaron Toponce has authored a 17-part blog on ZFS which is an excellent read.<br />
# [https://pthree.org/2012/12/04/zfs-administration-part-i-vdevs/ VDEVs]<br />
# [https://pthree.org/2012/12/05/zfs-administration-part-ii-raidz/ RAIDZ Levels]<br />
# [https://pthree.org/2012/12/06/zfs-administration-part-iii-the-zfs-intent-log/ The ZFS Intent Log]<br />
# [https://pthree.org/2012/12/07/zfs-administration-part-iv-the-adjustable-replacement-cache/ The ARC]<br />
# [https://pthree.org/2012/12/10/zfs-administration-part-v-exporting-and-importing-zpools/ Import/export zpools]<br />
# [https://pthree.org/2012/12/11/zfs-administration-part-vi-scrub-and-resilver/ Scrub and Resilver]<br />
# [https://pthree.org/2012/12/12/zfs-administration-part-vii-zpool-properties/ Zpool Properties]<br />
# [https://pthree.org/2012/12/13/zfs-administration-part-viii-zpool-best-practices-and-caveats/ Zpool Best Practices]<br />
# [https://pthree.org/2012/12/14/zfs-administration-part-ix-copy-on-write/ Copy on Write]<br />
# [https://pthree.org/2012/12/17/zfs-administration-part-x-creating-filesystems/ Creating Filesystems]<br />
# [https://pthree.org/2012/12/18/zfs-administration-part-xi-compression-and-deduplication/ Compression and Deduplication]<br />
# [https://pthree.org/2012/12/19/zfs-administration-part-xii-snapshots-and-clones/ Snapshots and Clones]<br />
# [https://pthree.org/2012/12/20/zfs-administration-part-xiii-sending-and-receiving-filesystems/ Send/receive Filesystems]<br />
# [https://pthree.org/2012/12/21/zfs-administration-part-xiv-zvols/ ZVOLs]<br />
# [https://pthree.org/2012/12/31/zfs-administration-part-xv-iscsi-nfs-and-samba/ iSCSI, NFS, and Samba]<br />
# [https://pthree.org/2013/01/02/zfs-administration-part-xvi-getting-and-setting-properties/ Get/Set Properties]<br />
# [https://pthree.org/2013/01/03/zfs-administration-part-xvii-best-practices-and-caveats/ ZFS Best Practices]</div>Justin8https://wiki.archlinux.org/index.php?title=Pacman/Tips_and_tricks&diff=418647Pacman/Tips and tricks2016-02-01T14:24:49Z<p>Justin8: Added gnome-packagekit</p>
<hr />
<div>{{Lowercase title}}<br />
[[Category:Package management]]<br />
[[es:Pacman/Tips and tricks]]<br />
[[fr:Astuces Pacman]]<br />
[[it:Pacman/Tips and tricks]]<br />
[[ja:Pacman ヒント]]<br />
[[ru:Pacman/Tips and tricks]]<br />
[[tr:Pacman ipuçları]]<br />
[[zh-cn:Pacman/Tips and tricks]]<br />
{{Related articles start}}<br />
{{Related|pacman}}<br />
{{Related|Mirrors}}<br />
{{Related|Creating packages}}<br />
{{Related articles end}}<br />
See [[pacman]] for the main article.<br />
<br />
For general methods to improve the flexibility of the provided tips or pacman itself, see [[Core utilities]] and [[Bash]].<br />
<br />
== Cosmetic and convenience ==<br />
<br />
=== Operations and Bash syntax ===<br />
<br />
{{Merge|pacman#Installing specific packages|Although this is definitely a tip/trick, it fits to the main [[pacman]] page much better than here. It shows generic examples of how to do things, not specific commands needed to achieve something non-trivial/unrelated.}}<br />
<br />
In addition to pacman's standard set of features, there are ways to extend its usability through rudimentary [[Bash]] commands/syntax.<br />
<br />
To install a number of packages sharing similar patterns in their names -- not the entire group nor all matching packages; eg. {{Grp|plasma}}:<br />
<br />
# pacman -S plasma-{desktop,mediacenter,nm}<br />
<br />
Of course, that is not limited and can be expanded to however many levels needed:<br />
<br />
# pacman -S plasma-{workspace{,-wallpapers},pa}<br />
<br />
Sometimes, {{Ic|-s}}'s builtin ERE can cause a lot of unwanted results, so it has to be limited to match the package name only; not the description nor any other field:<br />
<br />
# pacman -Ss '^vim-'<br />
<br />
pacman has the {{ic|-q}} operand to hide the version column, so it is possible to query and reinstall packages with "compiz" as part of their name:<br />
<br />
# pacman -S $(pacman -Qq | grep compiz)<br />
<br />
=== Graphical front-ends ===<br />
<br />
* {{App|Discover|A collection of package management tools for KDE, using PackageKit.|https://projects.kde.org/projects/kde/workspace/discover|{{Pkg|discover}}}}<br />
* {{App|GNOME packagekit|GTK based package management tool|http://www.freedesktop.org/software/PackageKit/|{{Pkg|gnome-packagekit}}}}<br />
* {{App|GNOME Software|Gnome Software App. (Curated selection for GNOME)|https://wiki.gnome.org/Apps/Software|{{pkg|gnome-software}}}}<br />
* {{App|pcurses|Package management in a curses frontend|https://github.com/schuay/pcurses|{{Pkg|pcurses}}}}<br />
* {{App|tkPacman|GUI front-end for pacman. Depends on Tcl/Tk and X11 but neither on GTK+, nor on QT. It only interacts with the package database via the CLI of 'pacman'. So, installing and removing packages with tkPacman or with pacman leads to exactly the same result.|http://sourceforge.net/projects/tkpacman|{{AUR|tkpacman}}}}<br />
<br />
=== Utilities ===<br />
<br />
* {{App|Lostfiles|Script for detecting orphaned files.|https://github.com/graysky2/lostfiles|{{AUR|lostfiles}}}}<br />
* {{App|[[Pacmatic]]|Pacman wrapper to check Arch News before upgrading, avoid partial upgrades, and warn about configuration file changes.|http://kmkeen.com/pacmatic|{{Pkg|pacmatic}}}}<br />
* {{App|[[pkgfile]]|Tool that finds what package owns a file.|http://github.com/falconindy/pkgfile|{{Pkg|pkgfile}}}}<br />
* {{App|[[pkgtools]]|Collection of scripts for Arch Linux packages.|https://github.com/Daenyth/pkgtools|{{AUR|pkgtools}}}}<br />
* {{App|srcpac|Simple tool that automates rebuilding packages from source.|https://projects.archlinux.org/srcpac.git|{{Pkg|srcpac}}}}<br />
<br />
== Maintenance ==<br />
<br />
See also [[System maintenance]].<br />
<br />
=== Listing packages ===<br />
<br />
You may want to get the list of installed packages with their version, which is useful when reporting bugs or discussing installed packages.<br />
<br />
* List all explicitly installed packages: {{ic| pacman -Qe }}.<br />
* List all foreign packages (typically manually downloaded and installed): {{ic| pacman -Qm }}.<br />
* List all native packages (installed from the sync database(s)): {{ic| pacman -Qn }}.<br />
* List packages by regex: {{ic|pacman -Qs ''regex''}}.<br />
* List packages by regex with custom output format: {{ic|expac -s "%-30n %v" ''regex''}} (needs {{Pkg|expac}}).<br />
<br />
==== With size ====<br />
<br />
To get a list of installed packages sorted by size, which may be useful when freeing space on your hard drive:<br />
<br />
* Install {{Pkg|expac}} and run {{ic|<nowiki>expac -H M '%m\t%n' | sort -h</nowiki>}}.<br />
* Run {{Pkg|pacgraph}} with the {{ic|-c}} option.<br />
<br />
To list the download size of several packages (leave {{ic|''packages''}} blank to list all packages):<br />
<br />
$ expac -S -H M '%k\t%n' ''packages''<br />
<br />
To list explicitly installed packages not in {{Grp|base}} nor {{Grp|base-devel}} with size and description:<br />
<br />
$ expac -H M "%011m\t%-20n\t%10d" $( comm -23 <(pacman -Qqen|sort) <(pacman -Qqg base base-devel|sort) ) | sort -n<br />
<br />
==== Latest installed packages ====<br />
<br />
Install {{Pkg|expac}} and run {{ic|<nowiki>expac --timefmt='%Y-%m-%d %T' '%l\t%n' | sort | tail -20</nowiki>}} or {{ic|<nowiki>expac --timefmt=%s '%l\t%n' | sort -n | tail -20</nowiki>}}<br />
<br />
==== All packages that nothing else depends on ====<br />
<br />
If you want to generate a list of all installed packages that nothing else depends on, you can use the following script. This is very helpful if you are trying to free hard drive space and have installed a lot of packages that you may not remember. You can browse through the output to find packages which you no longer need.<br />
<br />
{{Note|This script will show all packages that nothing else depends on, including those explicitly installed. To get a list of packages installed as dependencies but no longer required by any installed package, see [[#Orphans]].}} <br />
<br />
{{bc|<nowiki><br />
ignoregrp="base base-devel"<br />
ignorepkg=""<br />
<br />
comm -23 <(pacman -Qqt | sort) <(echo $ignorepkg | tr ' ' '\n' | cat <(pacman -Sqg $ignoregrp) - | sort -u)<br />
</nowiki>}}<br />
<br />
For list with descriptions for packages:<br />
<br />
{{bc|<nowiki><br />
expac -HM "%-20n\t%10d" $( comm -23 <(pacman -Qqt|sort) <(pacman -Qqg base base-devel|sort) )<br />
</nowiki>}}<br />
<br />
==== Installed packages that are not in a specified group or repository ====<br />
<br />
The following command will list any installed packages that are not in either {{Grp|base}} or {{Grp|base-devel}}, and as such were likely installed manually by the user:<br />
<br />
$ comm -23 <(pacman -Qeq | sort) <(pacman -Qgq base base-devel | sort)<br />
<br />
List all installed packages that are not in specified repository ({{ic|''repo_name''}} in example):<br />
<br />
$ comm -23 <(pacman -Qtq | sort) <(pacman -Slq ''repo_name'' | sort)<br />
<br />
List all installed packages that are in the {{ic|''repo_name''}} repository:<br />
<br />
$ comm -12 <(pacman -Qtq | sort) <(pacman -Slq ''repo_name'' | sort)<br />
<br />
=== Listing files owned by a package with size ===<br />
<br />
This one might come in handy if you have found that a specific package uses a huge amount of space and you want to find out which files make up the most of that.<br />
<br />
$ pacman -Qlq ''package'' | grep -v '/$' | xargs du -h | sort -h<br />
<br />
=== Identify files not owned by any package ===<br />
<br />
If your system has stray files not owned by any package (a common case if you do not [[Enhance system stability#Use the package manager to install software|use the package manager to install software]]), you may want to find such files in order to clean them up. The general process for doing so is:<br />
<br />
# Create a sorted list of the files you want to check ownership of: {{bc|<nowiki>$ find /etc /opt /usr | sort > all_files.txt</nowiki>}}<br />
# Create a sorted list of the files tracked by pacman (and remove the trailing slashes from directories): {{bc|<nowiki>$ pacman -Qlq | sed 's|/$||' | sort > owned_files.txt</nowiki>}}<br />
# Find lines in the first list that are not in the second: {{bc|$ comm -23 all_files.txt owned_files.txt}}<br />
<br />
This process is tricky in practice because many important files are not part of any package (e.g. files generated at runtime, custom configs) and so will be included in the final output, making it difficult to pick out the files that can be safely deleted.<br />
<br />
The {{AUR|lostfiles}} script performs similar steps, but also includes an extensive blacklist to remove common false positives from the output.<br />
<br />
=== Removing unused packages ===<br />
<br />
==== Orphans ====<br />
<br />
For ''recursively'' removing orphans and their configuration files:<br />
<br />
# pacman -Rns $(pacman -Qtdq)<br />
<br />
If no orphans were found, pacman errors with {{ic|error: no targets specified}}. This is expected as no arguments were passed to {{ic|pacman -Rns}}.<br />
<br />
{{Note|Since pacman version 4.2.0 only true orphans are listed. To make pacman also list packages which are only optionally required by another package, pass the {{ic|-t}}/{{ic|--unrequired}} flag twice:<br />
$ pacman -Qdttq<br />
Use this carefully, as it is not taken into account whether the package is an optional dependency and therefore bears the risk to remove packages which actually are not real orphans.}}<br />
<br />
==== Explicitly installed ====<br />
<br />
Because a lighter system is easier to maintain, occasionally looking through explicitly installed packages and ''manually'' selecting unused packages to be removed can be helpful.<br />
<br />
To list explicitly installed packages available in the official repositories:<br />
<br />
$ pacman -Qen<br />
<br />
To list explicitly installed packages not available in official repositories:<br />
<br />
$ pacman -Qem<br />
<br />
=== Removing everything but base group ===<br />
<br />
If it is ever necessary to remove all packages except the base group, try this one liner:<br />
<br />
# pacman -R $(comm -23 <(pacman -Qq|sort) <((for i in $(pacman -Qqg base); do pactree -ul $i; done)|sort -u|cut -d ' ' -f 1))<br />
<br />
The one-liner was originally devised in [https://bbs.archlinux.org/viewtopic.php?id=130176 this discussion], and later improved in this article.<br />
<br />
Notes:<br />
<br />
# {{ic|comm}} requires sorted input otherwise you get e.g. {{ic|comm: file 1 is not in sorted order}}.<br />
# {{ic|pactree}} prints the package name followed by what it provides. For example:<br />
<br />
{{hc|$ pactree -lu logrotate|<br />
logrotate<br />
popt<br />
glibc<br />
linux-api-headers<br />
tzdata<br />
dcron cron<br />
bash<br />
readline<br />
ncurses<br />
gzip}}<br />
<br />
The {{ic|dcron cron}} line seems to cause problems, that is why {{ic|cut -d ' ' -f 1}} is needed - to keep just the package name.<br />
<br />
=== Getting the dependencies list of several packages ===<br />
<br />
Dependencies are alphabetically sorted and doubles are removed.<br />
Note that you can use {{ic|pacman -Qi}} to improve response time a little. But<br />
you will not be able to query as many packages. Unfound packages are simply skipped<br />
(hence the {{ic|2>/dev/null}}).<br />
<br />
$ pacman -Si $@ 2>/dev/null | awk -F ": " -v filter="^Depends" \ '$0 ~ filter {gsub(/[>=<][^ ]*/,"",$2) ; gsub(/ +/,"\n",$2) ; print $2}' | sort -u<br />
<br />
Alternatively, you can use {{ic|expac}}: {{ic|expac -l '\n' %E -S $@ &#124; sort -u}}.<br />
<br />
=== Listing changed backup files ===<br />
<br />
If you want to backup your system configuration files you could copy all files in {{ic|/etc/}}, but usually you are only interested in the files that you have changed. Modified [[Pacnew_and_Pacsave_files#Package_backup_files|backup files]] can be viewed with the following command:<br />
<br />
# pacman -Qii | awk '/^MODIFIED/ {print $2}'<br />
<br />
Running this command with root permissions will ensure that files readable only by root (such as {{ic|/etc/sudoers}}) are included in the output.<br />
<br />
{{Tip|See [[#Listing all changed files from packages]] to list all changed files pacman knows, not only backup files.}}<br />
<br />
=== Back-up the pacman database ===<br />
<br />
The following command can be used to back up the local pacman database:<br />
<br />
$ tar -cjf pacman_database.tar.bz2 /var/lib/pacman/local<br />
<br />
Store the backup pacman database file on one or more offline media, such as a USB stick, external hard drive, or CD-R. See also [[Pacman tips#Backing up Local database with systemd]] for an alternative method.<br />
<br />
The database can be restored by moving the {{ic|pacman_database.tar.bz2}} file into the {{ic|/}} directory and executing the following command:<br />
<br />
# tar -xjvf pacman_database.tar.bz2<br />
<br />
{{Note|If the pacman database files are corrupted, and there is no backup file available, there exists some hope of rebuilding the pacman database. Consult [[Pacman tips#Restore pacman's local database]].}}<br />
<br />
==== Using systemd ====<br />
<br />
[[systemd]] can take snapshots of the pacman local database, each time it is modified.<br />
<br />
{{Tip|For a more configurable version, use: {{AUR|pakbak-git}}}}<br />
<br />
Use the following scripts, changing the value of {{ic|$pakbak}} for the backup location accordingly. The {{ic|pakbak.service}} can also automaticall be [[enable]]d on boot:<br />
<br />
{{hc|/usr/lib/systemd/scripts/pakbak_script|2=<br />
#!/bin/bash<br />
<br />
declare -r pakbak=''"/pakbak.tar.xz"''; ## set backup location<br />
tar -cJf "$pakbak" "/var/lib/pacman/local"; ## compress & store pacman local database in $pakbak<br />
}}<br />
<br />
{{hc|/usr/lib/systemd/system/pakbak.service|2=<br />
[Unit]<br />
Description=Back up pacman database<br />
<br />
[Service]<br />
Type=oneshot<br />
ExecStart=/bin/bash /usr/lib/systemd/scripts/pakbak_script<br />
RemainAfterExit=no<br />
}}<br />
<br />
{{hc|/usr/lib/systemd/system/pakbak.path|2=<br />
[Unit]<br />
Description=Back up pacman database<br />
<br />
[Path]<br />
PathChanged=/var/lib/pacman/local<br />
Unit=pakbak.service<br />
<br />
[Install]<br />
WantedBy=multi-user.target<br />
}}<br />
<br />
=== Check changelogs easily ===<br />
<br />
When maintainers update packages, commits are often commented in a useful fashion. Users can quickly check these from the command line by installing {{AUR|paclog}}. This utility lists recent commit messages for packages from the official repositories or the AUR, by using {{ic|paclog package}}.<br />
<br />
== Installation and recovery ==<br />
<br />
Alternative ways of getting and restoring packages.<br />
<br />
=== Installing packages from a CD/DVD or USB stick ===<br />
<br />
To download packages, or groups of packages:<br />
<br />
# cd ~/Packages<br />
# pacman -Syw base base-devel grub-bios xorg gimp --cachedir .<br />
# repo-add ./custom.db.tar.gz ./*<br />
<br />
Then you can burn the "Packages" folder to a CD/DVD or transfer it to a USB stick, external HDD, etc.<br />
<br />
To install:<br />
<br />
'''1.''' Mount the media:<br />
<br />
# mkdir /mnt/repo<br />
# mount /dev/sr0 /mnt/repo #For a CD/DVD.<br />
# mount /dev/sdxY /mnt/repo #For a USB stick.<br />
<br />
'''2.''' Edit {{ic|pacman.conf}} and add this repository ''before'' the other ones (e.g. extra, core, etc.). This is important. Do not just uncomment the one on the bottom. This way it ensures that the files from the CD/DVD/USB take precedence over those in the standard repositories:<br />
<br />
{{hc|/etc/pacman.conf|2=<br />
[custom]<br />
SigLevel = PackageRequired<br />
Server = file:///mnt/repo/Packages}}<br />
<br />
'''3.''' Finally, synchronize the pacman database to be able to use the new repository:<br />
<br />
# pacman -Syu<br />
<br />
=== Custom local repository ===<br />
<br />
Use the ''repo-add'' script included with Pacman to generate a database for a personal repository. Use {{ic|repo-add --help}} for more details on its usage. Simply store all of the built packages to be included in the repository in one directory, and execute the following command (where ''repo'' is the name of the custom repository):<br />
<br />
$ repo-add /path/to/repo.db.tar.gz /path/to/*.pkg.tar.xz<br />
<br />
{{Note|A package database is a tar file, optionally compressed. Valid extensions are “.db” or “.files” followed by an archive extension of “.tar”, “.tar.gz”, “.tar.bz2”, “.tar.xz”, or “.tar.Z”. The file does not need to exist, but all parent directories must exist.<br />
Furthermore when using {{ic|repo-add}} keep in mind that the database and the packages do not need to be in the same directory. But when using pacman with that database, they should be together.}}<br />
<br />
To add a new package to the database, or to replace the old version of an existing package in the database, run:<br />
<br />
$ repo-add /path/to/repo.db.tar.gz /path/to/packagetoadd-1.0-1-i686.pkg.tar.xz<br />
<br />
''repo-remove'' is used in the exact same manner as ''repo-add'', except that the packages listed on the command line are removed from the repository database.<br />
<br />
Once the local repository database has been created, add the repository to {{ic|pacman.conf}} for each system that is to use the repository. An example of a custom repository is in {{ic|pacman.conf}}. The repository's name is the database filename with the file extension omitted. In the case of the example above the repository's name would simply be ''repo''. Reference the repository's location using a {{ic|file://}} url, or via FTP using ftp://localhost/path/to/directory.<br />
<br />
If willing, add the custom repository to the [[Unofficial user repositories|list of unofficial user repositories]], so that the community can benefit from it.<br />
<br />
=== Network shared pacman cache ===<br />
<br />
If you happen to run several Arch boxes on your LAN, you can share packages so that you can greatly decrease your download times. Keep in mind you should not share between different architectures (i.e. i686 and x86_64) or you'll get into troubles.<br />
<br />
==== Read-only cache ====<br />
<br />
If you are looking for a quick and dirty solution, you can simply run a standalone webserver which other computers can use as a first mirror: {{ic|darkhttpd /var/cache/pacman/pkg}}. Just add this server at the top of your mirror list. Be aware that you might get a lot of 404 errors, due to cache misses, depending on what you do, but pacman will try the next (real) mirrors when that happens.<br />
<br />
==== Read-write cache ====<br />
<br />
{{Tip|See [[pacserve]] for an alternative (and probably simpler) solution than what follows.}}<br />
<br />
In order to share packages between multiple computers, simply share {{ic|/var/cache/pacman/}} using any network-based mount protocol. This section shows how to use shfs or sshfs to share a package cache plus the related library-directories between multiple computers on the same local network. Keep in mind that a network shared cache can be slow depending on the file-system choice, among other factors.<br />
<br />
First, install any network-supporting filesystem; for example [[sshfs]], [[shfs]], ftpfs, [[smbfs]] or [[nfs]].<br />
<br />
{{Tip|To use sshfs or shfs, consider reading [[Using SSH Keys]].}}<br />
<br />
Then, to share the actual packages, mount {{ic|/var/cache/pacman/pkg}} from the server to {{ic|/var/cache/pacman/pkg}} on every client machine.<br />
<br />
==== Dynamic reverse proxy cache using nginx ====<br />
<br />
[[nginx]] can be used to proxy requests to official upstream mirrors and cache the results to local disk. All subsequent requests for that file will be served directly from the local cache, minimizing the amount of internet traffic needed to update a large number of servers with minimal effort. <br />
<br />
{{Warning| This method has a limitation. You must use mirrors that use the same relative path to package files and you must configure your cache to use that same path. In this example, we are using mirrors that use the relative path {{ic|/archlinux/$repo/os/$arch}} and our cache's {{ic|Server}} setting in {{ic|mirrorlist}} is configured similarly.}}<br />
<br />
In this example, we will run the cache server on {{ic|<nowiki>http://cache.domain.local:8080/</nowiki>}} and storing the packages in {{ic|/srv/http/pacman-cache/}}. <br />
<br />
Create the directory for the cache and adjust the permissions so nginx can write files to it:<br />
<br />
# mkdir /srv/http/pacman-cache<br />
# chown http:http /srv/http/pacman-cache<br />
<br />
Next, configure nginx as our dynamic cache (read the comments for an explanation of the commands):<br />
<br />
{{hc|/etc/nginx/nginx.conf|<nowiki><br />
http<br />
{<br />
...<br />
<br />
# nginx may need to resolve domain names at run time<br />
resolver 8.8.8.8 8.8.4.4;<br />
<br />
# Pacman Cache<br />
server<br />
{<br />
listen 8080;<br />
server_name cache.domain.local;<br />
root /srv/http/pacman-cache;<br />
autoindex on;<br />
<br />
# Requests for package db and signature files should redirect upstream without caching<br />
location ~ \.(db|sig)$ {<br />
proxy_pass http://mirrors$request_uri;<br />
}<br />
<br />
# Requests for actual packages should be served directly from cache if available.<br />
# If not available, retrieve and save the package from an upstream mirror.<br />
location ~ \.tar\.xz$ {<br />
try_files $uri @pkg_mirror;<br />
}<br />
<br />
# Retrieve package from upstream mirrors and cache for future requests<br />
location @pkg_mirror {<br />
proxy_store on;<br />
proxy_redirect off;<br />
proxy_store_access user:rw group:rw all:r;<br />
proxy_next_upstream error timeout http_404;<br />
proxy_pass http://mirrors$request_uri;<br />
}<br />
}<br />
<br />
# Upstream Arch Linux Mirrors<br />
# - Configure as many backend mirrors as you want in the blocks below<br />
# - Servers are used in a round-robin fashion by nginx<br />
# - Add "backup" if you want to only use the mirror upon failure of the other mirrors<br />
# - Separate "server" configurations are required for each upstream mirror so we can set the "Host" header appropriately<br />
upstream mirrors {<br />
server localhost:8001;<br />
server localhost:8002 backup;<br />
server localhost:8003 backup;<br />
}<br />
<br />
# Arch Mirror 1 Proxy Configuration<br />
server<br />
{<br />
listen 8001;<br />
server_name localhost;<br />
<br />
location / {<br />
proxy_pass http://mirror.rit.edu$request_uri;<br />
proxy_set_header Host mirror.rit.edu;<br />
}<br />
}<br />
<br />
# Arch Mirror 2 Proxy Configuration<br />
server<br />
{<br />
listen 8002;<br />
server_name localhost;<br />
<br />
location / {<br />
proxy_pass http://mirrors.acm.wpi.edu$request_uri;<br />
proxy_set_header Host mirrors.acm.wpi.edu;<br />
}<br />
}<br />
<br />
# Arch Mirror 3 Proxy Configuration<br />
server<br />
{<br />
listen 8003;<br />
server_name localhost;<br />
<br />
location / {<br />
proxy_pass http://lug.mtu.edu$request_uri;<br />
proxy_set_header Host lug.mtu.edu;<br />
}<br />
}<br />
}<br />
</nowiki>}}<br />
<br />
Finally, update your other Arch Linux servers to use this new cache by adding the following line to the {{ic|mirrorlist}} file:<br />
<br />
{{hc|/etc/pacman.d/mirrorlist|<nowiki><br />
Server = http://cache.domain.local:8080/archlinux/$repo/os/$arch<br />
...<br />
</nowiki>}}<br />
<br />
{{Note| You will need to create a method to clear old packages, as this directory will continue to grow over time. {{ic|paccache}} (which is included with {{ic|pacman}}) can be used to automate this using retention criteria of your choosing. For example, {{ic|find /srv/http/pacman-cache/ -type d -exec paccache -v -r -k 2 -c {} \;}} will keep the last 2 versions of packages in your cache directory.}}<br />
<br />
==== Synchronize pacman package cache using BitTorrent Sync ====<br />
<br />
[[BitTorrent Sync]] is a new way of synchronizing folder via network (it works in LAN and over the internet). It is peer-to-peer so you do not need to set up a server: follow the link for more information.<br />
How to share a pacman cache using BitTorrent Sync:<br />
* First install the {{AUR|btsync}} package from the AUR on the machines you want to sync<br />
* Follow the installation instructions of the AUR package or on the [[BitTorrent Sync]] wiki page <br />
** set up BitTorrent Sync to work for the root account. This process requires read/write to the pacman package cache. <br />
** make sure to set a good password on btsync's web UI <br />
** start the systemd daemon for btsync.<br />
** in the btsync Web GUI add a new synchronized folder on the first machine and generate a new Secret. Point the folder to {{ic|/var/cache/pacman/pkg}}<br />
** Add the folder on all the other machines using the same Secret to share the cached packages between all systems. Or, to set the first system as a master and the others as slaves, use the Read Only Secret. Be sure to point it to {{ic|/var/cache/pacman/pkg}}<br />
<br />
Now the machines should connect and start synchronizing their cache. Pacman works as expected even during synchronization. The process of syncing is entirely automatic.<br />
<br />
==== Preventing unwanted cache purges ====<br />
<br />
By default, {{Ic|pacman -Sc}} removes package tarballs from the cache that correspond to packages that are not installed on the machine the command was issued on. Because pacman cannot predict what packages are installed on all machines that share the cache, it will end up deleting files that should not be.<br />
<br />
To clean up the cache so that only ''outdated'' tarballs are deleted, add this entry in the {{ic|[options]}} section of {{ic|/etc/pacman.conf}}:<br />
<br />
CleanMethod = KeepCurrent<br />
<br />
=== Recreate a package from the file system ===<br />
<br />
To recreate a package from the file system, use ''bacman'' (included with pacman). Files from the system are taken as they are, hence any modifications will be present in the assembled package. Distributing the recreated package is therefore discouraged; see [[ABS]] and [[Arch Rollback Machine]] for alternatives.<br />
<br />
{{Tip|''bacman'' honours the {{ic|PACKAGER}}, {{ic|PKGDEST}} and {{ic|PKGEXT}} options from {{ic|makepkg.conf}}. Custom options for the compression tools can be configured by exporting the relevant environment variable, for example {{ic|1=XZ_OPT="-T 0"}} will enable parallel compression for ''xz''.}}<br />
<br />
An alternative tool would be {{AUR|fakepkg}}. It supports parallelization and can handle multiple input packages in one command, which ''bacman'' both does not support.<br />
<br />
=== Backing up and retrieving a list of installed packages ===<br />
<br />
{{Expansion|Optional dependencies that are not required by any other package ({{ic|comm -13 <(pacman -Qdtq) <(pacman -Qdttq)}}) are ignored by this procedure.}}<br />
<br />
{{Tip|1=You may want to use {{AUR|pacbackup}} or [https://bbs.archlinux.org/viewtopic.php?id=200067 bacpac] to automatise the below tasks.}}<br />
<br />
It is good practice to keep periodic backups of all pacman-installed packages. In the event of a system crash which is unrecoverable by other means, pacman can then easily reinstall the very same packages onto a new installation.<br />
<br />
* First, backup the current list of non-local packages: {{ic|$ pacman -Qqen > pkglist.txt}}<br />
<br />
* Store the {{ic|pkglist.txt}} on a USB key or other convenient medium or gist.github.com or Evernote, Dropbox, etc.<br />
<br />
* Copy the {{ic|pkglist.txt}} file to the new installation, and navigate to the directory containing it.<br />
<br />
* Issue the following command to install from the backup list: {{ic|# pacman -S $(< pkglist.txt)}}<br />
<br />
In the case you have a list which was not generated like mentioned above, there may be foreign packages in it (i.e. packages not belonging to any repos you have configured, or packages from the AUR).<br />
<br />
In such a case, you may still want to install all available packages from that list:<br />
<br />
# pacman -S --needed $(comm -12 <(pacman -Slq|sort) <(sort badpkdlist) )<br />
<br />
Explanation:<br />
<br />
* {{ic|pacman -Slq}} lists all available softwares, but the list is sorted by repository first, hence the {{ic|sort}} command.<br />
* Sorted files are required in order to make the {{ic|comm}} command work.<br />
* The {{ic|-12}} parameter display lines common to both entries.<br />
* The {{ic|--needed}} switch is used to skip already installed packages.<br />
<br />
Finally, you may want to remove all the packages on your system that are not mentioned in the list.<br />
<br />
{{Warning|Use this command wisely, and always check the result prompted by pacman.}}<br />
<br />
# pacman -Rsu $(comm -23 <(pacman -Qq|sort) <(sort pkglist))<br />
<br />
=== Listing all changed files from packages ===<br />
If you are suspecting file corruption (e.g. by software / hardware failure), but don't know for sure whether / which files really got corrupted, you might want to compare with the hash sums in the packages. This can be done with the following script.<br />
<br />
The script depends on the accuracy of pacman's database in {{ic|/var/lib/pacman/local/}} and the used programs such as ''bash'', ''grep'' and so on. For recovery of the database see [[#Restore pacman's local database]]. The {{ic|mtree}} files can also be [[#Viewing a single file inside a .pkg file|extracted as {{ic|.MTREE}} from the respective package files]].<br />
<br />
{{Note|<br />
* This should '''not''' be used as is when suspecting malicious changes! In this case security precautions such as using a live medium and an independent source for the hash sums are advised.<br />
* This could take a long time, depending on the hardware and installed packages.<br />
}}<br />
<br />
{{bc|<nowiki><br />
#!/bin/bash -e<br />
<br />
# Select the hash algorithm. Currently available (see mtree files and mtree(5)):<br />
# md5, sha256<br />
algo="md5"<br />
<br />
for package in /var/lib/pacman/local/*; do<br />
[ "$package" = "/var/lib/pacman/local/ALPM_DB_VERSION" ] && continue<br />
<br />
# get files and hash sums<br />
zgrep " ${algo}digest=" "$package/mtree" | grep -Ev '^\./\.[A-Z]+' | \<br />
sed 's/^\([^ ]*\).*'"${algo}"'digest=\([a-f0-9]*\).*/\1 \2/' | \<br />
while read -r file hash<br />
do<br />
# expand "\nnn" (in mtree) / "\0nnn" (for echo) escapes of ASCII<br />
# characters (octal representation)<br />
for ascii in $(grep -Eo '\\[0-9]{1,3}' <<< "$file"); do<br />
file="$(sed "s/\\$ascii/$(echo -e "\0${ascii:1}")/" <<< "$file")"<br />
done<br />
<br />
# check file hash<br />
if [ "$("${algo}sum" /"$file" | awk '{ print $1; }')" != "$hash" ]; then<br />
echo "$(basename "$package")" /"$file"<br />
fi<br />
done<br />
done<br />
</nowiki>}}<br />
<br />
=== Reinstalling all packages ===<br />
To reinstall all native packages, use:<br />
<br />
# pacman -Qenq | pacman -S -<br />
<br />
Foreign (AUR) packages must be reinstalled separately; you can list them with {{ic|pacman -Qemq}}.<br />
<br />
Pacman preserves the installation reason by default.<br />
<br />
=== Restore pacman's local database ===<br />
<br />
{{Out of date|1=''testdb'' has been removed in pacman 5.0 [https://projects.archlinux.org/pacman.git/tree/NEWS?h=v5.0.0].}}<br />
<br />
Signs that pacman needs a local database restoration:<br />
<br />
* {{ic|pacman -Q}} gives absolutely no output, and {{Ic|pacman -Syu}} erroneously reports that the system is up to date.<br />
* When trying to install a package using {{ic|pacman -S package}}, and it outputs a list of already satisfied dependencies.<br />
* When {{ic|testdb}} (part of {{Pkg|pacman}}) reports database inconsistency.<br />
<br />
Most likely, pacman's database of installed software, {{ic|/var/lib/pacman/local}}, has been corrupted or deleted. While this is a serious problem, it can be restored by following the instructions below.<br />
<br />
Firstly, make sure pacman's log file is present:<br />
<br />
$ ls /var/log/pacman.log<br />
<br />
If it does not exist, it is ''not'' possible to continue with this method. You may be able to use [https://bbs.archlinux.org/viewtopic.php?pid=670876 Xyne's package detection script] to recreate the database. If not, then the likely solution is to re-install the entire system.<br />
<br />
==== Generating the package recovery list ====<br />
<br />
{{Warning|If for some reason your [[pacman]] cache or [[makepkg]] package destination contain packages for other architectures, remove them before continuation.}}<br />
<br />
Run the script (optionally passing additional directories with packages as parameters):<br />
<br />
$ paclog-pkglist /var/log/pacman.log | ./pacrecover >files.list 2>pkglist.orig<br />
<br />
This way two files will be created: {{Ic|files.list}} with package files, still present on machine and {{Ic|pkglist.orig}}, packages from which should be downloaded. Later operation may result in mismatch between files of older versions of package, still present on machine, and files, found in new version. Such mismatches will have to be fixed manually.<br />
<br />
Here is a way to automatically restrict second list to packages available in a repository:<br />
<br />
$ { cat pkglist.orig; pacman -Slq; } | sort | uniq -d > pkglist<br />
<br />
Check if some important ''base'' package are missing, and add them to the list:<br />
<br />
$ comm -23 <(pacman -Sgq base) pkglist.orig >> pkglist<br />
<br />
Proceed once the contents of both lists are satisfactory, since they will be used to restore pacman's installed package database; {{ic|/var/lib/pacman/local/}}.<br />
<br />
==== Performing the recovery ====<br />
<br />
Define bash alias for recovery purposes:<br />
<br />
# recovery-pacman() {<br />
pacman "$@" \<br />
--log /dev/null \<br />
--noscriptlet \<br />
--dbonly \<br />
--force \<br />
--nodeps \<br />
--needed \<br />
#<br />
}<br />
<br />
{{ic|--log /dev/null}} allows to avoid needless pollution of pacman log, {{Ic|--needed}} will save some time by skipping packages, already present in database, {{Ic|--nodeps}} will allow installation of cached packages, even if packages being installed depend on newer versions. Rest of options will allow '''pacman''' to operate without reading/writing filesystem.<br />
<br />
Populate the sync database:<br />
<br />
# pacman -Sy<br />
<br />
Start database generation by installing locally available package files from {{ic|files.list}}:<br />
<br />
# recovery-pacman -U $(< files.list)<br />
<br />
Install the rest from {{ic|pkglist}}:<br />
<br />
# recovery-pacman -S $(< pkglist)<br />
<br />
Update the local database so that packages that are not required by any other package are marked as explicitly installed and the other as dependences. You will need be extra careful in the future when removing packages, but with the original database lost is the best we can do.<br />
<br />
# pacman -D --asdeps $(pacman -Qq)<br />
# pacman -D --asexplicit $(pacman -Qtq)<br />
<br />
Optionally check all installed packages for corruption:<br />
<br />
# pacman -Qk<br />
<br />
Optionally [[#Identify files not owned by any package]].<br />
<br />
Update all packages:<br />
<br />
# pacman -Su<br />
<br />
=== Recovering a USB key from existing install ===<br />
<br />
If you have Arch installed on a USB key and manage to mess it up (e.g. removing it while it is still being written to), then it is possible to re-install all the packages and hopefully get it back up and working again (assuming USB key is mounted in /newarch)<br />
<br />
# pacman -S $(pacman -Qq --dbpath /newarch/var/lib/pacman) --root /newarch --dbpath /newarch/var/lib/pacman<br />
<br />
=== Extracting contents of a .pkg file ===<br />
<br />
The {{ic|.pkg}} files ending in {{ic|.xz}} are simply tar'ed archives that can be decompressed with:<br />
<br />
$ tar xvf package.tar.xz<br />
<br />
If you want to extract a couple of files out of a {{ic|.pkg}} file, this would be a way to do it.<br />
<br />
=== Viewing a single file inside a .pkg file ===<br />
<br />
For example, if you want to see the contents of {{ic|/etc/systemd/logind.conf}} supplied within the {{Pkg|systemd}} package:<br />
<br />
$ tar -xOf /var/cache/pacman/pkg/systemd-204-3-x86_64.pkg.tar.xz etc/systemd/logind.conf<br />
<br />
Or you can use {{pkg|vim}}, then browse the archive:<br />
<br />
$ vim /var/cache/pacman/pkg/systemd-204-3-x86_64.pkg.tar.xz<br />
<br />
=== Find applications that use libraries from older packages ===<br />
<br />
Even if you installed a package the existing long-running programs (like daemons and servers) still keep using code from old package libraries. And it is a bad idea to let these programs running if the old library contains a security bug.<br />
<br />
Here is a way how to find all the programs that use old packages code:<br />
<br />
# lsof +c 0 | grep -w DEL | awk '1 { print $1 ": " $NF }' | sort -u<br />
It will print running program name and old library that was removed or replaced with newer content.<br />
<br />
== Performance ==<br />
<br />
=== Database access speeds ===<br />
<br />
Pacman stores all package information in a collection of small files, one for each package. Improving database access speeds reduces the time taken in database-related tasks, e.g. searching packages and resolving package dependencies. The safest and easiest method is to run as root:<br />
<br />
# pacman-optimize<br />
<br />
This will attempt to put all the small files together in one (physical) location on the hard disk so that the hard disk head does not have to move so much when accessing all the data. This method is safe, but is not foolproof: it depends on your filesystem, disk usage and empty space fragmentation. Another, more aggressive, option would be to first remove uninstalled packages from cache and to remove unused repositories before database optimization:<br />
<br />
# pacman -Sc && pacman-optimize<br />
<br />
=== Download speeds ===<br />
<br />
{{Note|If your download speeds have been reduced to a crawl, ensure you are using one of the many [[mirrors]] and not ftp.archlinux.org, which is [https://www.archlinux.org/news/302/ throttled since March 2007].}}<br />
<br />
When downloading packages pacman uses the mirrors in the order they are in {{ic|/etc/pacman.d/mirrorlist}}. The mirror which is at the top of the list by default however may not be the fastest for you. To select a faster mirror, see [[Mirrors]].<br />
<br />
Pacman's speed in downloading packages can also be improved by using a different application to download packages, instead of Pacman's built-in file downloader.<br />
<br />
In all cases, make sure you have the latest Pacman before doing any modifications.<br />
<br />
# pacman -Syu<br />
<br />
==== Powerpill ====<br />
<br />
Powerpill is a full wrapper for Pacman that uses parallel and segmented downloads to speed up the download process. Normally Pacman will download one package at a time, waiting for it to complete before beginning the next download. Powerpill takes a different approach: it tries to download as many packages as possible at once.<br />
<br />
The [[Powerpill|Powerpill wiki page]] provides basic configuration and usage examples along with package and upstream links.<br />
<br />
==== wget ====<br />
<br />
This is also very handy if you need more powerful proxy settings than pacman's built-in capabilities. <br />
<br />
To use {{ic|wget}}, first [[install]] the {{Pkg|wget}} package then modify {{ic|/etc/pacman.conf}} by uncommenting the following line in the {{ic|[options]}} section:<br />
<br />
XferCommand = /usr/bin/wget -c -q --show-progress --passive-ftp -O %o %u<br />
<br />
Instead of uncommenting the {{ic|wget}} parameters in {{ic|/etc/pacman.conf}}, you can also modify the {{ic|wget}} configuration file directly (the system-wide file is {{ic|/etc/wgetrc}}, per user files are {{ic|$HOME/.wgetrc}}.<br />
<br />
==== aria2 ====<br />
<br />
[[aria2]] is a lightweight download utility with support for resumable and segmented HTTP/HTTPS and FTP downloads. aria2 allows for multiple and simultaneous HTTP/HTTPS and FTP connections to an Arch mirror, which should result in an increase in download speeds for both file and package retrieval.<br />
<br />
{{Note|Using aria2c in Pacman's XferCommand will '''not''' result in parallel downloads of multiple packages. Pacman invokes the XferCommand with a single package at a time and waits for it to complete before invoking the next. To download multiple packages in parallel, see the [[#Using_Powerpill|powerpill]] section above.}}<br />
<br />
Install {{Pkg|aria2}}, then edit {{ic|/etc/pacman.conf}} by adding the following line to the {{ic|[options]}} section:<br />
<br />
XferCommand = /usr/bin/aria2c --allow-overwrite=true --continue=true --file-allocation=none --log-level=error --max-tries=2 --max-connection-per-server=2 --max-file-not-found=5 --min-split-size=5M --no-conf --remote-time=true --summary-interval=60 --timeout=5 --dir=/ --out %o %u<br />
<br />
{{Tip|1=[https://bbs.archlinux.org/viewtopic.php?pid=1491879#p1491879 This alternative configuration for using pacman with aria2] tries to simplify configuration and adds more configuration options.}}<br />
<br />
See [http://aria2.sourceforge.net/manual/en/html/aria2c.html#options OPTIONS] in {{ic|man aria2c}} for used aria2c options.<br />
<br />
{{ic|-d, --dir}}<br />
:The directory to store the downloaded file(s) as specified by [[pacman]].<br />
<br />
{{ic|-o, --out}}<br />
:The output file name(s) of the downloaded file(s). <br />
<br />
{{ic|%o}}<br />
:Variable which represents the local filename(s) as specified by pacman.<br />
<br />
{{ic|%u}}<br />
:Variable which represents the download URL as specified by pacman.<br />
<br />
==== Other applications ====<br />
<br />
There are other downloading applications that you can use with Pacman. Here they are, and their associated XferCommand settings:<br />
<br />
* {{ic|snarf}}: {{ic|1=XferCommand = /usr/bin/snarf -N %u}}<br />
* {{ic|lftp}}: {{ic|1=XferCommand = /usr/bin/lftp -c pget %u}}<br />
* {{ic|axel}}: {{ic|1=XferCommand = /usr/bin/axel -n 2 -v -a -o %o %u}}</div>Justin8https://wiki.archlinux.org/index.php?title=Bluetooth_headset&diff=408403Bluetooth headset2015-11-07T12:17:15Z<p>Justin8: </p>
<hr />
<div>[[Category:Sound]]<br />
[[Category:Bluetooth]]<br />
[[ja:Bluetooth ヘッドセット]]<br />
[[ru:Bluetooth headset]]<br />
[[zh-CN:Bluetooth headset]]<br />
{{Related articles start}}<br />
{{Related|Bluetooth}}<br />
{{Related|Bluez4}}<br />
{{Related articles end}}<br />
<br />
Currently, Arch Linux supports the A2DP profile (Audio Sink) for remote audio playback with the default installation. <br />
<br />
{{Note|1=<nowiki></nowiki><br />
* The most recent version of [[Bluez]] does not support the Headset/Handsfree profiles (see [https://bugs.freedesktop.org/show_bug.cgi?id=73325 this bug report] for example). This means that microphone input will not work, and also no sound output on headsets that do not support the A2DP profile. For using a headset with the Headset/Handsfree profiles, you will need to jump down to the legacy methods which require using [[AUR]] to fetch alternative packages.<br />
* Bluez5 is only supported by [[PulseAudio]] and not by [[ALSA]]. If you do not want to use PulseAudio, you need to install an older Bluez version from the AUR.<br />
}}<br />
<br />
== Headset via Bluez5/PulseAudio ==<br />
<br />
PulseAudio 5.x supports A2DP per default.<br />
Make sure the following packages are [[install]]ed: {{Pkg|pulseaudio-alsa}}, {{Pkg|pulseaudio-bluetooth}}, {{Pkg|bluez}}, {{Pkg|bluez-libs}}, {{Pkg|bluez-utils}}, {{Pkg|bluez-firmware}}. Without {{Pkg|pulseaudio-bluetooth}} you won't be able to connect after the next pairing and you won't get any usable error messages.<br />
<br />
Start the Bluetooth system:<br />
<br />
# systemctl start bluetooth<br />
<br />
Now we can use the ''bluetoothctl'' command line utility to pair and connect. For troubleshooting and more detailed explanations of ''bluetoothctl'' see the [[Bluetooth]] article. Run<br />
<br />
# bluetoothctl<br />
<br />
to be greeted by its internal command prompt. Then enter:<br />
<br />
# power on<br />
# agent on<br />
# default-agent<br />
# scan on<br />
<br />
Now make sure that your headset is in pairing mode. It should be discovered shortly. For example,<br />
[NEW] Device 00:1D:43:6D:03:26 Lasmex LBT10<br />
shows a device that calls itself "Lasmex LBT10" and has MAC address ''00:1D:43:6D:03:26''. We will now use that MAC address to initiate the pairing:<br />
<br />
# pair 00:1D:43:6D:03:26<br />
<br />
After pairing, you also need to explicitly connect the device (every time?):<br />
<br />
# connect 00:1D:43:6D:03:26<br />
<br />
If everything works correctly, you now have a separate output device in [[PulseAudio]].<br />
{{Note|The device may be off by default. Select its audio profile (''OFF'', A2DP, HFP) in the "Configuration" tab of {{Pkg|pavucontrol}}.}}<br />
You can now redirect any audio through that device using the "Playback" and "Recording" tabs of {{Pkg|pavucontrol}}.<br />
<br />
You can now disable scanning again and exit the program:<br />
# scan off<br />
# exit<br />
<br />
=== Troubleshooting ===<br />
<br />
Many users report frustration with getting A2DP/Bluetooth Headsets to work. <br />
<br />
==== Selected audio profile, but headset inactive and audio cannot be redirected ====<br />
<br />
Deceptively, this menu is available before the device has been connected; annoyingly it will have no effect. The menu seems to be created as soon as the receiver recognizes the device.<br />
<br />
Make sure to run bluetoothctl (with sudo/as root) and connect the device manually. There may be configuration options to remove the need to do this each time, but neither pairing nor trusting induce automatic connecting for me.<br />
<br />
==== Pairing fails with AuthenticationFailed ====<br />
<br />
If pairing fails, you can try [https://stackoverflow.com/questions/12888589/linux-command-line-howto-accept-pairing-for-bluetooth-device-without-pin disabling SSPMode] with:<br />
# hciconfig hci0 sspmode 0<br />
<br />
==== Pairing works, but connecting does not ====<br />
<br />
You might see the following error in ''bluetoothctl'':<br />
<br />
[bluetooth]# connect 00:1D:43:6D:03:26<br />
Attempting to connect to 00:1D:43:6D:03:26<br />
Failed to connect: org.bluez.Error.Failed<br />
<br />
To further investigate, have a look at the log via one of the following commands:<br />
<br />
# systemctl status bluetooth<br />
# journalctl -n 20<br />
<br />
You might see a message like this:<br />
<br />
bluetoothd[5556]: a2dp-sink profile connect failed for 00:1D:43:6D:03:26: Protocol not available<br />
<br />
This may be due to the {{Pkg|pulseaudio-bluetooth}} package not being installed. Install it if it missing, then restart pulseaudio.<br />
<br />
If the issue is not due to the missing package, the problem in this case is that PulseAudio is not catching up. A common solution to this problem is to restart PulseAudio. Note that it is perfectly fine to run ''bluetoothctl'' as root while PulseAudio runs as user. After restarting PulseAudio, retry to connect. It is not necessary to repeat the pairing.<br />
<br />
If restarting PulseAudio does not work, you need to load module-bluetooth-discover.<br />
<br />
# pactl load-module module-bluetooth-discover<br />
<br />
The same load-module command can be added to {{ic|/etc/pulse/default.pa}}.<br />
<br />
If that still does not work, or you are using PulseAudio's system-wide mode, also load the following PulseAudio modules (again these can be loaded via your default.pa or system.pa):<br />
<br />
module-bluetooth-policy<br />
module-bluez5-device<br />
module-bluez5-discover<br />
<br />
To have your headset auto connect you need to enable PulseAudio's switch-on-connect module. Add the following:<br />
{{hc|/etc/pulse/default.pa|<br />
# automatically switch to newly-connected devices<br />
load-module module-switch-on-connect<br />
}}<br />
<br />
You then need to tell ''bluetoothctl'' to trust your Bluetooth headset, or you will see errors like this:<br />
bluetoothd[487]: Authentication attempt without agent<br />
bluetoothd[487]: Access denied: org.bluez.Error.Rejected<br />
<br />
[bluetooth]# trust 00:1D:43:6D:03:26<br />
<br />
After a reboot, your Bluetooth adapter will not power on by default. You need to add a udev rule to power it on:<br />
{{hc|/etc/udev/rules.d/10-local.rules|2=<br />
# Set bluetooth power up<br />
ACTION=="add", SUBSYSTEM=="bluetooth", KERNEL=="hci[0-9]*", RUN+="/usr/bin/hciconfig %k up"<br />
}}<br />
<br />
==== Connecting works, but I cannot play sound ====<br />
<br />
Make sure that you see the following messages in your system log:<br />
<br />
bluetoothd[5556]: Endpoint registered: sender=:1.83 path=/MediaEndpoint/A2DPSource<br />
bluetoothd[5556]: Endpoint registered: sender=:1.83 path=/MediaEndpoint/A2DPSink<br />
<br />
If you see a message similar to this, you can go on and investigate your PulseAudio configuration. Otherwise, go back and ensure the connection is successful.<br />
<br />
When using [[GDM]], another instance of PulseAudio is started, which "captures" your bluetooth device connection. This can be prevented by masking the pulseaudio socket for the GDM user by doing the following:<br />
<br />
$ mkdir -p ~gdm/.config/systemd/user<br />
$ ln -s /dev/null ~gdm/.config/systemd/user/pulseaudio.socket<br />
<br />
On next reboot the second instance of pulseaudio will not be started<br />
<br />
==== UUIDs has unsupported type ====<br />
<br />
During pairing you might see this output in ''bluetoothctl'':<br />
<br />
[CHG] Device 00:1D:43:6D:03:26 UUIDs has unsupported type<br />
<br />
This message is a very common one and can be ignored.<br />
<br />
== Legacy method: ALSA-BTSCO ==<br />
<br />
{{Out of date|Instructions rely on [[bluez4]].}}<br />
<br />
It is much easier to set up your bluetooth headset today, with bluez >= 3.16. You may want to try the out-of-box python script in [http://fosswire.com/2008/01/11/a2dp-stereo-linux/ this blog] (you need edit the script to work with gconftool-2). There is also a piece of equivalent bash script [http://lymanrb.blogspot.com/2008/05/linux.html here].<br />
<br />
You need your headset's bdaddr. It is of the form ''12:34:56:78:9A:BC''. Either find it in the documentation of your headset, on the headset itself or with the '''hcitool scan''' command.<br />
<br />
Install {{AUR|btsco}}{{Broken package link|{{aur-mirror|btsco}}}}.<br />
<br />
To load the kernel module, type:<br />
# modprobe snd-bt-sco<br />
There will now be an extra audio device. Use {{ic|alsamixer -cN}} (where N is most likely 1) to set the volume. You can access the device with any alsa-capable application by choosing the device ''BT headset'', or with any OSS application by using {{ic|/dev/dspN}} as the audio device.<br />
<br />
But to actually get any sound, you have to connect your headset to the computer first.<br />
<br />
=== Connecting the headset ===<br />
<br />
If you connect your headset for the first time, read the section about pairing first. To connect to your headset to the computer, use the command<br />
$ btsco -f <bdaddr><br />
for example<br />
$ btsco -f 12:34:56:78:9A:BC<br />
<br />
==== Pairing the headset with your computer ====<br />
<br />
The first time you connect the headset, you have to pair it with the computer. To do this, you need your headset's PIN. Depending on your headset you may have to reset the headset and repeat the pairing everytime you used the headset with another bluetooth device.<br />
<br />
There are two ways to pair your headset with the computer:<br />
<br />
===== Using ''bluez-gnome'' =====<br />
<br />
{{Out of date|Package does no longer exist in the repositories.}}<br />
<br />
Install the ''bluez-gnome'' package from the community repository. Then start the '''bt-applet''' program. Once you try to connect to the headset, a window will open and ask for the PIN.<br />
<br />
===== Using ''passkey-agent'' =====<br />
<br />
Before connecting to the headset, enter the command<br />
$ passkey-agent --default <pin><br />
where ''<pin>'' is your headset's PIN. Then try to connect to the headset.<br />
<br />
=== Headset and ALSA Devices ===<br />
<br />
1. First if you have not already, [[pacman|install]] {{Pkg|bluez}} from the [[official repositories]].<br />
<br />
2. Scan for your device<br />
$ hcitool (-i <optional hci#>***) scan<br />
<br />
3. Pair your headset with your device:<br />
$ bluez-simple-agent (optional hci# ***) XX:XX:XX:XX:XX:XX<br />
and put in your pin (0000 or 1234, etc)<br />
<br />
4. Make sure your {{ic|/etc/bluetooth/audio.conf}} allows A2DP Audio Sinks. Place this line just bellow the [General] heading:<br />
Enable=Source,Sink,Media,Socket<br />
<br />
5. Add this to your {{ic|/etc/asound.conf}} file:<br />
#/etc/asound.conf<br />
<br />
pcm.btheadset {<br />
type plug<br />
slave {<br />
pcm {<br />
type bluetooth<br />
device XX:XX:XX:XX:XX:XX <br />
profile "auto"<br />
} <br />
} <br />
hint {<br />
show on<br />
description "BT Headset"<br />
} <br />
}<br />
ctl.btheadset {<br />
type bluetooth<br />
} <br />
<br />
6. Check to see if it has been added to alsa devices<br />
$ aplay -L<br />
<br />
7. Now play with aplay:<br />
$ aplay -D btheadset /path/to/audio/file<br />
<br />
or Mplayer:<br />
$ mplayer -ao alsa:device=btheadset /path/to/audio/or/video/file<br />
<br />
{{Tip|To find hci# for a usb dongle, type in <br />
$ hcitool dev<br />
}}<br />
<br />
=== Headset's multimedia buttons ===<br />
<br />
In order to get your bluetooth headset's multimedia buttons (play, pause, next, previous) working you need to create {{ic|/etc/modules-load.d/uinput.conf}} containing {{ic|uinput}}.<br />
<br />
== Legacy method: PulseAudio ==<br />
<br />
{{Out of date|Instructions rely on [[bluez4]] (references to {{ic|/etc/bluetooth/audio.conf}} and ''bluez-simple-agent'').}}<br />
<br />
This one is much easier and more elegant. PulseAudio will seamlessly switch between output devices when the headset is turned on. If you have ALSA as the sound server, you need the following packages installed:<br />
{{Pkg|pulseaudio}} and {{Pkg|pulseaudio-alsa}}.<br />
<br />
Now, to configure the audio output to use bluetooth, just install {{Pkg|pavucontrol}} and run it to configure the audio output:<br />
$ pavucontrol<br />
Make sure to take a look at the [[PulseAudio]] wiki entry for setting up PulseAudio, especially if you are running KDE.<br />
<br />
=== Troubleshooting ===<br />
<br />
==== Audio sink fails ====<br />
<br />
Bluetooth headset is connected, but ALSA/PulseAudio fails to pick up the connected device. You will get "Audio sink fails".<br />
According to [http://wiki.gentoo.org/wiki/Bluetooth_Headset gentoo wiki], you have to verify than in {{ic|/etc/bluetooth/audio.conf}} there is {{ic|<nowiki>Enable=Socket</nowiki>}} under the {{ic|[General]}} section heading.<br />
<br />
Just do a {{ic|# systemctl restart bluetooth}} to apply it.<br />
<br />
==== Page timeout issue ====<br />
<br />
If you receive this error whilst trying to pair your headset with your system using bluez-simple-agent, then you can try to restart your system and use the graphical bluez applet of your desktop environment.<br />
<br />
== Legacy documentation: ALSA, bluez5 and PulseAudio method ==<br />
<br />
{{Accuracy|Describes two different methods, see the [[Talk:Bluetooth headset#bluez5 method: overcomplicated instructions|talk page]] for details.|Talk:Bluetooth_Headset#bluez5_method:_overcomplicated_instructions}}<br />
<br />
[[ALSA]], [[bluez|bluez5]], and [[PulseAudio]] work together to allow a wireless [[Bluetooth]] headset to play audio. The following method works with a Lenovo T61p laptop and SoundBot SB220 wireless bluetooth headset. The required software stack is extensive and failure to include all components can produce errors which are difficult to understand. The following list of software packages might not be the minimum required set and needs to be examined more closely.<br />
<br />
Bluez5 has a regression causing HSP/HFP Telephone profile to not be available. This regression is documented in the [http://www.freedesktop.org/wiki/Software/PulseAudio/Notes/5.0/ draft release notes for Pulseaudio 5.0] which say (in "Notes for packagers"): "PulseAudio now supports BlueZ 5, but only the A2DP profile. BlueZ 4 is still the only way to make HSP/HFP work." ([https://fedoraproject.org/wiki/Common_F20_bugs#bluez5-profile from here])<br />
<br />
=== Install Software Packages ===<br />
<br />
The core software components are [[ALSA]], Bluez5, [[PulseAudio]]. However there are additional libraries which are required. As well as a player which can play audio files. The following section lists the software packages installed in order to connect the headset and play audio over the headset.<br />
<br />
==== Install ALSA and associated libraries ====<br />
<br />
[[ALSA]] works with the linux kernel to provide audio services to user mode software. The following packages are used with the [[Bluetooth]] headset: {{Pkg|alsa-utils}}, {{Pkg|alsa-plugins}}, {{Pkg|alsa-tools}}.<br />
<br />
==== Install Bluez5 ====<br />
<br />
Bluez5 is the latest [[Bluetooth]] stack. It is required for [[PulseAudio]] to interface with wireless headsets. Required packages: {{Pkg|bluez}}, {{Pkg|bluez-utils}}, {{Pkg|bluez-libs}}.<br />
<br />
==== Install Audacious ====<br />
<br />
[[Audacious]] is a program which plays audio files. It can work directly with [[ALSA]] or with [[PulseAudio]]. Required packages: {{Pkg|audacious}}, {{Pkg|audacious-plugins}}.<br />
<br />
=== Procedure ===<br />
<br />
Once the required packages are installed, use this procedure to play audio with a bluetooth headset. The high level overview of the procedure is to pair the headset, connect the headset, configure the player and pulse audio controller and then play audio.<br />
<br />
Start the bluetooth service as root:<br />
# systemctl start bluetooth<br />
<br />
Verify Bluetooth is started<br />
# systemctl status bluetooth<br />
bluetooth.service - Bluetooth service<br />
Loaded: loaded (/usr/lib/systemd/system/bluetooth.service; disabled)<br />
Active: active (running) since Sat 2013-12-07 12:31:14 PST; 12s ago<br />
Docs: man:bluetoothd(8)<br />
Main PID: 3136 (bluetoothd)<br />
Status: "Running"<br />
CGroup: /system.slice/bluetooth.service<br />
└─3136 /usr/lib/bluetooth/bluetoothd<br />
<br />
Dec 07 12:31:14 t61p systemd[1]: Starting Bluetooth service...<br />
Dec 07 12:31:14 t61p bluetoothd[3136]: Bluetooth daemon 5.11<br />
Dec 07 12:31:14 t61p systemd[1]: Started Bluetooth service.<br />
Dec 07 12:31:14 t61p bluetoothd[3136]: Starting SDP server<br />
Dec 07 12:31:14 t61p bluetoothd[3136]: Bluetooth management interface 1.3 i...ed<br />
Hint: Some lines were ellipsized, use -l to show in full.<br />
<br />
Start the PulseAudio daemon. This must be done after X windows is started and as a normal user.<br />
$ pulseaudio -D<br />
<br />
Verify the PulseAudio daemon is running.<br />
$ pulseaudio --check -v<br />
I: [pulseaudio] main.c: Daemon running as PID 3186<br />
<br />
Start up bluetoothctl as root and pair and connect your headset. As a regular user, bluetoothctl will pair but not connect. Perhaps this is related to the config file (shown below) which is setup for what appears to be the root user.<br />
Note: the procedure shown below is for an initial pair and connect of the headphone. If the headset is already paired, then the procedure below can be shortened to: power on, agent on, default-agent, connect <mac address>. The mac address can be seen from the devices command output.<br />
<br />
$ bluetoothctl <br />
[NEW] Controller 00:1E:4C:F4:98:5B t61p-0 [default]<br />
[NEW] Device 00:1A:7D:12:36:B9 SoundBot SB220<br />
[bluetooth]# show<br />
Controller 00:1E:4C:F4:98:5B<br />
Name: t61p<br />
Alias: t61p-0<br />
Class: 0x000000<br />
Powered: no<br />
Discoverable: no<br />
Pairable: yes<br />
UUID: PnP Information (00001200-0000-1000-8000-00805f9b34fb)<br />
UUID: Generic Access Profile (00001800-0000-1000-8000-00805f9b34fb)<br />
UUID: Generic Attribute Profile (00001801-0000-1000-8000-00805f9b34fb)<br />
UUID: A/V Remote Control (0000110e-0000-1000-8000-00805f9b34fb)<br />
UUID: A/V Remote Control Target (0000110c-0000-1000-8000-00805f9b34fb)<br />
UUID: Audio Source (0000110a-0000-1000-8000-00805f9b34fb)<br />
UUID: Audio Sink (0000110b-0000-1000-8000-00805f9b34fb)<br />
Modalias: usb:v1D6Bp0246d050B<br />
Discovering: no<br />
[bluetooth]# power on<br />
[CHG] Controller 00:1E:4C:F4:98:5B Class: 0x0c010c<br />
Changing power on succeeded<br />
[CHG] Controller 00:1E:4C:F4:98:5B Powered: yes<br />
[bluetooth]# agent on<br />
Agent registered<br />
[bluetooth]# default-agent<br />
Default agent request successful<br />
<br />
<power on your headset in pairing mode. Eventually you will see what appears to be a mac address.><br />
[bluetooth]# scan on<br />
Discovery started<br />
[CHG] Controller 00:1E:4C:F4:98:5B Discovering: yes<br />
[CHG] Device 00:1A:7D:12:36:B9 RSSI: -61<br />
[bluetooth]# pair 00:1A:7D:12:36:B9<br />
Attempting to pair with 00:1A:7D:12:36:B9<br />
[CHG] Device 00:1A:7D:12:36:B9 Connected: yes<br />
[CHG] Device 00:1A:7D:12:36:B9 UUIDs has unsupported type<br />
[CHG] Device 00:1A:7D:12:36:B9 Paired: yes<br />
Pairing successful<br />
[bluetooth]# connect 00:1A:7D:12:36:B9<br />
[CHG] Device 00:1A:7D:12:36:B9 Connected: yes<br />
Connection successful<br />
[bluetooth]# info 00:1A:7D:12:36:B9<br />
Device 00:1A:7D:12:36:B9<br />
Name: SoundBot SB220<br />
Alias: SoundBot SB220<br />
Class: 0x240404<br />
Icon: audio-card<br />
Paired: yes<br />
Trusted: no<br />
Blocked: no<br />
Connected: yes<br />
LegacyPairing: yes<br />
UUID: Headset (00001108-0000-1000-8000-00805f9b34fb)<br />
UUID: Audio Sink (0000110b-0000-1000-8000-00805f9b34fb)<br />
UUID: A/V Remote Control Target (0000110c-0000-1000-8000-00805f9b34fb)<br />
UUID: A/V Remote Control (0000110e-0000-1000-8000-00805f9b34fb)<br />
UUID: Handsfree (0000111e-0000-1000-8000-00805f9b34fb)<br />
<br />
Start up alsamixer, for simplicity un-mute all your outputs. Oddly enough some can be muted though. The ones I had muted during playback were:<br />
* Headphones<br />
* SPIDF<br />
<br />
Start up audacious. Use the menu to select PulseAudio as your output. Somewhere I read that bluez5 requires pulseaudio-git and this jives with my experience.<br />
<br />
Start up pavucontrol in a terminal. In the Outputs tab select the bluetooth headset.<br />
<br />
[http://netskink.blogspot.com/2013/12/pulseaudio-pavucontrol-and-audacious.html screenshot of application settings]<br />
<br />
==== Miscellaneous configuration files ====<br />
<br />
For reference, these settings were also done.<br />
<br />
===== ALSA /etc/asound.conf =====<br />
<br />
The settings shown at the top of this page was used, but the additional modification for Intel laptop sound cards.<br />
<br />
{{bc|<nowiki><br />
pcm.btheadset {<br />
type plug<br />
slave {<br />
pcm {<br />
type bluetooth<br />
device 00:1A:7D:12:36:B9<br />
profile "auto"<br />
}<br />
}<br />
hint {<br />
show on<br />
description "BT Headset"<br />
}<br />
}<br />
ctl.btheadset {<br />
type bluetooth<br />
}<br />
options snd-hda-intel model=laptop<br />
</nowiki>}}<br />
<br />
===== /etc/dbus-1/system.d/bluetooth.conf =====<br />
<br />
The settings here seem to be enabled for root only. See the policy user="root" section. However, if a regular user is specified here, the system fails to start. Someone with more knowledge could explain why.<br />
<br />
{{hc|/etc/dbus-1/system.d/bluetooth.conf|<nowiki><br />
<!-- This configuration file specifies the required security policies for Bluetooth core daemon to work. --><br />
<br />
<!DOCTYPE busconfig PUBLIC "-//freedesktop//DTD D-BUS Bus Configuration 1.0//EN"<br />
"http://www.freedesktop.org/standards/dbus/1.0/busconfig.dtd"><br />
<busconfig><br />
<br />
<!-- ../system.conf have denied everything, so we just punch some holes --><br />
<br />
<policy user="root"><br />
<allow own="org.bluez"/><br />
<allow send_destination="org.bluez"/><br />
<allow send_interface="org.bluez.Agent1"/><br />
<allow send_interface="org.bluez.MediaEndpoint1"/><br />
<allow send_interface="org.bluez.MediaPlayer1"/><br />
<allow send_interface="org.bluez.ThermometerWatcher1"/><br />
<allow send_interface="org.bluez.AlertAgent1"/><br />
<allow send_interface="org.bluez.Profile1"/><br />
<allow send_interface="org.bluez.HeartRateWatcher1"/><br />
<allow send_interface="org.bluez.CyclingSpeedWatcher1"/><br />
</policy><br />
<br />
<policy at_console="true"><br />
<allow send_destination="org.bluez"/><br />
</policy><br />
<br />
<!-- allow users of lp group (printing subsystem) to communicate with bluetoothd --><br />
<policy group="lp"><br />
<allow send_destination="org.bluez"/><br />
</policy><br />
<br />
<policy context="default"><br />
<deny send_destination="org.bluez"/><br />
</policy><br />
<br />
</busconfig><br />
</nowiki>}}<br />
<br />
===== Tested applications =====<br />
<br />
As noted above this will work easily with audacious. YouTube videos with Chromium and Flash Player will work on some videos. If the video has ads it will not work, but if the video does not have ads it will work. Just make sure that after audacious is working with Bluetooth headset, start Chromium, and navigate to YouTube. Find a video without leading ads, and it should play the audio. If the settings icon has the a menu with two drop-down combo boxes for Speed and Quality it will play.<br />
<br />
== Switch between HSV and A2DP setting ==<br />
<br />
This can easily be achieved by the following command where 2 needs to be changed with the correct device number.<br />
<br />
pacmd set-card-profile 2 a2dp_sink<br />
<br />
=== A2DP not working with PulseAudio ===<br />
<br />
==== Socket interface problem ====<br />
<br />
If PulseAudio fails when changing the profile to A2DP with bluez 4.1+ and PulseAudio 3.0+, you can try disabling the Socket interface from {{ic|/etc/bluetooth/audio.conf}} by removing the line {{ic|1=Enable=Socket}} and adding line {{ic|1=Disable=Socket}}.<br />
<br />
==== Gnome with GDM ====<br />
{{Note|Below was tested with Gnome 3.16.2 and PulseAudio 7.0}}<br />
<br />
If PulseAudio fails when changing the profile to A2DP while using GNOME with GDM, you need to prevent GDM from starting its own instance of PulseAudio. Apply the same fix as shown in [[#Connecting works, but I cannot play sound]].<br />
<br />
{{Note|1=Discussion about this problem can be found [https://bbs.archlinux.org/viewtopic.php?id=194006 here] and [https://bbs.archlinux.org/viewtopic.php?id=196689 here]}}<br />
<br />
== Tested headsets ==<br />
<br />
{| class="wikitable"<br />
! Model<br />
! Version<br />
! Comments<br />
! Compatible<br />
|-<br />
| '''Philips SHB9150'''<br />
| bluez5, pulseaudio 5<br />
| Pause and resume does not work. With at least mpv and Banshee hitting the pause button stops audio output but does not pause the player.<br />
| {{R|Limited}}<br />
|-<br />
| '''Philips SHB9100'''<br />
| <br />
| Pause and resume is flaky. See [https://bbs.archlinux.org/viewtopic.php?pid=1315428#p1315428] for the underlying issue and a temporary solution to improve audio quality.<br />
| {{R|Limited}}<br />
|-<br />
| '''Philips SHB7000'''<br />
| <br />
| Pause and resume is flaky.<br />
| {{R|Limited}}<br />
|-<br />
| '''Philips SHB7100'''<br />
| bluez 5.32, pulseaudio 6.0<br />
| Next/previous buttons work. Pause and resume is flaky (sometimes works in VLC, not at all in Audacious). Tested only A2DP and Handsfree audio out, built-in mic was broken.<br />
| {{R|Limited}}<br />
|-<br />
| '''Philips SHB7150'''<br />
| bluez 5.32, pulseaudio 6.0<br />
| Next/previous buttons work. Pause and resume work in VLC. Tested only A2DP profile.<br />
| {{Yes}}<br />
|-<br />
| '''Philips SHB5500BK/00'''<br />
| bluez 5.28, PulseAudio 6.0<br />
| Pause and resume is not working.<br />
| {{R|Limited}}<br />
|-<br />
| '''Parrot Zik'''<br />
| <br />
| Firmware 1.04. The microphone is detected, but does not work. Sometimes it lags (but does not stutter); usually this is not noticeable unless playing games, in which case you may switch to a wired connection.<br />
| {{R|Limited}}<br />
|-<br />
| '''Sony DR-BT50'''<br />
| bluez{4,5}<br />
| Works for a2dp, see [http://vlsd.blogspot.com/2013/11/bluetooth-headphones-and-arch-linux.html]). Adapter: D-Link DBT-120 USB dongle.<br />
| {{Yes}}<br />
|-<br />
| '''Sony SBH50'''<br />
| bluez5<br />
| Works for a2dp, Adapter: Broadcom Bluetooth 2.1 Device (Vendor=0a5c ProdID=219b Rev=03.43). Requires the {{ic|btusb}} [[modprobe|module]].<br />
| {{Yes}}<br />
|-<br />
| '''Sony MDR-XB950BT'''<br />
| pulseaudio<br />
| Tested a2dp. Adapter: Grand-X BT40G. Doesn't auto-connect, need to connect manually. Other functionality works fine.<br />
| {{R|Limited}}<br />
|-<br />
| '''Sony MUC-M1BT1'''<br />
| bluez5, {{AUR|pulseaudio-git}}<br />
| Both A2DP & HSP/HFP work fine.<br />
| {{Yes}}<br />
|-<br />
| '''SoundBot SB220'''<br />
| bluez5, {{AUR|pulseaudio-git}}<br />
| <br />
| {{Yes}}<br />
|-<br />
| '''Auna Air 300'''<br />
| bluez5, pulseaudio-git<br />
| For some reason, a few restarts were required, and eventually it just started working.<br />
| {{R|Limited}}<br />
|-<br />
| '''Sennheiser MM 400-X'''<br />
| bluez5, pulseaudio 4.0-6<br />
| <br />
| {{Yes}}<br />
|-<br />
| '''Sennheiser MM 550-X Travel'''<br />
| bluez 5.27-1, pulseaudio 5.0-1<br />
| Next/Previous buttons work out-of-the-box, Play/Pause does not<br />
| {{Yes}}<br />
|-<br />
| '''Audionic BlueBeats (B-777)'''<br />
| bluez5, pulseaudio 4.0-6<br />
| <br />
| {{Yes}}<br />
|-<br />
| '''Logitech Wireless Headset'''<br />
| bluez 5.14, pulseaudio-git<br />
| part number PN 981-000381, advertised for use with iPad<br />
| {{Yes}}<br />
|-<br />
| '''HMDX Jam Classic Bluetooth'''<br />
| bluez, pulseaudio-git<br />
| <br />
| {{Yes}}<br />
|-<br />
| '''PT-810'''<br />
| bluez 5.14, pulseaudio-git<br />
| Generic USB-Powered Bluetooth Audio Receiver with 3.5mm headset jack and a2dp profile. Widely available as "USB Bluetooth Receiver." IDs as PT-810.<br />
| {{Yes}}<br />
|-<br />
| '''Philips SHB4000WT'''<br />
| bluez5<br />
| A2DP works, HDP distorted.<br />
| {{Yes}}<br />
|-<br />
| '''Philips AEA2000/12'''<br />
| bluez5<br />
| <br />
| {{Yes}}<br />
|-<br />
| '''Nokia BH-104'''<br />
| bluez4<br />
| <br />
| {{Yes}}<br />
|-<br />
| '''Creative AirwaveHD'''<br />
| bluez 5.23<br />
| Bluetooth adapter Atheros Communications usb: 0cf3:0036<br />
| {{Yes}}<br />
|-<br />
| '''Creative HITZ WP380'''<br />
| bluez 5.27, pulseaudio 5.0-1<br />
| A2DP Profile only. Buttons work (Play, Pause, Prev, Next). Volume buttons are hardware-only. Auto-connect works but you should include the bluetooth module in "pulseaudio" to switch to it automatically. Clear HD Music Audio (This device support APTx codec but it isn't supported in linux yet). You may have some latency problems which needs pulseaudio restart.<br />
| {{Yes}}<br />
|-<br />
| '''deleyCON Bluetooth Headset'''<br />
| bluez 5.23<br />
| Adapter: CSL - USB nano Bluetooth-Adapter V4.0. Tested a2dp profile. Untested microphone. Does not auto-connect (even when paired and trusted), must connect manually. Play/pause button mutes/unmutes the headphones, not the playback. Playback fwd/bwd buttons do not work (nothing visible with ''xev'').<br />
| {{R|Limited}}<br />
|-<br />
| '''UE BOOM'''<br />
| bluez 5.27, pulseaudio-git 5.99<br />
| Update to latest UE BOOM fw 1.3.58. Sound latency in video solved by configuring pavucontrol. Works with UE BOOM x2.<br />
| {{Yes}}<br />
|-<br />
| '''LG HBS-730'''<br />
| bluez 5.30, pulseaudio 6.0<br />
| Works out of box with A2DP profile.<br />
| {{Yes}}<br />
|-<br />
| '''LG HBS-750'''<br />
| bluez 5.30, pulseaudio-git 6.0<br />
| Works out of box with A2DP profile.<br />
| {{Yes}}<br />
|-<br />
| '''Beats Studio Wireless'''<br />
| bluez 5.28, pulseaudio 6.0<br />
| Works out of box. Not tested multimedia buttons.<br />
| {{Yes}}<br />
|-<br />
| '''AKG Y45BT'''<br />
| bluez 5.30, pulseaudio 6.0<br />
| Pause and resume does not work. Needs {{ic|1=Enable=Socket}} in {{ic|/etc/bluetooth/audio.conf}} and {{ic|load-module module-bluetooth-discover}} in {{ic|/etc/pulse/default.pa}}.<br />
| {{Yes}}<br />
|-<br />
| '''Bluedio Turbine/Turbine 2+'''<br />
| bluez 5.3, pulseaudio 6.0<br />
| HSP/HFP work fine, A2DP work fine.<br />
| {{Yes}}<br />
|-<br />
| '''Sony SBH20'''<br />
| bluez 5.30, pulseaudio 6.0<br />
| Works out of box with A2DP profile.<br />
| {{Yes}}<br />
|-<br />
| '''Nokia BH-111'''<br />
| bluez 5.30, pulseaudio 6.0<br />
| Works with both HSP/HFP and A2DP. Buttons work in certain apps.<br />
| {{Yes}}<br />
|-<br />
| '''Sony MDR-ZX330BT'''<br />
| bluez 5.31, pulseaudio 6.0<br />
| Works out of box (HSP/HFP and A2DP). Buttons work in certain apps.<br />
| {{Yes}}<br />
|-<br />
| '''Samsung Level Link'''<br />
| bluez 5.33, pulseaudio 6.0<br />
| Works out of box (HSP/HFP and A2DP). Buttons work in certain apps.<br />
| {{Yes}}<br />
|-<br />
| '''Bernafon Soundgate 3'''<br />
| bluez 5.34-2, pulseaudio 7.0-1<br />
| Works (A2DP) after installing blueman-git 2.0.r53.g2a812a8-1. <br />
| {{Yes}}<br />
|-<br />
| '''JBL E40 BT'''<br />
| bluez 5.35, pulseaudio 7.0<br />
| Connection works after installing blueberry and blueman. A2DP works after adding Disable=Socket to {{ic|/etc/bluetooth/audio.conf}}. Media buttons works.<br />
| {{Yes}}<br />
|}<br />
<br />
== See also ==<br />
<br />
<br />
Using the same device on Windows and Linux without pairing the device over and over again<br />
<br />
* [http://ubuntuforums.org/showthread.php?p=9363229#post9363229 Dual booting with a Bluetooth keyboard]</div>Justin8https://wiki.archlinux.org/index.php?title=Talk:Bluetooth_headset&diff=404205Talk:Bluetooth headset2015-10-11T12:18:54Z<p>Justin8: </p>
<hr />
<div>== Is note about BlueZ5 still valid? (Asked on Oct 11, 2015) ==<br />
Currently, the first note on the page is:<br />
{{Note|1=<nowiki></nowiki><br />
* The most recent version of [[Bluez]] does not support the Headset/Handsfree profiles (see [https://bugs.freedesktop.org/show_bug.cgi?id=73325 this bug report] for example). This means that microphone input will not work, and also no sound output on headsets that do not support the A2DP profile. For using a headset with the Headset/Handsfree profiles, you will need to jump down to the legacy methods which require using [[AUR]] to fetch alternative packages.}}<br />
<br />
''But is this still valid?'' (I could not figure out when it was posted; and the mentioned bug was marked as resolved a few months earlier. I've never used this, and would like to buy an Arch-compatible headset, that's why I'm asking.)<br />
<br />
* The latest pulseaudio supports HSP instead of bluez doing it itself; so it does work now. My headset has poor support, but it does connect in HSP mode using bluez 5.35 and pulseaudio 7.0<br />
<br />
== Tested headsets ==<br />
<br />
Added tested headsets section. Not really Arch Specific, but it is very helpful for Linux users out there looking to use a bluetooth wireless headset with Linux. Couldn't find any wiki or site that lists similar tests so I placed it here... --[[User:Divansantana|Divan Santana]] ([[User talk:Divansantana|talk]]) 17:50, 12 August 2013 (UTC)<br />
<br />
* Parrot Zik doesn't actually work that brilliantly - I have a terrible lag, and there is no microphone support.<br />
:: Agreed that there is no mic support. Mine pretty much never stutters, but does lag behind in real time sometimes. Most of the time it's not noticeable for me unless playing a game. [[User:Divansantana|Divan Santana]] ([[User talk:Divansantana|talk]]) 16:04, 13 September 2013 (UTC)<br />
<br />
Might be nice to turn this section into a table, with exact models, adapters, what works, what doesn't, etc. [[User:Vlsd|Vlsd]] ([[User talk:Vlsd|talk]]) 20:17, 28 January 2014 (UTC)<br />
<br />
== bluez5 method: overcomplicated instructions ==<br />
<br />
The method in [[Bluetooth_Headset#ALSA.2C_bluez5_and_PulseAudio_method]] describes two different methods:<br />
<br />
# using [[ALSA]] as a backend, similar to [[Bluetooth_Headset#ALSA-BTSCO_method]] but relying on {{Pkg|bluez}} instead of {{Pkg|bluez4}}<br />
# using [[PulseAudio]] as a backend - this is tested in the article with [[Audacious]], which is configured to use PulseAudio as a backend<br />
<br />
I see two possible solutions:<br />
<br />
# split the section into two sections, each describing different backend<br />
# drop the ALSA instructions completely, suggest to install {{Pkg|pulseaudio-alsa}}<br />
<br />
I also think that suggesting to install {{Pkg|audacious}} just for testing the output is unnecessary, any application using PulseAudio can be used for this purpose.<br />
<br />
-- [[User:Lahwaacz|Lahwaacz]] ([[User talk:Lahwaacz|talk]]) 14:25, 27 December 2013 (UTC)<br />
<br />
:Point 1. Regarding bluez5 vs bluez4 -- Its fine to have a bluez5 implmentation since bluez4 method differs. The problems I encountered had numerous unanswered replies on the forums. ie. a user can pair but not connect. <br />
:Point 2. Regarding pulseaudio -- My understanding is that pulseaudio is not required in bluez4 but is in bluez5.<br />
:Point 3. -- Any app can be chosen to test, but the point is that a app needs to be chosen and the user needs to know how to select pulseaudio otherwise they will not know.<br />
:-- [[User:Netskink|Netskink]] ([[User talk:Netskink|talk]]) 12:02, 11 January 2014 (UTC)<br />
<br />
::Regarding ALSA: there is no current or planned way to get {{Pkg|bluez}} working with ALSA. The only feasible method is to use {{Pkg|pulseaudio-git}}. ALSA instructions should be removed from bluez5 sections until such support becomes available (if it ever becomes available).<br />
::Regarding {{Pkg|audacious}}: a more compact way would be to use speaker-test, although this requires {{Pkg|alsa}}, {{Pkg|alsa-utils}} and {{Pkg|pulseaudio-alsa}} installed.<br />
::[[User:Vlsd|Vlsd]] ([[User talk:Vlsd|talk]]) 20:29, 28 January 2014 (UTC)<br />
<br />
:::This page should focus on bluez5, any bluez4 information will be removed as soon as there is working bluez5 alternative.<br />
:::The section in question is really bloated, let's drop ALSA completely as [[User:Vlsd|Vlsd]] suggested - applications without PulseAudio support should work just fine through {{Pkg|pulseaudio-alsa}}, right?<br />
:::-- [[User:Lahwaacz|Lahwaacz]] ([[User talk:Lahwaacz|talk]]) 13:02, 1 February 2014 (UTC)<br />
<br />
::::ALSA is vital for [[JACK Audio Connection Kit]] users. <br />
::::[[User:Jbrickman0000|Jbrickman0000]] ([[User talk:Jbrickman0000|talk]]) 17:28, 12 April 2014 (UTC)<br />
<br />
== Is the tip about pulseaudio-git still valid? ==<br />
<br />
The version of the official pulseaudio package is now 6.0, which is greater than the pulseaudio-git AUR package. Can someone clarify if this tip is still valid?</div>Justin8https://wiki.archlinux.org/index.php?title=Bluetooth_headset&diff=402615Bluetooth headset2015-10-01T20:15:16Z<p>Justin8: </p>
<hr />
<div>[[Category:Sound]]<br />
[[Category:Bluetooth]]<br />
[[ja:Bluetooth ヘッドセット]]<br />
[[ru:Bluetooth headset]]<br />
[[zh-CN:Bluetooth headset]]<br />
{{Related articles start}}<br />
{{Related|Bluetooth}}<br />
{{Related|Bluez4}}<br />
{{Related articles end}}<br />
<br />
Currently, Arch Linux supports the A2DP profile (Audio Sink) for remote audio playback with the default installation. <br />
<br />
{{Note|1=<nowiki></nowiki><br />
* The most recent version of [[Bluez]] does not support the Headset/Handsfree profiles (see [https://bugs.freedesktop.org/show_bug.cgi?id=73325 this bug report] for example). This means that microphone input will not work, and also no sound output on headsets that do not support the A2DP profile. For using a headset with the Headset/Handsfree profiles, you will need to jump down to the legacy methods which require using [[AUR]] to fetch alternative packages.<br />
* Bluez5 is only supported by [[PulseAudio]] and not by [[ALSA]]. If you do not want to use PulseAudio, you need to install an older Bluez version from the AUR.<br />
}}<br />
<br />
== Headset via Bluez5/PulseAudio ==<br />
<br />
PulseAudio 5.x supports A2DP per default.<br />
Make sure the following packages are [[install]]ed: {{Pkg|pulseaudio-alsa}}, {{Pkg|pulseaudio-bluetooth}}, {{Pkg|bluez}}, {{Pkg|bluez-libs}}, {{Pkg|bluez-utils}}, {{Pkg|bluez-firmware}}. Without {{Pkg|pulseaudio-bluetooth}} you won't be able to connect after the next pairing and you won't get any usable error messages.<br />
<br />
Start the Bluetooth system:<br />
<br />
# systemctl start bluetooth<br />
<br />
Now we can use the ''bluetoothctl'' command line utility to pair and connect. For troubleshooting and more detailed explanations of ''bluetoothctl'' see the [[Bluetooth]] article. Run<br />
<br />
# bluetoothctl<br />
<br />
to be greeted by its internal command prompt. Then enter:<br />
<br />
# power on<br />
# agent on<br />
# default-agent<br />
# scan on<br />
<br />
Now make sure that your headset is in pairing mode. It should be discovered shortly. For example,<br />
[NEW] Device 00:1D:43:6D:03:26 Lasmex LBT10<br />
shows a device that calls itself "Lasmex LBT10" and has MAC address ''00:1D:43:6D:03:26''. We will now use that MAC address to initiate the pairing:<br />
<br />
# pair 00:1D:43:6D:03:26<br />
<br />
After pairing, you also need to explicitly connect the device (every time?):<br />
<br />
# connect 00:1D:43:6D:03:26<br />
<br />
If everything works correctly, you now have a separate output device in [[PulseAudio]].<br />
{{Note|The device may be off by default. Select its audio profile (''OFF'', A2DP, HFP) in the "Configuration" tab of {{Pkg|pavucontrol}}.}}<br />
You can now redirect any audio through that device using the "Playback" and "Recording" tabs of {{Pkg|pavucontrol}}.<br />
<br />
You can now disable scanning again and exit the program:<br />
# scan off<br />
# exit<br />
<br />
=== Troubleshooting ===<br />
<br />
Many users report frustration with getting A2DP/Bluetooth Headsets to work. <br />
<br />
==== Selected audio profile, but headset inactive and audio cannot be redirected ====<br />
<br />
Deceptively, this menu is available before the device has been connected; annoyingly it will have no effect. The menu seems to be created as soon as the receiver recognizes the device.<br />
<br />
Make sure to run bluetoothctl (with sudo/as root) and connect the device manually. There may be configuration options to remove the need to do this each time, but neither pairing nor trusting induce automatic connecting for me.<br />
<br />
==== Pairing fails with AuthenticationFailed ====<br />
<br />
If pairing fails, you can try [https://stackoverflow.com/questions/12888589/linux-command-line-howto-accept-pairing-for-bluetooth-device-without-pin disabling SSPMode] with:<br />
# hciconfig hci0 sspmode 0<br />
<br />
==== Pairing works, but connecting does not ====<br />
<br />
You might see the following error in ''bluetoothctl'':<br />
<br />
[bluetooth]# connect 00:1D:43:6D:03:26<br />
Attempting to connect to 00:1D:43:6D:03:26<br />
Failed to connect: org.bluez.Error.Failed<br />
<br />
To further investigate, have a look at the log via one of the following commands:<br />
<br />
# systemctl status bluetooth<br />
# journalctl -n 20<br />
<br />
You might see a message like this:<br />
<br />
bluetoothd[5556]: a2dp-sink profile connect failed for 00:1D:43:6D:03:26: Protocol not available<br />
<br />
This may be due to the {{Pkg|pulseaudio-bluetooth}} package not being installed. Install it if it missing, then restart pulseaudio.<br />
<br />
If the issue is not due to the missing package, the problem in this case is that PulseAudio is not catching up. A common solution to this problem is to restart PulseAudio. Note that it is perfectly fine to run ''bluetoothctl'' as root while PulseAudio runs as user. After restarting PulseAudio, retry to connect. It is not necessary to repeat the pairing.<br />
<br />
If restarting PulseAudio does not work, you need to load module-bluetooth-discover.<br />
<br />
# pactl load-module module-bluetooth-discover<br />
<br />
The same load-module command can be added to {{ic|/etc/pulse/default.pa}}.<br />
<br />
If that still does not work, or you are using PulseAudio's system-wide mode, also load the following PulseAudio modules (again these can be loaded via your default.pa or system.pa):<br />
<br />
module-bluetooth-policy<br />
module-bluez5-device<br />
module-bluez5-discover<br />
<br />
To have your headset auto connect you need to enable PulseAudio's switch-on-connect module. Add the following:<br />
{{hc|/etc/pulse/default.pa|<br />
# automatically switch to newly-connected devices<br />
load-module module-switch-on-connect<br />
}}<br />
<br />
You then need to tell ''bluetoothctl'' to trust your Bluetooth headset, or you will see errors like this:<br />
bluetoothd[487]: Authentication attempt without agent<br />
bluetoothd[487]: Access denied: org.bluez.Error.Rejected<br />
<br />
[bluetooth]# trust 00:1D:43:6D:03:26<br />
<br />
After a reboot, your Bluetooth adapter will not power on by default. You need to add a udev rule to power it on:<br />
{{hc|/etc/udev/rules.d/10-local.rules|2=<br />
# Set bluetooth power up<br />
ACTION=="add", SUBSYSTEM=="bluetooth", KERNEL=="hci[0-9]*", RUN+="/usr/bin/hciconfig %k up"<br />
}}<br />
<br />
==== Connecting works, but I cannot play sound ====<br />
<br />
Make sure that you see the following messages in your system log:<br />
<br />
bluetoothd[5556]: Endpoint registered: sender=:1.83 path=/MediaEndpoint/A2DPSource<br />
bluetoothd[5556]: Endpoint registered: sender=:1.83 path=/MediaEndpoint/A2DPSink<br />
<br />
If you see a message similar to this, you can go on and investigate your PulseAudio configuration. Otherwise, go back and ensure the connection is successful.<br />
<br />
When using [[GDM]], another instance of PulseAudio is started, which "captures" your bluetooth device connection. This can be prevented by removing the gdm user's permissions to execute pulseaudio using the following command:<br />
<br />
$ setfacl -m u:gdm:r /usr/bin/pulseaudio<br />
<br />
Now kill the pulseaudio instances, and gdm's will fail to restart (yours will and grab control of any bluetooth headset that is connected at that time):<br />
<br />
$ sudo pkill pulseaudio<br />
<br />
==== UUIDs has unsupported type ====<br />
<br />
During pairing you might see this output in ''bluetoothctl'':<br />
<br />
[CHG] Device 00:1D:43:6D:03:26 UUIDs has unsupported type<br />
<br />
This message is a very common one and can be ignored.<br />
<br />
== Legacy method: ALSA-BTSCO ==<br />
<br />
{{Out of date|Instructions rely on [[bluez4]].}}<br />
<br />
It is much easier to set up your bluetooth headset today, with bluez >= 3.16. You may want to try the out-of-box python script in [http://fosswire.com/2008/01/11/a2dp-stereo-linux/ this blog] (you need edit the script to work with gconftool-2). There is also a piece of equivalent bash script [http://lymanrb.blogspot.com/2008/05/linux.html here].<br />
<br />
You need your headset's bdaddr. It is of the form ''12:34:56:78:9A:BC''. Either find it in the documentation of your headset, on the headset itself or with the '''hcitool scan''' command.<br />
<br />
Install {{AUR|btsco}}{{Broken package link|{{aur-mirror|btsco}}}}.<br />
<br />
To load the kernel module, type:<br />
# modprobe snd-bt-sco<br />
There will now be an extra audio device. Use {{ic|alsamixer -cN}} (where N is most likely 1) to set the volume. You can access the device with any alsa-capable application by choosing the device ''BT headset'', or with any OSS application by using {{ic|/dev/dspN}} as the audio device.<br />
<br />
But to actually get any sound, you have to connect your headset to the computer first.<br />
<br />
=== Connecting the headset ===<br />
<br />
If you connect your headset for the first time, read the section about pairing first. To connect to your headset to the computer, use the command<br />
$ btsco -f <bdaddr><br />
for example<br />
$ btsco -f 12:34:56:78:9A:BC<br />
<br />
==== Pairing the headset with your computer ====<br />
<br />
The first time you connect the headset, you have to pair it with the computer. To do this, you need your headset's PIN. Depending on your headset you may have to reset the headset and repeat the pairing everytime you used the headset with another bluetooth device.<br />
<br />
There are two ways to pair your headset with the computer:<br />
<br />
===== Using ''bluez-gnome'' =====<br />
<br />
{{Out of date|Package does no longer exist in the repositories.}}<br />
<br />
Install the ''bluez-gnome'' package from the community repository. Then start the '''bt-applet''' program. Once you try to connect to the headset, a window will open and ask for the PIN.<br />
<br />
===== Using ''passkey-agent'' =====<br />
<br />
Before connecting to the headset, enter the command<br />
$ passkey-agent --default <pin><br />
where ''<pin>'' is your headset's PIN. Then try to connect to the headset.<br />
<br />
=== Headset and ALSA Devices ===<br />
<br />
1. First if you have not already, [[pacman|install]] {{Pkg|bluez}} from the [[official repositories]].<br />
<br />
2. Scan for your device<br />
$ hcitool (-i <optional hci#>***) scan<br />
<br />
3. Pair your headset with your device:<br />
$ bluez-simple-agent (optional hci# ***) XX:XX:XX:XX:XX:XX<br />
and put in your pin (0000 or 1234, etc)<br />
<br />
4. Make sure your {{ic|/etc/bluetooth/audio.conf}} allows A2DP Audio Sinks. Place this line just bellow the [General] heading:<br />
Enable=Source,Sink,Media,Socket<br />
<br />
5. Add this to your {{ic|/etc/asound.conf}} file:<br />
#/etc/asound.conf<br />
<br />
pcm.btheadset {<br />
type plug<br />
slave {<br />
pcm {<br />
type bluetooth<br />
device XX:XX:XX:XX:XX:XX <br />
profile "auto"<br />
} <br />
} <br />
hint {<br />
show on<br />
description "BT Headset"<br />
} <br />
}<br />
ctl.btheadset {<br />
type bluetooth<br />
} <br />
<br />
6. Check to see if it has been added to alsa devices<br />
$ aplay -L<br />
<br />
7. Now play with aplay:<br />
$ aplay -D btheadset /path/to/audio/file<br />
<br />
or Mplayer:<br />
$ mplayer -ao alsa:device=btheadset /path/to/audio/or/video/file<br />
<br />
{{Tip|To find hci# for a usb dongle, type in <br />
$ hcitool dev<br />
}}<br />
<br />
=== Headset's multimedia buttons ===<br />
<br />
In order to get your bluetooth headset's multimedia buttons (play, pause, next, previous) working you need to create {{ic|/etc/modules-load.d/uinput.conf}} containing {{ic|uinput}}.<br />
<br />
== Legacy method: PulseAudio ==<br />
<br />
{{Out of date|Instructions rely on [[bluez4]] (references to {{ic|/etc/bluetooth/audio.conf}} and ''bluez-simple-agent'').}}<br />
<br />
This one is much easier and more elegant. PulseAudio will seamlessly switch between output devices when the headset is turned on. If you have ALSA as the sound server, you need the following packages installed:<br />
{{Pkg|pulseaudio}} and {{Pkg|pulseaudio-alsa}}.<br />
<br />
Now, to configure the audio output to use bluetooth, just install {{Pkg|pavucontrol}} and run it to configure the audio output:<br />
$ pavucontrol<br />
Make sure to take a look at the [[PulseAudio]] wiki entry for setting up PulseAudio, especially if you are running KDE.<br />
<br />
=== Troubleshooting ===<br />
<br />
==== Audio sink fails ====<br />
<br />
Bluetooth headset is connected, but ALSA/PulseAudio fails to pick up the connected device. You will get "Audio sink fails".<br />
According to [http://wiki.gentoo.org/wiki/Bluetooth_Headset gentoo wiki], you have to verify than in {{ic|/etc/bluetooth/audio.conf}} there is {{ic|<nowiki>Enable=Socket</nowiki>}} under the {{ic|[General]}} section heading.<br />
<br />
Just do a {{ic|# systemctl restart bluetooth}} to apply it.<br />
<br />
==== Page timeout issue ====<br />
<br />
If you receive this error whilst trying to pair your headset with your system using bluez-simple-agent, then you can try to restart your system and use the graphical bluez applet of your desktop environment.<br />
<br />
== Legacy documentation: ALSA, bluez5 and PulseAudio method ==<br />
<br />
{{Accuracy|Describes two different methods, see the [[Talk:Bluetooth headset#bluez5 method: overcomplicated instructions|talk page]] for details.|Talk:Bluetooth_Headset#bluez5_method:_overcomplicated_instructions}}<br />
<br />
[[ALSA]], [[bluez|bluez5]], and [[PulseAudio]] work together to allow a wireless [[Bluetooth]] headset to play audio. The following method works with a Lenovo T61p laptop and SoundBot SB220 wireless bluetooth headset. The required software stack is extensive and failure to include all components can produce errors which are difficult to understand. The following list of software packages might not be the minimum required set and needs to be examined more closely.<br />
<br />
Bluez5 has a regression causing HSP/HFP Telephone profile to not be available. This regression is documented in the [http://www.freedesktop.org/wiki/Software/PulseAudio/Notes/5.0/ draft release notes for Pulseaudio 5.0] which say (in "Notes for packagers"): "PulseAudio now supports BlueZ 5, but only the A2DP profile. BlueZ 4 is still the only way to make HSP/HFP work." ([https://fedoraproject.org/wiki/Common_F20_bugs#bluez5-profile from here])<br />
<br />
=== Install Software Packages ===<br />
<br />
The core software components are [[ALSA]], Bluez5, [[PulseAudio]]. However there are additional libraries which are required. As well as a player which can play audio files. The following section lists the software packages installed in order to connect the headset and play audio over the headset.<br />
<br />
==== Install ALSA and associated libraries ====<br />
<br />
[[ALSA]] works with the linux kernel to provide audio services to user mode software. The following packages are used with the [[Bluetooth]] headset: {{Pkg|alsa-utils}}, {{Pkg|alsa-plugins}}, {{Pkg|alsa-tools}}.<br />
<br />
==== Install Bluez5 ====<br />
<br />
Bluez5 is the latest [[Bluetooth]] stack. It is required for [[PulseAudio]] to interface with wireless headsets. Required packages: {{Pkg|bluez}}, {{Pkg|bluez-utils}}, {{Pkg|bluez-libs}}.<br />
<br />
==== Install Audacious ====<br />
<br />
[[Audacious]] is a program which plays audio files. It can work directly with [[ALSA]] or with [[PulseAudio]]. Required packages: {{Pkg|audacious}}, {{Pkg|audacious-plugins}}.<br />
<br />
=== Procedure ===<br />
<br />
Once the required packages are installed, use this procedure to play audio with a bluetooth headset. The high level overview of the procedure is to pair the headset, connect the headset, configure the player and pulse audio controller and then play audio.<br />
<br />
Start the bluetooth service as root:<br />
# systemctl start bluetooth<br />
<br />
Verify Bluetooth is started<br />
# systemctl status bluetooth<br />
bluetooth.service - Bluetooth service<br />
Loaded: loaded (/usr/lib/systemd/system/bluetooth.service; disabled)<br />
Active: active (running) since Sat 2013-12-07 12:31:14 PST; 12s ago<br />
Docs: man:bluetoothd(8)<br />
Main PID: 3136 (bluetoothd)<br />
Status: "Running"<br />
CGroup: /system.slice/bluetooth.service<br />
└─3136 /usr/lib/bluetooth/bluetoothd<br />
<br />
Dec 07 12:31:14 t61p systemd[1]: Starting Bluetooth service...<br />
Dec 07 12:31:14 t61p bluetoothd[3136]: Bluetooth daemon 5.11<br />
Dec 07 12:31:14 t61p systemd[1]: Started Bluetooth service.<br />
Dec 07 12:31:14 t61p bluetoothd[3136]: Starting SDP server<br />
Dec 07 12:31:14 t61p bluetoothd[3136]: Bluetooth management interface 1.3 i...ed<br />
Hint: Some lines were ellipsized, use -l to show in full.<br />
<br />
Start the PulseAudio daemon. This must be done after X windows is started and as a normal user.<br />
$ pulseaudio -D<br />
<br />
Verify the PulseAudio daemon is running.<br />
$ pulseaudio --check -v<br />
I: [pulseaudio] main.c: Daemon running as PID 3186<br />
<br />
Start up bluetoothctl as root and pair and connect your headset. As a regular user, bluetoothctl will pair but not connect. Perhaps this is related to the config file (shown below) which is setup for what appears to be the root user.<br />
Note: the procedure shown below is for an initial pair and connect of the headphone. If the headset is already paired, then the procedure below can be shortened to: power on, agent on, default-agent, connect <mac address>. The mac address can be seen from the devices command output.<br />
<br />
$ bluetoothctl <br />
[NEW] Controller 00:1E:4C:F4:98:5B t61p-0 [default]<br />
[NEW] Device 00:1A:7D:12:36:B9 SoundBot SB220<br />
[bluetooth]# show<br />
Controller 00:1E:4C:F4:98:5B<br />
Name: t61p<br />
Alias: t61p-0<br />
Class: 0x000000<br />
Powered: no<br />
Discoverable: no<br />
Pairable: yes<br />
UUID: PnP Information (00001200-0000-1000-8000-00805f9b34fb)<br />
UUID: Generic Access Profile (00001800-0000-1000-8000-00805f9b34fb)<br />
UUID: Generic Attribute Profile (00001801-0000-1000-8000-00805f9b34fb)<br />
UUID: A/V Remote Control (0000110e-0000-1000-8000-00805f9b34fb)<br />
UUID: A/V Remote Control Target (0000110c-0000-1000-8000-00805f9b34fb)<br />
UUID: Audio Source (0000110a-0000-1000-8000-00805f9b34fb)<br />
UUID: Audio Sink (0000110b-0000-1000-8000-00805f9b34fb)<br />
Modalias: usb:v1D6Bp0246d050B<br />
Discovering: no<br />
[bluetooth]# power on<br />
[CHG] Controller 00:1E:4C:F4:98:5B Class: 0x0c010c<br />
Changing power on succeeded<br />
[CHG] Controller 00:1E:4C:F4:98:5B Powered: yes<br />
[bluetooth]# agent on<br />
Agent registered<br />
[bluetooth]# default-agent<br />
Default agent request successful<br />
<br />
<power on your headset in pairing mode. Eventually you will see what appears to be a mac address.><br />
[bluetooth]# scan on<br />
Discovery started<br />
[CHG] Controller 00:1E:4C:F4:98:5B Discovering: yes<br />
[CHG] Device 00:1A:7D:12:36:B9 RSSI: -61<br />
[bluetooth]# pair 00:1A:7D:12:36:B9<br />
Attempting to pair with 00:1A:7D:12:36:B9<br />
[CHG] Device 00:1A:7D:12:36:B9 Connected: yes<br />
[CHG] Device 00:1A:7D:12:36:B9 UUIDs has unsupported type<br />
[CHG] Device 00:1A:7D:12:36:B9 Paired: yes<br />
Pairing successful<br />
[bluetooth]# connect 00:1A:7D:12:36:B9<br />
[CHG] Device 00:1A:7D:12:36:B9 Connected: yes<br />
Connection successful<br />
[bluetooth]# info 00:1A:7D:12:36:B9<br />
Device 00:1A:7D:12:36:B9<br />
Name: SoundBot SB220<br />
Alias: SoundBot SB220<br />
Class: 0x240404<br />
Icon: audio-card<br />
Paired: yes<br />
Trusted: no<br />
Blocked: no<br />
Connected: yes<br />
LegacyPairing: yes<br />
UUID: Headset (00001108-0000-1000-8000-00805f9b34fb)<br />
UUID: Audio Sink (0000110b-0000-1000-8000-00805f9b34fb)<br />
UUID: A/V Remote Control Target (0000110c-0000-1000-8000-00805f9b34fb)<br />
UUID: A/V Remote Control (0000110e-0000-1000-8000-00805f9b34fb)<br />
UUID: Handsfree (0000111e-0000-1000-8000-00805f9b34fb)<br />
<br />
Start up alsamixer, for simplicity un-mute all your outputs. Oddly enough some can be muted though. The ones I had muted during playback were:<br />
* Headphones<br />
* SPIDF<br />
<br />
Start up audacious. Use the menu to select PulseAudio as your output. Somewhere I read that bluez5 requires pulseaudio-git and this jives with my experience.<br />
<br />
Start up pavucontrol in a terminal. In the Outputs tab select the bluetooth headset.<br />
<br />
[http://netskink.blogspot.com/2013/12/pulseaudio-pavucontrol-and-audacious.html screenshot of application settings]<br />
<br />
==== Miscellaneous configuration files ====<br />
<br />
For reference, these settings were also done.<br />
<br />
===== ALSA /etc/asound.conf =====<br />
<br />
The settings shown at the top of this page was used, but the additional modification for Intel laptop sound cards.<br />
<br />
{{bc|<nowiki><br />
pcm.btheadset {<br />
type plug<br />
slave {<br />
pcm {<br />
type bluetooth<br />
device 00:1A:7D:12:36:B9<br />
profile "auto"<br />
}<br />
}<br />
hint {<br />
show on<br />
description "BT Headset"<br />
}<br />
}<br />
ctl.btheadset {<br />
type bluetooth<br />
}<br />
options snd-hda-intel model=laptop<br />
</nowiki>}}<br />
<br />
===== /etc/dbus-1/system.d/bluetooth.conf =====<br />
<br />
The settings here seem to be enabled for root only. See the policy user="root" section. However, if a regular user is specified here, the system fails to start. Someone with more knowledge could explain why.<br />
<br />
{{hc|/etc/dbus-1/system.d/bluetooth.conf|<nowiki><br />
<!-- This configuration file specifies the required security policies for Bluetooth core daemon to work. --><br />
<br />
<!DOCTYPE busconfig PUBLIC "-//freedesktop//DTD D-BUS Bus Configuration 1.0//EN"<br />
"http://www.freedesktop.org/standards/dbus/1.0/busconfig.dtd"><br />
<busconfig><br />
<br />
<!-- ../system.conf have denied everything, so we just punch some holes --><br />
<br />
<policy user="root"><br />
<allow own="org.bluez"/><br />
<allow send_destination="org.bluez"/><br />
<allow send_interface="org.bluez.Agent1"/><br />
<allow send_interface="org.bluez.MediaEndpoint1"/><br />
<allow send_interface="org.bluez.MediaPlayer1"/><br />
<allow send_interface="org.bluez.ThermometerWatcher1"/><br />
<allow send_interface="org.bluez.AlertAgent1"/><br />
<allow send_interface="org.bluez.Profile1"/><br />
<allow send_interface="org.bluez.HeartRateWatcher1"/><br />
<allow send_interface="org.bluez.CyclingSpeedWatcher1"/><br />
</policy><br />
<br />
<policy at_console="true"><br />
<allow send_destination="org.bluez"/><br />
</policy><br />
<br />
<!-- allow users of lp group (printing subsystem) to communicate with bluetoothd --><br />
<policy group="lp"><br />
<allow send_destination="org.bluez"/><br />
</policy><br />
<br />
<policy context="default"><br />
<deny send_destination="org.bluez"/><br />
</policy><br />
<br />
</busconfig><br />
</nowiki>}}<br />
<br />
===== Tested applications =====<br />
<br />
As noted above this will work easily with audacious. YouTube videos with Chromium and Flash Player will work on some videos. If the video has ads it will not work, but if the video does not have ads it will work. Just make sure that after audacious is working with Bluetooth headset, start Chromium, and navigate to YouTube. Find a video without leading ads, and it should play the audio. If the settings icon has the a menu with two drop-down combo boxes for Speed and Quality it will play.<br />
<br />
== Switch between HSV and A2DP setting ==<br />
<br />
This can easily be achieved by the following command where 2 needs to be changed with the correct device number.<br />
<br />
pacmd set-card-profile 2 a2dp_sink<br />
<br />
=== A2DP not working with PulseAudio ===<br />
<br />
==== Socket interface problem ====<br />
<br />
If PulseAudio fails when changing the profile to A2DP with bluez 4.1+ and PulseAudio 3.0+, you can try disabling the Socket interface from {{ic|/etc/bluetooth/audio.conf}} by removing the line {{ic|1=Enable=Socket}} and adding line {{ic|1=Disable=Socket}}.<br />
<br />
==== Gnome with GDM ====<br />
{{Note|Below was tested with Gnome 3.16.2 and PulseAudio 7.0}}<br />
<br />
If PulseAudio fails when changing the profile to A2DP while using GNOME with GDM, you need to prevent GDM from starting its own instance of PulseAudio. Apply the same fix as shown in [[#Connecting works, but I cannot play sound]].<br />
<br />
{{Note|1=Discussion about this problem can be found [https://bbs.archlinux.org/viewtopic.php?id=194006 here] and [https://bbs.archlinux.org/viewtopic.php?id=196689 here]}}<br />
<br />
== Tested headsets ==<br />
<br />
{| class="wikitable"<br />
! Model<br />
! Version<br />
! Comments<br />
! Compatible<br />
|-<br />
| '''Philips SHB9150'''<br />
| bluez5, pulseaudio 5<br />
| Pause and resume does not work. With at least mpv and Banshee hitting the pause button stops audio output but does not pause the player.<br />
| {{R|Limited}}<br />
|-<br />
| '''Philips SHB9100'''<br />
| <br />
| Pause and resume is flaky. See [https://bbs.archlinux.org/viewtopic.php?pid=1315428#p1315428] for the underlying issue and a temporary solution to improve audio quality.<br />
| {{R|Limited}}<br />
|-<br />
| '''Philips SHB7000'''<br />
| <br />
| Pause and resume is flaky.<br />
| {{R|Limited}}<br />
|-<br />
| '''Philips SHB7100'''<br />
| bluez 5.32, pulseaudio 6.0<br />
| Next/previous buttons work. Pause and resume is flaky (sometimes works in VLC, not at all in Audacious). Tested only A2DP and Handsfree audio out, built-in mic was broken.<br />
| {{R|Limited}}<br />
|-<br />
| '''Philips SHB7150'''<br />
| bluez 5.32, pulseaudio 6.0<br />
| Next/previous buttons work. Pause and resume work in VLC. Tested only A2DP profile.<br />
| {{Yes}}<br />
|-<br />
| '''Philips SHB5500BK/00'''<br />
| bluez 5.28, PulseAudio 6.0<br />
| Pause and resume is not working.<br />
| {{R|Limited}}<br />
|-<br />
| '''Parrot Zik'''<br />
| <br />
| Firmware 1.04. The microphone is detected, but does not work. Sometimes it lags (but does not stutter); usually this is not noticeable unless playing games, in which case you may switch to a wired connection.<br />
| {{R|Limited}}<br />
|-<br />
| '''Sony DR-BT50'''<br />
| bluez{4,5}<br />
| Works for a2dp, see [http://vlsd.blogspot.com/2013/11/bluetooth-headphones-and-arch-linux.html]). Adapter: D-Link DBT-120 USB dongle.<br />
| {{Yes}}<br />
|-<br />
| '''Sony SBH50'''<br />
| bluez5<br />
| Works for a2dp, Adapter: Broadcom Bluetooth 2.1 Device (Vendor=0a5c ProdID=219b Rev=03.43). Requires the {{ic|btusb}} [[modprobe|module]].<br />
| {{Yes}}<br />
|-<br />
| '''Sony MDR-XB950BT'''<br />
| pulseaudio<br />
| Tested a2dp. Adapter: Grand-X BT40G. Doesn't auto-connect, need to connect manually. Other functionality works fine.<br />
| {{R|Limited}}<br />
|-<br />
| '''Sony MUC-M1BT1'''<br />
| bluez5, {{AUR|pulseaudio-git}}<br />
| Both A2DP & HSP/HFP work fine.<br />
| {{Yes}}<br />
|-<br />
| '''SoundBot SB220'''<br />
| bluez5, {{AUR|pulseaudio-git}}<br />
| <br />
| {{Yes}}<br />
|-<br />
| '''Auna Air 300'''<br />
| bluez5, pulseaudio-git<br />
| For some reason, a few restarts were required, and eventually it just started working.<br />
| {{R|Limited}}<br />
|-<br />
| '''Sennheiser MM 400-X'''<br />
| bluez5, pulseaudio 4.0-6<br />
| <br />
| {{Yes}}<br />
|-<br />
| '''Sennheiser MM 550-X Travel'''<br />
| bluez 5.27-1, pulseaudio 5.0-1<br />
| Next/Previous buttons work out-of-the-box, Play/Pause does not<br />
| {{Yes}}<br />
|-<br />
| '''Audionic BlueBeats (B-777)'''<br />
| bluez5, pulseaudio 4.0-6<br />
| <br />
| {{Yes}}<br />
|-<br />
| '''Logitech Wireless Headset'''<br />
| bluez 5.14, pulseaudio-git<br />
| part number PN 981-000381, advertised for use with iPad<br />
| {{Yes}}<br />
|-<br />
| '''HMDX Jam Classic Bluetooth'''<br />
| bluez, pulseaudio-git<br />
| <br />
| {{Yes}}<br />
|-<br />
| '''PT-810'''<br />
| bluez 5.14, pulseaudio-git<br />
| Generic USB-Powered Bluetooth Audio Receiver with 3.5mm headset jack and a2dp profile. Widely available as "USB Bluetooth Receiver." IDs as PT-810.<br />
| {{Yes}}<br />
|-<br />
| '''Philips SHB4000WT'''<br />
| bluez5<br />
| A2DP works, HDP distorted.<br />
| {{Yes}}<br />
|-<br />
| '''Philips AEA2000/12'''<br />
| bluez5<br />
| <br />
| {{Yes}}<br />
|-<br />
| '''Nokia BH-104'''<br />
| bluez4<br />
| <br />
| {{Yes}}<br />
|-<br />
| '''Creative AirwaveHD'''<br />
| bluez 5.23<br />
| Bluetooth adapter Atheros Communications usb: 0cf3:0036<br />
| {{Yes}}<br />
|-<br />
| '''Creative HITZ WP380'''<br />
| bluez 5.27, pulseaudio 5.0-1<br />
| A2DP Profile only. Buttons work (Play, Pause, Prev, Next). Volume buttons are hardware-only. Auto-connect works but you should include the bluetooth module in "pulseaudio" to switch to it automatically. Clear HD Music Audio (This device support APTx codec but it isn't supported in linux yet). You may have some latency problems which needs pulseaudio restart.<br />
| {{Yes}}<br />
|-<br />
| '''deleyCON Bluetooth Headset'''<br />
| bluez 5.23<br />
| Adapter: CSL - USB nano Bluetooth-Adapter V4.0. Tested a2dp profile. Untested microphone. Does not auto-connect (even when paired and trusted), must connect manually. Play/pause button mutes/unmutes the headphones, not the playback. Playback fwd/bwd buttons do not work (nothing visible with ''xev'').<br />
| {{R|Limited}}<br />
|-<br />
| '''UE BOOM'''<br />
| bluez 5.27, pulseaudio-git 5.99<br />
| Update to latest UE BOOM fw 1.3.58. Sound latency in video solved by configuring pavucontrol. Works with UE BOOM x2.<br />
| {{Yes}}<br />
|-<br />
| '''LG HBS-730'''<br />
| bluez 5.30, pulseaudio 6.0<br />
| Works out of box with A2DP profile.<br />
| {{Yes}}<br />
|-<br />
| '''LG HBS-750'''<br />
| bluez 5.30, pulseaudio-git 6.0<br />
| Works out of box with A2DP profile.<br />
| {{Yes}}<br />
|-<br />
| '''Beats Studio Wireless'''<br />
| bluez 5.28, pulseaudio 6.0<br />
| Works out of box. Not tested multimedia buttons.<br />
| {{Yes}}<br />
|-<br />
| '''AKG Y45BT'''<br />
| bluez 5.30, pulseaudio 6.0<br />
| Pause and resume does not work. Needs {{ic|1=Enable=Socket}} in {{ic|/etc/bluetooth/audio.conf}} and {{ic|load-module module-bluetooth-discover}} in {{ic|/etc/pulse/default.pa}}.<br />
| {{Yes}}<br />
|-<br />
| '''Bluedio Turbine/Turbine 2+'''<br />
| bluez 5.3, pulseaudio 6.0<br />
| HSP/HFP work fine, A2DP work fine.<br />
| {{Yes}}<br />
|-<br />
| '''Sony SBH20'''<br />
| bluez 5.30, pulseaudio 6.0<br />
| Works out of box with A2DP profile.<br />
| {{Yes}}<br />
|-<br />
| '''Nokia BH-111'''<br />
| bluez 5.30, pulseaudio 6.0<br />
| Works with both HSP/HFP and A2DP. Buttons work in certain apps.<br />
| {{Yes}}<br />
|-<br />
| '''Sony MDR-ZX330BT'''<br />
| bluez 5.31, pulseaudio 6.0<br />
| Works out of box (HSP/HFP and A2DP). Buttons work in certain apps.<br />
| {{Yes}}<br />
|-<br />
| '''Samsung Level Link'''<br />
| bluez 5.33, pulseaudio 6.0<br />
| Works out of box (HSP/HFP and A2DP). Buttons work in certain apps.<br />
| {{Yes}}<br />
|-<br />
| '''Bernafon Soundgate 3'''<br />
| bluez 5.34-2, pulseaudio 7.0-1<br />
| Works (A2DP) after installing blueman-git 2.0.r53.g2a812a8-1. <br />
| {{Yes}}<br />
|}<br />
<br />
== See also ==<br />
<br />
<br />
Using the same device on Windows and Linux without pairing the device over and over again<br />
<br />
* [http://ubuntuforums.org/showthread.php?p=9363229#post9363229 Dual booting with a Bluetooth keyboard]</div>Justin8https://wiki.archlinux.org/index.php?title=Bluetooth_headset&diff=402614Bluetooth headset2015-10-01T20:11:49Z<p>Justin8: Setting autospawn = no in gdm's pulseaudio client.conf does not seem to work any more.</p>
<hr />
<div>[[Category:Sound]]<br />
[[Category:Bluetooth]]<br />
[[ja:Bluetooth ヘッドセット]]<br />
[[ru:Bluetooth headset]]<br />
[[zh-CN:Bluetooth headset]]<br />
{{Related articles start}}<br />
{{Related|Bluetooth}}<br />
{{Related|Bluez4}}<br />
{{Related articles end}}<br />
<br />
Currently, Arch Linux supports the A2DP profile (Audio Sink) for remote audio playback with the default installation. <br />
<br />
{{Note|1=<nowiki></nowiki><br />
* The most recent version of [[Bluez]] does not support the Headset/Handsfree profiles (see [https://bugs.freedesktop.org/show_bug.cgi?id=73325 this bug report] for example). This means that microphone input will not work, and also no sound output on headsets that do not support the A2DP profile. For using a headset with the Headset/Handsfree profiles, you will need to jump down to the legacy methods which require using [[AUR]] to fetch alternative packages.<br />
* Bluez5 is only supported by [[PulseAudio]] and not by [[ALSA]]. If you do not want to use PulseAudio, you need to install an older Bluez version from the AUR.<br />
}}<br />
<br />
== Headset via Bluez5/PulseAudio ==<br />
<br />
PulseAudio 5.x supports A2DP per default.<br />
Make sure the following packages are [[install]]ed: {{Pkg|pulseaudio-alsa}}, {{Pkg|pulseaudio-bluetooth}}, {{Pkg|bluez}}, {{Pkg|bluez-libs}}, {{Pkg|bluez-utils}}, {{Pkg|bluez-firmware}}. Without {{Pkg|pulseaudio-bluetooth}} you won't be able to connect after the next pairing and you won't get any usable error messages.<br />
<br />
Start the Bluetooth system:<br />
<br />
# systemctl start bluetooth<br />
<br />
Now we can use the ''bluetoothctl'' command line utility to pair and connect. For troubleshooting and more detailed explanations of ''bluetoothctl'' see the [[Bluetooth]] article. Run<br />
<br />
# bluetoothctl<br />
<br />
to be greeted by its internal command prompt. Then enter:<br />
<br />
# power on<br />
# agent on<br />
# default-agent<br />
# scan on<br />
<br />
Now make sure that your headset is in pairing mode. It should be discovered shortly. For example,<br />
[NEW] Device 00:1D:43:6D:03:26 Lasmex LBT10<br />
shows a device that calls itself "Lasmex LBT10" and has MAC address ''00:1D:43:6D:03:26''. We will now use that MAC address to initiate the pairing:<br />
<br />
# pair 00:1D:43:6D:03:26<br />
<br />
After pairing, you also need to explicitly connect the device (every time?):<br />
<br />
# connect 00:1D:43:6D:03:26<br />
<br />
If everything works correctly, you now have a separate output device in [[PulseAudio]].<br />
{{Note|The device may be off by default. Select its audio profile (''OFF'', A2DP, HFP) in the "Configuration" tab of {{Pkg|pavucontrol}}.}}<br />
You can now redirect any audio through that device using the "Playback" and "Recording" tabs of {{Pkg|pavucontrol}}.<br />
<br />
You can now disable scanning again and exit the program:<br />
# scan off<br />
# exit<br />
<br />
=== Troubleshooting ===<br />
<br />
Many users report frustration with getting A2DP/Bluetooth Headsets to work. <br />
<br />
==== Selected audio profile, but headset inactive and audio cannot be redirected ====<br />
<br />
Deceptively, this menu is available before the device has been connected; annoyingly it will have no effect. The menu seems to be created as soon as the receiver recognizes the device.<br />
<br />
Make sure to run bluetoothctl (with sudo/as root) and connect the device manually. There may be configuration options to remove the need to do this each time, but neither pairing nor trusting induce automatic connecting for me.<br />
<br />
==== Pairing fails with AuthenticationFailed ====<br />
<br />
If pairing fails, you can try [https://stackoverflow.com/questions/12888589/linux-command-line-howto-accept-pairing-for-bluetooth-device-without-pin disabling SSPMode] with:<br />
# hciconfig hci0 sspmode 0<br />
<br />
==== Pairing works, but connecting does not ====<br />
<br />
You might see the following error in ''bluetoothctl'':<br />
<br />
[bluetooth]# connect 00:1D:43:6D:03:26<br />
Attempting to connect to 00:1D:43:6D:03:26<br />
Failed to connect: org.bluez.Error.Failed<br />
<br />
To further investigate, have a look at the log via one of the following commands:<br />
<br />
# systemctl status bluetooth<br />
# journalctl -n 20<br />
<br />
You might see a message like this:<br />
<br />
bluetoothd[5556]: a2dp-sink profile connect failed for 00:1D:43:6D:03:26: Protocol not available<br />
<br />
This may be due to the {{Pkg|pulseaudio-bluetooth}} package not being installed. Install it if it missing, then restart pulseaudio.<br />
<br />
If the issue is not due to the missing package, the problem in this case is that PulseAudio is not catching up. A common solution to this problem is to restart PulseAudio. Note that it is perfectly fine to run ''bluetoothctl'' as root while PulseAudio runs as user. After restarting PulseAudio, retry to connect. It is not necessary to repeat the pairing.<br />
<br />
If restarting PulseAudio does not work, you need to load module-bluetooth-discover.<br />
<br />
# pactl load-module module-bluetooth-discover<br />
<br />
The same load-module command can be added to {{ic|/etc/pulse/default.pa}}.<br />
<br />
If that still does not work, or you are using PulseAudio's system-wide mode, also load the following PulseAudio modules (again these can be loaded via your default.pa or system.pa):<br />
<br />
module-bluetooth-policy<br />
module-bluez5-device<br />
module-bluez5-discover<br />
<br />
To have your headset auto connect you need to enable PulseAudio's switch-on-connect module. Add the following:<br />
{{hc|/etc/pulse/default.pa|<br />
# automatically switch to newly-connected devices<br />
load-module module-switch-on-connect<br />
}}<br />
<br />
You then need to tell ''bluetoothctl'' to trust your Bluetooth headset, or you will see errors like this:<br />
bluetoothd[487]: Authentication attempt without agent<br />
bluetoothd[487]: Access denied: org.bluez.Error.Rejected<br />
<br />
[bluetooth]# trust 00:1D:43:6D:03:26<br />
<br />
After a reboot, your Bluetooth adapter will not power on by default. You need to add a udev rule to power it on:<br />
{{hc|/etc/udev/rules.d/10-local.rules|2=<br />
# Set bluetooth power up<br />
ACTION=="add", SUBSYSTEM=="bluetooth", KERNEL=="hci[0-9]*", RUN+="/usr/bin/hciconfig %k up"<br />
}}<br />
<br />
==== Connecting works, but I cannot play sound ====<br />
<br />
Make sure that you see the following messages in your system log:<br />
<br />
bluetoothd[5556]: Endpoint registered: sender=:1.83 path=/MediaEndpoint/A2DPSource<br />
bluetoothd[5556]: Endpoint registered: sender=:1.83 path=/MediaEndpoint/A2DPSink<br />
<br />
If you see a message similar to this, you can go on and investigate your PulseAudio configuration. Otherwise, go back and ensure the connection is successful.<br />
<br />
When using [[GDM]], another instance of PulseAudio is started, which "captures" your bluetooth device connection. This can be prevented by removing the gdm user's permissions to execute pulseaudio using the following command:<br />
<br />
$ setfacl -m u:gdm:r /usr/bin/pulseaudio<br />
<br />
==== UUIDs has unsupported type ====<br />
<br />
During pairing you might see this output in ''bluetoothctl'':<br />
<br />
[CHG] Device 00:1D:43:6D:03:26 UUIDs has unsupported type<br />
<br />
This message is a very common one and can be ignored.<br />
<br />
== Legacy method: ALSA-BTSCO ==<br />
<br />
{{Out of date|Instructions rely on [[bluez4]].}}<br />
<br />
It is much easier to set up your bluetooth headset today, with bluez >= 3.16. You may want to try the out-of-box python script in [http://fosswire.com/2008/01/11/a2dp-stereo-linux/ this blog] (you need edit the script to work with gconftool-2). There is also a piece of equivalent bash script [http://lymanrb.blogspot.com/2008/05/linux.html here].<br />
<br />
You need your headset's bdaddr. It is of the form ''12:34:56:78:9A:BC''. Either find it in the documentation of your headset, on the headset itself or with the '''hcitool scan''' command.<br />
<br />
Install {{AUR|btsco}}{{Broken package link|{{aur-mirror|btsco}}}}.<br />
<br />
To load the kernel module, type:<br />
# modprobe snd-bt-sco<br />
There will now be an extra audio device. Use {{ic|alsamixer -cN}} (where N is most likely 1) to set the volume. You can access the device with any alsa-capable application by choosing the device ''BT headset'', or with any OSS application by using {{ic|/dev/dspN}} as the audio device.<br />
<br />
But to actually get any sound, you have to connect your headset to the computer first.<br />
<br />
=== Connecting the headset ===<br />
<br />
If you connect your headset for the first time, read the section about pairing first. To connect to your headset to the computer, use the command<br />
$ btsco -f <bdaddr><br />
for example<br />
$ btsco -f 12:34:56:78:9A:BC<br />
<br />
==== Pairing the headset with your computer ====<br />
<br />
The first time you connect the headset, you have to pair it with the computer. To do this, you need your headset's PIN. Depending on your headset you may have to reset the headset and repeat the pairing everytime you used the headset with another bluetooth device.<br />
<br />
There are two ways to pair your headset with the computer:<br />
<br />
===== Using ''bluez-gnome'' =====<br />
<br />
{{Out of date|Package does no longer exist in the repositories.}}<br />
<br />
Install the ''bluez-gnome'' package from the community repository. Then start the '''bt-applet''' program. Once you try to connect to the headset, a window will open and ask for the PIN.<br />
<br />
===== Using ''passkey-agent'' =====<br />
<br />
Before connecting to the headset, enter the command<br />
$ passkey-agent --default <pin><br />
where ''<pin>'' is your headset's PIN. Then try to connect to the headset.<br />
<br />
=== Headset and ALSA Devices ===<br />
<br />
1. First if you have not already, [[pacman|install]] {{Pkg|bluez}} from the [[official repositories]].<br />
<br />
2. Scan for your device<br />
$ hcitool (-i <optional hci#>***) scan<br />
<br />
3. Pair your headset with your device:<br />
$ bluez-simple-agent (optional hci# ***) XX:XX:XX:XX:XX:XX<br />
and put in your pin (0000 or 1234, etc)<br />
<br />
4. Make sure your {{ic|/etc/bluetooth/audio.conf}} allows A2DP Audio Sinks. Place this line just bellow the [General] heading:<br />
Enable=Source,Sink,Media,Socket<br />
<br />
5. Add this to your {{ic|/etc/asound.conf}} file:<br />
#/etc/asound.conf<br />
<br />
pcm.btheadset {<br />
type plug<br />
slave {<br />
pcm {<br />
type bluetooth<br />
device XX:XX:XX:XX:XX:XX <br />
profile "auto"<br />
} <br />
} <br />
hint {<br />
show on<br />
description "BT Headset"<br />
} <br />
}<br />
ctl.btheadset {<br />
type bluetooth<br />
} <br />
<br />
6. Check to see if it has been added to alsa devices<br />
$ aplay -L<br />
<br />
7. Now play with aplay:<br />
$ aplay -D btheadset /path/to/audio/file<br />
<br />
or Mplayer:<br />
$ mplayer -ao alsa:device=btheadset /path/to/audio/or/video/file<br />
<br />
{{Tip|To find hci# for a usb dongle, type in <br />
$ hcitool dev<br />
}}<br />
<br />
=== Headset's multimedia buttons ===<br />
<br />
In order to get your bluetooth headset's multimedia buttons (play, pause, next, previous) working you need to create {{ic|/etc/modules-load.d/uinput.conf}} containing {{ic|uinput}}.<br />
<br />
== Legacy method: PulseAudio ==<br />
<br />
{{Out of date|Instructions rely on [[bluez4]] (references to {{ic|/etc/bluetooth/audio.conf}} and ''bluez-simple-agent'').}}<br />
<br />
This one is much easier and more elegant. PulseAudio will seamlessly switch between output devices when the headset is turned on. If you have ALSA as the sound server, you need the following packages installed:<br />
{{Pkg|pulseaudio}} and {{Pkg|pulseaudio-alsa}}.<br />
<br />
Now, to configure the audio output to use bluetooth, just install {{Pkg|pavucontrol}} and run it to configure the audio output:<br />
$ pavucontrol<br />
Make sure to take a look at the [[PulseAudio]] wiki entry for setting up PulseAudio, especially if you are running KDE.<br />
<br />
=== Troubleshooting ===<br />
<br />
==== Audio sink fails ====<br />
<br />
Bluetooth headset is connected, but ALSA/PulseAudio fails to pick up the connected device. You will get "Audio sink fails".<br />
According to [http://wiki.gentoo.org/wiki/Bluetooth_Headset gentoo wiki], you have to verify than in {{ic|/etc/bluetooth/audio.conf}} there is {{ic|<nowiki>Enable=Socket</nowiki>}} under the {{ic|[General]}} section heading.<br />
<br />
Just do a {{ic|# systemctl restart bluetooth}} to apply it.<br />
<br />
==== Page timeout issue ====<br />
<br />
If you receive this error whilst trying to pair your headset with your system using bluez-simple-agent, then you can try to restart your system and use the graphical bluez applet of your desktop environment.<br />
<br />
== Legacy documentation: ALSA, bluez5 and PulseAudio method ==<br />
<br />
{{Accuracy|Describes two different methods, see the [[Talk:Bluetooth headset#bluez5 method: overcomplicated instructions|talk page]] for details.|Talk:Bluetooth_Headset#bluez5_method:_overcomplicated_instructions}}<br />
<br />
[[ALSA]], [[bluez|bluez5]], and [[PulseAudio]] work together to allow a wireless [[Bluetooth]] headset to play audio. The following method works with a Lenovo T61p laptop and SoundBot SB220 wireless bluetooth headset. The required software stack is extensive and failure to include all components can produce errors which are difficult to understand. The following list of software packages might not be the minimum required set and needs to be examined more closely.<br />
<br />
Bluez5 has a regression causing HSP/HFP Telephone profile to not be available. This regression is documented in the [http://www.freedesktop.org/wiki/Software/PulseAudio/Notes/5.0/ draft release notes for Pulseaudio 5.0] which say (in "Notes for packagers"): "PulseAudio now supports BlueZ 5, but only the A2DP profile. BlueZ 4 is still the only way to make HSP/HFP work." ([https://fedoraproject.org/wiki/Common_F20_bugs#bluez5-profile from here])<br />
<br />
=== Install Software Packages ===<br />
<br />
The core software components are [[ALSA]], Bluez5, [[PulseAudio]]. However there are additional libraries which are required. As well as a player which can play audio files. The following section lists the software packages installed in order to connect the headset and play audio over the headset.<br />
<br />
==== Install ALSA and associated libraries ====<br />
<br />
[[ALSA]] works with the linux kernel to provide audio services to user mode software. The following packages are used with the [[Bluetooth]] headset: {{Pkg|alsa-utils}}, {{Pkg|alsa-plugins}}, {{Pkg|alsa-tools}}.<br />
<br />
==== Install Bluez5 ====<br />
<br />
Bluez5 is the latest [[Bluetooth]] stack. It is required for [[PulseAudio]] to interface with wireless headsets. Required packages: {{Pkg|bluez}}, {{Pkg|bluez-utils}}, {{Pkg|bluez-libs}}.<br />
<br />
==== Install Audacious ====<br />
<br />
[[Audacious]] is a program which plays audio files. It can work directly with [[ALSA]] or with [[PulseAudio]]. Required packages: {{Pkg|audacious}}, {{Pkg|audacious-plugins}}.<br />
<br />
=== Procedure ===<br />
<br />
Once the required packages are installed, use this procedure to play audio with a bluetooth headset. The high level overview of the procedure is to pair the headset, connect the headset, configure the player and pulse audio controller and then play audio.<br />
<br />
Start the bluetooth service as root:<br />
# systemctl start bluetooth<br />
<br />
Verify Bluetooth is started<br />
# systemctl status bluetooth<br />
bluetooth.service - Bluetooth service<br />
Loaded: loaded (/usr/lib/systemd/system/bluetooth.service; disabled)<br />
Active: active (running) since Sat 2013-12-07 12:31:14 PST; 12s ago<br />
Docs: man:bluetoothd(8)<br />
Main PID: 3136 (bluetoothd)<br />
Status: "Running"<br />
CGroup: /system.slice/bluetooth.service<br />
└─3136 /usr/lib/bluetooth/bluetoothd<br />
<br />
Dec 07 12:31:14 t61p systemd[1]: Starting Bluetooth service...<br />
Dec 07 12:31:14 t61p bluetoothd[3136]: Bluetooth daemon 5.11<br />
Dec 07 12:31:14 t61p systemd[1]: Started Bluetooth service.<br />
Dec 07 12:31:14 t61p bluetoothd[3136]: Starting SDP server<br />
Dec 07 12:31:14 t61p bluetoothd[3136]: Bluetooth management interface 1.3 i...ed<br />
Hint: Some lines were ellipsized, use -l to show in full.<br />
<br />
Start the PulseAudio daemon. This must be done after X windows is started and as a normal user.<br />
$ pulseaudio -D<br />
<br />
Verify the PulseAudio daemon is running.<br />
$ pulseaudio --check -v<br />
I: [pulseaudio] main.c: Daemon running as PID 3186<br />
<br />
Start up bluetoothctl as root and pair and connect your headset. As a regular user, bluetoothctl will pair but not connect. Perhaps this is related to the config file (shown below) which is setup for what appears to be the root user.<br />
Note: the procedure shown below is for an initial pair and connect of the headphone. If the headset is already paired, then the procedure below can be shortened to: power on, agent on, default-agent, connect <mac address>. The mac address can be seen from the devices command output.<br />
<br />
$ bluetoothctl <br />
[NEW] Controller 00:1E:4C:F4:98:5B t61p-0 [default]<br />
[NEW] Device 00:1A:7D:12:36:B9 SoundBot SB220<br />
[bluetooth]# show<br />
Controller 00:1E:4C:F4:98:5B<br />
Name: t61p<br />
Alias: t61p-0<br />
Class: 0x000000<br />
Powered: no<br />
Discoverable: no<br />
Pairable: yes<br />
UUID: PnP Information (00001200-0000-1000-8000-00805f9b34fb)<br />
UUID: Generic Access Profile (00001800-0000-1000-8000-00805f9b34fb)<br />
UUID: Generic Attribute Profile (00001801-0000-1000-8000-00805f9b34fb)<br />
UUID: A/V Remote Control (0000110e-0000-1000-8000-00805f9b34fb)<br />
UUID: A/V Remote Control Target (0000110c-0000-1000-8000-00805f9b34fb)<br />
UUID: Audio Source (0000110a-0000-1000-8000-00805f9b34fb)<br />
UUID: Audio Sink (0000110b-0000-1000-8000-00805f9b34fb)<br />
Modalias: usb:v1D6Bp0246d050B<br />
Discovering: no<br />
[bluetooth]# power on<br />
[CHG] Controller 00:1E:4C:F4:98:5B Class: 0x0c010c<br />
Changing power on succeeded<br />
[CHG] Controller 00:1E:4C:F4:98:5B Powered: yes<br />
[bluetooth]# agent on<br />
Agent registered<br />
[bluetooth]# default-agent<br />
Default agent request successful<br />
<br />
<power on your headset in pairing mode. Eventually you will see what appears to be a mac address.><br />
[bluetooth]# scan on<br />
Discovery started<br />
[CHG] Controller 00:1E:4C:F4:98:5B Discovering: yes<br />
[CHG] Device 00:1A:7D:12:36:B9 RSSI: -61<br />
[bluetooth]# pair 00:1A:7D:12:36:B9<br />
Attempting to pair with 00:1A:7D:12:36:B9<br />
[CHG] Device 00:1A:7D:12:36:B9 Connected: yes<br />
[CHG] Device 00:1A:7D:12:36:B9 UUIDs has unsupported type<br />
[CHG] Device 00:1A:7D:12:36:B9 Paired: yes<br />
Pairing successful<br />
[bluetooth]# connect 00:1A:7D:12:36:B9<br />
[CHG] Device 00:1A:7D:12:36:B9 Connected: yes<br />
Connection successful<br />
[bluetooth]# info 00:1A:7D:12:36:B9<br />
Device 00:1A:7D:12:36:B9<br />
Name: SoundBot SB220<br />
Alias: SoundBot SB220<br />
Class: 0x240404<br />
Icon: audio-card<br />
Paired: yes<br />
Trusted: no<br />
Blocked: no<br />
Connected: yes<br />
LegacyPairing: yes<br />
UUID: Headset (00001108-0000-1000-8000-00805f9b34fb)<br />
UUID: Audio Sink (0000110b-0000-1000-8000-00805f9b34fb)<br />
UUID: A/V Remote Control Target (0000110c-0000-1000-8000-00805f9b34fb)<br />
UUID: A/V Remote Control (0000110e-0000-1000-8000-00805f9b34fb)<br />
UUID: Handsfree (0000111e-0000-1000-8000-00805f9b34fb)<br />
<br />
Start up alsamixer, for simplicity un-mute all your outputs. Oddly enough some can be muted though. The ones I had muted during playback were:<br />
* Headphones<br />
* SPIDF<br />
<br />
Start up audacious. Use the menu to select PulseAudio as your output. Somewhere I read that bluez5 requires pulseaudio-git and this jives with my experience.<br />
<br />
Start up pavucontrol in a terminal. In the Outputs tab select the bluetooth headset.<br />
<br />
[http://netskink.blogspot.com/2013/12/pulseaudio-pavucontrol-and-audacious.html screenshot of application settings]<br />
<br />
==== Miscellaneous configuration files ====<br />
<br />
For reference, these settings were also done.<br />
<br />
===== ALSA /etc/asound.conf =====<br />
<br />
The settings shown at the top of this page was used, but the additional modification for Intel laptop sound cards.<br />
<br />
{{bc|<nowiki><br />
pcm.btheadset {<br />
type plug<br />
slave {<br />
pcm {<br />
type bluetooth<br />
device 00:1A:7D:12:36:B9<br />
profile "auto"<br />
}<br />
}<br />
hint {<br />
show on<br />
description "BT Headset"<br />
}<br />
}<br />
ctl.btheadset {<br />
type bluetooth<br />
}<br />
options snd-hda-intel model=laptop<br />
</nowiki>}}<br />
<br />
===== /etc/dbus-1/system.d/bluetooth.conf =====<br />
<br />
The settings here seem to be enabled for root only. See the policy user="root" section. However, if a regular user is specified here, the system fails to start. Someone with more knowledge could explain why.<br />
<br />
{{hc|/etc/dbus-1/system.d/bluetooth.conf|<nowiki><br />
<!-- This configuration file specifies the required security policies for Bluetooth core daemon to work. --><br />
<br />
<!DOCTYPE busconfig PUBLIC "-//freedesktop//DTD D-BUS Bus Configuration 1.0//EN"<br />
"http://www.freedesktop.org/standards/dbus/1.0/busconfig.dtd"><br />
<busconfig><br />
<br />
<!-- ../system.conf have denied everything, so we just punch some holes --><br />
<br />
<policy user="root"><br />
<allow own="org.bluez"/><br />
<allow send_destination="org.bluez"/><br />
<allow send_interface="org.bluez.Agent1"/><br />
<allow send_interface="org.bluez.MediaEndpoint1"/><br />
<allow send_interface="org.bluez.MediaPlayer1"/><br />
<allow send_interface="org.bluez.ThermometerWatcher1"/><br />
<allow send_interface="org.bluez.AlertAgent1"/><br />
<allow send_interface="org.bluez.Profile1"/><br />
<allow send_interface="org.bluez.HeartRateWatcher1"/><br />
<allow send_interface="org.bluez.CyclingSpeedWatcher1"/><br />
</policy><br />
<br />
<policy at_console="true"><br />
<allow send_destination="org.bluez"/><br />
</policy><br />
<br />
<!-- allow users of lp group (printing subsystem) to communicate with bluetoothd --><br />
<policy group="lp"><br />
<allow send_destination="org.bluez"/><br />
</policy><br />
<br />
<policy context="default"><br />
<deny send_destination="org.bluez"/><br />
</policy><br />
<br />
</busconfig><br />
</nowiki>}}<br />
<br />
===== Tested applications =====<br />
<br />
As noted above this will work easily with audacious. YouTube videos with Chromium and Flash Player will work on some videos. If the video has ads it will not work, but if the video does not have ads it will work. Just make sure that after audacious is working with Bluetooth headset, start Chromium, and navigate to YouTube. Find a video without leading ads, and it should play the audio. If the settings icon has the a menu with two drop-down combo boxes for Speed and Quality it will play.<br />
<br />
== Switch between HSV and A2DP setting ==<br />
<br />
This can easily be achieved by the following command where 2 needs to be changed with the correct device number.<br />
<br />
pacmd set-card-profile 2 a2dp_sink<br />
<br />
=== A2DP not working with PulseAudio ===<br />
<br />
==== Socket interface problem ====<br />
<br />
If PulseAudio fails when changing the profile to A2DP with bluez 4.1+ and PulseAudio 3.0+, you can try disabling the Socket interface from {{ic|/etc/bluetooth/audio.conf}} by removing the line {{ic|1=Enable=Socket}} and adding line {{ic|1=Disable=Socket}}.<br />
<br />
==== Gnome with GDM ====<br />
{{Note|Below was tested with Gnome 3.16.2 and PulseAudio 7.0}}<br />
<br />
If PulseAudio fails when changing the profile to A2DP while using GNOME with GDM, you need to prevent GDM from starting its own instance of PulseAudio. Apply the same fix as shown in [[#Connecting works, but I cannot play sound]].<br />
<br />
{{Note|1=Discussion about this problem can be found [https://bbs.archlinux.org/viewtopic.php?id=194006 here] and [https://bbs.archlinux.org/viewtopic.php?id=196689 here]}}<br />
<br />
== Tested headsets ==<br />
<br />
{| class="wikitable"<br />
! Model<br />
! Version<br />
! Comments<br />
! Compatible<br />
|-<br />
| '''Philips SHB9150'''<br />
| bluez5, pulseaudio 5<br />
| Pause and resume does not work. With at least mpv and Banshee hitting the pause button stops audio output but does not pause the player.<br />
| {{R|Limited}}<br />
|-<br />
| '''Philips SHB9100'''<br />
| <br />
| Pause and resume is flaky. See [https://bbs.archlinux.org/viewtopic.php?pid=1315428#p1315428] for the underlying issue and a temporary solution to improve audio quality.<br />
| {{R|Limited}}<br />
|-<br />
| '''Philips SHB7000'''<br />
| <br />
| Pause and resume is flaky.<br />
| {{R|Limited}}<br />
|-<br />
| '''Philips SHB7100'''<br />
| bluez 5.32, pulseaudio 6.0<br />
| Next/previous buttons work. Pause and resume is flaky (sometimes works in VLC, not at all in Audacious). Tested only A2DP and Handsfree audio out, built-in mic was broken.<br />
| {{R|Limited}}<br />
|-<br />
| '''Philips SHB7150'''<br />
| bluez 5.32, pulseaudio 6.0<br />
| Next/previous buttons work. Pause and resume work in VLC. Tested only A2DP profile.<br />
| {{Yes}}<br />
|-<br />
| '''Philips SHB5500BK/00'''<br />
| bluez 5.28, PulseAudio 6.0<br />
| Pause and resume is not working.<br />
| {{R|Limited}}<br />
|-<br />
| '''Parrot Zik'''<br />
| <br />
| Firmware 1.04. The microphone is detected, but does not work. Sometimes it lags (but does not stutter); usually this is not noticeable unless playing games, in which case you may switch to a wired connection.<br />
| {{R|Limited}}<br />
|-<br />
| '''Sony DR-BT50'''<br />
| bluez{4,5}<br />
| Works for a2dp, see [http://vlsd.blogspot.com/2013/11/bluetooth-headphones-and-arch-linux.html]). Adapter: D-Link DBT-120 USB dongle.<br />
| {{Yes}}<br />
|-<br />
| '''Sony SBH50'''<br />
| bluez5<br />
| Works for a2dp, Adapter: Broadcom Bluetooth 2.1 Device (Vendor=0a5c ProdID=219b Rev=03.43). Requires the {{ic|btusb}} [[modprobe|module]].<br />
| {{Yes}}<br />
|-<br />
| '''Sony MDR-XB950BT'''<br />
| pulseaudio<br />
| Tested a2dp. Adapter: Grand-X BT40G. Doesn't auto-connect, need to connect manually. Other functionality works fine.<br />
| {{R|Limited}}<br />
|-<br />
| '''Sony MUC-M1BT1'''<br />
| bluez5, {{AUR|pulseaudio-git}}<br />
| Both A2DP & HSP/HFP work fine.<br />
| {{Yes}}<br />
|-<br />
| '''SoundBot SB220'''<br />
| bluez5, {{AUR|pulseaudio-git}}<br />
| <br />
| {{Yes}}<br />
|-<br />
| '''Auna Air 300'''<br />
| bluez5, pulseaudio-git<br />
| For some reason, a few restarts were required, and eventually it just started working.<br />
| {{R|Limited}}<br />
|-<br />
| '''Sennheiser MM 400-X'''<br />
| bluez5, pulseaudio 4.0-6<br />
| <br />
| {{Yes}}<br />
|-<br />
| '''Sennheiser MM 550-X Travel'''<br />
| bluez 5.27-1, pulseaudio 5.0-1<br />
| Next/Previous buttons work out-of-the-box, Play/Pause does not<br />
| {{Yes}}<br />
|-<br />
| '''Audionic BlueBeats (B-777)'''<br />
| bluez5, pulseaudio 4.0-6<br />
| <br />
| {{Yes}}<br />
|-<br />
| '''Logitech Wireless Headset'''<br />
| bluez 5.14, pulseaudio-git<br />
| part number PN 981-000381, advertised for use with iPad<br />
| {{Yes}}<br />
|-<br />
| '''HMDX Jam Classic Bluetooth'''<br />
| bluez, pulseaudio-git<br />
| <br />
| {{Yes}}<br />
|-<br />
| '''PT-810'''<br />
| bluez 5.14, pulseaudio-git<br />
| Generic USB-Powered Bluetooth Audio Receiver with 3.5mm headset jack and a2dp profile. Widely available as "USB Bluetooth Receiver." IDs as PT-810.<br />
| {{Yes}}<br />
|-<br />
| '''Philips SHB4000WT'''<br />
| bluez5<br />
| A2DP works, HDP distorted.<br />
| {{Yes}}<br />
|-<br />
| '''Philips AEA2000/12'''<br />
| bluez5<br />
| <br />
| {{Yes}}<br />
|-<br />
| '''Nokia BH-104'''<br />
| bluez4<br />
| <br />
| {{Yes}}<br />
|-<br />
| '''Creative AirwaveHD'''<br />
| bluez 5.23<br />
| Bluetooth adapter Atheros Communications usb: 0cf3:0036<br />
| {{Yes}}<br />
|-<br />
| '''Creative HITZ WP380'''<br />
| bluez 5.27, pulseaudio 5.0-1<br />
| A2DP Profile only. Buttons work (Play, Pause, Prev, Next). Volume buttons are hardware-only. Auto-connect works but you should include the bluetooth module in "pulseaudio" to switch to it automatically. Clear HD Music Audio (This device support APTx codec but it isn't supported in linux yet). You may have some latency problems which needs pulseaudio restart.<br />
| {{Yes}}<br />
|-<br />
| '''deleyCON Bluetooth Headset'''<br />
| bluez 5.23<br />
| Adapter: CSL - USB nano Bluetooth-Adapter V4.0. Tested a2dp profile. Untested microphone. Does not auto-connect (even when paired and trusted), must connect manually. Play/pause button mutes/unmutes the headphones, not the playback. Playback fwd/bwd buttons do not work (nothing visible with ''xev'').<br />
| {{R|Limited}}<br />
|-<br />
| '''UE BOOM'''<br />
| bluez 5.27, pulseaudio-git 5.99<br />
| Update to latest UE BOOM fw 1.3.58. Sound latency in video solved by configuring pavucontrol. Works with UE BOOM x2.<br />
| {{Yes}}<br />
|-<br />
| '''LG HBS-730'''<br />
| bluez 5.30, pulseaudio 6.0<br />
| Works out of box with A2DP profile.<br />
| {{Yes}}<br />
|-<br />
| '''LG HBS-750'''<br />
| bluez 5.30, pulseaudio-git 6.0<br />
| Works out of box with A2DP profile.<br />
| {{Yes}}<br />
|-<br />
| '''Beats Studio Wireless'''<br />
| bluez 5.28, pulseaudio 6.0<br />
| Works out of box. Not tested multimedia buttons.<br />
| {{Yes}}<br />
|-<br />
| '''AKG Y45BT'''<br />
| bluez 5.30, pulseaudio 6.0<br />
| Pause and resume does not work. Needs {{ic|1=Enable=Socket}} in {{ic|/etc/bluetooth/audio.conf}} and {{ic|load-module module-bluetooth-discover}} in {{ic|/etc/pulse/default.pa}}.<br />
| {{Yes}}<br />
|-<br />
| '''Bluedio Turbine/Turbine 2+'''<br />
| bluez 5.3, pulseaudio 6.0<br />
| HSP/HFP work fine, A2DP work fine.<br />
| {{Yes}}<br />
|-<br />
| '''Sony SBH20'''<br />
| bluez 5.30, pulseaudio 6.0<br />
| Works out of box with A2DP profile.<br />
| {{Yes}}<br />
|-<br />
| '''Nokia BH-111'''<br />
| bluez 5.30, pulseaudio 6.0<br />
| Works with both HSP/HFP and A2DP. Buttons work in certain apps.<br />
| {{Yes}}<br />
|-<br />
| '''Sony MDR-ZX330BT'''<br />
| bluez 5.31, pulseaudio 6.0<br />
| Works out of box (HSP/HFP and A2DP). Buttons work in certain apps.<br />
| {{Yes}}<br />
|-<br />
| '''Samsung Level Link'''<br />
| bluez 5.33, pulseaudio 6.0<br />
| Works out of box (HSP/HFP and A2DP). Buttons work in certain apps.<br />
| {{Yes}}<br />
|-<br />
| '''Bernafon Soundgate 3'''<br />
| bluez 5.34-2, pulseaudio 7.0-1<br />
| Works (A2DP) after installing blueman-git 2.0.r53.g2a812a8-1. <br />
| {{Yes}}<br />
|}<br />
<br />
== See also ==<br />
<br />
<br />
Using the same device on Windows and Linux without pairing the device over and over again<br />
<br />
* [http://ubuntuforums.org/showthread.php?p=9363229#post9363229 Dual booting with a Bluetooth keyboard]</div>Justin8https://wiki.archlinux.org/index.php?title=Mac&diff=391866Mac2015-08-20T10:03:25Z<p>Justin8: Updated some MacbookPro11,4 details</p>
<hr />
<div>[[Category:Apple]]<br />
[[de:ArchLinux auf einem MacBook]]<br />
[[fr:MacBook]]<br />
[[it:MacBook]]<br />
[[ja:MacBook]]<br />
[[zh-CN:MacBook]]<br />
{{Related articles start}}<br />
{{Related|Installation guide}}<br />
{{Related|Beginners' guide}}<br />
{{Related|General recommendations}}<br />
{{Related|MacBook4,2 (late 2008)}}<br />
{{Related|MacBook5,2 (early-mid 2009)}}<br />
{{Related|MacBookPro7,1}}<br />
{{Related|MacBookPro8,1/8,2/8,3 (2011)}}<br />
{{Related|MacBookPro9,2 (Mid-2012)}}<br />
{{Related|MacBookPro10,x}}<br />
{{Related|MacBookPro11,x}}<br />
{{Related|iMac Aluminum}}<br />
{{Related|Apple Fusion Drive}}<br />
{{Related articles end}}<br />
<br />
Installing Arch Linux on a MacBook (Air/Pro) is quite similar to installing it on any other computer. However, due to the specific hardware configuration on a MacBook, there are a few deviations and special considerations which warrant a separate guide. For more background information, please see the [[Installation guide]], [[Beginners' guide]] and [[UEFI]]. This guide contains installation-instructions that can be used on any Apple computer whose hardware is supported by the Linux kernel. Please see 'related' pages (on the top right of this page) for model-specific tips and troubleshooting.<br />
<br />
== Overview ==<br />
<br />
Specifically, the procedure for installing Arch Linux on a MacBook is:<br />
<br />
# '''[[#Updating OS X|Update OS X]]''': It always helps to start from a clean, backed up, and up-to-date install of OS X.<br />
# '''[[#Partitions|Partition]]''': Resizing or deleting the OS X partition to create partitions for Arch Linux.<br />
# '''[[#Setup bootloader|Setup bootloader]]''': Making sure that the new partition is bootable.<br />
# '''[[#Installation|Install Arch Linux]]''': Actually installing Arch Linux.<br />
# '''[[#Post-installation|Post-installation]]''': MacBook-specific configuration.<br />
<br />
== Updating OS X ==<br />
<br />
{{Note|If you uninstalled OS X or want to reinstall it, do that first. [http://www.apple.com Apple] has great instructions.}}<br />
In OS X, '''install every update''' (in the App Store). '''Reboot''' your computer, and then '''update again''' to make sure that you installed everything.<br />
<br />
* If you plan to remove OS X completely, make backups of these files, which you will need in Linux for adjusting the [[#Color Profile|color profile]]:<br />
{{Note|It is advisable to keep OS X, because the firmware can only be updated using OS X.}}<br />
<br />
/Library/ColorSync/Profiles/Displays/<FILES HERE><br />
<br />
You will also need the following file for iSight functionality:<br />
{{Note|This does not apply to devices using the FaceTime HD webcam (MacBookAir5,1/5,2/6,1/6,2), because the webcam is not yet supported by the Linux kernel.}}<br />
<br />
/System/Library/Extensions/IOUSBFamily.kext/Contents/PlugIns/AppleUSBVideoSupport.kext/Contents/MacOS/AppleUSBVideoSupport<br />
<br />
* If you plan to dual boot and want to keep OS X, continue to [[#Partitions]]<br />
<br />
== Partitions ==<br />
<br />
The next step in the installation is to re-partition the hard drive. If OS X was installed using the typical procedure, then your drive should have a GPT format and the following partitions:<br />
<br />
* '''EFI''': a 200 MB partition at the beginning of the disk. It is often read as '''msdos''' or '''FAT''' by some partitioning tools and usually labeled ''#1''.<br />
* '''OS X''': the ''(HFS+)'' partition that should take up all of the remaining disk space. Usually labeled ''#2''.<br />
* '''Recovery''': A recovery partition (only for OS X 10.7+).<br />
<br />
{{Note|If your hardware includes a Fusion Drive you might have a look at the [[Apple Fusion Drive]] page.}}<br />
<br />
How to partition depends on how many operating systems you want install. The following options will be explained:<br />
<br />
* Single boot: [[#Arch Linux only]]<br />
* Dual boot: [[#OS X with Arch Linux]] ''(recommended so you can still return to OS X when needed)''<br />
* Triple boot: [[#OS X, Windows XP, and Arch Linux triple boot]]<br />
<br />
----<br />
<br />
=== Arch Linux only ===<br />
<br />
This situation is the easiest to deal with. Partitioning is the same as any other hardware that Arch Linux can be installed on.<br />
<br />
The only special consideration is the MacBook firmware boot sound. To ensure that this sound is off: '''mute''' the volume in OS X before continuing further. The MacBook firmware relies on the value in OS X, if available. Note that if you choose to get rid of the OS X partition, there is no easy way to update your machine's firmware unless you use an external drive to boot OS X.<br />
You can boot in EFI mode (recommended) or bios-compatibility mode, if in doubt choose EFI.<br />
<br />
==== Option 1: EFI (recommended) ====<br />
<br />
* Run '''cgdisk''' ({{Pkg|gptfdisk}} package).<br />
* Create the necessary partitions.<br />
<br />
{{Note|<br />
* The swap partition is optional, on machines with a RAM of size 4GB or more, good performance can be expected without a swap partition. Also, a '''swap file''' can be created later, see [[Swap#Swap file|Swap file]]. You'll need a swap partition/file if you expect to [[Suspend and hibernate|Hibernate]] your machine in future.<br />
* For more information on partitioning, see [[Beginners' guide#Partitioning hard disks: General information|Partitioning hard disks: General information]].<br />
* As of Aug 2014 {{pkg|refind-efi}} has a bug that does not allow to run {{ic|refind-install}} if EFI partition is mounted to {{ic|/boot}}. Other bootloaders (like {{pkg|gummiboot}}) have no such problem and thus there is no need to split {{ic|/boot/efi}} from {{ic|/boot}}.<br />
}}<br />
Simple example (no LVM, crypto):<br />
partition mountpoint size type label<br />
/dev/sda1 /boot/efi 200MiB vfat EFI<br />
/dev/sda2 /boot 100MiB ext2 boot<br />
/dev/sda3 - adjust swap swap<br />
/dev/sda4 / 10GiB ext4 root<br />
/dev/sda5 /home remain. ext4 home<br />
<br />
* Done, you can continue to [[#Installation]]<br />
<br />
==== Option 2: BIOS-compatibility ====<br />
<br />
* Boot installation medium and switch to a free tty.<br />
* Run '''parted'''. The simplest way is to change the partition table to '''msdos''' and then partition as normal. GRUB is compatible with GPT.<br />
<br />
* Create the necessary partitions.<br />
<br />
* Done, you can continue to [[#Installation]]<br />
<br />
=== OS X with Arch Linux ===<br />
<br />
The easiest way to partition your hard drive, so that OS X and Arch Linux will co-exist, is to use partitioning tools in OS X and then finish with Arch Linux tools.<br />
<br />
{{Warning|It is highly recommended that this only be attempted after a clean install of OS X. Using these methods on a pre-existing system may have undesired results.}}<br />
<br />
'''Procedure''':<br />
* In OS X, run '''Disk Utility.app''' (located in {{ic|/Applications/Utilities}})<br />
<br />
* Select the drive to be partitioned in the left-hand column (not the partitions!). Click on the '''partition''' tab on the right.<br />
<br />
* Select the volume to be resized in the '''Volume scheme'''.<br />
<br />
* Decide how much space you wish to have for your OS X partition, and how much for Arch Linux. Remember that a typical installation of OS X requires around 15-20 GiB, depending on the number of software applications and files.<br />
<br />
* Finally, in the size box, type the smaller size to which you want to resize your OS X partition, and click '''Apply'''. This will create a new partition out of the empty space. You will delete this partition later.<br />
<br />
{{Note|If you wish to have a shared partition between OS X and Arch Linux, then additional steps will need to happen here. Please see [[#HFS partition sharing]].}}<br />
<br />
* If the above completed successfully, then you can continue. If not, then you may need to fix your partitions from within OS X first.<br />
{{Note|If you have any problems, try using the [http://gparted.sourceforge.net/ gparted] live CD (i.e. ''instead'' of using Disk Utility and/or cgdisk). It is capable of shrinking the OS X partition and creating Linux partitions ready for installation.}}<br />
<br />
* Boot the Arch installation [[Beginners'_guide#Prepare_the_latest_installation_medium|LiveCD]] or [[USB flash installation media|LiveUSB]] by holding down the alt key '''during boot'''. Follow '''one''' of the procedures below according to your choice of boot method.<br />
<br />
==== Option 1: EFI ====<br />
<br />
* Run '''cgdisk'''<br />
<br />
* Delete the partition you made in Disk Utility.app and create the necessary partitions for Arch Linux. OS X likes to see a 128 MiB gap after partitions, so when you create the first partition after the last OS X-partition, type in '''+128M''' when cgdisk asks for the first sector for the partition. A simple example (no LVM, crypto):<br />
{{Note|<br />
* The swap partition is optional on machines with 4GB of RAM or more. A '''[[Swap#Swap file|swap file]]''' can be created later.<br />
* The easiest dual-boot option is to install rEFInd from inside OS X, to its root directory (default for install.sh). Following that, copy the driver folder from the installation tarball into the new rEFInd location, and uncomment the lines "scan_all_linux_kernels" and "also_scan_dirs" options in refind.conf. Configuration of boot options can then be done from a refind_linux.conf in Arch's /boot directory.<br />
* If you want to be able to boot GRUB from the Apple boot loader, you can create a small hfs+ partition (for convenience, use OS X to format it in Disk Utility.app afterwards). Follow the GRUB EFI install procedure, and mount your {{ic|/boot/efi}} directory to the hfs+ partition you created. Finally, finish up again in OS X by blessing the partition. This will set GRUB as the default boot option (holding alt at startup goes to the mac boot options screen still. See http://mjg59.dreamwidth.org/7468.html)<br />
* OS X's EFI partition can be shared with Arch Linux, making the creation of an additional EFI partition dedicated to arch completely optional<br />
}}<br />
{{Note|For more information on partitioning, see [[Partitioning]]}}<br />
partition mountpoint size type label<br />
/dev/sda1 /boot/efi 200MiB vfat EFI<br />
/dev/sda2 - ? hfs+ OS X<br />
/dev/sda3 - ? hfs+ Recovery<br />
/dev/sda4 - 100MiB hfs+ Boot Arch Linux from the Apple boot loader (optional)<br />
/dev/sda5 /boot 100MiB boot boot<br />
/dev/sda6 - ? swap swap (optional)<br />
/dev/sda7 / 10GiB ext4 root<br />
/dev/sda8 /home remaining ext4 home<br />
<br />
* Done, you can continue to [[#Installation]]<br />
<br />
==== Option 2: BIOS-compatibility ====<br />
<br />
* Run '''parted''' as root.<br />
<br />
* Delete the empty space partition and partition the space as you would for any other installation. Note that MBR is limited to 4 primary partitions (including the efi partition). That leaves 2 primary partitions for arch. One strategy is to have a system and home partition, and use a swap file (I have not tried to use logical partitions). Another is to dedicate one partition to a shared partition (see below).<br />
<br />
* Next, create new filesystems on those partitions which need them, especially the partition which will contain /boot. If you are not sure how to do this using {{ic|mkfs.ext2}} (or whatever), run {{ic|/arch/setup}} and work through until you get to Prepare Hard Drive and use the "Manually configure block devices..." option, then exit the installer. This is necessary so that rEFIt will set the right partition type in the MBR in the next step (without an existing filesystem, it seems to ignore the partition type set by parted), without which GRUB will refuse to install to the right partition.<br />
<br />
* At this point you should reboot your computer and have rEFIt fix the partition tables on your hard drive. (If you do not do this, you may have to reinstall GRUB later on in order to have your Mac recognize the Linux partition.) When you are into the rEFIt menu, select '''update partition table''', then press Y. Reboot.<br />
<br />
* Done, you can continue with [[#Installation]].<br />
<br />
=== OS X, Windows XP, and Arch Linux triple boot ===<br />
<br />
This may not work for everyone but it has been successfully tested on a MacBook from late 2009.<br />
<br />
The easiest way to partition your hard drive, so that all these operating systems can co-exist, is to use disk utility in OS X, use the formatter on windows XP, install XP and then finish with Arch Linux tools.<br />
<br />
{{Warning|It is highly recommended that this only be attempted after a clean install of OS X. Using these methods on a pre-existing system may have undesired results. At least back your stuff up with timemachine or clonezilla before you begin.}}<br />
<br />
'''Procedure''':<br />
* In OS X, run '''Disk Utility''' (located in {{ic|/Applications/Utilities}}).<br />
<br />
* Select the drive to be partitioned in the left-hand column (not the partitions!). Click on the '''partition''' tab on the right.<br />
<br />
* Select the volume to be resized in the '''volume scheme.'''<br />
<br />
* Decide how much space you wish to have for your OS X partition, how much for XP, and how much for Arch Linux. Remember that a typical installation of OS X requires around 15-20 GiB, and XP about the same, depending on the number of software applications and files. Something like OS X 200Gb, XP 25Gb, Arch 25Gb should be fine.<br />
<br />
* Put your decisions into action by pressing the + button and adding the new partitions, Label them as you like and make sure that your XP partition is the last one on the disk and is formatted for FAT32. It is probably best to have Arch formatted in HFS format as to not confuse you later, it will be reformatted anyway.<br />
<br />
So in linux terms your partitions will be something like:<br />
<br />
:*sda (disk)<br />
:*sda1 (Mac boot partition - you cannot see this one in OS X)<br />
:*sda2 (OS X install in HFS+)<br />
:*sda3 (Arch install temporarly in HFS)<br />
:*sda4 (XP install in FAT32)<br />
<br />
* Finally, click '''apply'''. This will create a new partition out of the empty space.<br />
<br />
{{Note|Using this method you may not be able to have a shared partition between OS X and Arch Linux, this is because the mac will only allow for 4 active partitions. You will however be able to mount a HFS partition in Arch for one workaround. There are other workarounds possible also.}}<br />
<br />
* If the above completed successfully, you can continue. If not, then you may need to fix your partitions from within OS X first.<br />
<br />
* You will not be needing boot camp this way, the program rEFIt is much more flexible (though not as flexible as GRUB). Download and install rEFIt [[http://refit.sourceforge.net/]]<br />
<br />
* Go into a terminal in OS X and perform the following, this will enable the rEFIt boot manager. <br />
<br />
cd /efi/refit<br />
./enable.sh<br />
<br />
* Reboot to check the rEFIt is working, it should appear on boot. When it comes up go to the rEFIt partition manager and agree to the changes.<br />
<br />
* Put your XP install CD and boot it with rEFIt - You may have to reboot a few times until it is recognized by the boot loader. Install XP and once it is installed use the OS X installation CD to get your drivers running nicely in XP.<br />
** Note: when installing XP make sure you select your XP partition and format it again inside the XP installer. If you do not reformat it will not work.<br />
<br />
* Boot the Arch install CD, log in as root and run {{ic|# /arch/setup}}.<br />
<br />
* Follow the install as normal but note that you will have to tell that arch installer to mount sda3 as the root partition and format it as ext3, there will not be a /boot or swap partition so ignore those warnings.<br />
<br />
* At this point, if you are dual booting, you should reboot your computer and have rEFIt fix the partition tables on your hard drive. (If you do not do this, you may have to reinstall GRUB later on in order to have your Mac recognize the Linux partition.) When you are into the rEFIt menu, select '''update partition table''', then press Y.<br />
# reboot<br />
<br />
* Done! You can continue to [[#Installation]] but make sure you read [[#Booting directly from GRUB]] for the stage "* (for booting with EFI) After the install boot loader stage, exit the installer and install GRUB."<br />
<br />
== Setup bootloader ==<br />
<br />
If you are going for an Arch Linux-only setup, installing the bootloader is no different than on any other machine: Install [[gummiboot]], [[rEFInd]] or other bootloader of your choice.<br />
<br />
If, on the other hand, you are dual/triple booting, then read on.<br />
<br />
{{Out of date | Section that describes bootloader setup for dual/triple boot should be revised and re-structured into more readable way}}<br />
{{Tip|rEFIt is a popular bootloader for EFI-firmware computers (including Macs). It can be installed at any time during the installation. For instructions, please see [[#rEFIt]]. }}<br />
<br />
<br />
=== Installing GRUB to EFI partition directly ===<br />
<br />
* If you would like to use GRUB as your main bootloader and use the "boot while holding the Alt/Option key" method to go back to OS X rather than using alternatives such as rEFIt (http://refit.sourceforge.net/, mentioned previously in [[#BIOS-compatibility]] and [[#OS X, Windows XP, and Arch Linux triple boot]]) then you must install {{Pkg|grub}} to your Mac's '''already-existing''' EFI partition (see below). <br />
<br />
{{Note| These instructions are known to work on a MacBook Pro (Early 2011). Please read the procedure carefully '''as well as the details following it'''.}}<br />
<br />
{{Note| With a new MacBook Pro (Mid 2014), this procedure worked only after installing the<br />
{{Pkg|efibootmgr}} package.}}<br />
<br />
'''Procedure''':<br />
<br />
* Install {{Pkg|grub}}<br />
<br />
* Make a directory named {{ic|efi}} in {{ic|/boot}} <br />
<br />
* Mount the '''already-existing''' EFI partition on your Mac to this {{ic|/boot/efi}} directory<br />
<br />
* Install GRUB to this directory<br />
<br />
* Make a directory named {{ic| locale}} in {{ic| /boot/grub}}<br />
<br />
* Copy {{ic| grub.mo}} from {{ic| /usr/share/locale/en\@quot/LC_MESSAGES/}} to {{ic| /boot/grub/locale}} <br />
<br />
* Generate a configuration for GRUB<br />
<br />
* Done! GRUB will now start on reboot and you can boot into your newly installed Arch Linux.<br />
<br />
* Remember to hold ALT/Option key '''while''' starting your computer if you want to boot back into OS X.<br />
<br />
'''Details (quoted from [[GRUB_EFI_Examples#M5A97]]):'''<br />
<br />
Finish the standard Arch install procedures, making sure that you install {{Pkg|grub}} and partition your boot hard disk as GPT.<br />
<br />
From [[GRUB#Install_to_UEFI_system_partition]]:<br />
<br />
The UEFI system partition will need to be mounted at {{ic|/boot/efi/}} for the GRUB install script to detect it:<br />
<br />
# mkdir -p /boot/efi<br />
# mount -t vfat /dev/sdXY /boot/efi<br />
<br />
Where X is your boot hard disk and Y is the efi partition you created earlier.<br />
<br />
Install GRUB UEFI application to and its modules to {{ic|/boot/grub/x86_64-efi}} using:<br />
<br />
# modprobe dm-mod<br />
# grub-install --target=x86_64-efi --efi-directory=/boot/efi --bootloader-id=arch_grub --recheck --debug<br />
# mkdir -p /boot/grub/locale<br />
# cp /usr/share/locale/en\@quot/LC_MESSAGES/grub.mo /boot/grub/locale/en.mo<br />
<br />
Generate a configuration for GRUB<br />
<br />
# grub-mkconfig -o /boot/grub/grub.cfg<br />
<br />
=== Using blessing ===<br />
<br />
It is possible to boot directly from GRUB in EFI mode without using rEFIt through what is known as "blessing" after placing GRUB on a '''separate partition'''. These instructions are known to work on a MacBook7,1. It is advisable to host GRUB on either a FAT32 or HFS+ partition, but ext2 or ext3 may also work. GRUB's appleloader command does not currently work with the 7,1, but support can be added with the patch available [https://savannah.gnu.org/bugs/index.php?33185 here].<br />
<br />
After the GRUB install is in the desired location, the firmware needs to be instructed to boot from that location. This can be done from either an existing OS X install or an OS X install disk. The following command assumes that the GRUB install is in {{ic|/efi/grub}} on an existing OS X partition:<br />
# bless --folder /efi/grub --file /efi/grub/grub.efi<br />
<br />
=== Compilation ===<br />
<br />
Some models may need EFI_ARCH set to i386.<br />
bzr branch --revision -2 bzr://bzr.savannah.gnu.org/grub/trunk/grub grub<br />
cd grub<br />
./autogen.sh<br />
patch -p1 < appleloader_macbook_7_1.patch<br />
export EFI_ARCH=x86_64<br />
./configure --with-platform=efi --target=${EFI_ARCH} --program-prefix=""<br />
make<br />
cd grub-core<br />
../grub-mkimage -O ${EFI_ARCH}-efi -d . -o grub.efi -p "" part_gpt part_msdos ntfs ntfscomp hfsplus fat ext2 normal chain boot configfile linux multiboot<br />
cp grub.efi *.mod *.lst yourinstalllocation<br />
<br />
== Installation ==<br />
<br />
{{Note|This section is only required if you want to have OS X installed along with Arch Linux. If not, follow the steps in the official install guide, then skip to [[#Post-installation]].}}<br />
<br />
* Boot from the Arch Linux install CD, from the latest [[Archboot]] iso (unofficial), or from a [[USB flash installation media#Using manual formatting|manually created]] bootable USB drive.<br />
{{Note|<br />
* On a MacBookPro7,1, I had an error booting the installation media Version 2012.12.01: "unable to handle kernel NULL pointer dereference at 0000000000000010" during pacpi_set_dmamode. To fix this problem, boot with the option: acpi&#61;off. After chrooting, add MODULES&#61;"ata_generic" into /etc/mkinitcpio.conf and execute mkinitcpio -p linux, see: [[Installation guide#Configure_the_system|Installation Guide, 9 Configure the system]].<br />
* Some MacBook users report strange keyboard output such as long delays and character doubling. To fix this problem, boot with the following options: arch noapic irqpoll acpi&#61;force}}<br />
<br />
* Proceed through the installation as described in the [[Installation guide]] '''except''' in the following areas:<br />
** In the [[Installation guide#Prepare Hard Drive|prepare hard drive]] stage, do only the [[Installation guide#Manually configure block devices, filesystems and mountpoints|set filesystem mountpoints]] step, taking care to assign the correct partitions. Partitions have already been created if you followed [[#Partition]]<br />
** '''(for booting with EFI''') After the [[Installation guide#Install Bootloader|install boot loader]] stage, exit the installer and install [[GRUB]].<br />
** '''(for booting with BIOS-compatibility)''' In the [[Installation guide#Install Bootloader|install boot loader]] stage, edit the menu.lst file and add '''reboot=pci''' to the end of the '''kernel''' lines, for example: {{bc|1=kernel /vmlinuz26 root=/dev/sda5 ro reboot=pci}} This will allow your MacBook to reboot correctly from Arch.<br />
** '''(for booting with BIOS-compatibility)''' Also in the [[Installation guide#Install Bootloader|install boot loader]] stage, install GRUB on whatever partition that {{ic|/boot}} is on. {{Warning|Do not install GRUB onto ''/dev/sda'' !!! Doing so is likely to lead to an unstable post-environment.}}<br />
** In the [[Installation guide#Configure System|configure system]] stage, edit /etc/mkinitcpio.conf and add the '''usbinput''' hook to the '''HOOKS''' line somewhere after the '''autodetect''' hook. This will load the drivers for your keyboard in case you need to use it before Arch boots (e.g. entering a [[LUKS]] password or using the troubleshooting shell).<br />
<br />
* When the install process is complete, reboot your computer.<br />
<br />
* If using optical media, hold down the eject key as your MacBook starts, this should eject the Arch Linux install disk.<br />
<br />
* If dual-booting OS X and Arch Linux, hold down the alt (option) key while the system boots to use the Mac bootloader to select which OS to boot.<br />
<br />
== Post-installation ==<br />
<br />
{{Style|Duplicated information, does not comply with [[Help:Style]].}}<br />
<br />
See [[General recommendations]] for system management directions and post-installation tutorials like setting up a graphical user interface, sound or a touchpad.<br />
<br />
==== Video ====<br />
<br />
Different MacBook models have different graphic cards.<br />
<br />
To see which graphics card you have type:<br />
<br />
$ lspci | grep VGA<br />
<br />
* If it returns a string containing '''intel''' you only need the {{Pkg|xf86-video-intel}} driver. Intel-based MacBooks work out-of-the-box.<br />
<br />
* If it returns '''nVidia''', read [[NVIDIA]].<br />
<br />
* Otherwise if it returns '''ATI''' or '''AMD''', read [[ATI]].<br />
<br />
===== NVIDIA note =====<br />
<br />
{{Tip|MBP 6.2 - With the proprietary [[NVIDIA]] drivers, support for [[NVIDIA#Enabling Pure Video HD (VDPAU/VAAPI)|PureVideo HD]] is available for hardware video decoding. }}<br />
<br />
For MacBooks with NVIDIA graphics, for the backlight to work properly you may need the {{AUR|nvidia-bl}} package.<br />
<br />
{{Tip|<br />
* If backlight control does not work after installing nvidia-bl, you should [[Kernel modules#Blacklisting|blacklist]] apple_bl kernel module.<br />
* Alternatively, you can choose to use the {{AUR|pommed-light}} package. If you do so, you may wish to change the step settings in {{ic|/etc/pommed.conf.mactel}} to something around 5000-10000 depending on how many levels of brightness you desire. The max brightness is around 80000, so take that into account.}}<br />
<br />
===== MacBookPro5,5, NVIDIA and secondary display =====<br />
<br />
As of January 1 2011, the latest NVIDIA drivers (290.10) might not work properly when a secondary display is used (tested with TwinView), NVIDIA's current [http://www.nvnews.net/vbulletin/showthread.php?t=122606 long-live supported] 275xx drivers seem to work fine. Install {{AUR|nvidia-275xx}} and {{AUR|nvidia-utils-275xx}}, and possibly {{AUR|lib32-nvidia-utils-275xx}} if you are on x86_64 system and want 32-bits support.<br />
<br />
MacBookPro5,5 has an NVIDIA 9400m graphics card. This problem might apply to other devices as well.<br />
<br />
==== Touchpad ====<br />
<br />
The touchpad should have basic functionality by default. A true multitouch driver which behaves very similarly to native OS X is included in the {{AUR|xf86-input-multitouch-git}} package. It supports 1, 2 and 3 finger gestures, including differentiation between horizontal and vertical 3 finger swipe. Additional details are available at [http://bitmath.org/code/multitouch/ the driver's project page].<br />
<br />
xf86-input-multitouch-git does not support any sort of configuration without editing the driver's source. Some users are also experiencing issues with false clicks from palm touches. There is now a much more configurable fork available as {{AUR|xf86-input-mtrack-git}}. Configuration options are documented in the [https://github.com/BlueDragonX/xf86-input-mtrack readme].<br />
<br />
The following mtrack options work well on a MacBook7,1:<br />
<br />
Option "Thumbsize" "50"<br />
Option "ScrollDistance" "100"<br />
<br />
Probably you need also to add:<br />
<br />
MatchDevicePath "/dev/input/event10"<br />
<br />
To disable tap-to-click (that is, to press down to click) by default, add the following to your mtrack configuration section<br />
<br />
Option "TapButton1" "0" <br />
Option "TapButton2" "0"<br />
Option "TapButton3" "0"<br />
<br />
'''Natural scrolling:''' To configure natural two finger scrolling similar to [http://www.apple.com/au/osx/what-is/gestures.html#gallery-gestures-scroll OS X], refer to [[Touchpad Synaptics#Natural scrolling]]. If you are using GNOME, it will override these settings - in this case refer to [[GNOME#Natural_scrolling_touchpad]].<br />
<br />
If you are using {{AUR|xf86-input-mtrack-git}}, you can simply swap the scroll up and scroll down buttons (along with the scroll left and scroll right):<br />
{{hc|/etc/X11/xorg.conf.d/10-mtrack.conf|<br />
...<br />
Option "ScrollUpButton" "5"<br />
Option "ScrollDownButton" "4"<br />
Option "ScrollLeftButton" "7"<br />
Option "ScrollRightButton" "6"<br />
...}}<br />
<br />
<br />
'''Special Note About Older Macbook Models (confirmed on MacBook2,1):''' On older Macbook models (pre-multitouch), the touchpad will not function properly until you install the xf86-input-synaptics package. Please see [[Touchpad Synaptics]] for more information on installing and configuring this package.<br />
<br />
'''Note on MacBookPro5,5:''' I found it is much simpler to use the {{Pkg|xf86-input-synaptics}} in Extra. Although it does not have much function as 3 finger swipe, this driver provides faster response. {{Pkg|gpointing-device-settings}} also provides a simple GUI config. Below is a Xorg config file /etc/X11/xorgconfig.d/60-synaptics.conf for reference only.<br />
Section "InputClass"<br />
Identifier "touchpad catchall"<br />
Driver "synaptics"<br />
MatchIsTouchpad "on"<br />
MatchDevicePath "/dev/input/event*"<br />
Option "SHMConfig" "on"<br />
Option "TapButton1" "1"<br />
Option "TapButton2" "3"<br />
Option "TapButton3" "2"<br />
Option "PalmDetect" "on"<br />
Option "VertEdgeScroll" "off"<br />
Option "HorizEdgeScroll" "off"<br />
Option "CornerCoasting" "off"<br />
Option "EdgeMotionUseAlways" "off"<br />
Option "AreaLeftEdge" "10"<br />
Option "AreaRightEdge" "1270"<br />
EndSection<br />
'''For some users, the two-finger right-click may not work correctly and trackpad may also become less responsive after these settings. For me, removing the 'AreaLeftEdge' and 'AreaRightEdge', solved that problem.'''<br />
'''OS X like MultiTouch Gestures''' ''currently broken due to newer synaptic drivers!'' For users looking to add more of OS X's multitouch gestures to Arch, [https://github.com/iberianpig/xSwipe xSwipe] is a highly customisable, light weight perl script, which does just that. Once installed and configured (see xSwipe wiki on Github) I would recommend adding xSwipe as a [[Autostarting|start up item]].<br />
<br />
==== Keyboard ====<br />
<br />
MacBook keyboards work by default. For swaping fn keys with Fx keys see [[Apple Keyboard]].<br />
<br />
To enable it you can map with right application like '''xbindkeys''' or through DE preferences; but another very good way, that we recommend, is to install the {{AUR|pommed}} package.<br />
<br />
Edit the {{ic|/etc/pommed.conf}} according to your hardware on MacBook, building<br />
it from {{ic|/etc/pommed.conf.mac}} or {{ic|/etc/pommed.conf.ppc}} example files.<br />
<br />
Note that you can also run it without a configuration file, the defaults may work for you. Then enable and start pommed [[Systemd]] service.<br />
<br />
systemctl enable pommed<br />
systemctl start pommed<br />
<br />
{{Tip|if you are using Gnome or KDE you can easily configure ''3rd level functionality'', ''multimedia key'', etc. in Keyboard Preferences.}}<br />
<br />
{{Note|See the [[Xorg input hotplugging]] page for other configuration information.}}<br />
<br />
===== Keyboard Backlight =====<br />
<br />
The keyboard backlight is controlled by {{ic|/sys/class/leds/smc::kbd_backlight}}. Write the desired value to {{ic|brightness}} in that directory.<br />
<br />
You may also use {{AUR|kbdlight}} to control keyboard backlight though scripts or by running it via [[sxhkd]]. It has the advantage of allowing keyboard light-level changes without being root.<br />
<br />
====== NVIDIA note ======<br />
<br />
If the brightness does not function correctly through pommed, make sure you have installed the {{AUR|nvidia-bl}} package and insert<br />
<br />
find . -name "*" -exec sed -i 's/mbp_backlight/nvidia_backlight/' '{}' \;<br />
<br />
into the second line of the pommed PKGBUILD build() function and remake the package. From [https://bbs.archlinux.org/viewtopic.php?id=105091 this forum post].<br />
<br />
Another possible solution is to modify the pommed PKGBUILD build():<br />
<br />
find . -name "*" -exec sed -i 's/nvidia_backlight/apple_backlight/' '{}' \;<br />
<br />
If the previous does not work try the following,<br />
<br />
run nvidia-settings, edit the file '/etc/X11/xorg.conf' and add this line into the Device section:<br />
<br />
Option "RegistryDwords" "EnableBrightnessControl=1"<br />
Save and reboot and check backlight buttons work.<br />
More information available at [https://help.ubuntu.com/community/MacBookPro5-5/Precise#LCD Ubuntu MacBookPro5,5]<br />
<br />
=== Wi-Fi ===<br />
<br />
Different MacBook models have different wireless cards.<br />
<br />
You can easily check what card do your MacBook have by:<br />
<br />
# lspci | grep Network<br />
<br />
* If you have an Atheros card, all should work out-of-the-box.<br />
<br />
* If you have a Broadcom card, follow the [[Broadcom BCM4312]] page.<br />
<br />
* 5.0 and 6.0 generation MacBooks may have a BCM43xx, follow the instructions for the broadcom-wl driver on the [[Broadcom wireless]] page. The interfaces can swap during reboot so its best to define them in a udev rule (instructions on the [[Broadcom wireless]] page).<br />
<br />
* 8.1 generation MacBooks have BCM4331, for which support is not present in either Linux (3.0 and 3.1) or the proprietary drivers by Broadcom. There is however preliminary support for it in Linux 3.2. To run the drivers on earlier kernels, you will need to use [https://backports.wiki.kernel.org/index.php/Documentation/compat-drivers compat-drivers]<br />
<br />
{{Note|If your connection frequently drops, you may have to turn off Wi-Fi power management. If you are running [[pm-utils]], you may override wireless power management by creating an executable file {{ic|/etc/pm/wireless}} with the lines:<br />
#!/bin/sh<br />
iwconfig wlp2s0 power off<br />
}}<br />
<br />
=== Power management ===<br />
<br />
[[Powerdown]] is a very simple to set up set of scripts what will maximize your battery duration. A MacBook Air 2013 with powerdown provides abour 11 hours of light usage with just powerdown installed.<br />
All the usual [[power management]] recomendations apply as well.<br />
<br />
==== Suspend and Hibernate ====<br />
<br />
Suspending (suspend to ram) and hibernating (suspend to disk) work fine out of the box:<br />
<br />
systemctl suspend<br />
<br />
Issues were reported where the machine would "suspend immediately after resume" in certain conditions when suspending by closing the lid. This was solved by de-selecting the option "event_when_closed_battery" in gconf-editor &rarr; gnome-power-manager &rarr; actions).<br />
<br />
See [[Suspend and hibernate]] for details on how to configure hibernation. Noticably, you'll need a swap partition or file (see the mentioned article for further instructions).<br />
<br />
If after suspend laptop is woken up after few seconds, may help to disable all stuff in /proc/acpi/wakeup, exclude LID0:<br />
# echo XHC1 > /proc/acpi/wakeup<br />
$ cat /proc/acpi/wakeup<br />
Device S-state Status Sysfs node<br />
P0P2 S3 *disabled<br />
EC S3 *disabled<br />
HDEF S3 *disabled pci:0000:00:1b.0<br />
RP01 S3 *disabled pci:0000:00:1c.0<br />
RP02 S3 *disabled pci:0000:00:1c.1<br />
RP03 S3 *disabled pci:0000:00:1c.2<br />
ARPT S4 *disabled pci:0000:03:00.0<br />
RP05 S3 *disabled pci:0000:00:1c.4<br />
RP06 S3 *disabled pci:0000:00:1c.5<br />
SPIT S3 *disabled<br />
XHC1 S3 *disabled pci:0000:00:14.0<br />
ADP1 S3 *disabled<br />
LID0 S3 *enabled<br />
And for permanent disabling:<br />
$ cat /etc/udev/rules.d/90-xhc_sleep.rules <br />
<br />
# disable wake from S3 on XHC1<br />
SUBSYSTEM=="pci", KERNEL=="0000:00:14.0", ATTR{power/wakeup}="disabled"<br />
<br />
=== Light sensor ===<br />
<br />
If you want to use the built in light sensor to automatically adjust screen and keyboard backlight brightness check out<br />
'''Lighter''' [https://github.com/Janhouse/lighter] (simple perl script, easy to fine-tune) and '''Lightum''' [https://github.com/poliva/lightum] (Requires Gnome or KDE but is older and more complete than Lighter).<br />
<br />
=== Sound ===<br />
<br />
{{Tip| If using [[ALSA]], the internal speaker might not be disabled when using the headphone jack. To solve this, enable "Auto-mute" using {{ic|alsamixer}}}}<br />
<br />
First of all follow [[ALSA]] wiki page, then if something does not work correctly, continue reading this part.<br />
<br />
Edit your {{ic|/etc/modprobe.d/50-sound.conf}} or {{ic|/etc/modprobe.d/modprobe.conf}} appending this line:<br />
<br />
options snd_hda_intel model=intel-mac-auto<br />
<br />
This should automatically specify the codec in your MacBook. Alternatively, for MacBookPro5,X, you can use:<br />
<br />
options snd_hda_intel model=mb5<br />
<br />
(note that the jack output is controlled with "HP").<br />
<br />
If you have an iMac8,1, you should instead use<br />
<br />
options snd-hda-intel model=mbp3 position_fix=2<br />
<br />
You can try to specify other options, that depend on your hardware. All other possible settings are listed in Kernel Documentation, avaible online:<br />
<br />
* [http://www.kernel.org/doc/Documentation/sound/alsa/ALSA-Configuration.txt ALSA-Configuration.txt]<br />
* [http://www.kernel.org/doc/Documentation/sound/alsa/HD-Audio.txt HD-Audio.txt]<br />
* [http://www.kernel.org/doc/Documentation/sound/alsa/HD-Audio-Models.txt HD-Audio-Models.txt].}}<br />
<br />
Then, reboot.<br />
<br />
=== Bluetooth ===<br />
<br />
Bluetooth should work out-of-the box. See the article on [[Bluetooth]] to install and configure all software needed.<br />
<br />
=== Webcam ===<br />
<br />
==== iSight ====<br />
<br />
{{Note|Linux kernel from 2.6.26 includes the Linux UVC driver natively. '''MBP 6,2+ (Kernel ~2.6.37+) iSight works out of the box''' without the need to use firmware from OS X. Only use {{ic|isight-firmware-tools}} if it doesn't work normally.}}<br />
<br />
iSight webcams on MacBooks or pre 6,2 MacBook Pros (6,2 came out around 2010) require the Apple's proprietary firmware that cannot be redistributed. It must be extracted from OS X and loaded onto Arch.<br />
<br />
You will need to install {{AUR|isight-firmware-tools}} to extract the firmware. This package also includes a udev rule and ELF binary that are necessary, even once you have extracted the firmware file into {{ic|/lib/firmware/isight.fw}}, for the file to be loaded every time you boot your computer (namely {{ic|/etc/udev/rules.d/isight.rules}} which uses {{ic|/usr/lib/udev/ift-load}}).<br />
<br />
Instructions:<br />
<br />
First you need to get the firmware out of a particular file located on your OS X install. It is located in {{ic|/System/Library/Extensions/IOUSBFamily.kext/Contents/PlugIns/AppleUSBVideoSupport.kext/Contents/MacOS/AppleUSBVideoSupport}}.<br />
<br />
{{ Tip | The {{ic|AppleUSBVideoSupport}} file from a OS X 10.6 (Snow Leopard) installation may not work properly. If possible, use the file from OS X 10.5 or earlier.}}<br />
<br />
To mount the OS X drive if multi-booting:<br />
<br />
# sudo mkdir /media/OSX<br />
# sudo mount -t hfsplus /dev/sda2 /media/OSX<br />
<br />
Then, install the {{AUR|isight-firmware-tools}} package.<br />
<br />
Locate the {{ic|AppleUSBVideoSupport}} file in the OS X directory listed above. Either copy it over to your Arch system (Any OS X installation should do, such as an iMac, not just one specific to your system) or, if multi-booting, mount the OS X drive and navigate to the directory. (On 10.7 (Lion) the directory is {{ic|/System/Library/Extensions/IOUSBFamily.kext/Contents/PlugIns/AppleUSBVideoSupport.kext/Contents/MacOS}}.) In that directory you can go ahead and extract the driver:<br />
<br />
# ift-extract --apple-driver AppleUSBVideoSupport<br />
<br />
When it's done, check that the firmware has been found:<br />
<br />
# ls /lib/firmware/isight.fw<br />
<br />
Once successful, completely '''SHUTDOWN''' your Mac and start it back up again (to clear the hardware state of the webcam). Do ''not'' reboot.<br />
<br />
It should be automatically loaded at boot; if it isn't you can load the '''uvcvideo''' module [[Kernel modules|manually or load it at boot]].<br />
<br />
You can use many applications to test the webcam:<br />
<br />
* MPlayer<br />
<br />
# mplayer tv:// -tv driver=v4l2:width=320:height=240:device=/dev/video0 -fps 30<br />
<br />
* Cheese<br />
* Skype<br />
* Ekiga<br />
<br />
A simple solution to take snapshots is:<br />
<br />
# mplayer tv:// -vf screenshot<br />
<br />
and the pressing the s key to take a snapshot. Files are of the format {{ic|shot\d\d\d\d.png}} and are reported in the standard output.<br />
<br />
==== Facetime HD ====<br />
The Facetime HD webcam (included on 2013 MBAs onwards) [http://mactaris.blogspot.co.uk/2013/07/webcam-settings-20-will-support.html is no longer UVC device], and therefore, does not work out of the box. It is actually a PCIE device. While a [https://github.com/patjak/bcwc_pcie bcwc_pcie] driver is being developed, it will probably take some time before it is ready. See also [https://bugzilla.kernel.org/show_bug.cgi?id=71131 Linux bug #71131] and [https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1276811 Ubuntu bug #1276711].<br />
<br />
=== Temperature Sensors ===<br />
<br />
For reading temperature just install {{Pkg|lm_sensors}}. See the [[Lm sensors]] page for more information.<br />
<br />
=== Color Profile ===<br />
<br />
We can use color profiles from OS X.<br />
<br />
First, install the {{AUR|xcalib}} package.<br />
<br />
Second copy pre-saved color profiles placed in {{ic|/Library/ColorSync/Profiles/Displays/}} on OS X partition to {{ic|~/colorprofiles/}} for example.<br />
<br />
There are color profile files agree with in MacBook models; select the right one:<br />
<br />
* '''Color LCD-4271800.icc''' for MacBook Pro with CoreDuo CPU<br />
* '''Color LCD-4271880.icc''' for MacBook with Core2Duo<br />
* '''Color LCD-4271780.icc''' for MacBook (non-Pro) based on CoreDuo or Core2Duo.<br />
<br />
{{Tip|Also OS X allows to save current color profile from ''Displays > Color'' section of the ''Mac OS System Preferences'', in this case file is saved to {{ic|/Users/<username>/Library/ColorSync/Profiles}}.}}<br />
<br />
Finally you can activate it by running<br />
<br />
# xcalib ~/colorprofile.icc<br />
<br />
{{Note|Previous command set the color profile only for the current session; this mean that you must run it every time you login in your system. For automating it you can execute the command by '''Autostart Application''', concording with your DE (or add the command to your login manager's initialization script, e.g. /etc/gdm/Init/Default).}}<br />
<br />
=== Apple Remote ===<br />
<br />
First, to correctly install and configure the '''lirc''' software that control IR see [[LIRC]] wiki.<br />
<br />
Then make LIRC use {{ic|/dev/usb/hiddev0}} (or {{ic|/dev/hiddev0}}) by editing {{ic|/etc/conf.d/lircd}}. Here is how mine look:<br />
<br />
#<br />
# Parameters for lirc daemon<br />
#<br />
LIRC_DEVICE="/dev/usb/hiddev0"<br />
LIRC_DRIVER="macmini"<br />
LIRC_EXTRAOPTS=""<br />
LIRC_CONFIGFILE="/etc/lirc/lircd.conf"<br />
<br />
Use '''irrecord''' (available when installing '''lirc''') to create a configuration file matching your remote control signals (alternatively, you can try to use the {{ic|lircd.conf}} below):<br />
<br />
# irrecord -d /dev/usb/hiddev0 -H macmini output_conf_file<br />
<br />
Start '''lircd''' and use '''irw''' to check if it works.<br />
<br />
Example of an {{ic|/etc/lirc/lircd.conf}}:<br />
<br />
begin remote<br />
<br />
name lircd.conf.macbook<br />
bits 8<br />
eps 30<br />
aeps 100<br />
<br />
one 0 0<br />
zero 0 0<br />
pre_data_bits 24<br />
pre_data 0x87EEFD<br />
gap 211994<br />
toggle_bit_mask 0x87EEFD01<br />
<br />
begin codes<br />
Repeat 0x01<br />
Menu 0x03<br />
Play 0x05<br />
Prev 0x09<br />
Next 0x06<br />
Up 0x0A<br />
Down 0x0C<br />
end codes<br />
<br />
end remote<br />
<br />
=== HFS partition sharing ===<br />
<br />
First, install the {{AUR|hfsprogs}} package. <br />
<br />
we have to list our partitions. Use<br />
<br />
fdisk -l /dev/sda<br />
<br />
example output:<br />
<br />
# fdisk -l /dev/sda<br />
Device Boot Start End Blocks Id Type<br />
/dev/sda1 1 26 204819 ee GPT<br />
/dev/sda2 26 13602 109051903+ af Unknown<br />
/dev/sda3 * 13602 14478 7031250 83 Linux<br />
/dev/sda4 14478 14594 932832+ 82 Linux swap / Solaris<br />
<br />
As we see, the "Unknown" partition is our OS X partition, which is located in {{ic|/dev/sda2}}.<br />
<br />
Create a "mac" folder in /media:<br />
<br />
# mkdir /media/mac<br />
<br />
Add at the end of ''/etc/fstab'' this line:<br />
<br />
/dev/sda2 /media/mac hfsplus auto,user,rw,exec 0 0<br />
<br />
Mount it :<br />
<br />
mount /media/mac<br />
<br />
and check it:<br />
<br />
ls /media/mac<br />
<br />
=== HFS+ Partitions ===<br />
<br />
HFS+ partitions, now the default in OS X, are not fully supported by Linux and are mounted as read-only by default. In order to write to an HFS+ partition, it is necessary to disable journaling. This can be accomplished using the OS X Disk Utility. Refer to this [http://support.apple.com/kb/ht2355 Apple support page] for more information.<br />
<br />
===Home Sharing===<br />
<br />
'''''UID Synchronization'''''<br />
<br />
==== In OS X ====<br />
<br />
{{Note|It is strongly recommended that UID/GID manipulation be done immediately after a new user account is created, in OS X as well as in Arch Linux. If you installed OS X from scratch, then this operation is guaranteed to work after logging into your account for the first time.}}<br />
<br />
===== Step 1: change UID and GID(s) =====<br />
<br />
'''''Pre-Leopard'''''<br />
<br />
# Open '''NetInfo Manager''' located in the ''/Applications/Utilities'' folder.<br />
# If not done for you already, enable access to user account transactions by clicking on the closed lock at the bottom of the window, and entering your account password, or root password if you have created a root account.<br />
# Navigate to ''/users/<new user name>'' where <new user name> is the name of the account that will have read/write access to the folder that will be shared with the primary user in Arch.<br />
# Change the '''UID''' value to 1000 (the value used by default for first user created in Arch).<br />
# Also change the '''GID''' value to 1000 (the value used by default for user account creation in Arch).<br />
# Navigate to {{ic|/groups/<new user name>}}, automatically saving the changes you have made so far.<br />
<br />
{{Note|If you get an error message that the transaction is not allowed, log out and log back in.}}<br />
<br />
'''''Leopard'''''<br />
<br />
In Leopard, the '''NetInfo Manager''' application is not present. A different set of steps is required for UID synchronization:<br />
<br />
# Open '''System Preferences'''.<br />
# Click on '''Users & Groups'''.<br />
# Unlock the pane if not already done so.<br />
# Right-click on the desired user and select '''Advanced Options'''.<br />
# Write down the value of the '''User ID''' field, you will need it later on. Change both the UID and GID to match the UID and GID of the account wished to be shared with in Arch (1000 by default for the first user created in Arch).<br />
<br />
===== Step 2: change "Home" permissions =====<br />
<br />
# Open up '''Terminal''' in the {{ic|/Applications/Utilities}} folder.<br />
<br />
# Enter the following command to reclaim the permission settings of your home folder, replacing <your user name>, <your user group> and <your old UID> with the user name whose UID and GID values you just changed, the group name whose GID value you just changed and the old UID number, respectively.<br />
<br />
# find /User/<your user name> -user <your old UID> -exec chown <your user name>:<your user group> {} \;<br />
<br />
==== In Arch ====<br />
<br />
To synchronize your UID in Arch Linux, you are advised to perform this operation ''while creating a new user account''.<br />
It is therefore recommended that you do this as soon as you install Arch Linux.<br />
<br />
Now you must substitute Arch's home with OS X's home, by modify entries of {{ic|/etc/fstab}}.<br />
<br />
=== Avoid long EFI wait before booting ===<br />
<br />
If your MacBook spends 30 seconds with "white screen" before booting you need to tell the firmware where is the booting partition.<br />
<br />
Boot OS X, if do not have it installed, you can use the install DVD (select language, then click Utilities->Terminal), or another MacBook with OS X (connect the two computers via firewire or thunderbolt, start the other MacBook keeping pressed T, boot your MacBook keeping pressed Options).<br />
<br />
Either way, once you got a OS X terminal running on your MacBook you need to execute, as root, a different command if the boot partition is EFI or it is not:<br />
<br />
# bless --device /dev/disk0s1 --setBoot # if the booting partition is EFI<br />
or<br />
# bless --device /dev/disk0s1 --setBoot --legacy # if the booting partition is not EFI<br />
<br />
(given that if your GRUB or EFI is on sda1, /dev/disk1s2 if it is on sdb2, etc). See also https://bbs.archlinux.org/viewtopic.php?pid=833215 and https://support.apple.com/kb/HT1533 .<br />
<br />
=== Mute startup chime ===<br />
<br />
If you forgot to mute before installing, you can still mute again if you have a OS X install disk. Boot from it, select language, then click ''Utilities > Terminal'', and enter<br />
<br />
# /usr/sbin/nvram SystemAudioVolume=%01<br />
<br />
(or whatever volume you want).<br />
<br />
=== kworker using high CPU ===<br />
Sometime with the addition of Yosemite, some users found that kworker CPU usage will spike, as disccused [https://bbs.archlinux.org/viewtopic.php?id=171883&p=11 here]. This is sometimes the result of runaway ACPI interrupts.<br />
<br />
To check and see, you can count the number of recent ACPI interrupts and see if any of them are out of control.<br />
<br />
grep . -r /sys/firmware/acpi/interrupts/<br />
<br />
<br />
If you see that one particular interrupt is out of control (possibly GPE66), i.e., registering hundreds of thousands of lines, you can try disabling it (replace XX with the runaway interrupt):<br />
<br />
<br />
echo "disable" > /sys/firmware/acpi/interrupts/gpeXX<br />
<br />
<br />
Disabling random ACPI interrupts could cause all kinds of problems, so do this at your own risk. If this fixes the problem, there is discussion about how to make a systemd service that automatically disables an interrupt at every boot [https://bbs.archlinux.org/viewtopic.php?pid=1488371 here].<br />
<br />
== rEFIt ==<br />
<br />
{{Note|<br />
* You probably want to have a look at [http://www.rodsbooks.com/refind/ rEFInd], which is some type of successor of rEFIt.<br />
* This is not a requirement. It only gives you a menu to choose between OS X and Arch Linux upon every boot.<br />
}}<br />
<br />
For more see, [http://refit.sourceforge.net/myths/ rEFIt myths].<br />
<br />
In OS X, download the ".dmg" from [http://refit.sourceforge.net/ rEFIt Homepage] and install it.<br />
<br />
{{Note|If you have already partitioned your hard disk in preparation for the Arch installation, rEFIt may not be enabled by default. You will have to run the "enable.sh" script installed in /efi/refit/.}}<br />
<br />
Open up '''Terminal''' and enter:<br />
<br />
cd /efi/refit;<br />
./enable.sh<br />
<br />
=== Problems with rEFIt ===<br />
<br />
If you experience problems after the install of Arch or rEFIt, especially is the right OS is not showing up to boot to or if it dumps you at a GRUB prompt stuck like the following:<br />
<br />
GRUB>_<br />
<br />
Then have a look at this link:<br />
<br />
http://mac.linux.be/content/problems-refit-and-grub-after-installation<br />
<br />
It can give you a basic idea on how to boot off the Arch live cd, mount the problem Arch install, chroot, use gptsync, and reinstall GRUB. This is probably for more advanced users who can translate the commands from a debian system to an Arch system and also apply it to the partitions on their machine. Be careful not to install GRUB in the wrong spot.<br />
<br />
If you need a copy of gptsync you can wget it from here:<br />
http://packages.debian.org/sid/gptsync<br />
or try these, for 64 bit:<br />
<br />
wget http://ftp.us.debian.org/debian/pool/main/r/refit/gptsync_0.14-2_amd64.deb<br />
<br />
and for i386:<br />
<br />
wget http://ftp.us.debian.org/debian/pool/main/r/refit/gptsync_0.14-2_i386.deb<br />
<br />
since they are .deb packages you will need the program {{AUR|deb2targz}}.<br />
<br />
==== Mavericks upgrade breaks Arch boot option ====<br />
For some multi-boot users who utilize a separate Linux boot partition, the OS X Mavericks upgrade may overwrite the boot partition with Apple's own recovery boot filesystem. This breaks the Arch Linux boot option in rEFIt/rEFInd. The best way to proceed in this situation is to abandon a separate boot partition and use the EFI system partition (ESP) to install the bootloader of your choice. It is also recommended that you use rEFInd instead of rEFIt as development on the latter has halted.<br />
<br />
Assuming grub2 as the bootloader:<br />
<br />
Use the Arch LiveCD to boot to a shell and [[Change root|chroot]] to your broken Arch Linux environment.<br />
<br />
Mount the ESP on /boot.<br />
<br />
Edit the fstab and remove the old boot partition and make ESP the new boot partition. Now mount the ESP as the new /boot parition.<br />
# mount -a<br />
<br />
[[install|Reinstall]] {{pkg|linux}}.<br />
<br />
Create a new initramfs and vmlinuz in /boot.<br />
# mkinitcpio -p linux<br />
<br />
Install grub.<br />
# grub-install --target=x86_64-efi --efi-directory=/boot --bootloader-id=grub --recheck --debug<br />
<br />
Create a new grub.cfg file.<br />
# grub-mkconfig -o /boot/EFI/grub/grub.cfg<br />
<br />
Make sure that grub.cfg is in the same directory as grubx64.efi.<br />
<br />
Generate a new refind_linux.conf file in /boot simply by running mkrlconf.sh which comes with rEFInd.<br />
<br />
Exit the chroot environment.<br />
<br />
Reboot. You should see a new entry for Arch Linux in rEFInd and it should boot to your Arch Linux installation.<br />
<br />
== Model-specific information ==<br />
<br />
=== MacBook ===<br />
<br />
==== Mid 2007 13" - Version 2,1 ====<br />
<br />
{{Note|I used the 201212 ISO image.}}<br />
<br />
Since older Macbooks have a 32bit EFI running, the usual installation image is not recognized. You need to either remove the UEFI support from the disc ([[Unified_Extensible_Firmware_Interface#Remove_UEFI_boot_support_from_ISO]]) or build a 32bit EFI version of the disc. The paragraphs below will take the first path to success, booting into BIOS mode and its pitfalls. For a try the other way round, read [[Unified_Extensible_Firmware_Interface#Create_UEFI_bootable_USB_from_ISO]] first.<br />
<br />
First prepare your harddisc according to your wishes. In this scenario it was a "Linux only" approach with<br />
<br />
/dev/sda1 HFS+ AF00 200M -> EFI boot system on Apple HFS+ partition<br />
/dev/sda2 ext4 8300 147G -> arch system<br />
/dev/sda3 swap 8200 1G -> swap<br />
<br />
The {{AUR|hfsprogs}} package contains the tools to handle HFS/HFS+ filesystems. The rEFInd bootloader recognizes it on its own. Usually the partition for the EFI bootloader is a FAT32 (vfat) partition. In this case I tried rEFIt first, which apparently needs the HFS+ filesystem to work, and kept it at that.<br />
<br />
The mount points are:<br />
<br />
/dev/sda2 -> /<br />
/dev/sda1 -> /boot/EFI<br />
<br />
The bootloader in use was [http://www.rodsbooks.com/refind/index.html rEFInd] instead of rEFIt. To install it, the rEFInd homepage provides a good guide. Usually it is simply done by copying rEFInd:<br />
<br />
cp -vr /usr/share/refind/drivers_ia32 /boot/EFI/refind/<br />
cp -vr /usr/share/refind/tools_ia32 /boot/EFI/refind/<br />
cp -vr /usr/share/refind/fonts /boot/EFI/refind/<br />
cp -vr /usr/share/refind/icons /boot/EFI/refind/<br />
cp -v /usr/share/refind/refind_ia32.efi /boot/EFI/refind/<br />
cp -v /usr/share/refind/refind.conf-sample /boot/EFI/refind/refind.conf<br />
cp -v /usr/share/refind/refind_linux.conf-sample /boot/refind_linux.conf<br />
<br />
{{Note|I'm using the 32bit version of Arch and refind, since the EFI of the old MacBooks is 32bit. I'm not sure about 32bit rEFInd booting a 64bit Arch...}}<br />
<br />
The pitfall here is, that the system bootet in BIOS compatibility mode and not in EFI mode. You cannot therefore use {{ic|efibootmgr}}, because the EFI variables (even with 'modprobe efivars') are not available. While installing the system get {{AUR|mactel-boot}}. The {{ic|hfs-bless}} utility comes in handy, when blessing the EFI bootloader. This is done by calling:<br />
<br />
hfs-bless /boot/EFI/refind/refind_ia32.efi<br />
<br />
Since the Linux kernel does come with EFI stub enabled, it seems a good idea to run it through a bootloader first. Especially if it runs not out of the box. But using rEFInd makes GRUB (or any other bootloader) obsolete, because of that.<br />
<br />
{{Note|In the refind_linux.conf you add any kernel option you may want as long as you use the EFI stub of your kernel. In refind.conf you adjust your needs for the bootloader itself, like menu entries. If you use them (menu entries), rEFInd should not look for these EFI stub kernels itself, so blacklist the directories used in here, like {{ic|/boot/}}.}}<br />
<br />
Not running out of the box is unfortunately the initial stage for the kernel. Since we installed it in BIOS mode, two modules are missing to grant access to the root partition while booting. Hence the 'initfsram-linux.img' can not be found/loaded. Adding the following modules to your 'MODULES' line in {{ic|/etc/mkinitcpio.conf}} solved this ([https://bbs.archlinux.org/viewtopic.php?pid=1139226#p1139226 original post]).<br />
<br />
{{hc|/etc/mkinitcpio.conf|2=<br />
MODULES="ahci sd_mod"}}<br />
<br />
Rebuild your kernel image:<br />
<br />
mkinitcpio -p linux<br />
<br />
The bootloader rEFInd can scan kernels even out of the '/boot/...' directory and assumes an efi kernel even without the extension '.efi'. If you do not want to try out special kernels, this should work without the hassle to copy each kernel after building to some spot special.<br />
<br />
If you happen to get multiple entries for one boot image, it often results of a previous installation of a bootloader within the MBR. To remove that, try the following - taken from the [http://ubuntuforums.org/showpost.php?p=7828260&postcount=4 original post]. This is valid for GPT partitioned discs, so please check your environment and save your MBR first.<br />
<br />
# dd if=/dev/zero of=/dev/sda bs=440 count=1<br />
<br />
=== MacBook Pro with Retina display ===<br />
<br />
==== Early 2015 13"/15" - Version 12,x/11,4+ ====<br />
<br />
===== Wireless =====<br />
The {{ic|brcmfmac}} driver is working (with some issues) out of the box as of 2015-05-02.<br />
* 5GHz is not working. <br />
* There seems to be an issue which might lead to a kernel oops. See [http://www.spinics.net/lists/linux-wireless/msg136089.html this] post for further information and a patch which seems to fix the problem.<br />
<br />
For Network controller BCM43602 14e4:43ba "Broadcom Corporation Device 43ba (rev 01)", on MacBookPro12,1 (output of "sudo dmidecode -s system-product-name"):<br />
* you need kernel version 3.17+ See [http://linuxwireless.org/en/users/Drivers/brcm80211/ this driver page] for details.<br />
* the kernel oops / freeze appeared - [http://www.spinics.net/lists/linux-wireless/msg136092.html This patch] does fix (as far as I could test) the problem that appeared when running multiple (in my case 6) concurrent downloads. This patch is included in kernel 4.0.5 :)<br />
* But there is another problem : Even with the patch applied I ran in another kernel (version 4.0.4) freeze triggered by the brcmfmac kernel module ! - See [https://bugzilla.kernel.org/show_bug.cgi?id=100201 this new bug report].<br />
<br />
===== Bluetooth =====<br />
Bluetooth is not working at all.<br />
# hcitool dev<br />
Devices:<br />
<br />
===== Keyboard & Trackpad =====<br />
Haptic feedback works out of the box due to the trackpad's built-in firmware. (Kernels 4.2-rc4 and lower)<br />
<br />
{{Pkg|xf86-input-synaptics}} is required for the trackpad to work, as is kernel 4.2.x+. mtrack-git does not yet support the new trackpad (as of 2015-08-20)<br />
<br />
As of 4.2-rc5 the trackpad and keyboard are working. You can install this kernel version using the {{AUR|linux-mainline}} package.<br />
<br />
The following config is necessary on some systems to enable trackpad movement on 4.2+ kernels, avoiding an issue where the trackpad could be clicked, but the cursor could not be moved:<br />
<br />
# /etc/X11/xorg.conf.d/60-magictrackpad.conf<br />
Section "InputClass"<br />
Identifier "Trackpad"<br />
Driver "synaptics"<br />
MatchIsTouchpad "on"<br />
MatchDevicePath "/dev/input/event*"<br />
EndSection<br />
<br />
Further, some US/ANSI keyboards suffer from an issue where the tilde key (~, the key vertically between Esc and Tab) registers as < and >. The following config file fixes this issue.<br />
<br />
# /etc/modprobe.d/hid_apple.conf<br />
options hid_apple iso_layout=0<br />
<br />
See [https://bugzilla.kernel.org/show_bug.cgi?id=96771 this kernel bugzilla] for more details and the relevant patches for earlier kernels.<br />
<br />
===== Graphics =====<br />
For Intel-only graphics, install {{Pkg|xf86-video-intel}}. For more information or OpenGL/3D support, follow instructions at [[Intel graphics]].<br />
<br />
For Dual Graphics see [https://wiki.archlinux.org/index.php/MacBookPro11,x#Graphics this section on the 11,3] <br />
For 8086:162b "Intel Corporation Broadwell-U Integrated Graphics (rev 09)" intel Iris 6100, on MacBookPro12,1 (output of "sudo dmidecode -s system-product-name"):<br />
* acpi_backlight=vendor with kernel 4.0.4 as kernel parameter is necessary to help fixing fn+brightness not working in X because of a ghost acpi_video0 brightness control. See [https://bugzilla.kernel.org/show_bug.cgi?id=100041 this bug report], and [https://bugzilla.kernel.org/attachment.cgi?id=180311&action=diff&context=patch&collapsed=&headers=1&format=raw the working patch], confirmed with kernel 4.0.4, that fixes the problem (so no acpi_backlight=vendor or video.* parameter necessary anymore to get working brightness controls).<br />
* i915.enable_ips=0 is necessary with kernel 4.0.4, as there is an issue on my MacBookPro12,1 (output of "sudo dmidecode -s system-product-name") that makes the ips - intermediate pixel storage (primarily a power saving feature) cause /flickering/ when mode switching (to tty console).<br />
* i915.lvds_downclock=1 ("Use panel (LVDS/eDP) downclocking for power savings"), sounds good, but will do nothing (because the MacBook has an eDP panel! there's a new auto-downclock for edp in recent kernels and it's enabled by default if available :), so don't use - recommendation from irc #intel-gfx)<br />
* intel_iommu=igfx_off if you have GPU HANG: ecode 8:0:0x85dffffb, in X [1737] blah-blah in dmesg and GPU hangs all the time during OpenGL operations.<br />
<br />
===== Apple Thunderbolt 2 Gigabit Ethernet Dongle =====<br />
On MacBookPro12,1 (output of "sudo dmidecode -s system-product-name") there is a bug (in kernel 4.0.4) in that the dongle gets recognized on boot-up and works just fine, but is not hotpluggable, so when unplugged it does not get recognized / work at all when replugged. See the following kernel bug report: https://bugzilla.kernel.org/show_bug.cgi?id=100191<br />
<br />
==== 2012 - 2014 models ====<br />
<br />
* [[MacBookPro11,x]] (Late 2013—Mid 2014)<br />
* [[MacBookPro10,x]] (Mid 2012—Early 2013)<br />
<br />
=== MacBook Air===<br />
<br />
==== Early 2014 11" - Version 6,1 ====<br />
This is almost the same as the 2013 version, where the only known difference is a slightly faster processor. The version numbers have not been changed since the 2013 version.<br />
<br />
It works excellently after following the instructions for the MBA 2013 13" here and in the forum thread.<br />
Bluetooth, which has been reported not working for some people with the 2013 version, works without trouble for the 2014 version, although it should be excactly the same.<br />
<br />
{{Note| Unless you have a local repository on a USB disk, you need a USB to ethernet adaptor or a USB wireless adaptor supported natively by the kernel to easily install Arch Linux, since you have to install the {{AUR|broadcom-wl-dkms}} package to make the internal wireless adaptor work.}}<br />
<br />
Unresolved issues:<br />
<br />
- There is no driver for the webcam yet.<br />
- rEFInd uses 30 seconds to start booting. Using the bless trick stops rEFInd from loading, and it has to be re-installed.<br />
<br />
==== Mid 2013 13" - Version 6,2 ====<br />
[https://bbs.archlinux.org/viewtopic.php?id=165899 Dedicated forum thread]<br />
===== Installing and booting =====<br />
Booting from a normal 2013.6 USB key works fine, but I could not seem to get either GRUB or Syslinux working.<br />
<br />
I was able to boot by first installing Arch Linux following the MacBook guide at the wiki (having a separate FAT32 /boot partition). Skip the bootloader installation. <br />
<br />
Installing [http://www.rodsbooks.com/refind/getting.html rEFInd] from OS X (important!) and installing the EFI stub loader made me able to boot fine.<br />
<br />
[https://bbs.archlinux.org/viewtopic.php?id=165710 Dedicated thread].<br />
<br />
{{Note| Installing [http://www.rodsbooks.com/refind/getting.html rEFInd] from Linux (or from OS X, but to the esp) also works fine}}<br />
<br />
===== Arch Only Installation =====<br />
This method works without rEFInd and uses grub to boot EFI. Partition as follows:<br />
<br />
/dev/sda1 200M Microsoft basic data<br />
/dev/sda2 256M Linux filesystem<br />
/dev/sda3 4G Linux swap<br />
/dev/sda4 108.6G Linux filesystem<br />
<br />
sda1 can also be a HFS+ partition for EFI. This example chooses to use FAT32 (vfat). Although swap is optional, it is required for hibernation. Instead of sda4 for root and home, an alternative partition scheme would be to make sda4 as root and sda5 as home.<br />
<br />
Format and mount:<br />
<br />
mkfs.vfat -F 32 /dev/sda1<br />
mkfs.ext2 /dev/sda2<br />
mkswap /dev/sda3<br />
swapon /dev/sda3<br />
mkfs.ext4 /dev/sda4<br />
<br />
mount /dev/sda4 /mnt<br />
mkdir /mnt/boot<br />
mount/dev/sda2 /mnt/boot<br />
mkdir /mnt/boot/efi<br />
mount /dev/sda1 /mnt/boot/efi<br />
<br />
Finish the installation according to the [https://wiki.archlinux.org/index.php/Beginners'_Guide#Select_a_mirror Beginner's Guide] and skip anything after the bootloader. After you have generated your initramfs and set root passwd follow below to setup grub:<br />
<br />
pacman -S grub efibootmgr<br />
mount -t efivarfs efivarfs /sys/firmware/efi/efivars<br />
grub-install --target=x86_64-efi --efi-directory=/boot/efi --bootloader-id=grub --recheck --debug<br />
grub-mkconfig -o /boot/efi/EFI/grub/grub.cfg<br />
cp /boot/efi/EFI/grub/grub.cfg /boot/grub/grub.cfg<br />
cp /boot/efi/EFI/grub/grubx64.efi /boot/efi/EFI/boot/bootx64.efi}}<br />
<br />
Now you can exit/unmount/reboot:<br />
exit<br />
umount -R /mnt<br />
reboot<br />
<br />
===== Stability problems =====<br />
{{Note| Passing {{ic|<nowiki>libata.force=1:noncq</nowiki>}} to the kernel parameters solves the problem.}}<br />
This is the big worry for me. Every now and then my system hangs for a brief moment and everything involving net or disk access just hangs there for a while and then it seems to work. <br />
So far it only seems to happen when I run something disk- or CPU-intensive. Also had an occassion when I could not start X and just got this repeating all over my screen:<br />
<br />
ata1.00: failed command: WRITE FPDMA QUEUED<br />
ata1.00: cmd 61/08:f0:10:8c:c2/00:00:0b:00:00/40 tag 30 ncq 4096 out<br />
res 40/00:00:00:00:00/00:00:00:00:00/00 Emask 0x4 (timeout)<br />
ata1.00: status: { DRDY }<br />
<br />
On the next attempt it worked fine.<br />
I did SMART short and long tests on my disk and they returned fine:<br />
<br />
[http://pastebin.com/vRE4T2Ld smartctl -a]<br />
<br />
There are some messages in my boot that indicate this could be disk and/or ACPI related.<br />
<br />
These are with 2013-06 ISO, 3.9.7-1 2013 x86_64 kernel.<br />
<br />
[http://pastebin.com/mjTJaPFa journalctl -b]<br />
Seems to only work with the headphone jack, not with the speakers.<br />
<br />
[http://pastebin.com/SdAcHuKh dmesg]<br />
<br />
===== Marvell ATA suspend bugs =====<br />
If you have 2013 MacBook Air with a Marvell 128 gig drive, you might get the following ata errors instead after pm-suspend/resumes:<br />
<br />
ata1: exception Emask 0x10 SAct 0x0 SErr 0x10000 action 0xe frozen<br />
ata1: irq_stat 0x00400000, PHY RDY changed<br />
ata1: SError: { PHYRdyChg }<br />
ata1: hard resetting link<br />
ata1: SATA link up 1.5 Gbps (SStatus 113 SControl 310)<br />
ata1.00: unexpected _GTF length (8)<br />
ata1.00: unexpected _GTF length (8)<br />
ata1.00: configured for UDMA/33<br />
ata1: EH complete<br />
<br />
Try what Patrick and Tejun figured out on the [https://bugzilla.kernel.org/show_bug.cgi?id=62351 linux bug]. I followed what Patrick describes with sata_alpm, and I haven't seen the issue since.<br />
<br />
===== Suspend/Resume =====<br />
Brightness is either 0% or 100% after resuming from suspend. Until the kernel is fixed, use patjak's fix by installing {{AUR|mba6x_bl-dkms}}. Patjak's github is at [https://github.com/patjak/mba6x_bl].<br />
===== WiFi =====<br />
WiFi does not work out of the box. Install {{AUR|broadcom-wl-dkms}} to connect to a network. <br />
<br />
===== Touchpad =====<br />
Since 3.10.3 kernel touchpad works perfectly with {{Pkg|xf86-input-synaptics}}.<br />
<br />
===== Audio =====<br />
As of Linux 3.12, sound works out of the box. If you do not get sound with only {{pkg|alsa-utils}}, you may need to create a /etc/asound.conf with below entries:<br />
<br />
defaults.pcm.card 1<br />
defaults.pcm.device 0<br />
defaults.ctl.card 1<br />
<br />
==== Mid 2012 13" — version 5,2 ====<br />
<br />
Kernel panics using default boot media under arch kernel 3.5. Adding <code>intremap=off</code> fixes this. Additionally, there are problems loading the <code>applesmc</code> module (meaning the temperature sensors, fan, and keyboard backlight do not work). These problems are fixed in the linux 3.6-rc4 mainline kernel (I have tested).<br />
<br />
==== Mid 2012 11.5" — Version 5,1 ====<br />
<br />
If you have issues with waking from sleep while in X11 such as a black screen or showing the console with a frozen mouse cursor then remove {{Pkg|xf86-input-synaptics}} and install {{AUR|mtrack-git}}. This fixed errors such as <br />
(EE) [dix] bcm5974: unable to find touch point 0<br />
and backtraces that causes X11 to crash. This might apply to Version 5,2 assuming they use the same trackpad.<br />
<br />
===== Installing using the Archboot 2012.06 image =====<br />
<br />
Several people have reported problems installing Arch Linux on the MBA version 5,2 (See [https://bbs.archlinux.org/viewtopic.php?id=144089 problems booting Arch Linux on a MacBook Air Mid 2012]). A common problem is that the screen is not detected and therefore goes black when the installer boots. To fix this problem one has to select the normal install (Not the LTS) during boot and press tab to edit the boot flags. Then add noapic flag to the boot line. This should fix the screen going black. Install the system as you normally would. It may help later in the configuration process if the support packages are installed already at this stage.<br />
<br />
When the install has finished again add the noapic flag to the GRUB boot line (if you use GRUB) and also add i915.diescreaming=1 (or perhaps i915.die). This should keep the screen from going black when booting the new system. After you enter the system the wireless driver should be loaded. If you installed the support packages during installation you should have the wifi-menu command. Run it and select the network you want to use. One could also use wpa_supplicant but wifi-menu is quite fast to use at this stage. Now you are ready to upgrade the system. As of writing there have been a lot of changes to Arch Linux since the 2012.06 image of Archboot was released ([https://www.archlinux.org/news/filesystem-upgrade-manual-intervention-required-1/ filesystem] and [https://www.archlinux.org/news/the-lib-directory-becomes-a-symlink/ glibc]). Therefore the upgrade process can be a bit difficult. The current solution has sucessfully upgraded a standart archboot version to a up-to-date version as of October 2012 and this step should be obsolete in future releases of archboot.<br />
<br />
First ignore the new "big" changes to Arch Linux,<br />
<br />
pacman -Suy --ignore glibc,libarchive,curl,filesystem <br />
<br />
If this only upgrades pacman then run the command again. Remember to make sure that pacman is ignoring the packages you do not want upgraded now. Otherwise you may break the system and have to reinstall! Now upgrade to the new filesystem,<br />
<br />
pacman -S filesystem --force<br />
<br />
As described in [[DeveloperWiki:usrlib|Glibc upgrade guide]] there may be conflicts with installed packages that require the /lib directory. Follow the guide and remove any packages that use /lib. The stock 3.4.2 kernel from Archboot should be on this list so first upgrade this,<br />
<br />
pacman -S linux<br />
<br />
This may give some errors saying that the system may not boot because of missing modules. Ignore this warning for now. The stock install may also contain gcc in the /lib directory so also remove this if needed and any other packages that have conflicts. Now Glibc should be the only package in /lib so run the system upgrade and accept all changes, <br />
<br />
pacman -Su<br />
<br />
Finally reinstall the kernel so that it can find the correct modules.<br />
<br />
Now this command should not give any errors like last time. You can also reinstall gcc at this point. After a rebooted the system should startup and the new kernel should have fixed the problem with the screen going black. If want to boot Xorg then you may need to remove the i915.diescreaming=1 line from GRUB. If not then attach a external screen and try to fix the problem that way. Some people have reported commands that may help on the [https://bbs.archlinux.org/viewtopic.php?id=144089 forum].<br />
<br />
==== Mid 2011 — version 4,x ====<br />
<br />
Works out-of-the-box since kernel 3.2. It is recommended to use [[Archboot]], install [[GRUB]] and use EFI.<br />
<br />
==== Early 2008 — version 1,1 ====<br />
<br />
Everything works out of the box though you will need {{Pkg|b43-fwcutter}} (or simply {{AUR|b43-firmware}}) for the wireless adapter to work.<br />
<br />
Since this model has only one USB port, you may find it easiest to install Arch with a powered USB hub. Plug a USB network adapter (wireless or ethernet adapter to plug into a USB port) and your Arch installation media into the USB hub.<br />
<br />
If you can't get any result by scanning wireless network after boot, unload modules <code>b43</code> and <code>ssb</code> and load them again:<br />
<br />
rmmod ssb<br />
rmmod b43<br />
modprobe b43<br />
<br />
There is a good chance you will find what's wrong with DMA from the dmesg log.<br />
<br />
Even if you can scan wireless networks after reloading the modules, it's still possible that you will only be able to connect to some networks, but not all of them. According to a more detailed discussion here: http://crunchbang.org/forums/viewtopic.php?id=17368, adding <code>pio=1,qos=0</code> options to the b43 module can solve this problem.<br />
<br />
I tested this for a 13' MacBookAir1,1 with a BCM4321 chipset, and it works.<br />
<br />
== See also ==<br />
* '''MacBook Air'''<br />
** [http://dabase.com/blog/Macbook_Air_Early_2014_Archlinux/ Macbook Air Early 2014 — dabase.com]<br />
** [http://www.frankshin.com/installing-archlinux-on-macbook-air-2013/ Installing Archlinux on Macbook Air 2013 — Frank Shin]<br />
** [http://blog.panks.me/posts/2013/06/arch-linux-installation-with-os-x-on-macbook-air-dual-boot/ Arch Linux Installation with OS X on Macbook Air (Dual Boot) — Pankaj Kumar]<br />
** [http://ryangehrig.com/index.php/arch-linux-on-macbook-air-2013/ Arch Linux – MacBook Air 2013 — Ryan Gehrig]<br />
** [http://www.nico.schottelius.org/blog/macbook-air-42-archlinux/ Installing Linux on a Macbook Air (4,2) — Nico Schottelius]<br />
** [http://www.dm9.se/?p=398 Arch linux single, pure efi boot on the macbook air3,1/3,2 — DIMENSION9]<br />
* '''MacBook Pro'''<br />
** http://www.netsoc.tcd.ie/~theorie/interblag/2010/01/30/installing-arch-linux-on-a-mac-pro/<br />
** http://allanmcrae.com/2010/04/installing-arch-on-a-macbook-pro-5-5/<br />
** http://allanmcrae.com/2012/04/installing-arch-on-a-macbook-pro-8-1/<br />
** http://linux-junky.blogspot.com/2011/08/triple-boot-archlinux-windows-7-and-mac.html</div>Justin8https://wiki.archlinux.org/index.php?title=Mac&diff=391548Mac2015-08-17T23:54:57Z<p>Justin8: Added requirement of synaptics driver for touchpad. Does not work out of the box on current image without this.</p>
<hr />
<div>[[Category:Apple]]<br />
[[de:ArchLinux auf einem MacBook]]<br />
[[fr:MacBook]]<br />
[[it:MacBook]]<br />
[[ja:MacBook]]<br />
[[zh-CN:MacBook]]<br />
{{Related articles start}}<br />
{{Related|Installation guide}}<br />
{{Related|Beginners' guide}}<br />
{{Related|General recommendations}}<br />
{{Related|MacBook4,2 (late 2008)}}<br />
{{Related|MacBook5,2 (early-mid 2009)}}<br />
{{Related|MacBookPro7,1}}<br />
{{Related|MacBookPro8,1/8,2/8,3 (2011)}}<br />
{{Related|MacBookPro9,2 (Mid-2012)}}<br />
{{Related|MacBookPro10,x}}<br />
{{Related|MacBookPro11,x}}<br />
{{Related|iMac Aluminum}}<br />
{{Related|Apple Fusion Drive}}<br />
{{Related articles end}}<br />
<br />
Installing Arch Linux on a MacBook (Air/Pro) is quite similar to installing it on any other computer. However, due to the specific hardware configuration on a MacBook, there are a few deviations and special considerations which warrant a separate guide. For more background information, please see the [[Installation guide]], [[Beginners' guide]] and [[UEFI]]. This guide contains installation-instructions that can be used on any Apple computer whose hardware is supported by the Linux kernel. Please see 'related' pages (on the top right of this page) for model-specific tips and troubleshooting.<br />
<br />
== Overview ==<br />
<br />
Specifically, the procedure for installing Arch Linux on a MacBook is:<br />
<br />
# '''[[#Updating OS X|Update OS X]]''': It always helps to start from a clean, backed up, and up-to-date install of OS X.<br />
# '''[[#Partitions|Partition]]''': Resizing or deleting the OS X partition to create partitions for Arch Linux.<br />
# '''[[#Setup bootloader|Setup bootloader]]''': Making sure that the new partition is bootable.<br />
# '''[[#Installation|Install Arch Linux]]''': Actually installing Arch Linux.<br />
# '''[[#Post-installation|Post-installation]]''': MacBook-specific configuration.<br />
<br />
== Updating OS X ==<br />
<br />
{{Note|If you uninstalled OS X or want to reinstall it, do that first. [http://www.apple.com Apple] has great instructions.}}<br />
In OS X, '''install every update''' (in the App Store). '''Reboot''' your computer, and then '''update again''' to make sure that you installed everything.<br />
<br />
* If you plan to remove OS X completely, make backups of these files, which you will need in Linux for adjusting the [[#Color Profile|color profile]]:<br />
{{Note|It is advisable to keep OS X, because the firmware can only be updated using OS X.}}<br />
<br />
/Library/ColorSync/Profiles/Displays/<FILES HERE><br />
<br />
You will also need the following file for iSight functionality:<br />
{{Note|This does not apply to devices using the FaceTime HD webcam (MacBookAir5,1/5,2/6,1/6,2), because the webcam is not yet supported by the Linux kernel.}}<br />
<br />
/System/Library/Extensions/IOUSBFamily.kext/Contents/PlugIns/AppleUSBVideoSupport.kext/Contents/MacOS/AppleUSBVideoSupport<br />
<br />
* If you plan to dual boot and want to keep OS X, continue to [[#Partitions]]<br />
<br />
== Partitions ==<br />
<br />
The next step in the installation is to re-partition the hard drive. If OS X was installed using the typical procedure, then your drive should have a GPT format and the following partitions:<br />
<br />
* '''EFI''': a 200 MB partition at the beginning of the disk. It is often read as '''msdos''' or '''FAT''' by some partitioning tools and usually labeled ''#1''.<br />
* '''OS X''': the ''(HFS+)'' partition that should take up all of the remaining disk space. Usually labeled ''#2''.<br />
* '''Recovery''': A recovery partition (only for OS X 10.7+).<br />
<br />
{{Note|If your hardware includes a Fusion Drive you might have a look at the [[Apple Fusion Drive]] page.}}<br />
<br />
How to partition depends on how many operating systems you want install. The following options will be explained:<br />
<br />
* Single boot: [[#Arch Linux only]]<br />
* Dual boot: [[#OS X with Arch Linux]] ''(recommended so you can still return to OS X when needed)''<br />
* Triple boot: [[#OS X, Windows XP, and Arch Linux triple boot]]<br />
<br />
----<br />
<br />
=== Arch Linux only ===<br />
<br />
This situation is the easiest to deal with. Partitioning is the same as any other hardware that Arch Linux can be installed on.<br />
<br />
The only special consideration is the MacBook firmware boot sound. To ensure that this sound is off: '''mute''' the volume in OS X before continuing further. The MacBook firmware relies on the value in OS X, if available. Note that if you choose to get rid of the OS X partition, there is no easy way to update your machine's firmware unless you use an external drive to boot OS X.<br />
You can boot in EFI mode (recommended) or bios-compatibility mode, if in doubt choose EFI.<br />
<br />
==== Option 1: EFI (recommended) ====<br />
<br />
* Run '''cgdisk''' ({{Pkg|gptfdisk}} package).<br />
* Create the necessary partitions.<br />
<br />
{{Note|<br />
* The swap partition is optional, on machines with a RAM of size 4GB or more, good performance can be expected without a swap partition. Also, a '''swap file''' can be created later, see [[Swap#Swap file|Swap file]]. You'll need a swap partition/file if you expect to [[Suspend and hibernate|Hibernate]] your machine in future.<br />
* For more information on partitioning, see [[Beginners' guide#Partitioning hard disks: General information|Partitioning hard disks: General information]].<br />
* As of Aug 2014 {{pkg|refind-efi}} has a bug that does not allow to run {{ic|refind-install}} if EFI partition is mounted to {{ic|/boot}}. Other bootloaders (like {{pkg|gummiboot}}) have no such problem and thus there is no need to split {{ic|/boot/efi}} from {{ic|/boot}}.<br />
}}<br />
Simple example (no LVM, crypto):<br />
partition mountpoint size type label<br />
/dev/sda1 /boot/efi 200MiB vfat EFI<br />
/dev/sda2 /boot 100MiB ext2 boot<br />
/dev/sda3 - adjust swap swap<br />
/dev/sda4 / 10GiB ext4 root<br />
/dev/sda5 /home remain. ext4 home<br />
<br />
* Done, you can continue to [[#Installation]]<br />
<br />
==== Option 2: BIOS-compatibility ====<br />
<br />
* Boot installation medium and switch to a free tty.<br />
* Run '''parted'''. The simplest way is to change the partition table to '''msdos''' and then partition as normal. GRUB is compatible with GPT.<br />
<br />
* Create the necessary partitions.<br />
<br />
* Done, you can continue to [[#Installation]]<br />
<br />
=== OS X with Arch Linux ===<br />
<br />
The easiest way to partition your hard drive, so that OS X and Arch Linux will co-exist, is to use partitioning tools in OS X and then finish with Arch Linux tools.<br />
<br />
{{Warning|It is highly recommended that this only be attempted after a clean install of OS X. Using these methods on a pre-existing system may have undesired results.}}<br />
<br />
'''Procedure''':<br />
* In OS X, run '''Disk Utility.app''' (located in {{ic|/Applications/Utilities}})<br />
<br />
* Select the drive to be partitioned in the left-hand column (not the partitions!). Click on the '''partition''' tab on the right.<br />
<br />
* Select the volume to be resized in the '''Volume scheme'''.<br />
<br />
* Decide how much space you wish to have for your OS X partition, and how much for Arch Linux. Remember that a typical installation of OS X requires around 15-20 GiB, depending on the number of software applications and files.<br />
<br />
* Finally, in the size box, type the smaller size to which you want to resize your OS X partition, and click '''Apply'''. This will create a new partition out of the empty space. You will delete this partition later.<br />
<br />
{{Note|If you wish to have a shared partition between OS X and Arch Linux, then additional steps will need to happen here. Please see [[#HFS partition sharing]].}}<br />
<br />
* If the above completed successfully, then you can continue. If not, then you may need to fix your partitions from within OS X first.<br />
{{Note|If you have any problems, try using the [http://gparted.sourceforge.net/ gparted] live CD (i.e. ''instead'' of using Disk Utility and/or cgdisk). It is capable of shrinking the OS X partition and creating Linux partitions ready for installation.}}<br />
<br />
* Boot the Arch installation [[Beginners'_guide#Prepare_the_latest_installation_medium|LiveCD]] or [[USB flash installation media|LiveUSB]] by holding down the alt key '''during boot'''. Follow '''one''' of the procedures below according to your choice of boot method.<br />
<br />
==== Option 1: EFI ====<br />
<br />
* Run '''cgdisk'''<br />
<br />
* Delete the partition you made in Disk Utility.app and create the necessary partitions for Arch Linux. OS X likes to see a 128 MiB gap after partitions, so when you create the first partition after the last OS X-partition, type in '''+128M''' when cgdisk asks for the first sector for the partition. A simple example (no LVM, crypto):<br />
{{Note|<br />
* The swap partition is optional on machines with 4GB of RAM or more. A '''[[Swap#Swap file|swap file]]''' can be created later.<br />
* The easiest dual-boot option is to install rEFInd from inside OS X, to its root directory (default for install.sh). Following that, copy the driver folder from the installation tarball into the new rEFInd location, and uncomment the lines "scan_all_linux_kernels" and "also_scan_dirs" options in refind.conf. Configuration of boot options can then be done from a refind_linux.conf in Arch's /boot directory.<br />
* If you want to be able to boot GRUB from the Apple boot loader, you can create a small hfs+ partition (for convenience, use OS X to format it in Disk Utility.app afterwards). Follow the GRUB EFI install procedure, and mount your {{ic|/boot/efi}} directory to the hfs+ partition you created. Finally, finish up again in OS X by blessing the partition. This will set GRUB as the default boot option (holding alt at startup goes to the mac boot options screen still. See http://mjg59.dreamwidth.org/7468.html)<br />
* OS X's EFI partition can be shared with Arch Linux, making the creation of an additional EFI partition dedicated to arch completely optional<br />
}}<br />
{{Note|For more information on partitioning, see [[Partitioning]]}}<br />
partition mountpoint size type label<br />
/dev/sda1 /boot/efi 200MiB vfat EFI<br />
/dev/sda2 - ? hfs+ OS X<br />
/dev/sda3 - ? hfs+ Recovery<br />
/dev/sda4 - 100MiB hfs+ Boot Arch Linux from the Apple boot loader (optional)<br />
/dev/sda5 /boot 100MiB boot boot<br />
/dev/sda6 - ? swap swap (optional)<br />
/dev/sda7 / 10GiB ext4 root<br />
/dev/sda8 /home remaining ext4 home<br />
<br />
* Done, you can continue to [[#Installation]]<br />
<br />
==== Option 2: BIOS-compatibility ====<br />
<br />
* Run '''parted''' as root.<br />
<br />
* Delete the empty space partition and partition the space as you would for any other installation. Note that MBR is limited to 4 primary partitions (including the efi partition). That leaves 2 primary partitions for arch. One strategy is to have a system and home partition, and use a swap file (I have not tried to use logical partitions). Another is to dedicate one partition to a shared partition (see below).<br />
<br />
* Next, create new filesystems on those partitions which need them, especially the partition which will contain /boot. If you are not sure how to do this using {{ic|mkfs.ext2}} (or whatever), run {{ic|/arch/setup}} and work through until you get to Prepare Hard Drive and use the "Manually configure block devices..." option, then exit the installer. This is necessary so that rEFIt will set the right partition type in the MBR in the next step (without an existing filesystem, it seems to ignore the partition type set by parted), without which GRUB will refuse to install to the right partition.<br />
<br />
* At this point you should reboot your computer and have rEFIt fix the partition tables on your hard drive. (If you do not do this, you may have to reinstall GRUB later on in order to have your Mac recognize the Linux partition.) When you are into the rEFIt menu, select '''update partition table''', then press Y. Reboot.<br />
<br />
* Done, you can continue with [[#Installation]].<br />
<br />
=== OS X, Windows XP, and Arch Linux triple boot ===<br />
<br />
This may not work for everyone but it has been successfully tested on a MacBook from late 2009.<br />
<br />
The easiest way to partition your hard drive, so that all these operating systems can co-exist, is to use disk utility in OS X, use the formatter on windows XP, install XP and then finish with Arch Linux tools.<br />
<br />
{{Warning|It is highly recommended that this only be attempted after a clean install of OS X. Using these methods on a pre-existing system may have undesired results. At least back your stuff up with timemachine or clonezilla before you begin.}}<br />
<br />
'''Procedure''':<br />
* In OS X, run '''Disk Utility''' (located in {{ic|/Applications/Utilities}}).<br />
<br />
* Select the drive to be partitioned in the left-hand column (not the partitions!). Click on the '''partition''' tab on the right.<br />
<br />
* Select the volume to be resized in the '''volume scheme.'''<br />
<br />
* Decide how much space you wish to have for your OS X partition, how much for XP, and how much for Arch Linux. Remember that a typical installation of OS X requires around 15-20 GiB, and XP about the same, depending on the number of software applications and files. Something like OS X 200Gb, XP 25Gb, Arch 25Gb should be fine.<br />
<br />
* Put your decisions into action by pressing the + button and adding the new partitions, Label them as you like and make sure that your XP partition is the last one on the disk and is formatted for FAT32. It is probably best to have Arch formatted in HFS format as to not confuse you later, it will be reformatted anyway.<br />
<br />
So in linux terms your partitions will be something like:<br />
<br />
:*sda (disk)<br />
:*sda1 (Mac boot partition - you cannot see this one in OS X)<br />
:*sda2 (OS X install in HFS+)<br />
:*sda3 (Arch install temporarly in HFS)<br />
:*sda4 (XP install in FAT32)<br />
<br />
* Finally, click '''apply'''. This will create a new partition out of the empty space.<br />
<br />
{{Note|Using this method you may not be able to have a shared partition between OS X and Arch Linux, this is because the mac will only allow for 4 active partitions. You will however be able to mount a HFS partition in Arch for one workaround. There are other workarounds possible also.}}<br />
<br />
* If the above completed successfully, you can continue. If not, then you may need to fix your partitions from within OS X first.<br />
<br />
* You will not be needing boot camp this way, the program rEFIt is much more flexible (though not as flexible as GRUB). Download and install rEFIt [[http://refit.sourceforge.net/]]<br />
<br />
* Go into a terminal in OS X and perform the following, this will enable the rEFIt boot manager. <br />
<br />
cd /efi/refit<br />
./enable.sh<br />
<br />
* Reboot to check the rEFIt is working, it should appear on boot. When it comes up go to the rEFIt partition manager and agree to the changes.<br />
<br />
* Put your XP install CD and boot it with rEFIt - You may have to reboot a few times until it is recognized by the boot loader. Install XP and once it is installed use the OS X installation CD to get your drivers running nicely in XP.<br />
** Note: when installing XP make sure you select your XP partition and format it again inside the XP installer. If you do not reformat it will not work.<br />
<br />
* Boot the Arch install CD, log in as root and run {{ic|# /arch/setup}}.<br />
<br />
* Follow the install as normal but note that you will have to tell that arch installer to mount sda3 as the root partition and format it as ext3, there will not be a /boot or swap partition so ignore those warnings.<br />
<br />
* At this point, if you are dual booting, you should reboot your computer and have rEFIt fix the partition tables on your hard drive. (If you do not do this, you may have to reinstall GRUB later on in order to have your Mac recognize the Linux partition.) When you are into the rEFIt menu, select '''update partition table''', then press Y.<br />
# reboot<br />
<br />
* Done! You can continue to [[#Installation]] but make sure you read [[#Booting directly from GRUB]] for the stage "* (for booting with EFI) After the install boot loader stage, exit the installer and install GRUB."<br />
<br />
== Setup bootloader ==<br />
<br />
If you are going for an Arch Linux-only setup, installing the bootloader is no different than on any other machine: Install [[gummiboot]], [[rEFInd]] or other bootloader of your choice.<br />
<br />
If, on the other hand, you are dual/triple booting, then read on.<br />
<br />
{{Out of date | Section that describes bootloader setup for dual/triple boot should be revised and re-structured into more readable way}}<br />
{{Tip|rEFIt is a popular bootloader for EFI-firmware computers (including Macs). It can be installed at any time during the installation. For instructions, please see [[#rEFIt]]. }}<br />
<br />
<br />
=== Installing GRUB to EFI partition directly ===<br />
<br />
* If you would like to use GRUB as your main bootloader and use the "boot while holding the Alt/Option key" method to go back to OS X rather than using alternatives such as rEFIt (http://refit.sourceforge.net/, mentioned previously in [[#BIOS-compatibility]] and [[#OS X, Windows XP, and Arch Linux triple boot]]) then you must install {{Pkg|grub}} to your Mac's '''already-existing''' EFI partition (see below). <br />
<br />
{{Note| These instructions are known to work on a MacBook Pro (Early 2011). Please read the procedure carefully '''as well as the details following it'''.}}<br />
<br />
{{Note| With a new MacBook Pro (Mid 2014), this procedure worked only after installing the<br />
{{Pkg|efibootmgr}} package.}}<br />
<br />
'''Procedure''':<br />
<br />
* Install {{Pkg|grub}}<br />
<br />
* Make a directory named {{ic|efi}} in {{ic|/boot}} <br />
<br />
* Mount the '''already-existing''' EFI partition on your Mac to this {{ic|/boot/efi}} directory<br />
<br />
* Install GRUB to this directory<br />
<br />
* Make a directory named {{ic| locale}} in {{ic| /boot/grub}}<br />
<br />
* Copy {{ic| grub.mo}} from {{ic| /usr/share/locale/en\@quot/LC_MESSAGES/}} to {{ic| /boot/grub/locale}} <br />
<br />
* Generate a configuration for GRUB<br />
<br />
* Done! GRUB will now start on reboot and you can boot into your newly installed Arch Linux.<br />
<br />
* Remember to hold ALT/Option key '''while''' starting your computer if you want to boot back into OS X.<br />
<br />
'''Details (quoted from [[GRUB_EFI_Examples#M5A97]]):'''<br />
<br />
Finish the standard Arch install procedures, making sure that you install {{Pkg|grub}} and partition your boot hard disk as GPT.<br />
<br />
From [[GRUB#Install_to_UEFI_system_partition]]:<br />
<br />
The UEFI system partition will need to be mounted at {{ic|/boot/efi/}} for the GRUB install script to detect it:<br />
<br />
# mkdir -p /boot/efi<br />
# mount -t vfat /dev/sdXY /boot/efi<br />
<br />
Where X is your boot hard disk and Y is the efi partition you created earlier.<br />
<br />
Install GRUB UEFI application to and its modules to {{ic|/boot/grub/x86_64-efi}} using:<br />
<br />
# modprobe dm-mod<br />
# grub-install --target=x86_64-efi --efi-directory=/boot/efi --bootloader-id=arch_grub --recheck --debug<br />
# mkdir -p /boot/grub/locale<br />
# cp /usr/share/locale/en\@quot/LC_MESSAGES/grub.mo /boot/grub/locale/en.mo<br />
<br />
Generate a configuration for GRUB<br />
<br />
# grub-mkconfig -o /boot/grub/grub.cfg<br />
<br />
=== Using blessing ===<br />
<br />
It is possible to boot directly from GRUB in EFI mode without using rEFIt through what is known as "blessing" after placing GRUB on a '''separate partition'''. These instructions are known to work on a MacBook7,1. It is advisable to host GRUB on either a FAT32 or HFS+ partition, but ext2 or ext3 may also work. GRUB's appleloader command does not currently work with the 7,1, but support can be added with the patch available [https://savannah.gnu.org/bugs/index.php?33185 here].<br />
<br />
After the GRUB install is in the desired location, the firmware needs to be instructed to boot from that location. This can be done from either an existing OS X install or an OS X install disk. The following command assumes that the GRUB install is in {{ic|/efi/grub}} on an existing OS X partition:<br />
# bless --folder /efi/grub --file /efi/grub/grub.efi<br />
<br />
=== Compilation ===<br />
<br />
Some models may need EFI_ARCH set to i386.<br />
bzr branch --revision -2 bzr://bzr.savannah.gnu.org/grub/trunk/grub grub<br />
cd grub<br />
./autogen.sh<br />
patch -p1 < appleloader_macbook_7_1.patch<br />
export EFI_ARCH=x86_64<br />
./configure --with-platform=efi --target=${EFI_ARCH} --program-prefix=""<br />
make<br />
cd grub-core<br />
../grub-mkimage -O ${EFI_ARCH}-efi -d . -o grub.efi -p "" part_gpt part_msdos ntfs ntfscomp hfsplus fat ext2 normal chain boot configfile linux multiboot<br />
cp grub.efi *.mod *.lst yourinstalllocation<br />
<br />
== Installation ==<br />
<br />
{{Note|This section is only required if you want to have OS X installed along with Arch Linux. If not, follow the steps in the official install guide, then skip to [[#Post-installation]].}}<br />
<br />
* Boot from the Arch Linux install CD, from the latest [[Archboot]] iso (unofficial), or from a [[USB flash installation media#Using manual formatting|manually created]] bootable USB drive.<br />
{{Note|<br />
* On a MacBookPro7,1, I had an error booting the installation media Version 2012.12.01: "unable to handle kernel NULL pointer dereference at 0000000000000010" during pacpi_set_dmamode. To fix this problem, boot with the option: acpi&#61;off. After chrooting, add MODULES&#61;"ata_generic" into /etc/mkinitcpio.conf and execute mkinitcpio -p linux, see: [[Installation guide#Configure_the_system|Installation Guide, 9 Configure the system]].<br />
* Some MacBook users report strange keyboard output such as long delays and character doubling. To fix this problem, boot with the following options: arch noapic irqpoll acpi&#61;force}}<br />
<br />
* Proceed through the installation as described in the [[Installation guide]] '''except''' in the following areas:<br />
** In the [[Installation guide#Prepare Hard Drive|prepare hard drive]] stage, do only the [[Installation guide#Manually configure block devices, filesystems and mountpoints|set filesystem mountpoints]] step, taking care to assign the correct partitions. Partitions have already been created if you followed [[#Partition]]<br />
** '''(for booting with EFI''') After the [[Installation guide#Install Bootloader|install boot loader]] stage, exit the installer and install [[GRUB]].<br />
** '''(for booting with BIOS-compatibility)''' In the [[Installation guide#Install Bootloader|install boot loader]] stage, edit the menu.lst file and add '''reboot=pci''' to the end of the '''kernel''' lines, for example: {{bc|1=kernel /vmlinuz26 root=/dev/sda5 ro reboot=pci}} This will allow your MacBook to reboot correctly from Arch.<br />
** '''(for booting with BIOS-compatibility)''' Also in the [[Installation guide#Install Bootloader|install boot loader]] stage, install GRUB on whatever partition that {{ic|/boot}} is on. {{Warning|Do not install GRUB onto ''/dev/sda'' !!! Doing so is likely to lead to an unstable post-environment.}}<br />
** In the [[Installation guide#Configure System|configure system]] stage, edit /etc/mkinitcpio.conf and add the '''usbinput''' hook to the '''HOOKS''' line somewhere after the '''autodetect''' hook. This will load the drivers for your keyboard in case you need to use it before Arch boots (e.g. entering a [[LUKS]] password or using the troubleshooting shell).<br />
<br />
* When the install process is complete, reboot your computer.<br />
<br />
* If using optical media, hold down the eject key as your MacBook starts, this should eject the Arch Linux install disk.<br />
<br />
* If dual-booting OS X and Arch Linux, hold down the alt (option) key while the system boots to use the Mac bootloader to select which OS to boot.<br />
<br />
== Post-installation ==<br />
<br />
{{Style|Duplicated information, does not comply with [[Help:Style]].}}<br />
<br />
See [[General recommendations]] for system management directions and post-installation tutorials like setting up a graphical user interface, sound or a touchpad.<br />
<br />
==== Video ====<br />
<br />
Different MacBook models have different graphic cards.<br />
<br />
To see which graphics card you have type:<br />
<br />
$ lspci | grep VGA<br />
<br />
* If it returns a string containing '''intel''' you only need the {{Pkg|xf86-video-intel}} driver. Intel-based MacBooks work out-of-the-box.<br />
<br />
* If it returns '''nVidia''', read [[NVIDIA]].<br />
<br />
* Otherwise if it returns '''ATI''' or '''AMD''', read [[ATI]].<br />
<br />
===== NVIDIA note =====<br />
<br />
{{Tip|MBP 6.2 - With the proprietary [[NVIDIA]] drivers, support for [[NVIDIA#Enabling Pure Video HD (VDPAU/VAAPI)|PureVideo HD]] is available for hardware video decoding. }}<br />
<br />
For MacBooks with NVIDIA graphics, for the backlight to work properly you may need the {{AUR|nvidia-bl}} package.<br />
<br />
{{Tip|<br />
* If backlight control does not work after installing nvidia-bl, you should [[Kernel modules#Blacklisting|blacklist]] apple_bl kernel module.<br />
* Alternatively, you can choose to use the {{AUR|pommed-light}} package. If you do so, you may wish to change the step settings in {{ic|/etc/pommed.conf.mactel}} to something around 5000-10000 depending on how many levels of brightness you desire. The max brightness is around 80000, so take that into account.}}<br />
<br />
===== MacBookPro5,5, NVIDIA and secondary display =====<br />
<br />
As of January 1 2011, the latest NVIDIA drivers (290.10) might not work properly when a secondary display is used (tested with TwinView), NVIDIA's current [http://www.nvnews.net/vbulletin/showthread.php?t=122606 long-live supported] 275xx drivers seem to work fine. Install {{AUR|nvidia-275xx}} and {{AUR|nvidia-utils-275xx}}, and possibly {{AUR|lib32-nvidia-utils-275xx}} if you are on x86_64 system and want 32-bits support.<br />
<br />
MacBookPro5,5 has an NVIDIA 9400m graphics card. This problem might apply to other devices as well.<br />
<br />
==== Touchpad ====<br />
<br />
The touchpad should have basic functionality by default. A true multitouch driver which behaves very similarly to native OS X is included in the {{AUR|xf86-input-multitouch-git}} package. It supports 1, 2 and 3 finger gestures, including differentiation between horizontal and vertical 3 finger swipe. Additional details are available at [http://bitmath.org/code/multitouch/ the driver's project page].<br />
<br />
xf86-input-multitouch-git does not support any sort of configuration without editing the driver's source. Some users are also experiencing issues with false clicks from palm touches. There is now a much more configurable fork available as {{AUR|xf86-input-mtrack-git}}. Configuration options are documented in the [https://github.com/BlueDragonX/xf86-input-mtrack readme].<br />
<br />
The following mtrack options work well on a MacBook7,1:<br />
<br />
Option "Thumbsize" "50"<br />
Option "ScrollDistance" "100"<br />
<br />
Probably you need also to add:<br />
<br />
MatchDevicePath "/dev/input/event10"<br />
<br />
To disable tap-to-click (that is, to press down to click) by default, add the following to your mtrack configuration section<br />
<br />
Option "TapButton1" "0" <br />
Option "TapButton2" "0"<br />
Option "TapButton3" "0"<br />
<br />
'''Natural scrolling:''' To configure natural two finger scrolling similar to [http://www.apple.com/au/osx/what-is/gestures.html#gallery-gestures-scroll OS X], refer to [[Touchpad Synaptics#Natural scrolling]]. If you are using GNOME, it will override these settings - in this case refer to [[GNOME#Natural_scrolling_touchpad]].<br />
<br />
If you are using {{AUR|xf86-input-mtrack-git}}, you can simply swap the scroll up and scroll down buttons (along with the scroll left and scroll right):<br />
{{hc|/etc/X11/xorg.conf.d/10-mtrack.conf|<br />
...<br />
Option "ScrollUpButton" "5"<br />
Option "ScrollDownButton" "4"<br />
Option "ScrollLeftButton" "7"<br />
Option "ScrollRightButton" "6"<br />
...}}<br />
<br />
<br />
'''Special Note About Older Macbook Models (confirmed on MacBook2,1):''' On older Macbook models (pre-multitouch), the touchpad will not function properly until you install the xf86-input-synaptics package. Please see [[Touchpad Synaptics]] for more information on installing and configuring this package.<br />
<br />
'''Note on MacBookPro5,5:''' I found it is much simpler to use the {{Pkg|xf86-input-synaptics}} in Extra. Although it does not have much function as 3 finger swipe, this driver provides faster response. {{Pkg|gpointing-device-settings}} also provides a simple GUI config. Below is a Xorg config file /etc/X11/xorgconfig.d/60-synaptics.conf for reference only.<br />
Section "InputClass"<br />
Identifier "touchpad catchall"<br />
Driver "synaptics"<br />
MatchIsTouchpad "on"<br />
MatchDevicePath "/dev/input/event*"<br />
Option "SHMConfig" "on"<br />
Option "TapButton1" "1"<br />
Option "TapButton2" "3"<br />
Option "TapButton3" "2"<br />
Option "PalmDetect" "on"<br />
Option "VertEdgeScroll" "off"<br />
Option "HorizEdgeScroll" "off"<br />
Option "CornerCoasting" "off"<br />
Option "EdgeMotionUseAlways" "off"<br />
Option "AreaLeftEdge" "10"<br />
Option "AreaRightEdge" "1270"<br />
EndSection<br />
'''For some users, the two-finger right-click may not work correctly and trackpad may also become less responsive after these settings. For me, removing the 'AreaLeftEdge' and 'AreaRightEdge', solved that problem.'''<br />
'''OS X like MultiTouch Gestures''' ''currently broken due to newer synaptic drivers!'' For users looking to add more of OS X's multitouch gestures to Arch, [https://github.com/iberianpig/xSwipe xSwipe] is a highly customisable, light weight perl script, which does just that. Once installed and configured (see xSwipe wiki on Github) I would recommend adding xSwipe as a [[Autostarting|start up item]].<br />
<br />
==== Keyboard ====<br />
<br />
MacBook keyboards work by default. For swaping fn keys with Fx keys see [[Apple Keyboard]].<br />
<br />
To enable it you can map with right application like '''xbindkeys''' or through DE preferences; but another very good way, that we recommend, is to install the {{AUR|pommed}} package.<br />
<br />
Edit the {{ic|/etc/pommed.conf}} according to your hardware on MacBook, building<br />
it from {{ic|/etc/pommed.conf.mac}} or {{ic|/etc/pommed.conf.ppc}} example files.<br />
<br />
Note that you can also run it without a configuration file, the defaults may work for you. Then enable and start pommed [[Systemd]] service.<br />
<br />
systemctl enable pommed<br />
systemctl start pommed<br />
<br />
{{Tip|if you are using Gnome or KDE you can easily configure ''3rd level functionality'', ''multimedia key'', etc. in Keyboard Preferences.}}<br />
<br />
{{Note|See the [[Xorg input hotplugging]] page for other configuration information.}}<br />
<br />
===== Keyboard Backlight =====<br />
<br />
The keyboard backlight is controlled by {{ic|/sys/class/leds/smc::kbd_backlight}}. Write the desired value to {{ic|brightness}} in that directory.<br />
<br />
You may also use {{AUR|kbdlight}} to control keyboard backlight though scripts or by running it via [[sxhkd]]. It has the advantage of allowing keyboard light-level changes without being root.<br />
<br />
====== NVIDIA note ======<br />
<br />
If the brightness does not function correctly through pommed, make sure you have installed the {{AUR|nvidia-bl}} package and insert<br />
<br />
find . -name "*" -exec sed -i 's/mbp_backlight/nvidia_backlight/' '{}' \;<br />
<br />
into the second line of the pommed PKGBUILD build() function and remake the package. From [https://bbs.archlinux.org/viewtopic.php?id=105091 this forum post].<br />
<br />
Another possible solution is to modify the pommed PKGBUILD build():<br />
<br />
find . -name "*" -exec sed -i 's/nvidia_backlight/apple_backlight/' '{}' \;<br />
<br />
If the previous does not work try the following,<br />
<br />
run nvidia-settings, edit the file '/etc/X11/xorg.conf' and add this line into the Device section:<br />
<br />
Option "RegistryDwords" "EnableBrightnessControl=1"<br />
Save and reboot and check backlight buttons work.<br />
More information available at [https://help.ubuntu.com/community/MacBookPro5-5/Precise#LCD Ubuntu MacBookPro5,5]<br />
<br />
=== Wi-Fi ===<br />
<br />
Different MacBook models have different wireless cards.<br />
<br />
You can easily check what card do your MacBook have by:<br />
<br />
# lspci | grep Network<br />
<br />
* If you have an Atheros card, all should work out-of-the-box.<br />
<br />
* If you have a Broadcom card, follow the [[Broadcom BCM4312]] page.<br />
<br />
* 5.0 and 6.0 generation MacBooks may have a BCM43xx, follow the instructions for the broadcom-wl driver on the [[Broadcom wireless]] page. The interfaces can swap during reboot so its best to define them in a udev rule (instructions on the [[Broadcom wireless]] page).<br />
<br />
* 8.1 generation MacBooks have BCM4331, for which support is not present in either Linux (3.0 and 3.1) or the proprietary drivers by Broadcom. There is however preliminary support for it in Linux 3.2. To run the drivers on earlier kernels, you will need to use [https://backports.wiki.kernel.org/index.php/Documentation/compat-drivers compat-drivers]<br />
<br />
{{Note|If your connection frequently drops, you may have to turn off Wi-Fi power management. If you are running [[pm-utils]], you may override wireless power management by creating an executable file {{ic|/etc/pm/wireless}} with the lines:<br />
#!/bin/sh<br />
iwconfig wlp2s0 power off<br />
}}<br />
<br />
=== Power management ===<br />
<br />
[[Powerdown]] is a very simple to set up set of scripts what will maximize your battery duration. A MacBook Air 2013 with powerdown provides abour 11 hours of light usage with just powerdown installed.<br />
All the usual [[power management]] recomendations apply as well.<br />
<br />
==== Suspend and Hibernate ====<br />
<br />
Suspending (suspend to ram) and hibernating (suspend to disk) work fine out of the box:<br />
<br />
systemctl suspend<br />
<br />
Issues were reported where the machine would "suspend immediately after resume" in certain conditions when suspending by closing the lid. This was solved by de-selecting the option "event_when_closed_battery" in gconf-editor &rarr; gnome-power-manager &rarr; actions).<br />
<br />
See [[Suspend and hibernate]] for details on how to configure hibernation. Noticably, you'll need a swap partition or file (see the mentioned article for further instructions).<br />
<br />
If after suspend laptop is woken up after few seconds, may help to disable all stuff in /proc/acpi/wakeup, exclude LID0:<br />
# echo XHC1 > /proc/acpi/wakeup<br />
$ cat /proc/acpi/wakeup<br />
Device S-state Status Sysfs node<br />
P0P2 S3 *disabled<br />
EC S3 *disabled<br />
HDEF S3 *disabled pci:0000:00:1b.0<br />
RP01 S3 *disabled pci:0000:00:1c.0<br />
RP02 S3 *disabled pci:0000:00:1c.1<br />
RP03 S3 *disabled pci:0000:00:1c.2<br />
ARPT S4 *disabled pci:0000:03:00.0<br />
RP05 S3 *disabled pci:0000:00:1c.4<br />
RP06 S3 *disabled pci:0000:00:1c.5<br />
SPIT S3 *disabled<br />
XHC1 S3 *disabled pci:0000:00:14.0<br />
ADP1 S3 *disabled<br />
LID0 S3 *enabled<br />
And for permanent disabling:<br />
$ cat /etc/udev/rules.d/90-xhc_sleep.rules <br />
<br />
# disable wake from S3 on XHC1<br />
SUBSYSTEM=="pci", KERNEL=="0000:00:14.0", ATTR{power/wakeup}="disabled"<br />
<br />
=== Light sensor ===<br />
<br />
If you want to use the built in light sensor to automatically adjust screen and keyboard backlight brightness check out<br />
'''Lighter''' [https://github.com/Janhouse/lighter] (simple perl script, easy to fine-tune) and '''Lightum''' [https://github.com/poliva/lightum] (Requires Gnome or KDE but is older and more complete than Lighter).<br />
<br />
=== Sound ===<br />
<br />
{{Tip| If using [[ALSA]], the internal speaker might not be disabled when using the headphone jack. To solve this, enable "Auto-mute" using {{ic|alsamixer}}}}<br />
<br />
First of all follow [[ALSA]] wiki page, then if something does not work correctly, continue reading this part.<br />
<br />
Edit your {{ic|/etc/modprobe.d/50-sound.conf}} or {{ic|/etc/modprobe.d/modprobe.conf}} appending this line:<br />
<br />
options snd_hda_intel model=intel-mac-auto<br />
<br />
This should automatically specify the codec in your MacBook. Alternatively, for MacBookPro5,X, you can use:<br />
<br />
options snd_hda_intel model=mb5<br />
<br />
(note that the jack output is controlled with "HP").<br />
<br />
If you have an iMac8,1, you should instead use<br />
<br />
options snd-hda-intel model=mbp3 position_fix=2<br />
<br />
You can try to specify other options, that depend on your hardware. All other possible settings are listed in Kernel Documentation, avaible online:<br />
<br />
* [http://www.kernel.org/doc/Documentation/sound/alsa/ALSA-Configuration.txt ALSA-Configuration.txt]<br />
* [http://www.kernel.org/doc/Documentation/sound/alsa/HD-Audio.txt HD-Audio.txt]<br />
* [http://www.kernel.org/doc/Documentation/sound/alsa/HD-Audio-Models.txt HD-Audio-Models.txt].}}<br />
<br />
Then, reboot.<br />
<br />
=== Bluetooth ===<br />
<br />
Bluetooth should work out-of-the box. See the article on [[Bluetooth]] to install and configure all software needed.<br />
<br />
=== Webcam ===<br />
<br />
==== iSight ====<br />
<br />
{{Note|Linux kernel from 2.6.26 includes the Linux UVC driver natively. '''MBP 6,2+ (Kernel ~2.6.37+) iSight works out of the box''' without the need to use firmware from OS X. Only use {{ic|isight-firmware-tools}} if it doesn't work normally.}}<br />
<br />
iSight webcams on MacBooks or pre 6,2 MacBook Pros (6,2 came out around 2010) require the Apple's proprietary firmware that cannot be redistributed. It must be extracted from OS X and loaded onto Arch.<br />
<br />
You will need to install {{AUR|isight-firmware-tools}} to extract the firmware. This package also includes a udev rule and ELF binary that are necessary, even once you have extracted the firmware file into {{ic|/lib/firmware/isight.fw}}, for the file to be loaded every time you boot your computer (namely {{ic|/etc/udev/rules.d/isight.rules}} which uses {{ic|/usr/lib/udev/ift-load}}).<br />
<br />
Instructions:<br />
<br />
First you need to get the firmware out of a particular file located on your OS X install. It is located in {{ic|/System/Library/Extensions/IOUSBFamily.kext/Contents/PlugIns/AppleUSBVideoSupport.kext/Contents/MacOS/AppleUSBVideoSupport}}.<br />
<br />
{{ Tip | The {{ic|AppleUSBVideoSupport}} file from a OS X 10.6 (Snow Leopard) installation may not work properly. If possible, use the file from OS X 10.5 or earlier.}}<br />
<br />
To mount the OS X drive if multi-booting:<br />
<br />
# sudo mkdir /media/OSX<br />
# sudo mount -t hfsplus /dev/sda2 /media/OSX<br />
<br />
Then, install the {{AUR|isight-firmware-tools}} package.<br />
<br />
Locate the {{ic|AppleUSBVideoSupport}} file in the OS X directory listed above. Either copy it over to your Arch system (Any OS X installation should do, such as an iMac, not just one specific to your system) or, if multi-booting, mount the OS X drive and navigate to the directory. (On 10.7 (Lion) the directory is {{ic|/System/Library/Extensions/IOUSBFamily.kext/Contents/PlugIns/AppleUSBVideoSupport.kext/Contents/MacOS}}.) In that directory you can go ahead and extract the driver:<br />
<br />
# ift-extract --apple-driver AppleUSBVideoSupport<br />
<br />
When it's done, check that the firmware has been found:<br />
<br />
# ls /lib/firmware/isight.fw<br />
<br />
Once successful, completely '''SHUTDOWN''' your Mac and start it back up again (to clear the hardware state of the webcam). Do ''not'' reboot.<br />
<br />
It should be automatically loaded at boot; if it isn't you can load the '''uvcvideo''' module [[Kernel modules|manually or load it at boot]].<br />
<br />
You can use many applications to test the webcam:<br />
<br />
* MPlayer<br />
<br />
# mplayer tv:// -tv driver=v4l2:width=320:height=240:device=/dev/video0 -fps 30<br />
<br />
* Cheese<br />
* Skype<br />
* Ekiga<br />
<br />
A simple solution to take snapshots is:<br />
<br />
# mplayer tv:// -vf screenshot<br />
<br />
and the pressing the s key to take a snapshot. Files are of the format {{ic|shot\d\d\d\d.png}} and are reported in the standard output.<br />
<br />
==== Facetime HD ====<br />
The Facetime HD webcam (included on 2013 MBAs onwards) [http://mactaris.blogspot.co.uk/2013/07/webcam-settings-20-will-support.html is no longer UVC device], and therefore, does not work out of the box. It is actually a PCIE device. While a [https://github.com/patjak/bcwc_pcie bcwc_pcie] driver is being developed, it will probably take some time before it is ready. See also [https://bugzilla.kernel.org/show_bug.cgi?id=71131 Linux bug #71131] and [https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1276811 Ubuntu bug #1276711].<br />
<br />
=== Temperature Sensors ===<br />
<br />
For reading temperature just install {{Pkg|lm_sensors}}. See the [[Lm sensors]] page for more information.<br />
<br />
=== Color Profile ===<br />
<br />
We can use color profiles from OS X.<br />
<br />
First, install the {{AUR|xcalib}} package.<br />
<br />
Second copy pre-saved color profiles placed in {{ic|/Library/ColorSync/Profiles/Displays/}} on OS X partition to {{ic|~/colorprofiles/}} for example.<br />
<br />
There are color profile files agree with in MacBook models; select the right one:<br />
<br />
* '''Color LCD-4271800.icc''' for MacBook Pro with CoreDuo CPU<br />
* '''Color LCD-4271880.icc''' for MacBook with Core2Duo<br />
* '''Color LCD-4271780.icc''' for MacBook (non-Pro) based on CoreDuo or Core2Duo.<br />
<br />
{{Tip|Also OS X allows to save current color profile from ''Displays > Color'' section of the ''Mac OS System Preferences'', in this case file is saved to {{ic|/Users/<username>/Library/ColorSync/Profiles}}.}}<br />
<br />
Finally you can activate it by running<br />
<br />
# xcalib ~/colorprofile.icc<br />
<br />
{{Note|Previous command set the color profile only for the current session; this mean that you must run it every time you login in your system. For automating it you can execute the command by '''Autostart Application''', concording with your DE (or add the command to your login manager's initialization script, e.g. /etc/gdm/Init/Default).}}<br />
<br />
=== Apple Remote ===<br />
<br />
First, to correctly install and configure the '''lirc''' software that control IR see [[LIRC]] wiki.<br />
<br />
Then make LIRC use {{ic|/dev/usb/hiddev0}} (or {{ic|/dev/hiddev0}}) by editing {{ic|/etc/conf.d/lircd}}. Here is how mine look:<br />
<br />
#<br />
# Parameters for lirc daemon<br />
#<br />
LIRC_DEVICE="/dev/usb/hiddev0"<br />
LIRC_DRIVER="macmini"<br />
LIRC_EXTRAOPTS=""<br />
LIRC_CONFIGFILE="/etc/lirc/lircd.conf"<br />
<br />
Use '''irrecord''' (available when installing '''lirc''') to create a configuration file matching your remote control signals (alternatively, you can try to use the {{ic|lircd.conf}} below):<br />
<br />
# irrecord -d /dev/usb/hiddev0 -H macmini output_conf_file<br />
<br />
Start '''lircd''' and use '''irw''' to check if it works.<br />
<br />
Example of an {{ic|/etc/lirc/lircd.conf}}:<br />
<br />
begin remote<br />
<br />
name lircd.conf.macbook<br />
bits 8<br />
eps 30<br />
aeps 100<br />
<br />
one 0 0<br />
zero 0 0<br />
pre_data_bits 24<br />
pre_data 0x87EEFD<br />
gap 211994<br />
toggle_bit_mask 0x87EEFD01<br />
<br />
begin codes<br />
Repeat 0x01<br />
Menu 0x03<br />
Play 0x05<br />
Prev 0x09<br />
Next 0x06<br />
Up 0x0A<br />
Down 0x0C<br />
end codes<br />
<br />
end remote<br />
<br />
=== HFS partition sharing ===<br />
<br />
First, install the {{AUR|hfsprogs}} package. <br />
<br />
we have to list our partitions. Use<br />
<br />
fdisk -l /dev/sda<br />
<br />
example output:<br />
<br />
# fdisk -l /dev/sda<br />
Device Boot Start End Blocks Id Type<br />
/dev/sda1 1 26 204819 ee GPT<br />
/dev/sda2 26 13602 109051903+ af Unknown<br />
/dev/sda3 * 13602 14478 7031250 83 Linux<br />
/dev/sda4 14478 14594 932832+ 82 Linux swap / Solaris<br />
<br />
As we see, the "Unknown" partition is our OS X partition, which is located in {{ic|/dev/sda2}}.<br />
<br />
Create a "mac" folder in /media:<br />
<br />
# mkdir /media/mac<br />
<br />
Add at the end of ''/etc/fstab'' this line:<br />
<br />
/dev/sda2 /media/mac hfsplus auto,user,rw,exec 0 0<br />
<br />
Mount it :<br />
<br />
mount /media/mac<br />
<br />
and check it:<br />
<br />
ls /media/mac<br />
<br />
=== HFS+ Partitions ===<br />
<br />
HFS+ partitions, now the default in OS X, are not fully supported by Linux and are mounted as read-only by default. In order to write to an HFS+ partition, it is necessary to disable journaling. This can be accomplished using the OS X Disk Utility. Refer to this [http://support.apple.com/kb/ht2355 Apple support page] for more information.<br />
<br />
===Home Sharing===<br />
<br />
'''''UID Synchronization'''''<br />
<br />
==== In OS X ====<br />
<br />
{{Note|It is strongly recommended that UID/GID manipulation be done immediately after a new user account is created, in OS X as well as in Arch Linux. If you installed OS X from scratch, then this operation is guaranteed to work after logging into your account for the first time.}}<br />
<br />
===== Step 1: change UID and GID(s) =====<br />
<br />
'''''Pre-Leopard'''''<br />
<br />
# Open '''NetInfo Manager''' located in the ''/Applications/Utilities'' folder.<br />
# If not done for you already, enable access to user account transactions by clicking on the closed lock at the bottom of the window, and entering your account password, or root password if you have created a root account.<br />
# Navigate to ''/users/<new user name>'' where <new user name> is the name of the account that will have read/write access to the folder that will be shared with the primary user in Arch.<br />
# Change the '''UID''' value to 1000 (the value used by default for first user created in Arch).<br />
# Also change the '''GID''' value to 1000 (the value used by default for user account creation in Arch).<br />
# Navigate to {{ic|/groups/<new user name>}}, automatically saving the changes you have made so far.<br />
<br />
{{Note|If you get an error message that the transaction is not allowed, log out and log back in.}}<br />
<br />
'''''Leopard'''''<br />
<br />
In Leopard, the '''NetInfo Manager''' application is not present. A different set of steps is required for UID synchronization:<br />
<br />
# Open '''System Preferences'''.<br />
# Click on '''Users & Groups'''.<br />
# Unlock the pane if not already done so.<br />
# Right-click on the desired user and select '''Advanced Options'''.<br />
# Write down the value of the '''User ID''' field, you will need it later on. Change both the UID and GID to match the UID and GID of the account wished to be shared with in Arch (1000 by default for the first user created in Arch).<br />
<br />
===== Step 2: change "Home" permissions =====<br />
<br />
# Open up '''Terminal''' in the {{ic|/Applications/Utilities}} folder.<br />
<br />
# Enter the following command to reclaim the permission settings of your home folder, replacing <your user name>, <your user group> and <your old UID> with the user name whose UID and GID values you just changed, the group name whose GID value you just changed and the old UID number, respectively.<br />
<br />
# find /User/<your user name> -user <your old UID> -exec chown <your user name>:<your user group> {} \;<br />
<br />
==== In Arch ====<br />
<br />
To synchronize your UID in Arch Linux, you are advised to perform this operation ''while creating a new user account''.<br />
It is therefore recommended that you do this as soon as you install Arch Linux.<br />
<br />
Now you must substitute Arch's home with OS X's home, by modify entries of {{ic|/etc/fstab}}.<br />
<br />
=== Avoid long EFI wait before booting ===<br />
<br />
If your MacBook spends 30 seconds with "white screen" before booting you need to tell the firmware where is the booting partition.<br />
<br />
Boot OS X, if do not have it installed, you can use the install DVD (select language, then click Utilities->Terminal), or another MacBook with OS X (connect the two computers via firewire or thunderbolt, start the other MacBook keeping pressed T, boot your MacBook keeping pressed Options).<br />
<br />
Either way, once you got a OS X terminal running on your MacBook you need to execute, as root, a different command if the boot partition is EFI or it is not:<br />
<br />
# bless --device /dev/disk0s1 --setBoot # if the booting partition is EFI<br />
or<br />
# bless --device /dev/disk0s1 --setBoot --legacy # if the booting partition is not EFI<br />
<br />
(given that if your GRUB or EFI is on sda1, /dev/disk1s2 if it is on sdb2, etc). See also https://bbs.archlinux.org/viewtopic.php?pid=833215 and https://support.apple.com/kb/HT1533 .<br />
<br />
=== Mute startup chime ===<br />
<br />
If you forgot to mute before installing, you can still mute again if you have a OS X install disk. Boot from it, select language, then click ''Utilities > Terminal'', and enter<br />
<br />
# /usr/sbin/nvram SystemAudioVolume=%01<br />
<br />
(or whatever volume you want).<br />
<br />
=== kworker using high CPU ===<br />
Sometime with the addition of Yosemite, some users found that kworker CPU usage will spike, as disccused [https://bbs.archlinux.org/viewtopic.php?id=171883&p=11 here]. This is sometimes the result of runaway ACPI interrupts.<br />
<br />
To check and see, you can count the number of recent ACPI interrupts and see if any of them are out of control.<br />
<br />
grep . -r /sys/firmware/acpi/interrupts/<br />
<br />
<br />
If you see that one particular interrupt is out of control (possibly GPE66), i.e., registering hundreds of thousands of lines, you can try disabling it (replace XX with the runaway interrupt):<br />
<br />
<br />
echo "disable" > /sys/firmware/acpi/interrupts/gpeXX<br />
<br />
<br />
Disabling random ACPI interrupts could cause all kinds of problems, so do this at your own risk. If this fixes the problem, there is discussion about how to make a systemd service that automatically disables an interrupt at every boot [https://bbs.archlinux.org/viewtopic.php?pid=1488371 here].<br />
<br />
== rEFIt ==<br />
<br />
{{Note|<br />
* You probably want to have a look at [http://www.rodsbooks.com/refind/ rEFInd], which is some type of successor of rEFIt.<br />
* This is not a requirement. It only gives you a menu to choose between OS X and Arch Linux upon every boot.<br />
}}<br />
<br />
For more see, [http://refit.sourceforge.net/myths/ rEFIt myths].<br />
<br />
In OS X, download the ".dmg" from [http://refit.sourceforge.net/ rEFIt Homepage] and install it.<br />
<br />
{{Note|If you have already partitioned your hard disk in preparation for the Arch installation, rEFIt may not be enabled by default. You will have to run the "enable.sh" script installed in /efi/refit/.}}<br />
<br />
Open up '''Terminal''' and enter:<br />
<br />
cd /efi/refit;<br />
./enable.sh<br />
<br />
=== Problems with rEFIt ===<br />
<br />
If you experience problems after the install of Arch or rEFIt, especially is the right OS is not showing up to boot to or if it dumps you at a GRUB prompt stuck like the following:<br />
<br />
GRUB>_<br />
<br />
Then have a look at this link:<br />
<br />
http://mac.linux.be/content/problems-refit-and-grub-after-installation<br />
<br />
It can give you a basic idea on how to boot off the Arch live cd, mount the problem Arch install, chroot, use gptsync, and reinstall GRUB. This is probably for more advanced users who can translate the commands from a debian system to an Arch system and also apply it to the partitions on their machine. Be careful not to install GRUB in the wrong spot.<br />
<br />
If you need a copy of gptsync you can wget it from here:<br />
http://packages.debian.org/sid/gptsync<br />
or try these, for 64 bit:<br />
<br />
wget http://ftp.us.debian.org/debian/pool/main/r/refit/gptsync_0.14-2_amd64.deb<br />
<br />
and for i386:<br />
<br />
wget http://ftp.us.debian.org/debian/pool/main/r/refit/gptsync_0.14-2_i386.deb<br />
<br />
since they are .deb packages you will need the program {{AUR|deb2targz}}.<br />
<br />
==== Mavericks upgrade breaks Arch boot option ====<br />
For some multi-boot users who utilize a separate Linux boot partition, the OS X Mavericks upgrade may overwrite the boot partition with Apple's own recovery boot filesystem. This breaks the Arch Linux boot option in rEFIt/rEFInd. The best way to proceed in this situation is to abandon a separate boot partition and use the EFI system partition (ESP) to install the bootloader of your choice. It is also recommended that you use rEFInd instead of rEFIt as development on the latter has halted.<br />
<br />
Assuming grub2 as the bootloader:<br />
<br />
Use the Arch LiveCD to boot to a shell and [[Change root|chroot]] to your broken Arch Linux environment.<br />
<br />
Mount the ESP on /boot.<br />
<br />
Edit the fstab and remove the old boot partition and make ESP the new boot partition. Now mount the ESP as the new /boot parition.<br />
# mount -a<br />
<br />
[[install|Reinstall]] {{pkg|linux}}.<br />
<br />
Create a new initramfs and vmlinuz in /boot.<br />
# mkinitcpio -p linux<br />
<br />
Install grub.<br />
# grub-install --target=x86_64-efi --efi-directory=/boot --bootloader-id=grub --recheck --debug<br />
<br />
Create a new grub.cfg file.<br />
# grub-mkconfig -o /boot/EFI/grub/grub.cfg<br />
<br />
Make sure that grub.cfg is in the same directory as grubx64.efi.<br />
<br />
Generate a new refind_linux.conf file in /boot simply by running mkrlconf.sh which comes with rEFInd.<br />
<br />
Exit the chroot environment.<br />
<br />
Reboot. You should see a new entry for Arch Linux in rEFInd and it should boot to your Arch Linux installation.<br />
<br />
== Model-specific information ==<br />
<br />
=== MacBook ===<br />
<br />
==== Mid 2007 13" - Version 2,1 ====<br />
<br />
{{Note|I used the 201212 ISO image.}}<br />
<br />
Since older Macbooks have a 32bit EFI running, the usual installation image is not recognized. You need to either remove the UEFI support from the disc ([[Unified_Extensible_Firmware_Interface#Remove_UEFI_boot_support_from_ISO]]) or build a 32bit EFI version of the disc. The paragraphs below will take the first path to success, booting into BIOS mode and its pitfalls. For a try the other way round, read [[Unified_Extensible_Firmware_Interface#Create_UEFI_bootable_USB_from_ISO]] first.<br />
<br />
First prepare your harddisc according to your wishes. In this scenario it was a "Linux only" approach with<br />
<br />
/dev/sda1 HFS+ AF00 200M -> EFI boot system on Apple HFS+ partition<br />
/dev/sda2 ext4 8300 147G -> arch system<br />
/dev/sda3 swap 8200 1G -> swap<br />
<br />
The {{AUR|hfsprogs}} package contains the tools to handle HFS/HFS+ filesystems. The rEFInd bootloader recognizes it on its own. Usually the partition for the EFI bootloader is a FAT32 (vfat) partition. In this case I tried rEFIt first, which apparently needs the HFS+ filesystem to work, and kept it at that.<br />
<br />
The mount points are:<br />
<br />
/dev/sda2 -> /<br />
/dev/sda1 -> /boot/EFI<br />
<br />
The bootloader in use was [http://www.rodsbooks.com/refind/index.html rEFInd] instead of rEFIt. To install it, the rEFInd homepage provides a good guide. Usually it is simply done by copying rEFInd:<br />
<br />
cp -vr /usr/share/refind/drivers_ia32 /boot/EFI/refind/<br />
cp -vr /usr/share/refind/tools_ia32 /boot/EFI/refind/<br />
cp -vr /usr/share/refind/fonts /boot/EFI/refind/<br />
cp -vr /usr/share/refind/icons /boot/EFI/refind/<br />
cp -v /usr/share/refind/refind_ia32.efi /boot/EFI/refind/<br />
cp -v /usr/share/refind/refind.conf-sample /boot/EFI/refind/refind.conf<br />
cp -v /usr/share/refind/refind_linux.conf-sample /boot/refind_linux.conf<br />
<br />
{{Note|I'm using the 32bit version of Arch and refind, since the EFI of the old MacBooks is 32bit. I'm not sure about 32bit rEFInd booting a 64bit Arch...}}<br />
<br />
The pitfall here is, that the system bootet in BIOS compatibility mode and not in EFI mode. You cannot therefore use {{ic|efibootmgr}}, because the EFI variables (even with 'modprobe efivars') are not available. While installing the system get {{AUR|mactel-boot}}. The {{ic|hfs-bless}} utility comes in handy, when blessing the EFI bootloader. This is done by calling:<br />
<br />
hfs-bless /boot/EFI/refind/refind_ia32.efi<br />
<br />
Since the Linux kernel does come with EFI stub enabled, it seems a good idea to run it through a bootloader first. Especially if it runs not out of the box. But using rEFInd makes GRUB (or any other bootloader) obsolete, because of that.<br />
<br />
{{Note|In the refind_linux.conf you add any kernel option you may want as long as you use the EFI stub of your kernel. In refind.conf you adjust your needs for the bootloader itself, like menu entries. If you use them (menu entries), rEFInd should not look for these EFI stub kernels itself, so blacklist the directories used in here, like {{ic|/boot/}}.}}<br />
<br />
Not running out of the box is unfortunately the initial stage for the kernel. Since we installed it in BIOS mode, two modules are missing to grant access to the root partition while booting. Hence the 'initfsram-linux.img' can not be found/loaded. Adding the following modules to your 'MODULES' line in {{ic|/etc/mkinitcpio.conf}} solved this ([https://bbs.archlinux.org/viewtopic.php?pid=1139226#p1139226 original post]).<br />
<br />
{{hc|/etc/mkinitcpio.conf|2=<br />
MODULES="ahci sd_mod"}}<br />
<br />
Rebuild your kernel image:<br />
<br />
mkinitcpio -p linux<br />
<br />
The bootloader rEFInd can scan kernels even out of the '/boot/...' directory and assumes an efi kernel even without the extension '.efi'. If you do not want to try out special kernels, this should work without the hassle to copy each kernel after building to some spot special.<br />
<br />
If you happen to get multiple entries for one boot image, it often results of a previous installation of a bootloader within the MBR. To remove that, try the following - taken from the [http://ubuntuforums.org/showpost.php?p=7828260&postcount=4 original post]. This is valid for GPT partitioned discs, so please check your environment and save your MBR first.<br />
<br />
# dd if=/dev/zero of=/dev/sda bs=440 count=1<br />
<br />
=== MacBook Pro with Retina display ===<br />
<br />
==== Early 2015 13"/15" - Version 12,x/11,4+ ====<br />
<br />
===== Wireless =====<br />
The {{ic|brcmfmac}} driver is working (with some issues) out of the box as of 2015-05-02.<br />
* 5GHz is not working. <br />
* There seems to be an issue which might lead to a kernel oops. See [http://www.spinics.net/lists/linux-wireless/msg136089.html this] post for further information and a patch which seems to fix the problem.<br />
<br />
For Network controller BCM43602 14e4:43ba "Broadcom Corporation Device 43ba (rev 01)", on MacBookPro12,1 (output of "sudo dmidecode -s system-product-name"):<br />
* you need kernel version 3.17+ See [http://linuxwireless.org/en/users/Drivers/brcm80211/ this driver page] for details.<br />
* the kernel oops / freeze appeared - [http://www.spinics.net/lists/linux-wireless/msg136092.html This patch] does fix (as far as I could test) the problem that appeared when running multiple (in my case 6) concurrent downloads. This patch is included in kernel 4.0.5 :)<br />
* But there is another problem : Even with the patch applied I ran in another kernel (version 4.0.4) freeze triggered by the brcmfmac kernel module ! - See [https://bugzilla.kernel.org/show_bug.cgi?id=100201 this new bug report].<br />
<br />
===== Bluetooth =====<br />
Bluetooth is not working at all.<br />
# hcitool dev<br />
Devices:<br />
<br />
===== Keyboard & Touchpad =====<br />
Haptic feedback works out of the box due to the touchpad's built-in firmware. (Kernels 4.2-rc4 and lower)<br />
<br />
{{Pkg|xf86-input-synaptics}} may be required on 4.1.x kernels for the touchpad to consistently work.<br />
<br />
As of 4.2-rc5 the touchpad and keyboard are working. You can install this kernel version using the {{AUR|linux-mainline}} package.<br />
<br />
The following config is necessary on some systems to enable trackpad movement on 4.2+ kernels, avoiding an issue where the trackpad could be clicked, but the cursor could not be moved:<br />
<br />
# /etc/X11/xorg.conf.d/60-magictrackpad.conf<br />
Section "InputClass"<br />
Identifier "Trackpad"<br />
Driver "synaptics"<br />
MatchIsTouchpad "on"<br />
MatchDevicePath "/dev/input/event*"<br />
EndSection<br />
<br />
Further, some US/ANSI keyboards suffer from an issue where the tilde key (~, the key vertically between Esc and Tab) registers as < and >. The following config file fixes this issue.<br />
<br />
# /etc/modprobe.d/hid_apple.conf<br />
options hid_apple iso_layout=0<br />
<br />
See [https://bugzilla.kernel.org/show_bug.cgi?id=96771 this kernel bugzilla] for more details and the relevant patches for earlier kernels.<br />
<br />
===== Graphics =====<br />
For Dual Graphics see [https://wiki.archlinux.org/index.php/MacBookPro11,x#Graphics this section on the 11,3] <br />
For 8086:162b "Intel Corporation Broadwell-U Integrated Graphics (rev 09)" intel Iris 6100, on MacBookPro12,1 (output of "sudo dmidecode -s system-product-name"):<br />
* acpi_backlight=vendor with kernel 4.0.4 as kernel parameter is necessary to help fixing fn+brightness not working in X because of a ghost acpi_video0 brightness control. See [https://bugzilla.kernel.org/show_bug.cgi?id=100041 this bug report], and [https://bugzilla.kernel.org/attachment.cgi?id=180311&action=diff&context=patch&collapsed=&headers=1&format=raw the working patch], confirmed with kernel 4.0.4, that fixes the problem (so no acpi_backlight=vendor or video.* parameter necessary anymore to get working brightness controls).<br />
* i915.enable_ips=0 is necessary with kernel 4.0.4, as there is an issue on my MacBookPro12,1 (output of "sudo dmidecode -s system-product-name") that makes the ips - intermediate pixel storage (primarily a power saving feature) cause /flickering/ when mode switching (to tty console).<br />
* i915.lvds_downclock=1 ("Use panel (LVDS/eDP) downclocking for power savings"), sounds good, but will do nothing (because the MacBook has an eDP panel! there's a new auto-downclock for edp in recent kernels and it's enabled by default if available :), so don't use - recommendation from irc #intel-gfx)<br />
* intel_iommu=igfx_off if you have GPU HANG: ecode 8:0:0x85dffffb, in X [1737] blah-blah in dmesg and GPU hangs all the time during OpenGL operations.<br />
<br />
===== Apple Thunderbolt 2 Gigabit Ethernet Dongle =====<br />
on MacBookPro12,1 (output of "sudo dmidecode -s system-product-name") there is a bug (in kernel 4.0.4) in that the dongle gets recognized on boot-up and works just fine, but is not hotpluggable, so when unplugged it does not get recognized / work at all when replugged. See the following kernel bug report: https://bugzilla.kernel.org/show_bug.cgi?id=100191<br />
<br />
==== 2012 - 2014 models ====<br />
<br />
* [[MacBookPro11,x]] (Late 2013—Mid 2014)<br />
* [[MacBookPro10,x]] (Mid 2012—Early 2013)<br />
<br />
=== MacBook Air===<br />
<br />
==== Early 2014 11" - Version 6,1 ====<br />
This is almost the same as the 2013 version, where the only known difference is a slightly faster processor. The version numbers have not been changed since the 2013 version.<br />
<br />
It works excellently after following the instructions for the MBA 2013 13" here and in the forum thread.<br />
Bluetooth, which has been reported not working for some people with the 2013 version, works without trouble for the 2014 version, although it should be excactly the same.<br />
<br />
{{Note| Unless you have a local repository on a USB disk, you need a USB to ethernet adaptor or a USB wireless adaptor supported natively by the kernel to easily install Arch Linux, since you have to install the {{AUR|broadcom-wl-dkms}} package to make the internal wireless adaptor work.}}<br />
<br />
Unresolved issues:<br />
<br />
- There is no driver for the webcam yet.<br />
- rEFInd uses 30 seconds to start booting. Using the bless trick stops rEFInd from loading, and it has to be re-installed.<br />
<br />
==== Mid 2013 13" - Version 6,2 ====<br />
[https://bbs.archlinux.org/viewtopic.php?id=165899 Dedicated forum thread]<br />
===== Installing and booting =====<br />
Booting from a normal 2013.6 USB key works fine, but I could not seem to get either GRUB or Syslinux working.<br />
<br />
I was able to boot by first installing Arch Linux following the MacBook guide at the wiki (having a separate FAT32 /boot partition). Skip the bootloader installation. <br />
<br />
Installing [http://www.rodsbooks.com/refind/getting.html rEFInd] from OS X (important!) and installing the EFI stub loader made me able to boot fine.<br />
<br />
[https://bbs.archlinux.org/viewtopic.php?id=165710 Dedicated thread].<br />
<br />
{{Note| Installing [http://www.rodsbooks.com/refind/getting.html rEFInd] from Linux (or from OS X, but to the esp) also works fine}}<br />
<br />
===== Arch Only Installation =====<br />
This method works without rEFInd and uses grub to boot EFI. Partition as follows:<br />
<br />
/dev/sda1 200M Microsoft basic data<br />
/dev/sda2 256M Linux filesystem<br />
/dev/sda3 4G Linux swap<br />
/dev/sda4 108.6G Linux filesystem<br />
<br />
sda1 can also be a HFS+ partition for EFI. This example chooses to use FAT32 (vfat). Although swap is optional, it is required for hibernation. Instead of sda4 for root and home, an alternative partition scheme would be to make sda4 as root and sda5 as home.<br />
<br />
Format and mount:<br />
<br />
mkfs.vfat -F 32 /dev/sda1<br />
mkfs.ext2 /dev/sda2<br />
mkswap /dev/sda3<br />
swapon /dev/sda3<br />
mkfs.ext4 /dev/sda4<br />
<br />
mount /dev/sda4 /mnt<br />
mkdir /mnt/boot<br />
mount/dev/sda2 /mnt/boot<br />
mkdir /mnt/boot/efi<br />
mount /dev/sda1 /mnt/boot/efi<br />
<br />
Finish the installation according to the [https://wiki.archlinux.org/index.php/Beginners'_Guide#Select_a_mirror Beginner's Guide] and skip anything after the bootloader. After you have generated your initramfs and set root passwd follow below to setup grub:<br />
<br />
pacman -S grub efibootmgr<br />
mount -t efivarfs efivarfs /sys/firmware/efi/efivars<br />
grub-install --target=x86_64-efi --efi-directory=/boot/efi --bootloader-id=grub --recheck --debug<br />
grub-mkconfig -o /boot/efi/EFI/grub/grub.cfg<br />
cp /boot/efi/EFI/grub/grub.cfg /boot/grub/grub.cfg<br />
cp /boot/efi/EFI/grub/grubx64.efi /boot/efi/EFI/boot/bootx64.efi}}<br />
<br />
Now you can exit/unmount/reboot:<br />
exit<br />
umount -R /mnt<br />
reboot<br />
<br />
===== Stability problems =====<br />
{{Note| Passing {{ic|<nowiki>libata.force=1:noncq</nowiki>}} to the kernel parameters solves the problem.}}<br />
This is the big worry for me. Every now and then my system hangs for a brief moment and everything involving net or disk access just hangs there for a while and then it seems to work. <br />
So far it only seems to happen when I run something disk- or CPU-intensive. Also had an occassion when I could not start X and just got this repeating all over my screen:<br />
<br />
ata1.00: failed command: WRITE FPDMA QUEUED<br />
ata1.00: cmd 61/08:f0:10:8c:c2/00:00:0b:00:00/40 tag 30 ncq 4096 out<br />
res 40/00:00:00:00:00/00:00:00:00:00/00 Emask 0x4 (timeout)<br />
ata1.00: status: { DRDY }<br />
<br />
On the next attempt it worked fine.<br />
I did SMART short and long tests on my disk and they returned fine:<br />
<br />
[http://pastebin.com/vRE4T2Ld smartctl -a]<br />
<br />
There are some messages in my boot that indicate this could be disk and/or ACPI related.<br />
<br />
These are with 2013-06 ISO, 3.9.7-1 2013 x86_64 kernel.<br />
<br />
[http://pastebin.com/mjTJaPFa journalctl -b]<br />
Seems to only work with the headphone jack, not with the speakers.<br />
<br />
[http://pastebin.com/SdAcHuKh dmesg]<br />
<br />
===== Marvell ATA suspend bugs =====<br />
If you have 2013 MacBook Air with a Marvell 128 gig drive, you might get the following ata errors instead after pm-suspend/resumes:<br />
<br />
ata1: exception Emask 0x10 SAct 0x0 SErr 0x10000 action 0xe frozen<br />
ata1: irq_stat 0x00400000, PHY RDY changed<br />
ata1: SError: { PHYRdyChg }<br />
ata1: hard resetting link<br />
ata1: SATA link up 1.5 Gbps (SStatus 113 SControl 310)<br />
ata1.00: unexpected _GTF length (8)<br />
ata1.00: unexpected _GTF length (8)<br />
ata1.00: configured for UDMA/33<br />
ata1: EH complete<br />
<br />
Try what Patrick and Tejun figured out on the [https://bugzilla.kernel.org/show_bug.cgi?id=62351 linux bug]. I followed what Patrick describes with sata_alpm, and I haven't seen the issue since.<br />
<br />
===== Suspend/Resume =====<br />
Brightness is either 0% or 100% after resuming from suspend. Until the kernel is fixed, use patjak's fix by installing {{AUR|mba6x_bl-dkms}}. Patjak's github is at [https://github.com/patjak/mba6x_bl].<br />
===== WiFi =====<br />
WiFi does not work out of the box. Install {{AUR|broadcom-wl-dkms}} to connect to a network. <br />
<br />
===== Touchpad =====<br />
Since 3.10.3 kernel touchpad works perfectly with {{Pkg|xf86-input-synaptics}}.<br />
<br />
===== Audio =====<br />
As of Linux 3.12, sound works out of the box. If you do not get sound with only {{pkg|alsa-utils}}, you may need to create a /etc/asound.conf with below entries:<br />
<br />
defaults.pcm.card 1<br />
defaults.pcm.device 0<br />
defaults.ctl.card 1<br />
<br />
==== Mid 2012 13" — version 5,2 ====<br />
<br />
Kernel panics using default boot media under arch kernel 3.5. Adding <code>intremap=off</code> fixes this. Additionally, there are problems loading the <code>applesmc</code> module (meaning the temperature sensors, fan, and keyboard backlight do not work). These problems are fixed in the linux 3.6-rc4 mainline kernel (I have tested).<br />
<br />
==== Mid 2012 11.5" — Version 5,1 ====<br />
<br />
If you have issues with waking from sleep while in X11 such as a black screen or showing the console with a frozen mouse cursor then remove {{Pkg|xf86-input-synaptics}} and install {{AUR|mtrack-git}}. This fixed errors such as <br />
(EE) [dix] bcm5974: unable to find touch point 0<br />
and backtraces that causes X11 to crash. This might apply to Version 5,2 assuming they use the same trackpad.<br />
<br />
===== Installing using the Archboot 2012.06 image =====<br />
<br />
Several people have reported problems installing Arch Linux on the MBA version 5,2 (See [https://bbs.archlinux.org/viewtopic.php?id=144089 problems booting Arch Linux on a MacBook Air Mid 2012]). A common problem is that the screen is not detected and therefore goes black when the installer boots. To fix this problem one has to select the normal install (Not the LTS) during boot and press tab to edit the boot flags. Then add noapic flag to the boot line. This should fix the screen going black. Install the system as you normally would. It may help later in the configuration process if the support packages are installed already at this stage.<br />
<br />
When the install has finished again add the noapic flag to the GRUB boot line (if you use GRUB) and also add i915.diescreaming=1 (or perhaps i915.die). This should keep the screen from going black when booting the new system. After you enter the system the wireless driver should be loaded. If you installed the support packages during installation you should have the wifi-menu command. Run it and select the network you want to use. One could also use wpa_supplicant but wifi-menu is quite fast to use at this stage. Now you are ready to upgrade the system. As of writing there have been a lot of changes to Arch Linux since the 2012.06 image of Archboot was released ([https://www.archlinux.org/news/filesystem-upgrade-manual-intervention-required-1/ filesystem] and [https://www.archlinux.org/news/the-lib-directory-becomes-a-symlink/ glibc]). Therefore the upgrade process can be a bit difficult. The current solution has sucessfully upgraded a standart archboot version to a up-to-date version as of October 2012 and this step should be obsolete in future releases of archboot.<br />
<br />
First ignore the new "big" changes to Arch Linux,<br />
<br />
pacman -Suy --ignore glibc,libarchive,curl,filesystem <br />
<br />
If this only upgrades pacman then run the command again. Remember to make sure that pacman is ignoring the packages you do not want upgraded now. Otherwise you may break the system and have to reinstall! Now upgrade to the new filesystem,<br />
<br />
pacman -S filesystem --force<br />
<br />
As described in [[DeveloperWiki:usrlib|Glibc upgrade guide]] there may be conflicts with installed packages that require the /lib directory. Follow the guide and remove any packages that use /lib. The stock 3.4.2 kernel from Archboot should be on this list so first upgrade this,<br />
<br />
pacman -S linux<br />
<br />
This may give some errors saying that the system may not boot because of missing modules. Ignore this warning for now. The stock install may also contain gcc in the /lib directory so also remove this if needed and any other packages that have conflicts. Now Glibc should be the only package in /lib so run the system upgrade and accept all changes, <br />
<br />
pacman -Su<br />
<br />
Finally reinstall the kernel so that it can find the correct modules.<br />
<br />
Now this command should not give any errors like last time. You can also reinstall gcc at this point. After a rebooted the system should startup and the new kernel should have fixed the problem with the screen going black. If want to boot Xorg then you may need to remove the i915.diescreaming=1 line from GRUB. If not then attach a external screen and try to fix the problem that way. Some people have reported commands that may help on the [https://bbs.archlinux.org/viewtopic.php?id=144089 forum].<br />
<br />
==== Mid 2011 — version 4,x ====<br />
<br />
Works out-of-the-box since kernel 3.2. It is recommended to use [[Archboot]], install [[GRUB]] and use EFI.<br />
<br />
==== Early 2008 — version 1,1 ====<br />
<br />
Everything works out of the box though you will need {{Pkg|b43-fwcutter}} (or simply {{AUR|b43-firmware}}) for the wireless adapter to work.<br />
<br />
Since this model has only one USB port, you may find it easiest to install Arch with a powered USB hub. Plug a USB network adapter (wireless or ethernet adapter to plug into a USB port) and your Arch installation media into the USB hub.<br />
<br />
If you can't get any result by scanning wireless network after boot, unload modules <code>b43</code> and <code>ssb</code> and load them again:<br />
<br />
rmmod ssb<br />
rmmod b43<br />
modprobe b43<br />
<br />
There is a good chance you will find what's wrong with DMA from the dmesg log.<br />
<br />
Even if you can scan wireless networks after reloading the modules, it's still possible that you will only be able to connect to some networks, but not all of them. According to a more detailed discussion here: http://crunchbang.org/forums/viewtopic.php?id=17368, adding <code>pio=1,qos=0</code> options to the b43 module can solve this problem.<br />
<br />
I tested this for a 13' MacBookAir1,1 with a BCM4321 chipset, and it works.<br />
<br />
== See also ==<br />
* '''MacBook Air'''<br />
** [http://dabase.com/blog/Macbook_Air_Early_2014_Archlinux/ Macbook Air Early 2014 — dabase.com]<br />
** [http://www.frankshin.com/installing-archlinux-on-macbook-air-2013/ Installing Archlinux on Macbook Air 2013 — Frank Shin]<br />
** [http://blog.panks.me/posts/2013/06/arch-linux-installation-with-os-x-on-macbook-air-dual-boot/ Arch Linux Installation with OS X on Macbook Air (Dual Boot) — Pankaj Kumar]<br />
** [http://ryangehrig.com/index.php/arch-linux-on-macbook-air-2013/ Arch Linux – MacBook Air 2013 — Ryan Gehrig]<br />
** [http://www.nico.schottelius.org/blog/macbook-air-42-archlinux/ Installing Linux on a Macbook Air (4,2) — Nico Schottelius]<br />
** [http://www.dm9.se/?p=398 Arch linux single, pure efi boot on the macbook air3,1/3,2 — DIMENSION9]<br />
* '''MacBook Pro'''<br />
** http://www.netsoc.tcd.ie/~theorie/interblag/2010/01/30/installing-arch-linux-on-a-mac-pro/<br />
** http://allanmcrae.com/2010/04/installing-arch-on-a-macbook-pro-5-5/<br />
** http://allanmcrae.com/2012/04/installing-arch-on-a-macbook-pro-8-1/<br />
** http://linux-junky.blogspot.com/2011/08/triple-boot-archlinux-windows-7-and-mac.html</div>Justin8https://wiki.archlinux.org/index.php?title=Pacman/Rosetta&diff=376673Pacman/Rosetta2015-06-01T05:48:36Z<p>Justin8: Replaced Rc with Rs as Rc will remove all packages that depend on the one in question, instead of its dependencies which most people will want. Also to conform with what the other package manager's do with their commands.</p>
<hr />
<div>[[Category:Package management]]<br />
[[es:Pacman Rosetta]]<br />
[[ja:Pacman 比較表]]<br />
[[sr:Pacman Rosetta]]<br />
[[zh-CN:Pacman Rosetta]]<br />
This page pulls heavily from [http://old-en.opensuse.org/Software_Management_Command_Line_Comparison openSUSE's Software Management Command Line Comparison]. It has been simplified and has added Arch to the comparison, as well as modified the order in which each distribution exists for the benefit of Arch users.<br />
<br />
{{Tip|Arch users having to temporarily deal with another Linux distribution can use [https://github.com/icy/pacapt pacapt], a simple wrapper around other package managers.}}<br />
<br />
{{Note|<br />
* Some of the tools described here are specific to a certain version of pacman. The -Qk option is new in pacman 4.1.<br />
* The command {{ic|pkgfile}} can be found in the {{Pkg|pkgfile}} package.<br />
}}<br />
{|<br />
| align="center" style="background:#f0f0f0;"|'''<font color="#707070">Action</font>'''<br />
| align="center" style="background:#f0f0f0;"|'''Arch'''<br />
| align="center" style="background:#f0f0f0;"|'''Red Hat/Fedora'''<br />
| align="center" style="background:#f0f0f0;"|'''Debian/Ubuntu'''<br />
| align="center" style="background:#f0f0f0;"|'''SUSE/openSUSE'''<br />
| align="center" style="background:#f0f0f0;"|'''Gentoo'''<br />
|-<br />
| Install a package(s) by name ||pacman -S||yum install ||apt-get install||zypper install<br>zypper in|| emerge [-a]<br />
|- style="background:#e4e4e4"<br />
| Remove a package(s) by name ||pacman -Rs||yum remove/erase ||apt-get remove||zypper remove<br>zypper rm ||emerge -C<br />
|-<br />
| Search for package(s) by searching the expression in name, description, short description. What exact fields are being searched by default varies in each tool. Mostly options bring tools on par. ||pacman -Ss||yum search ||apt-cache search||zypper search<br>zypper se [-s]||emerge -S <br />
|- style="background:#e4e4e4"<br />
| Upgrade Packages - Install packages which have an older version already installed ||pacman -Syu||yum update ||apt-get update; apt-get upgrade||zypper update zypper up||emerge -u world<br />
|-<br />
| Upgrade Packages - Another form of the update command, which can perform more complex updates -- like distribution upgrades. When the usual update command will omit package updates, which include changes in dependencies, this command can perform those updates. ||pacman -Syu||yum distro-sync ||apt-get dist-upgrade||zypper dup||emerge -uDN world<br />
|- style="background:#e4e4e4"<br />
| Reinstall given Package - Will reinstall the given package without dependency hassle. ||pacman -S||yum reinstall||apt-get install --reinstall||zypper install --force||emerge [-a]<br />
|-<br />
| Installs local package file, e.g. app.rpm and uses the installation sources to resolve dependencies ||pacman -U||yum localinstall ||dpkg -i && apt-get install -f||zypper in /path/to/local.rpm||emerge<br />
|- style="background:#e4e4e4"<br />
| Updates package(s) with local packages and uses the installation sources to resolve dependencies ||pacman -U||yum localupdate ||debi||emerge||<br />
|-<br />
| Use some magic to fix broken dependencies in a system || pacman dep level - testdb, shared lib level - findbrokenpkgs or lddd||package-cleanup --problems||apt-get --fix-broken<br>aptitude install||zypper verify ||revdep-rebuild<br />
|- style="background:#e4e4e4"<br />
| Only downloads the given package(s) without unpacking or installing them ||pacman -Sw||yumdownloader (found in yum-utils package)||apt-get install --download-only (into the package cache)<br>apt-get download (bypass the package cache)|| zypper --download-only ||emerge --fetchonly <br />
|-<br />
| Remove dependencies that are no longer needed, because e.g. the package which needed the dependencies was removed. ||<nowiki>pacman -Qdtq | pacman -Rs -</nowiki>||yum autoremove||apt-get autoremove ||zypper rm -u||emerge --depclean <br />
|- style="background:#e4e4e4"<br />
| Downloads the corresponding source package(s) to the given package name(s) || Use [[ABS]] && makepkg -o ||yumdownloader --source||apt-get source / debcheckout||zypper source-install||emerge --fetchonly<br />
|- style="background:#e4e4e4"<br />
| Remove packages no longer included in any repositories. ||||package-cleanup --orphans||aptitude purge '~o'||||<br />
|-<br />
| Install/Remove packages to satisfy build-dependencies. Uses information in the source package. ||automatic||yum-builddep||apt-get build-dep ||zypper si -d||emerge -o <br />
|- style="background:#e4e4e4"<br />
| Add a package lock rule to keep its current state from being changed ||${EDITOR} /etc/pacman.conf<br/>modify IgnorePkg array||yum.conf <--”exclude” option (add/amend)||apt-mark hold pkg||Put package name in /etc/zypp/locks, or zypper al||/etc/portage/package.mask<br />
|-<br />
| Delete a package lock rule ||remove package from IgnorePkg line in /etc/pacman.conf||yum.conf <--”exclude” option (remove/amend)||apt-mark unhold pkg||Remove package name from /etc/zypp/locks||/etc/portage/package.mask (or package.unmask) <br />
|- style="background:#e4e4e4"<br />
| Show a listing of all lock rules ||cat /etc/pacman.conf||yum.conf (research needed)||/etc/apt/preferences ||View /etc/zypp/locks||cat /etc/portage/package.mask<br />
|-<br />
| Add a checkpoint to the package system for later rollback ||||(unnecessary, done on every transaction)||||n/a||<br />
|- style="background:#e4e4e4"<br />
| Remove a checkpoint from the system ||N/A||N/A||||n/a||<br />
|-<br />
| Provide a list of all system checkpoints ||N/A||yum history list||||n/a ||<br />
|- style="background:#e4e4e4"<br />
| Rolls entire packages back to a certain date or checkpoint. ||N/A||yum history rollback||||n/a ||<br />
|-<br />
|- <br />
| Undo a single specified transaction. ||N/A||yum history undo||||n/a||||<br />
|- style="background:#e4e4e4"<br />
| Mark a package previously installed as a dependency as explicitly required. ||pacman -D --asexplicit||||apt-mark manual||||emerge --select <br />
|- <br />
| Install package(s) as dependency / without marking as explicitly required. ||pacman -S --asdeps||||aptitude install 'pkg&M'||||emerge -1<br />
| ||||||||||<br />
|-<br />
| ||||||||||<br />
|-<br />
| '''''Package information management''''' ||||||||||<br />
|- style="background:#e4e4e4"<br />
| Get a dump of the whole system information - Prints, Saves or similar the current state of the package management system. Preferred output is text or XML. (Note: Why either-or here? No tool offers the option to choose the output format.) ||(see /var/lib/pacman/local)||(see /var/lib/rpm/Packages)||apt-cache stats||n/a ||emerge --info<br />
|-<br />
| Show all or most information about a package. The tools' verbosity for the default command vary. But with options, the tools are on par with each other. ||pacman -[S<nowiki>|</nowiki>Q]i ||yum list or info ||apt-cache show / apt-cache policy||zypper info zypper if||emerge -S; emerge -pv; eix<br />
|-<br />
|- style="background:#e4e4e4"<br />
| Search for package(s) by searching the expression in name, description, short description. What exact fields are being searched by default varies in each tool. Mostly options bring tools on par. ||pacman -Ss ||yum search ||apt-cache search||zypper search zypper se [-s]||emerge -S <br />
|-<br />
| Display changelogs||||yum changelog (found in yum-plugin-changelog package)||apt-get changelog||||<br />
|- style="background:#e4e4e4"<br />
| e-mail delivery of package changes||||||apt-get install apt-listchanges||||<br />
|-<br />
| Lists packages which have an update available. Note: Some provide special commands to limit the output to certain installation sources, others use options. ||pacman -Qu ||yum list updates yum check-update ||apt-get upgrade -> n||zypper list-updates zypper patch-check (just for patches) ||emerge -uDNp world<br />
|- style="background:#e4e4e4"<br />
| Display a list of all packages in all installation sources that are handled by the packages management. Some tools provide options or additional commands to limit the output to a specific installation source. ||pacman -Sl ||yum list available||apt-cache dumpavail apt-cache dump (Cache only) apt-cache pkgnames||zypper packages ||emerge -ep world<br />
|-<br />
| Displays packages which provide the given exp. aka reverse provides. Mainly a shortcut to search a specific field. Other tools might offer this functionality through the search command. ||pkgfile <filename>||yum provides / yum whatprovides ||apt-file search <filename>||zypper what-provides zypper wp|| equery belongs (only installed packages); pfl<br />
|- style="background:#e4e4e4"<br />
| Display packages which require X to be installed, aka show reverse/ dependencies.||pacman -Sii||yum resolvedep ||apt-cache rdepends / aptitude search ~Dpattern||IN PROGRESS || equery depends<br />
|-<br />
| Display packages which conflict with given expression (often package). Search can be used as well to mimic this function.||(none)||repoquery --whatconflicts||aptitude search '~Cpattern'||IN PROGRESS ||<br />
|- style="background:#e4e4e4"<br />
| List all packages which are required for the given package, aka show dependencies. ||pacman -[S<nowiki>|</nowiki>Q]i||yum deplist ||apt-cache depends / apt-cache show||IN PROGRESS || emerge -ep<br />
|-<br />
| List what the current package provides ||||yum provides ||dpkg -s / aptitude show||IN PROGRESS||equery files<br />
|- style="background:#e4e4e4"<br />
| List the files that the package holds. Again, this functionality can be mimicked by other more complex commands. ||pacman -Ql $pkgname <br/>pkgfile -l ||repoquery -l $pkgname ||dpkg-query -L $pkgname ||IN PROGRESS ||equery files<br />
|-<br />
| List all packages that require a particular package ||||repoquery --whatrequires [--recursive]||aptitude search \~D{depends,recommends,suggests}:pattern / aptitude why pkg||||equery depends -a<br />
|- style="background:#e4e4e4"<br />
| Search all packages to find the one which holds the specified file. auto-apt is using this functionality. ||pkgfile -s||yum provides / yum whatprovides ||apt-file search||IN PROGRESS ||equery belongs<br />
|- style="background:#e4e4e4"<br />
| Display all packages that the specified packages obsoletes. ||||yum list obsoletes ||apt-cache show||IN PROGRESS|| <br />
|-<br />
| Verify dependencies of the complete system. Used if installation process was forcefully killed. ||testdb||yum deplist ||apt-get check||n/a|| emerge -uDN world<br />
|- style="background:#e4e4e4"<br />
| Generates a list of installed packages || pacman -Q || yum list installed || dpkg --get-selections || zypper ||emerge -ep world<br />
|-<br />
| List packages that are installed but are not available in any installation source (anymore). ||pacman -Qm||yum list extras || deborphan || |zypper se -si | grep 'System Packages'||eix-test-obsolete<br />
|- style="background:#e4e4e4"<br />
| List packages that were recently added to one of the installation sources, i.e. which are new to it. ||(none)||yum list recent ||aptitude search '~N' / aptitude forget-new||n/a||eix-diff<br />
|-<br />
| Show a log of actions taken by the software management. ||cat /var/log/pacman.log ||yum history cat /var/log/yum.log||cat /var/log/dpkg.log||cat /var/log/zypp/history || located in /var/log/portage<br />
|- style="background:#e4e4e4"<br />
| Clean up all local caches. Options might limit what is actually cleaned. Autoclean removes only unneeded, obsolete information. ||pacman -Sc<br/>pacman -Scc ||yum clean all ||apt-get clean / apt-get autoclean / aptitude clean|| zypper clean || eclean distfiles<br />
|-<br />
| Add a local package to the local package cache mostly for debugging purposes. ||cp $pkgname /var/cache/pacman/pkg/||||apt-cache add ||n/a|| cp $srcfile /usr/portage/distfiles<br />
|- style="background:#e4e4e4"<br />
| Display the source package to the given package name(s) ||||repoquery -s||apt-cache showsrc ||n/a||<br />
|-<br />
| Generates an output suitable for processing with dotty for the given package(s). ||||||apt-cache dotty ||n/a||<br />
|- style="background:#e4e4e4"<br />
| Set the priority of the given package to avoid upgrade, force downgrade or to overwrite any default behavior. Can also be used to prefer a package version from a certain installation source. ||${EDITOR} /etc/pacman.conf<br/>Modify HoldPkg and/or IgnorePkg arrays||yum-plugin-priorities and yum-plugin-protect-packages||/etc/apt/preferences, apt-cache policy|| zypper mr -p || ${EDITOR} /etc/portage/package.keywords<br/>Add a line with =category/package-version<br />
|-<br />
| Remove a previously set priority ||||||/etc/apt/preferences ||zypper mr -p || ${EDITOR} /etc/portage/package.keywords<br/>remove offending line<br />
|- style="background:#e4e4e4"<br />
| Show a list of set priorities. ||||||apt-cache policy /etc/apt/preferences ||n/a|| cat /etc/portage/package.keywords<br />
|-<br />
| Ignores problems that priorities may trigger. ||||||||n/a||<br />
|-<br />
| ||||||||||<br />
|-<br />
| ||||||||||<br />
|- style="background:#e4e4e4"<br />
| Installation sources management ||${EDITOR} /etc/pacman.conf||${EDITOR} /etc/yum.repos.d/${REPO}.repo||${EDITOR} /etc/apt/sources.list|| ||layman<br />
|-<br />
| Add an installation source to the system. Some tools provide additional commands for certain sources, others allow all types of source URI for the add command. Again others, like apt and yum force editing a sources list. apt-cdrom is a special command, which offers special options design for CDs/DVDs as source. ||${EDITOR} /etc/pacman.conf||${EDITOR} /etc/yum.repos.d/${REPO}.repo||apt-cdrom add||zypper service-add ||layman, overlays<br />
|- style="background:#e4e4e4"<br />
| Refresh the information about the specified installation source(s) or all installation sources. ||pacman -Sy ([[Pacman#Partial upgrades are unsupported|always upgrade the whole system afterwards]]) ||yum clean expire-cache && yum check-update ||apt-get update||zypper refresh zypper ref||layman -f<br />
|-<br />
| Prints a list of all installation sources including important information like URI, alias etc. ||cat /etc/pacman.d/mirrorlist||cat /etc/yum.repos.d/*||apt-cache policy||zypper service-list ||layman -l<br />
|- style="background:#e4e4e4"<br />
| Disable an installation source for an operation ||||yum --disablerepo=${REPO}||||||emerge package::repo-to-use<br />
|- <br />
| Download packages from a different version of the distribution than the one installed. ||||yum --releasever=${VERSION} ||apt-get install -t release package/ apt-get install package/release (deps not covered)||||echo "category/package ~amd64" >> /etc/portage/package.keywords && emerge package<br />
|- style="background:#e4e4e4"<br />
| '''''Other commands''''' ||||||||||<br />
|-<br />
| Start a shell to enter multiple commands in one session ||||yum shell ||apt-config shell||zypper shell ||<br />
|-<br />
| ||||||||||<br />
|-<br />
| ||||||||||<br />
|- style="background:#e4e4e4"<br />
| '''''Package Verification'''''||||||||||<br />
|-<br />
| Single package||pacman -Qk[k] <package>||rpm -V <package>||debsums||rpm -V <package>||equery check<br />
|- style="background:#e4e4e4"<br />
| All packages||pacman -Qk[k]||rpm -Va||debsums||rpm -Va||equery check<br />
|-<br />
| ||||||||||<br />
|-<br />
| ||||||||||<br />
|-<br />
| '''''Package Querying'''''||||||||||<br />
|- style="background:#e4e4e4"<br />
| List installed local packages along with version||pacman -Q||rpm -qa||dpkg -l||||emerge -e world<br />
|-<br />
| Display local package information: Name, version, description, etc.||pacman -Qi ||rpm -qi ||dpkg -s / aptitude show||||emerge -pv and emerge -S<br />
|- style="background:#e4e4e4"<br />
| Display remote package information: Name, version, description, etc.||pacman -Si ||yum info ||apt-cache show / aptitude show||||emerge -pv and emerge -S<br />
|- <br />
| Display files provided by local package||pacman -Ql ||rpm -ql||dpkg -L||||equery files<br />
|- style="background:#e4e4e4"<br />
| Display files provided by a remote package||pkgfile -l||repoquery -l||apt-file list pattern||||pfl<br />
|- <br />
| Query the package which provides FILE ||pacman -Qo ||rpm -qf (installed only) or yum whatprovides (everything) ||dpkg -S / dlocate||||equery belongs<br />
|- style="background:#e4e4e4"<br />
| Query a package supplied on the command line rather than an entry in the package management database||pacman -Qp||rpm -qp||dpkg -I||||<br />
|-<br />
| Show the changelog of a package||pacman -Qc||rpm -q --changelog||apt-get changelog|||||equery changes -f<br />
|- style="background:#e4e4e4"<br />
| Search locally installed package for names or descriptions ||pacman -Qs||rpm -qa '*<str>*' ||aptitude search <nowiki>'~i(~n name|~d description)'</nowiki>|||||eix -S -I<br />
|-<br />
| List packages not required by any other package||pacman -Qt||package-cleanup --all --leaves||deborphan -anp1||||<br />
|-<br />
| ||||||||||<br />
|-<br />
| '''''Building Packages'''''||||||||||<br />
|- style="background:#e4e4e4"<br />
| Build a package||makepkg -s||rpmbuild -ba (normal)<br>mock (in chroot)||debuild||rpmbuild -ba ||ebuild; quickpkg<br />
|-<br />
| Check for possible packaging issues||namcap||rpmlint ||lintian||||repoman<br />
|- style="background:#e4e4e4"<br />
| List the contents of a package file||pacman -Qpl <file>||rpmls rpm -qpl||dpkg -c||rpm -qpl||<br />
|-<br />
| Extract a package ||tar -Jxvf||<nowiki>rpm2cpio | cpio -vid</nowiki>||dpkg-deb -x||<nowiki>rpm2cpio | cpio -vid</nowiki>||tar -jxvf<br />
|- style="background:#e4e4e4"<br />
| Query a package supplied on the command line rather than an entry in the package management database||pacman -Qp||rpm -qp||dpkg -I||||<br />
|-<br />
| align="center" style="background:#f0f0f0;"|'''<font color="#707070">Action</font>'''<br />
| align="center" style="background:#f0f0f0;"|'''Arch'''<br />
| align="center" style="background:#f0f0f0;"|'''Red Hat/Fedora'''<br />
| align="center" style="background:#f0f0f0;"|'''Debian/Ubuntu'''<br />
| align="center" style="background:#f0f0f0;"|'''SUSE/openSUSE'''<br />
| align="center" style="background:#f0f0f0;"|'''Gentoo'''<br />
|}</div>Justin8https://wiki.archlinux.org/index.php?title=PDF_forms&diff=374147PDF forms2015-05-19T11:54:41Z<p>Justin8: Checkboxes work in current evince</p>
<hr />
<div>{{stub}}<br />
[[Category:Office]]<br />
This article is meant to guide (Arch)linux users to use PDF forms.<br />
Some of the information here is from [https://bbs.archlinux.org/viewtopic.php?id=52084 this] thread.<br />
<br />
==Reading and filling PDF forms==<br />
<br />
===Adobe Acrobat Reader===<br />
Naturally, [http://www.adobe.com/products/acrobat/readstep2.html Adobe's Reader] (available through the AUR) will be able to read and fill PDF files. This is currently your only option to save pdf form data into the pdf file itself, compliant with pdf specification, so that you will be able to send the filled documents to others.<br />
<br />
===Evince===<br />
[http://www.gnome.org/projects/evince/ Evince] (available in the extra repository) is a GNOME application that can read, fill, and save PDF forms.<br />
<br />
For those of you who do not use GNOME, {{AUR|evince-gtk}} is available from [[AUR]] without gnome-keyring and gconf as dependencies.<br />
<br />
Please note that evince isn't able to save the pdf form data into the pdf file itself, as per pdf spec.<br />
<br />
===Okular===<br />
[http://okular.kde.org/ Okular] (available in the extra repository as package {{pkg|kdegraphics-okular}}) is a universal document viewer based on KPDF for KDE 4. The last stable release is Okular 0.21, shipped in the kdegraphics module of KDE Development Platform 4.14. Its development began as part of Google's Summer of Code program. The description of the project is located at KDE Developer's Corner.<br />
<br />
Okular combines the excellent functionalities of KPDF with the versatility of supporting different kind of documents, like PDF, Postscript, DjVu, CHM, and others.<br />
The document format handlers page has a chart describing in more detail the supported formats and the features supported in each of them.<br />
<br />
Please note that okular isn't able to save the pdf form data into the pdf file itself, as per pdf spec.<br />
<br />
===Cabaret Stage===<br />
[http://www.cabaret-solutions.com/en/ This] freeware application claims to be able to read, fill, and save PDF forms. It is available in the AUR as {{AUR|cabaretstage}}.<br />
<br />
===Text2pdf===<br />
[http://www.eprg.org/pdfcorner/text2pdf/ Text2pdf] it's a nice application that can convert text files to pdf format available throught AUR there {{AUR|text2pdf}}...<br />
<br />
===flpsed===<br />
Quoting flpsed’s author: “'''[http://www.ecademix.com/JohannesHofmann/flpsed.html flpsed]''' is a WYSIWYG PostScript annotator. You can’t remove or modify existing elements of a document. But flpsed lets you add arbitrary text lines to existing PostScript documents [...]. Added lines can later be re-edited with flpsed. [...] flpsed is useful for filling in forms, adding notes etc.”<br />
<br />
And quoting the aforementioned [https://bbs.archlinux.org/viewtopic.php?pid=556501#p556501 thread]: “flpsed, in the AUR, will place text on top of a plain pdf (without the special form fields). (Note: it took me a while to figure out how to move or edit text that has already been placed, but all you have to do is click on the first letter in the field. If you click on the middle of the field, nothing will happen)”<br />
<br />
{{AUR|flpsed}} is available from the [[AUR]].<br />
<br />
===Inkscape===<br />
[http://www.inkscape.org/ Inkscape] is an image editing program that can be used to fill PDF forms by importing the PDF file and simply inserting text fields where you want them to be. Its not really filling the form (the fields will probably still be blank if the forms are to be read electronically), but should work well enough if you intend to print them.</div>Justin8https://wiki.archlinux.org/index.php?title=Pulse_Connect_Secure&diff=362503Pulse Connect Secure2015-02-23T22:53:11Z<p>Justin8: jnc64 doesn't exist. no point linking to a missing package.</p>
<hr />
<div>[[Category:Virtual Private Network]]<br />
{{Poor writing|Using first person, needs more [[Help:Style|style]] fixes.}}<br />
==Preferred installation method==<br />
(NOTE: In [http://kb.juniper.net/InfoCenter/index?page=content&id=KB20490&actp=RSS some cases], depending on your corporate policy configuration, you _must_ login through the browser. If this is the case, command-line tools (jnc, junipernc) will not work.)<br />
<br />
1) Go to your companys' VPN site, log in and download / install the juniper client. <br />
<br />
2) install {{AUR|jnc}} from the [[Arch User Repository]]. For 64-bit Arch, you will need to install 32-bit packages ([[Multilib]]), see [http://www.scc.kit.edu/scc/net/juniper-vpn/linux/ upstream website].<br />
<br />
3) Make a directory for the .config file:<br />
{{bc|$ mkdir -p ~/.juniper_networks/network_connect/config}}<br />
<br />
4) Copy and adapt this .config file in this directory:<br />
{{hc|~/.juniper_networks/network_connect/config/.config|<nowiki><br />
host=foo.bar.com<br />
user=username<br />
password=secret<br />
realm= realm with spaces<br />
cafile=/etc/ssl/bar-chain.pem<br />
certfile=</nowiki>}}<br />
<br />
; cafile: ca chain to verify the host certificate<br />
; certfile: host certificate in DER format. Cafile or certfile must be configured, you can download them from your VPN sign-in page (certificate information, export certificate).<br />
; realm: You can find out your realm by viewing the page source of your VPN sign-in page: just search for the word realm in it.<br />
<br />
5) Start / stop network connect:<br />
{{bc|$ jnc --nox}}<br />
for use without GUI. To stop the client, execute<br />
{{bc|$ jnc stop}}<br />
<br />
==64-bit Hack==<br />
This was the final fix after veritable hours of trying to make it work more properly, and it is very simple:<br />
<br />
1) Install {{AUR|bin32-jre}} from the AUR - make sure the PKGBUILD installs it to {{ic|/opt/bin32-jre}}, rather than {{ic|/opt/java}}, where it will conflict with the 64-bit JRE.<br />
<br />
2) Install {{AUR|jre}} from the AUR.<br />
<br />
3) Move the java binary to java.orig:<br />
{{bc|# mv /opt/java/jre/bin/java /opt/java/jre/bin/java.orig}}<br />
<br />
4) Create a bash script {{ic|java}} and make it executable:<br />
{{bc|# touch /opt/java/jre/bin/java<br />
# chmod 755 /opt/java/jre/bin/java}}<br />
<br />
5) Edit the bash script and you are done:<br />
{{hc|/opt/java/jre/bin/java|<nowiki><br />
#!/bin/bash<br />
if [ $3x = "NCx" ]<br />
then<br />
/opt/bin32-jre/jre/bin/java "$@"<br />
else<br />
/opt/java/jre/bin/java.orig "$@"<br />
fi</nowiki>}}<br />
<br />
Bear in mind, this is a terrible hack, and if you update JRE it will break and you will have to repeat a few steps. That said, it worked fantastically for me, with minimal setup if I need to hop on a VPN from another Arch PC.<br />
<br />
==Another installation method==<br />
<br />
Here is what I did to connect to the Juniper VPN at my company:<br />
<br />
References:<br />
[http://www.gentoo-wiki.info/HOWTO_Juniper_SSL_Network_Connect_VPN Gentoo Wiki Archives]<br />
<br />
#Get [https://www.archlinux.org/packages/search/?q=jre JRE]<br />
#Get the really old GCC libs<br />
##Either with [https://aur.archlinux.org/packages.php?ID=27768 gcc3] and [https://aur.archlinux.org/packages.php?ID=2299 gcc2]<br />
##If you are lazy like me or just cannot get it to produce the super-old libstdc++-libc6.2-2.so.3, just steal the whole lib-compat from gentoo with this PKGBUILD:<br />
# Contributor: Clement Siuchung Cheung <clement.cheung@umich.edu><br />
pkgname=lib-compat<br />
pkgver=1.4.1<br />
pkgrel=1<br />
pkgdesc="Gentoo lib compat for old programs only available in binary"<br />
arch=(x86)<br />
url="http://www.gentoo.org/"<br />
source=(ftp://ftp.ibiblio.org/pub/linux/distributions/gentoo/distfiles/${pkgname}-${pkgver}.tar.bz2)<br />
md5sums=('ec4a4528295b5879ad055e44c4a6d463')<br />
<br />
build() {<br />
cd $startdir/src/${pkgname}-${pkgver}/x86<br />
<br />
# Install /lib files<br />
mkdir -p $startdir/pkg/lib<br />
mv ld-linux.so.1* $startdir/pkg/lib<br />
<br />
# Install /usr/lib files<br />
mkdir -p $startdir/pkg/usr/lib<br />
mv *.so* $startdir/pkg/usr/lib<br />
<br />
# Fix files<br />
cd $startdir/pkg/usr/lib<br />
mv -f libstdc++-libc6.2-2.so.3 libstdc++-3-libc6.2-2-2.10.0.so<br />
ln -s libstdc++-3-libc6.2-2-2.10.0.so libstdc++-libc6.2-2.so.3<br />
mv -f libstdc++-libc6.1-1.so.2 libstdc++-2-libc6.1-1-2.9.0.so<br />
ln -s libstdc++-2-libc6.1-1-2.9.0.so libstdc++-libc6.1-1.so.2<br />
ln -s libstdc++.so.2.8.0 libstdc++.so.2.8<br />
ln -s libstdc++.so.2.7.2.8 libstdc++.so.2.7.2<br />
ln -s libg++.so.2.7.2.8 libg++.so.2.7.2<br />
rm -f libstdc++.so.2.9.dummy libstdc++.so.2.9.0<br />
rm -f libsmpeg-0.4.so.0.dummy<br />
}<br />
<br />
#Get the smelly old Motif libs<br />
##Install lesstif. Then symlink to fool the system that it is motif like they say in the Gentoo wiki.<br />
##Sadly I was not able to get it work through the openmotif route because our openmotif package is too new and will give you libXm.so.4 instead of libXm.so.3. Add your instructions here if you manage to get this work.<br />
#Get the su work. They use xterm to ask for root password to do the install. So do either of the following:<br />
##Install {{Pkg|xterm}}<br />
##Setup your user to be able to su without password (google for the instructions)<br />
#Do "sudo modprobe tun". You will need to do it every time before you connect. So you might want to setup the tun module to be autoloaded at start up to save you time and trouble.<br />
<br />
==Troubleshooting==<br />
<br />
There are many things that can go wrong. Please share your experience here if there is something non-obvious that wasted you weeks to track down so that others can save their time. ;-)<br />
<br />
===It keeps saying password incorrect===<br />
First of all, make sure the username and password is actually correct. ;-) Check caps lock, etc. If you swear it is correct and it still says incorrect, that means the POST request to the Juniper IVE box "somehow" failed.<br />
<br />
The [https://addons.mozilla.org/en-US/firefox/addon/966 Tamper Data] addon for Firefox can be used to debug. Try changing the fields in the headers.<br />
<br />
One thing that had me scratching my head for months is incorrect charset. Juniper IVE apparently does not support UTF-8. For some reasons, my "intl.charset.default" setting in "about:config" for Firefox is UTF-8, causing my POST request to have *ONLY* UTF-8 in the charset. Setting it to ISO-8859-1 fixes the problem. Also double check "intl.accept_charsets". You can have UTF-8, Chinese and European charsets all you want. But make sure you have ISO-8859-1 as fallback. Use the Tamper Data addon to make sure you really are accepting ISO-8859-1 in the HTTP header.<br />
<br />
Another thing is the useragent must be "Firefox", not "Bon Echo". You may need to change this under "general.useragent.extra.firefox" in about:config.<br />
<br />
===I can login but Network Connect will not launch===<br />
#Check your JRE<br />
#Go to ".juniper_networks/network_connect" in your home directory.<br />
#Check that ncsvc is setuid root. Fix it if not.<br />
#ldd ncsvc and see if there are any missing libraries<br />
#Follow instructions [http://www.juniperforum.com/index.php/topic,2043.0.html here] to run it from command line. Use the "-L 5" switch to log everything, use strace as root, etc. Peek at ncsvc.log and see if there is anything wrong.<br />
<br />
===Network Connect launched but the VPN does not work===<br />
Run "ip route" and see if the route is there. Network connect has a diagnosis tool in the GUI. You can also checks the logs (also available in the GUI).<br />
<br />
If it initially works but stops working later on, see caveat below.<br />
<br />
===Network Connect launched and a configuration error message is displayed===<br />
Check that you have net-tools installed.<br />
<br />
=== ncapp.error Failed to connect/authenticate with IVE.===<br />
See [http://ubuntuforums.org/showthread.php?p=12127450#post12127450 my post] on the ubuntu form. I was trying some of the several 'command-line' options and it turns out that in certain cases, policy will not permit it. It had to install both bin32-jre and bin32-firefox and authenticate through the browser. <br />
<br />
==Caveats==<br />
/etc/resolv.conf will get overwritten every once in a while by DHCPCD so your VPN will stop working eventually. If that happens, just restart Network Connect. There is no known solution to the problem but I do find a discussion on Redhat bugs website about this. We need to somehow teach DHCPCD the concept of merging configs and being a good neighbor...<br />
<br />
Until then, restart the connection every once in a while, save /etc/resolv.conf somewhere or somehow whip up some super-clever script yourself to restore the VPN settings every time your DHCP lease is renewed.<br />
<br />
==Alternative Method==<br />
<br />
Another method to get Juniper VPN to work for 64 bit Arch linux is suggested for your reference. I use this method to connect to my university's vpn network. <br />
<br />
The key reference:<br />
http://wireless.siu.edu/install-ubuntu-64.htm<br />
<br />
Basics<br />
<br />
The key issue is that 64 bit java plugin do not work with the Juniper software. (firefox, sun java jre)<br />
<br />
One way to do it is to install an alternative version of java and link the java plugin for the firefox manually. This saves us from the trouble of having to deal with the chroot environment as suggested in other sites. <br />
<br />
These are the steps I follow:<br />
<br />
I have firefox and sun java jre installed. I assume the system is 64 bit Arch linux. <br />
<br />
1.) install xterm:<br />
<br />
pacman -S xterm<br />
<br />
2.) install a custom 64 bit java<br />
<br />
go to http://www.java.com/en/download<br />
select the Linux x64 verson<br />
<br />
Decide on a location for the installation, extract the binary and put it in the desired location, and make the binary executable with<br />
chmod +x << binary >>. <br />
<br />
Finally run it to install java. <br />
<br />
3.) install the customized 32 bit java<br />
<br />
again, go to http://www.java.com/en/download<br />
this time, select Linux(self-extracting) option<br />
<br />
Extract the new binary to the same location created above, make it executable, and run the binary. It will ask you whether you want to replace the files to 32 bit, '''Type "A" to overwrite all the 64-bit files with the 32-bit ones.'''<br />
<br />
4.) link the library<br />
<br />
the relevant library for firefox is libnpjp2.so, to link it, <br />
<br />
ln -s << location of java you installed above >>/lib/amd64/libnpjp2.so /usr/lib/mozilla/plugins/libnpjp2.so<br />
<br />
The newest firefox 5 does look at /usr/lib/mozilla/plugins for plugins, instead of the ~/.mozilla/plugins in the previous versions.<br />
<br />
==Yet Another Method using the Mad Scientist's "msjnc" script==<br />
<br />
Follow instructions here: http://mad-scientist.us/juniper.html<br />
<br />
For arch, you must [[install]] {{Pkg|gtk2-perl}}, {{Pkg|glib-perl}} and {{Pkg|unzip}}.<br />
<br />
=== Special instructions for 64-bit users ===<br />
[[Multilib#Enabling|Enable multilib]].<br />
<br />
[[Install]] {{Pkg|lib32-zlib}}, {{Pkg|net-tools}}, {{Pkg|glib-perl}}, {{Pkg|perl-libwww}} and {{Pkg|gtk2-perl}}.<br />
<br />
Access the the Juniper VPN website you need to use. Log in and allow the installation to attempt and fail (due to non-32 bit java). I get the following error:<br />
{{bc|Setup failed.<br />
Please install 32 bit Java and update alternatives links using update-alternatives command.<br />
For more details, refer KB article KB25230}}<br />
<br />
You should now have the file {{ic|~/.juniper_networks/ncLinuxApp.jar}} present.<br />
<br />
Download the msjnc script, make it executable, and put it in your path.<br />
{{bc|$ wget -q -O /tmp/msjnc https://raw.github.com/madscientist/msjnc/master/msjnc<br />
$ chmod 755 /tmp/msjnc<br />
# cp /tmp/msjnc /usr/bin}}<br />
<br />
==== Automatic installation of ncsvc using msjnc ====<br />
<br />
The first time you launch msjnc (before ncsvc is installed), it will extract {{ic|ncLinuxApp.jar}} and prompt for your password in order to install the service. This requires sudo to be configured to allow all commands to your user.<br />
<br />
After the service is installed to {{ic|~/.juniper_networks/network_connect/ncsvc}} with suid, create a profile and connect!<br />
<br />
==== Manual installation of msjnc ====<br />
<br />
Create these directories:<br />
{{bc|$ mkdir -p ~/.juniper_networks/network_connect<br />
$ mkdir -p ~/.juniper_networks/tmp}}<br />
<br />
Extract the software:<br />
{{bc|$ unzip ~/.juniper_networks/ncLinuxApp.jar -d ~/.juniper_networks/tmp}}<br />
<br />
Copy NC.jar to the network_connect directory:<br />
{{bc|$ cp ~/.juniper_networks/tmp/NC.jar ~/.juniper_networks/network_connect}}<br />
<br />
Install the service:<br />
{{bc|$ sh ~/.juniper_networks/tmp/installNC.sh ~/.juniper_networks/network_connect}}<br />
<br />
Launch msjnc, create a profile, and connect!<br />
<br />
==== Note regarding Server/URL ====<br />
For the Server/URL, you may have to provide the URL that processes the login form rather than the login page itself. For example, my company's login form is on {{ic|/dana-na/auth/url_0/welcome.cgi}} but the form is actually processed by {{ic|/dana-na/auth/url_0/login.cgi}}. You may have to inspect the html of the login page to find the form's action attribute.<br />
<br />
== Yet another alternative using jvpn script (support 64bits and host checker) ==<br />
<br />
Jvpn perl script establishes a Juniper VPN connection and supports the below features:<br />
* connection using Host Checker<br />
* automatic download of the required Juniper java and daemon files (ncsvc) when run under su/sudo<br />
https://github.com/samm-git/jvpn<br />
<br />
=== Installation ===<br />
<br />
1. [[Install]] the perl dependencies {{Pkg|perl-term-readkey}} and {{Pkg|perl-lwp-protocol-https}}<br />
<br />
2. Choose whether to run jvpn with (easiest method) or without sudo and run the below steps accordingly <br />
<br />
'''If running the script with sudo:'''<br />
<br />
2.1- Run the command<br />
{{bc|<nowiki># curl -L https://github.com/samm-git/jvpn/archive/v0.7.0.tar.gz | tar xz</nowiki>}}<br />
The command creates a file {{ic|jvpn-0.7.0}} in current directory<br />
<br />
'''If running the script without sudo:'''<br />
<br />
2.1 Use your regular web browser (no need for a 32-bit java) to connect to the VPN website and download the appropriate software.<br />
The files downloaded will be located in {{ic|~/.juniper_networks/network_connect/}} (even if the VPN connection actually fails).<br />
This step is considered more complex because you have to have a functional java plugin in your browser (configured wit appropriate security settings).<br />
During installation of Network Connect, the browser will request a root password to set the setuid flag on ncsvc (Juniper daemon).<br />
<br />
2.2 Install jvpn into the folder with command<br />
{{bc|$ cd ~/.juniper_networks/network_connect<br />
$ curl -L https://github.com/samm-git/jvpn/archive/v0.7.0.tar.gz | tar xz --strip-components=1}}<br />
<br />
3. Edit {{ic|jvpn.ini}} (directions included in the file)<br />
<br />
=== Run ===<br />
<br />
'''If running the script with sudo:'''<br />
<br />
Simply run the commands:<br />
<pre>su<br />
./jvpn.pl</pre><br />
On first run, the script will download all the necessary files<br />
<br />
'''If running the script without sudo:'''<br />
<br />
Simply run the commands:<br />
<pre><br />
cd ~/.juniper_networks/network_connect<br />
./jvpn.pl<br />
</pre><br />
<br />
== Native Open Source support with OpenConnect ==<br />
<br />
The [http://www.infradead.org/openconnect/ OpenConnect] VPN client has recently added support for Juniper VPN, supporting both TCP and UDP data transports. See the [http://lists.infradead.org/pipermail/openconnect-devel/2015-January/002628.html initial announcement] on the mailing list for more details.<br />
<br />
You'll need to build OpenConnect from the git repository, and the simplest way to use it for now is using the tool at https://github.com/russdill/juniper-vpn-py which is based on some of the other python scripts mentioned above.</div>Justin8https://wiki.archlinux.org/index.php?title=Zabbix&diff=357693Zabbix2015-01-23T01:00:40Z<p>Justin8: Removed database config steps as this is done via the setup UI on first install now.</p>
<hr />
<div>[[Category:Network monitoring]]<br />
[[ja:Zabbix]]<br />
[[ru:Zabbix]]<br />
[http://zabbix.com Zabbix] is a full-featured monitoring solution for larger networks. It can discover all kind of networking devices using different methods, check machine states and applications, sending pre-defined alarm messages and visualize complex data correlations.<br />
<br />
==Server setup ==<br />
=== Installation ===<br />
If you want to use the Zabbix server with [[MariaDB]], install {{AUR|zabbix-server-mysql}} from the [[AUR]]. For [[PostgreSQL]] as database backend, you should use {{AUR|zabbix-server}}. You also have to choose a web server with PHP support, e.g.:<br />
*[[LAMP|Apache]]<br />
*[[Lighttpd]]<br />
*[[NginX]]<br />
Or one of the other servers found in the [[:Category:Web Server|web server]] category.<br />
You may edit the [[PKGBUILD]]s if you plan to use Nginx as web-server, since by default they have {{Pkg|apache}} and {{Pkg|php-apache}} as dependency.<br />
<br />
===Configuration===<br />
Symlink the Zabbix web application directory to your http document root, e.g.:<br />
{{bc|ln -s /usr/share/webapps/zabbix /var/www}}<br />
Make sure to adjust following variables to these minimal values in your {{ic|php.ini}}:<br />
{{hc|/etc/php/php.ini|<br />
<nowiki>extension=bcmath.so<br />
extension=gd.so<br />
extension=sockets.so<br />
post_max_size = 16M<br />
max_execution_time = 300<br />
max_input_time = 300<br />
date.timezone = "UTC"<br />
always_populate_raw_post_data = -1<br />
</nowiki>}}<br />
In this example, we create on localhost a MariaDB database called {{ic|zabbix}} for the user {{ic|zabbix}} identified by the password {{ic|test}} and then import the database templates. This connection will be later used by the Zabbix server and web application:<br />
{{bc|mysql -u root -p -e "create database zabbix"<br />
mysql -u root -p -e "grant all on zabbix.* to zabbix@localhost identified by 'test'"<br />
mysql -u zabbix -p zabbix < /etc/zabbix/database/schema.sql<br />
mysql -u zabbix -p zabbix < /etc/zabbix/database/images.sql<br />
mysql -u zabbix -p zabbix < /etc/zabbix/database/data.sql}}<br />
<br />
=== Starting ===<br />
[[systemd#Using units|Enable and start]] the {{ic|zabbix-server}} service.<br />
<br />
Finally you can access Zabbix via your local web server, e.g.: http://127.0.0.1/zabbix, finish the installation wizard and access the frontend the first time. The default username is {{ic|Admin}} and password {{ic|zabbix}}.<br />
<br />
See appendix for a link to the official documentation, which explains all further steps in using it.<br />
<br />
== Agent setup ==<br />
=== Installation ===<br />
Currently, the server package already includes {{AUR|zabbix-agent}}, so you do not have to install this package on your monitoring server. However, for monitoring targets, the client part is more minimal, standalone and easy to deploy, just install {{AUR|zabbix-agent}}.<br />
<br />
=== Configuration ===<br />
Simply edit the {{ic|zabbix_agentd.conf}} and replace the server variable with the IP of your monitoring server. Only servers from this/these IP will be allowed to access the agent.<br />
{{hc|/etc/zabbix/zabbix_agentd.conf|<br />
<nowiki>Server=127.0.0.1<br />
ServerActive=</nowiki>}}<br />
Further make sure the port {{ic|10050}} on your device being monitored is not blocked and is properly forwarded.<br />
<br />
=== Starting ===<br />
[[systemd#Using units|Enable and start]] the {{ic|zabbix-agentd}} service.<br />
<br />
== Tips and tricks ==<br />
=== Debugging a Zabbix agent ===<br />
On the client site, you can check the state of an item like this:<br />
zabbix_agentd -t hdd.smart[sda,Temperature_Celsius]<br />
On the server/monitoring site, try this:<br />
zabbix_get -s ''host'' -k hdd.smart[sda,Temperature_Celsius]<br />
<br />
=== Monitor ArchLinux system updates ===<br />
Here's an approach on how to monitor your ArchLinux clients for available system update using a custom UserParameter<br />
{{hc|/etc/zabbix/zabbix_agentd.conf|<nowiki>Include=/etc/zabbix/zabbix_agentd.conf.d/*.conf</nowiki>}}<br />
{{hc|/etc/zabbix/zabbix_agentd.conf.d/archlinuxupdates.conf|<nowiki>UserParameter=archlinuxupdates,checkupdates | wc -l</nowiki>}}<br />
You have to restart {{ic|zabbix-agentd}} to apply the new configuration. The keyword for the item you later use in the web frontend is {{ic|archlinuxupdates}}. It returns an integer representing the count of available updates.<br />
<br />
== See also ==<br />
* [https://www.zabbix.com/documentation/doku.php?id=2.0 Official manual for version 2.0]</div>Justin8https://wiki.archlinux.org/index.php?title=Zabbix&diff=357692Zabbix2015-01-23T00:59:16Z<p>Justin8: </p>
<hr />
<div>[[Category:Network monitoring]]<br />
[[ja:Zabbix]]<br />
[[ru:Zabbix]]<br />
[http://zabbix.com Zabbix] is a full-featured monitoring solution for larger networks. It can discover all kind of networking devices using different methods, check machine states and applications, sending pre-defined alarm messages and visualize complex data correlations.<br />
<br />
==Server setup ==<br />
=== Installation ===<br />
If you want to use the Zabbix server with [[MariaDB]], install {{AUR|zabbix-server-mysql}} from the [[AUR]]. For [[PostgreSQL]] as database backend, you should use {{AUR|zabbix-server}}. You also have to choose a web server with PHP support, e.g.:<br />
*[[LAMP|Apache]]<br />
*[[Lighttpd]]<br />
*[[NginX]]<br />
Or one of the other servers found in the [[:Category:Web Server|web server]] category.<br />
You may edit the [[PKGBUILD]]s if you plan to use Nginx as web-server, since by default they have {{Pkg|apache}} and {{Pkg|php-apache}} as dependency.<br />
<br />
===Configuration===<br />
Symlink the Zabbix web application directory to your http document root, e.g.:<br />
{{bc|ln -s /usr/share/webapps/zabbix /var/www}}<br />
Make sure to adjust following variables to these minimal values in your {{ic|php.ini}}:<br />
{{hc|/etc/php/php.ini|<br />
<nowiki>extension=bcmath.so<br />
extension=gd.so<br />
extension=sockets.so<br />
post_max_size = 16M<br />
max_execution_time = 300<br />
max_input_time = 300<br />
date.timezone = "UTC"<br />
always_populate_raw_post_data = -1<br />
</nowiki>}}<br />
In this example, we create on localhost a MariaDB database called {{ic|zabbix}} for the user {{ic|zabbix}} identified by the password {{ic|test}} and then import the database templates. This connection will be later used by the Zabbix server and web application:<br />
{{bc|mysql -u root -p -e "create database zabbix"<br />
mysql -u root -p -e "grant all on zabbix.* to zabbix@localhost identified by 'test'"<br />
mysql -u zabbix -p zabbix < /etc/zabbix/database/schema.sql<br />
mysql -u zabbix -p zabbix < /etc/zabbix/database/images.sql<br />
mysql -u zabbix -p zabbix < /etc/zabbix/database/data.sql}}<br />
Adjust database settings in {{ic|zabbix_server.conf}}:<br />
{{hc|/etc/zabbix/zabbix_server.conf|<br />
<nowiki>DBHost=mysql.local<br />
DBName=zabbix<br />
DBUser=zabbix<br />
DBPassword=test<br />
</nowiki>}}<br />
<br />
=== Starting ===<br />
[[systemd#Using units|Enable and start]] the {{ic|zabbix-server}} service.<br />
<br />
Finally you can access Zabbix via your local web server, e.g.: http://127.0.0.1/zabbix, finish the installation wizard and access the frontend the first time. The default username is {{ic|Admin}} and password {{ic|zabbix}}.<br />
<br />
See appendix for a link to the official documentation, which explains all further steps in using it.<br />
<br />
== Agent setup ==<br />
=== Installation ===<br />
Currently, the server package already includes {{AUR|zabbix-agent}}, so you do not have to install this package on your monitoring server. However, for monitoring targets, the client part is more minimal, standalone and easy to deploy, just install {{AUR|zabbix-agent}}.<br />
<br />
=== Configuration ===<br />
Simply edit the {{ic|zabbix_agentd.conf}} and replace the server variable with the IP of your monitoring server. Only servers from this/these IP will be allowed to access the agent.<br />
{{hc|/etc/zabbix/zabbix_agentd.conf|<br />
<nowiki>Server=127.0.0.1<br />
ServerActive=</nowiki>}}<br />
Further make sure the port {{ic|10050}} on your device being monitored is not blocked and is properly forwarded.<br />
<br />
=== Starting ===<br />
[[systemd#Using units|Enable and start]] the {{ic|zabbix-agentd}} service.<br />
<br />
== Tips and tricks ==<br />
=== Debugging a Zabbix agent ===<br />
On the client site, you can check the state of an item like this:<br />
zabbix_agentd -t hdd.smart[sda,Temperature_Celsius]<br />
On the server/monitoring site, try this:<br />
zabbix_get -s ''host'' -k hdd.smart[sda,Temperature_Celsius]<br />
<br />
=== Monitor ArchLinux system updates ===<br />
Here's an approach on how to monitor your ArchLinux clients for available system update using a custom UserParameter<br />
{{hc|/etc/zabbix/zabbix_agentd.conf|<nowiki>Include=/etc/zabbix/zabbix_agentd.conf.d/*.conf</nowiki>}}<br />
{{hc|/etc/zabbix/zabbix_agentd.conf.d/archlinuxupdates.conf|<nowiki>UserParameter=archlinuxupdates,checkupdates | wc -l</nowiki>}}<br />
You have to restart {{ic|zabbix-agentd}} to apply the new configuration. The keyword for the item you later use in the web frontend is {{ic|archlinuxupdates}}. It returns an integer representing the count of available updates.<br />
<br />
== See also ==<br />
* [https://www.zabbix.com/documentation/doku.php?id=2.0 Official manual for version 2.0]</div>Justin8https://wiki.archlinux.org/index.php?title=AUR_helpers/Graphical&diff=322352AUR helpers/Graphical2014-06-30T01:01:32Z<p>Justin8: Changed octopi-git to octopi since a stable package exists.</p>
<hr />
<div>[[Category:Arch User Repository]]<br />
[[Category:Package management]]<br />
[[cs:Pacman GUI Frontends]]<br />
[[el:Pacman GUI Frontends]]<br />
[[it:Pacman GUI Frontends]]<br />
[[ja:Pacman GUI Frontends]]<br />
[[ru:Pacman GUI Frontends]]<br />
[[tr:Pacman_Önyüzleri]]<br />
[[zh-CN:Pacman GUI Frontends]]<br />
This is a list of frontends for the [[pacman]] CLI tool. The list includes full featured GUI frontends, informational tools, and a variety of system tray notifiers. The list also includes categories for GTK2 based and Qt based software.<br />
<br />
{{Warning|None of these tools are officially supported by Arch Linux/Pacman developers.}}<br />
<br />
== Pacman frontends ==<br />
<br />
=== X11 ===<br />
<br />
* {{App|1=PacmanXG4|2=GUI front-end for pacman. Depends neither GTK+ nor Qt, just X11. This graphical tool allows to do the following:<br />
:* Install/remove/upgrade packages<br />
:* Search packages/filter packages<br />
:* Retrieve package info include screenshots<br />
:* Downgrade packages (need downgrade utility from AUR)<br />
:* Refresh package database, synchronize mirrors.<br />
:* Update system in one click<br />
:* Find out which package a specific file belongs to (include file with pkgfile utility)<br />
:* Package Cache management<br />
:* yaourt support<br />
:'''Screenshots''' http://almin-soft.ru/index.php?programmy/pacmanxg/pacmanxg-screenshots<br><br />
:'''Direct link to binary:''' http://almin-soft.ru/data/files/pacmanxg/download.php?get=pacmanXG4.tar.bz2 <br />
|3='''Web page:''' http://almin-soft.ru/index.php?programmy/pacmanxg/tags/pacmanxg (ru, present "Google site translate" ability)<br />
|4='''AUR : ''' {{AUR|pacmanxg4-bin}}}}<br />
<br />
* {{App|1=PacmanExpress|2=GUI front-end for pacman. Depends neither GTK+ nor Qt, just X11. This graphical tool is a lightweight version of PacmanXG<br />
:* Interface "all in one box"<br />
:* No query. Install/remove packages takes place immediately.<br />
:* Ability to run multiple operations/remove packages (be careful!)<br />
:* yaourt support<br />
<br />
:'''Direct link to binary:''' http://almin-soft.ru/data/files/pacmanxg/download.php?get=pacmanexpress.tar.bz2<br />
|3='''Web page:''' http://almin-soft.ru/index.php?programmy/pacmanexpress/tags/pacmanexpress (ru, present "Google site translate" ability)<br />
|4='''AUR : ''' {{AUR|pacmanexpress}}}}<br />
<br />
* {{App|1=tkPacman|2=GUI front-end for pacman. Depends on Tcl/Tk and X11 but neither on GTK+, nor on QT. It only interacts with the package database via the CLI of 'pacman'. So, installing and removing packages with tkPacman or with pacman leads to exactly the same result.<br />
:* Browse packages available in repositories;<br />
:* Browse installed packages;<br />
:* Many ways for filtering packages (word, group, repo, upgrades, orphans, explicit, foreign, fileowner);<br />
:* Display detailed information of packages;<br />
:* Display list of files belonging to installed packages;<br />
:* Refresh package database;<br />
:* Full system upgrade;<br />
:* Install a package from a local file;<br />
:* View pacman.log file.<br />
|3='''Web page:'''http://sourceforge.net/projects/tkpacman<br />
|4='''AUR : ''' {{AUR|tkpacman}}}}<br />
<br />
=== GNOME/GTK+ ===<br />
<br />
* {{App|GNOME PackageKit|Distribution-agnostic collection of utilities for managing packages. Using the alpm backend, it supports the following features:<br />
:* Install and remove packages from the repos.<br />
:* Periodically refresh package databases and prompt for updates.<br />
:* Install packages from tarballs.<br />
:* Search for packages by name, description, category or file.<br />
:* Show package dependencies, files and reverse dependencies.<br />
:* Ignore IgnorePkgs and hold HoldPkgs.<br />
:* Report optional dependencies, .pacnew files, etc.<br />
:You can change the remove operation from -Rc to -Rsc by setting the DConf key org.gnome.packagekit.enable-autoremove.<br />
|http://packagekit.org/|{{Pkg|gnome-packagekit}}}}<br />
* {{App|zenity_pacgui|Zenity GUI for Pacman.|http://sourceforge.net/projects/zenitypacgui/|{{AUR|zenity_pacgui}}}}<br />
* {{App|Argon|lightweight python GUI package manager<br />
:* package installation/removal and system update, including AUR (via pacaur)<br />
:* top-level package list<br />
:* configurable update notifier<br />
|http://code.google.com/p/arch-argon|{{AUR|argon}}}}<br />
<br />
=== KDE/Qt ===<br />
<br />
* {{App|1=KPackageKit/Apper|2=GUI frontend for [http://www.packagekit.org/ PackageKit]. Pacman integration is accomplished via the {{Pkg|packagekit}}, which gained upstream support for pacman. This graphical tool allows to do the following from KDE's systemsettings:<br />
:* Install/remove/upgrade packages<br />
:* Search packages/filter packages<br />
:* Retrieve package info<br />
:* Refresh package database<br />
:* Choose which repositories will be updated<br />
:* Automatically refresh database (Hourly, daily etc.)<br />
:* Automatically update packages<br />
:While pacman support in PackageKit is relatively new, it works with no major problems, providing ease of use, simplicity, and good integration with KDE (and PolicyKit).<br />
:'''Screenshots:''' http://kde-apps.org/content/show.php/Apper?content=84745<br />
|3=http://kde-apps.org/content/show.php/Apper?content=84745|4={{Pkg|apper}}}}<br />
<br />
* {{App|1=AppSet|2=Advanced and feature rich GUI front-end for Package Managers. AppSet has the following features:<br />
:* Software sections (games, office, multimedia, internet etc.)<br />
:* Shows homepages for selected packages in an embedded web browser<br />
:* Shows distributions news with an embedded feed reader<br />
:* Upgrades, installs and removes packages<br />
:* Shows available upgrades with a Tray Icon<br />
:* Updates database periodically<br />
:* Informs about dependencies (for example when trying to remove a package needed by others)<br />
:* Cache clean command (to free disk space)<br />
:* Intelligent launcher that uses what is already installed to get administrative privileges (by searching for kdesu, gksu or at last for an xterm where it starts with a sudo command)<br />
:* Now with AUR support with Packer as backend<br />
:AppSet needs only QT libs as dependence for installation. It can be used in any desktop environment. Currently only works for Archlinux using pacman.<br />
:'''Screenshots''' http://sourceforge.net/project/screenshots.php?group_id=376825<br />
|3=http://appset.sourceforge.net/|4={{AUR|appset-qt}}}}<br />
<br />
* {{App|1=Octopi|2=Powerful Pacman frontend written in Qt. Features include:<br />
:* LOW in resource consumption (including memory)<br />
:* FAST<br />
:* Supports Cinnamon, KDE 4.x, XFCE, LXDE, LXQt, MATE, Openbox and TDE<br />
:* Systemtray icon providing notifier support<br />
:* Pacman sync database, system upgrade and clean cache support<br />
:* Yaourt / Pacaur support<br />
:* Install/Re install/Upgrade/Remove selected packages – watching the output of these commands on demand – in a trasaction based abstraction<br />
:* View the contents of installed packages (including opening and editing its files)<br />
:* View the description of packages in tooltips, just moving the mouse over them<br />
:Octopi needs QT(4/5) libs as dependence for installation.<br />
:'''Screenshots''' http://octopiproject.wordpress.com/screenshots/<br />
|3=http://octopiproject.wordpress.com/|4={{AUR|octopi}}}}<br />
<br />
=== Ncurses ===<br />
<br />
* {{App|1=pcurses|2=Package management in a curses frontend, including:<br />
:* regexp filtering and searching any package property<br />
:* customizable colorcoding<br />
:* customizable sorting<br />
:* external command execution with package list string replacements<br />
:* user defined macros and hotkeys<br />
:'''Screenshots''' https://bbs.archlinux.org/viewtopic.php?id=122749<br />
|3=https://github.com/schuay/pcurses|4={{Pkg|pcurses}}}}<br />
<br />
* {{App|1=yaourt-gui|2=Designed for new users who want to start using Arch Linux. Written in Bash, it offers a GUI from terminal to the common tasks of yaourt and pacman.<br />
:'''Screenshots''' http://sourceforge.net/projects/yaourt-gui/ <br><br />
:'''Direct link to source:''' http://sourceforge.net/projects/yaourt-gui/files/yaourt-gui-0.9.tar.gz <br />
|3='''Web page:'''http://alexiobash.com/yaourt-gui-a-bash-gui-per-yaourt-3/<br />
|4='''AUR : ''' {{AUR|yaourt-gui}}}}<br />
<br />
== Pacman/AUR package browser ==<br />
<br />
* {{App|1=PkgBrowser|2=Application for searching and browsing Arch packages, showing details on selected packages.<br />
:* Search and browse Arch packages including the AUR<br />
:* Purely an informational application that cannot be used to install, remove or update packages <br />
:* By design, is an accessory to CLI package management via pacman<br />
:* Further details on use via manual accessed from help menu<br />
:'''Forum page:''' https://bbs.archlinux.org/viewtopic.php?id=117297 <br><br />
|3=https://code.google.com/p/pkgbrowser/|4={{AUR|pkgbrowser}}}}<br />
<br />
* {{App|Pacinfo|Application to browse the installed packages and show information like screenshot, installed files, installation date and others. Written in Mono/GTK#<br />
|https://code.google.com/p/pacinfo/|{{AUR|pacinfo}}}}<br />
<br />
== System tray notifiers ==<br />
<br />
* {{App|1=Aarchup|2=Fork of archup. Has the same options as archup plus a few other features. For differences between both please check [https://bbs.archlinux.org/viewtopic.php?id=119129 changelog].<br />
:'''Screenshots''': http://i.imgur.com/yTNvg.png<br />
|3=https://github.com/aericson/aarchup/|4={{AUR|aarchup}}}}<br />
<br />
* {{App|pacman-notifier|Written in Ruby, uses GTK+. Shows an icon in the system tray and popup notifications (using libnotify) for new packages.<br />
:'''Screenshots''': https://github.com/v01d/pacman-notifier/wiki<br />
|https://github.com/v01d/pacman-notifier/wiki|{{AUR|pacman-notifier}}}}<br />
<br />
* {{App|Pacupdate|Small application that notifies the user about new updates for Arch Linux. If Pacupdate finds out that a update is available, it will display a notification in SystemTray|https://code.google.com/p/pacupdate/|{{AUR|pacupdate-svn}}}}<br />
<br />
* {{App|1=Yapan|2=Written in C++ and Qt. It shows an icon in the system tray and popup notifications for new packages and supports AUR helpers.<br />
:'''Forum page''': https://bbs.archlinux.org/viewtopic.php?id=113078<br />
|3=http://code.google.com/p/arch-yapan/|4={{AUR|yapan}}}}<br />
<br />
* {{App|1=ZenMan|2=Pacman frontend (tray update notifier) for GTK+/GNOME/zenity/libnotify.<br />
:'''Screenshots''': http://show.harvie.cz/screenshots/zenman-screenshot-2.png<br />
|3=https://aur.archlinux.org/packages.php?ID=25948|4={{AUR|zenman}}}}<br />
<br />
* {{App|1=pkgnotify.sh|2=Simple 14 line shell script that displays the number of available updates in the dzen2 title window and a list of these updates in the slave window. Depends on dzen2, inotify-tools, package-query and optionally an AUR helper (yaourt by default).<br />
:'''Screenshots''': http://andreasbwagner.tumblr.com/post/853471635/arch-linux-update-notifier-for-dzen2<br />
|3=http://pointfree.net/repo/?r=dzen2_scripts;a=headblob;f=/src/pkgnotify/pkgnotify.sh|4={{AUR?|pkgnotify}}}}<br />
<br />
* {{App|1=kalu|2=Small C application that adds an icon in the systray and can show notifications for Arch Linux News, Upgrades, AUR upgrades, and watched (AUR) upgrades (upgrades for packages not installed). Also includes a GUI system upgrader.<br />
:'''Screenshots''': http://jjacky.com/kalu<br />
:'''Forum''': https://bbs.archlinux.org/viewtopic.php?id=135773<br />
|3=https://github.com/jjk-jacky/kalu|4={{AUR|kalu}}}}<br />
<br />
== Inactive software packages ==<br />
<br />
* [https://code.google.com/p/wakka-package-manager/ Wakka] - Next generation of gtkpacman<br />
* [http://gtkpacman.berlios.de/ GtkPacman] - GTK+ frontend<br />
* [http://guzuta.berlios.de/ Guzuta] - GTK+ frontend.<br />
* [http://chakra-linux.org/wiki/index.php/Shaman Shaman] - GUI using Pacman’s ''libalpm'' library<br />
* [http://code.google.com/p/pacmon/ pacmon] - notification GUI.<br />
* [https://gna.org/projects/paku/ Paku] - GUI alternative to Pacman.<br />
* [http://www.kde-apps.org/content/show.php/YAPG+-+Yet+Another+Pacman+Gui+?content=60052 YAPG].</div>Justin8https://wiki.archlinux.org/index.php?title=Arch_Linux_on_a_VPS&diff=317119Arch Linux on a VPS2014-05-29T11:21:04Z<p>Justin8: Removed Digital Ocean as they have deprecated Arch support.</p>
<hr />
<div>[[Category:Getting and installing Arch]]<br />
[[Category:Virtualization]]<br />
{{Related articles start}}<br />
{{Related|Comprehensive Server Guide}}<br />
{{Related articles end}}<br />
From [[Wikipedia:Virtual private server]]:<br />
<br />
:''Virtual private server (VPS) is a term used by Internet hosting services to refer to a virtual machine. The term is used for emphasizing that the virtual machine, although running in software on the same physical computer as other customers' virtual machines, is in many respects functionally equivalent to a separate physical computer, is dedicated to the individual customer's needs, has the privacy of a separate physical computer, and can be configured to run server software.''<br />
<br />
This article discusses the use of Arch Linux on Virtual Private Servers, and includes some fixes and installation instructions specific to VPSes.<br />
<br />
{{Warning|1=[https://bbs.archlinux.org/viewtopic.php?id=176171 It appears] that systemd does not support Linux 2.6.32 since systemd-205. Since many container-based virtualization environments rely on older kernels (the latest OpenVZ runs on a modified RHEL6-2.6.32 for example), it may be impossible to keep an Arch Linux install up to date. Most of the instructions regarding OpenVZ on this page were written for systemd-204 and earlier.}}<br />
<br />
==Providers that offer Arch Linux==<br />
<br />
{{Warning|We cannot vouch for the honesty or quality of any provider. Please conduct due diligence before ordering.}}<br />
{{Note|This list is for providers with a convenient Arch Linux image. Using Arch on other providers is probably possible, but would require loading custom ISOs or disk images or [[Installation guide|installing under chroot]].}}<br />
<br />
{| class="wikitable"<br />
! Provider !! Arch Release !! Virtualization !! Locations !! Notes<br />
|-<br />
| [http://www.milesweb.com/vps-hosting.php A MilesWeb VPS] || 2013.10.14 || OpenVZ || Europe, India, US || Latest Arch Linux available on OpenVZ platform. Quick setup, 24/7 support via Live Chat, Email and Phone. VPS starts from $20 / mo<br />
|-<br />
| [http://123systems.net 123 Systems] || 2010.05.xx || OpenVZ || Dallas, US-TX || Arch available as a selection upon reinstall. Very old (2.6.18-308) kernel - See [[Virtual_Private_Server#OpenVZ:_kernel_too_old_for_glibc|OpenVZ troubleshooting]]. Limited information available before purchase. Cannot verify Arch Linux version without purchase.<br />
|-<br />
| [http://ausweb.com.au AUSWEB] || Latest Only (clarify?) || VMware ESXi || Sydney, AU || Latest ISO (clarify?) of Arch Available. Enterprise Service. <br />
|-<br />
| [https://www.affinity.net.nz affinity.net.nz] || 2013.08.01 || KVM || Auckland, New Zealand (NZ) || IRC channel is #affinity on ircs.kiwicon.org<br />
|-<br />
| [http://afterburst.com/ Afterburst] || 2012.12.01 || OpenVZ || Miami, US-FL; Nuremberg, DE || Formerly FanaticalVPS, kernel version depends on what node your VPS is on, the ones in Miami are fine (2.6.32-042stab072.10) but some of the ones in Germany require a [[Virtual_Private_Server#OpenVZ:_kernel_too_old_for_glibc|custom glibc]].<br />
|-<br />
| [http://www.buyvm.net/ BuyVM] || 2013.07.01 || KVM || LA, Buffalo NY || Must chose a different OS at sign up. Once accessible, choose to mount the latest Arch ISO and reboot to install manually. <br />
|-<br />
| [http://en.edis.at/ Edis] || [http://www.edis.at/en/support-and-service/faq/server-faq/which-distributions-are-available-with-edis-kvm-vps-plans/ 2013.03.01] || vServer, KVM, OpenVZ || [http://www.edis.at/en/server/kvm-vps/austria/ Multiple international locations]. || Also offer dedicated server options as well as an "off-shore" location at the Isle of Man (IM).<br />
|-<br />
| [https://www.directvps.nl/ DirectVPS] || 2014.01.xx || OpenVZ || Amsterdam, AN; Rotterdam, AN || Dutch language site. Version verifyable by clicking through https://www.directvps.nl/try-1.plp?p=31<br />
|-<br />
| [https://www.gandi.net/hosting/ Gandi] || 2013.10.27 || Xen || Paris, FR; Baltimore, MD, US; Bissen, LU || Very granular scaling of system resources (e.g. RAM, disk space); IPv6-only option available; you can supply your own install image, version based on keyring package version ||<br />
|-<br />
| [https://www.gigatux.com/virtual.php GigaTux] || [https://www.gigatux.com/distro/ 2013.06.01] || Xen || Chicago, US-IL; Frankfurt, DE; London, GB; San Jose, US-CA ||<br />
|-<br />
| [http://www.vr.org/ Host Virtual] || [http://www.vr.org/os/linux-vps/archlinux-vps 2011.08.19] || KVM || [http://www.vr.org/cloud-locations/ Multiple International Locations] || Appears to use KVM virtualization. Site lists "Xen based virtualization" and [http://www.vr.org/features/ features] lists ability to install from ISO.<br />
|-<br />
| [https://hostigation.com/ Hostigation] || [https://hostigation.com/wiki/index.php?title=KVM:Install 2010.05 i686] || OpenVZ, KVM || Charlotte, US-NC; Los Angeles, US-CA || You can [[Migrating Between Architectures Without Reinstalling|migrate to x86_64]].<br />
|-<br />
| [http://www.intovps.com IntoVPS] || 2012.09.xx || OpenVZ || Amsterdam, NA; Bucharest, RO; Dallas, US-TX; Fremont, US-CA; London, GB || Blog has not been updated since September, 2012 which included the Arch Linux update.<br />
|-<br />
| [https://leapswitch.com Leapswitch Networks] || [2013.10.xx] || OpenVZ/KVM || USA, India, Portugal, Spain, Ukraine, Germany || ArchLinux currently available in Control Panel for reinstall, not on order form. <br />
|-<br />
| [https://www.linode.com Linode.com] || [https://www.linode.com/faq.cfm 2013.06.xx] || OpenVZ|| [https://www.linode.com/speedtest/ Tokyo, JP; Multiple US; London, GB] || To run a custom kernel, install {{AUR|linux-linode}}. ({{pkg|linux}} will break on a 32-bit Linode.)<br />
|-<br />
| [http://lylix.net/home Lylix] || [https://customer.lylix.net/announcements/21/New-64-bit-Linux-Distributions-and-32-bit-Ubuntu-1304.html 2013.xx.xx] || Unlisted || Unlisted || Core2Duo and Woodcrest based processors. <br />
|-<br />
| [http://www.nodedeploy.com Node Deploy] || xxxx.xx.xx || OpenVZ, KVM || Germany (DE); Los Angeles, US-CA; Atlanta, US-GA; Phoenix, US-AZ || "At NodeDeploy we support virtually every linux distribution." Arch Linux is listed under their Operating Systems. No version information.<br />
|-<br />
| [http://netcup.de Netcup] || 2012.11.xx || KVM || Germany (DE)|| German language site. <br />
|-<br />
| [http://onepoundwebhosting.co.uk OnePoundWebHosting] || 2013.05.xx || Xen PV, Xen HVM || United Kingdom (UK) || They are a registrar too. Unable to verify server locations.<br />
|-<br />
| [https://www.proplay.biz/ proPlay.de] || 2012.12.xx || OpenVZ, KVM || Germany (DE) || German language site.<br />
|-<br />
| [https://www.quickvz.com QuickVZ] || 2013.10 || OpenVZ, Xen || Amsterdam, Netherlands (NL); Stockholm, Sweden (SE) || Provide hardened Arch Linux images along with Enterprise services (e,g. VPN, Virtual Private LAN Service (VPLS) and Virtual Routers.<br />
|-<br />
| [http://www.rackspace.com/cloud/cloud_hosting_products/servers/ Rackspace Cloud] || 2013.6 || Xen || [https://www.rackspace.com/whyrackspace/network/datacenters/ Multiple international locations] || Billed per hour. Use their "next gen" VPSes (using the mycloud.rackspace.com panel); the Arch image on the first gen Rackspace VPSes is out of date.<br />
|-<br />
| [http://www.ramhost.us RamHost.us] || [http://www.ramhost.us/?page=news 2013.05.01] || OpenVZ, KVM || Los Angeles, US-CA; Great Britain (GB); Atlanta, US-GA; Germany (DE) || You can request a newer ISO on RamHost's IRC network.<br />
|-<br />
| [http://www.ramnode.com RamNode] || [https://clientarea.ramnode.com/knowledgebase.php?action=displayarticle&id=48 2013.07.01] || [https://clientarea.ramnode.com/knowledgebase.php?action=displayarticle&id=39 SSD and SSD Cached:] [https://clientarea.ramnode.com/knowledgebase.php?action=displayarticle&id=52 OpenVZ, KVM] || [https://clientarea.ramnode.com/knowledgebase.php?action=displayarticle&id=18 Seattle, WA USA, Atlanta, GA USA] || [https://clientarea.ramnode.com/knowledgebase.php?action=displayarticle&id=66 You can request Host/CPU passthrough with KVM service.] [http://www.ramnode.com/about.php Customer service has been prompt and professional.] [https://twitter.com/search?q=ramnode%20code&src=typd Regular discount codes can be found (15-35% off).] [http://www.ramnode.com/index.php Modern hardware.] [https://clientarea.ramnode.com/cart.php?carttpl=svz Competitive pricing (before discounts).]<br />
|-<br />
| [http://www.tilaa.nl/ Tilaa] || 2013.06.01 || [https://www.tilaa.com/pages/vps/technology KVM] || Amsterdam, NL || English or Dutch language site.<br />
|-<br />
| [https://www.transip.eu/ TransIP] || [https://www.transip.eu/vps/vps-os/ 2013.05.01] || [https://www.transip.eu/vps/vps-technology/ KVM] || Amsterdam, NL || English language site. Registrar.<br />
|-<br />
| [http://www.xenvz.co.uk/ XenVZ] || 2009.12.07 || OpenVZ, Xen || United Kingdom (UK), United States (US) || [http://www.xenvz.co.uk/faq.php#use2 Hardware]<br />
|-<br />
| [http://www.virpus.com/ Virpus] || [http://virpus.com/tour/ 2013.05.xx] || OpenVZ, Xen || Kansas City, US-KS; Los Angeles, US-CA ||<br />
|-<br />
| [http://www.vmline.pl/ Vmline] || 2013.09.01 || KVM, OpenVZ || Kraków, PL || [http://www.s-net.pl/en/ S-Net] reseller. Full virtualization. Polish language site.<br />
|-<br />
| [https://vpsbg.eu/ VPSBG.eu] || 2013.10 || OpenVZ || [https://vpsbg.eu/en/index.php?page=vps-datacenter Sofia, Bulgaria] || Offshore VPS in Bulgaria - anonymous registrations and Bitcoin are accepted.<br />
|-<br />
| [https://vps6.net/ VPS6.NET] || 2013.01.xx || OpenVZ, Xen, HVM-ISO || [http://vps6.net/network/ Multiple US]; Frankfurt, DE; Bucharest, RO; Istanbul, TR || Registrar.<br />
|-<br />
|}<br />
<br />
==Installation==<br />
<br />
===KVM===<br />
{{Expansion|Are there instructions specific to VPSes?}}<br />
See [[QEMU#Preparing an (Arch) Linux guest]].<br />
<br />
===OpenVZ===<br />
<br />
====Updating a 2010.05 installation image====<br />
These instructions assume you have a 2010.05 image from your VPS provider and you would like to get it updated. The biggest work involves preparing {{ic|/lib}} for the symlink upgrade ({{pkg|glibc}} 2.16, and later {{pkg|filesystem}} 2013.01).<br />
<br />
{{Warning|If you are on a older kernel than 2.6.32, please refer [[Virtual_Private_Server#OpenVZ:_kernel_too_old_for_glibc|further down the page]] to get the ''glibc-vps'' repository working (just add the repository and you can follow these steps).}}<br />
<br />
To start, grab the latest BusyBox from http://busybox.net/downloads/binaries/latest/. This allows you to force glibc (losing {{ic|/lib}} temporarily) without losing your OS (BusyBox comes with its own GNU tools which are statically linked).<br />
# wget http://busybox.net/downloads/binaries/latest/busybox-i686<br />
# chmod +x busybox-i686<br />
<br />
First, you can get a list of packages that own files in {{ic|/lib}} with the following command:<br />
{{bc|<nowiki><br />
$ pacman -Qo /lib/* | cut -d' ' -f 5 | egrep -v 'glibc' | uniq | xargs<br />
</nowiki>}}<br />
<br />
For the current 2010.05 that comes from ibiru's page, these are the packages that were required to be removed for me:<br />
<br />
{{bc|pacman -S acl attr util-linux-ng bzip2 libcap e2fsprogs libgcrypt libgpg-error udev readline ncurses pam pcre popt procps readline shadow e2fsprogs sysfsutils udev util-linux-ng sysvinit coreutils}}<br />
<br />
You may have to remove {{ic|/lib/udev/devices/loop0}} (a simple {{ic|rm}} works).<br />
<br />
After the upgrade finishes, you must remove any extra empty directories in {{ic|/lib}} ({{ic|/lib/modules}} is the common offender):<br />
# rm -rf /lib/modules<br />
<br />
Install {{pkg|tzdata}} to fix some dependencies, and remove {{ic|/etc/profile.d/locale.sh}}:<br />
# pacman -S tzdata<br />
# rm /etc/profile.d/locale.sh<br />
<br />
Remove {{ic|/var/run}} (you should have nothing running that matters):<br />
# rm -rf /var/run<br />
<br />
Force glibc, which will pull in the latest filesystem package, but will BREAK everything (other than BusyBox):<br />
# pacman -S --force glibc<br />
<br />
Now, you will have a broken system, so symlink {{ic|/usr/lib}} to {{ic|/lib}} with BusyBox's ln program:<br />
# ./busybox-i686 ln -s /usr/lib /lib<br />
<br />
And you should have a fully functional system where you can now update pacman.<br />
# pacman -S pacman<br />
# pacman-key --init<br />
# pacman-key --populate archlinux<br />
# pacman-db-upgrade<br />
# pacman -Syy<br />
<br />
Now, update initscripts to get the {{pkg|iproute2}} package:<br />
# pacman -S initscripts<br />
<br />
Install the {{pkg|makedev}} package:<br />
{{bc|pacman -S makedev}}<br />
<br />
Add the following to your {{ic|/etc/rc.local}}:<br />
/usr/sbin/MAKEDEV tty<br />
/usr/sbin/MAKEDEV pty<br />
<br />
Comment the following lines in {{ic|/etc/inittab}}:<br />
#c1:2345:respawn:/sbin/agetty -8 -s 38400 tty1 linux<br />
#c2:2345:respawn:/sbin/agetty -8 -s 38400 tty2 linux<br />
#c3:2345:respawn:/sbin/agetty -8 -s 38400 tty3 linux<br />
#c4:2345:respawn:/sbin/agetty -8 -s 38400 tty4 linux<br />
#c5:2345:respawn:/sbin/agetty -8 -s 38400 tty5 linux<br />
#c6:2345:respawn:/sbin/agetty -8 -s 38400 tty6 linux<br />
<br />
Finally, you should be able to upgrade the whole system:<br />
# pacman -Syu<br />
<br />
You may run into some issues with krb5 and heimdal, as krb5 no longer provides+conflicts+replaces heimdal (https://projects.archlinux.org/svntogit/packages.git/commit/trunk/PKGBUILD?h=packages/krb5&id=f5e6d77fd14ced15ebf5b6a78a7c76e0db0625f7). The old openssh depends on heimdal (and the new openssh depends on krb5), so force install krb5, then upgrade openssh, then remove heimdal and reinstall krb5.<br />
# pacman -S --force krb5<br />
# pacman -S openssh openssl<br />
# pacman -R heimdal<br />
# pacman -S krb5<br />
<br />
Fix {{pkg|syslog-ng}}. Set the src to {{ic|unix-dgram("/dev/log")}} and add {{ic|--no-caps}} to both check and run args in {{ic|/etc/conf.d/syslog-ng}}.<br />
<br />
Make sure your {{ic|/etc/rc.conf}} is not messed up with broken network definitions, or else be sure serial access works on your VPS before you reboot.<br />
<br />
====Moving your VPS from network configuration in rc.conf to netcfg (tested with OpenVZ)====<br />
<br />
1) Install netcfg<br />
<br />
{{bc|pacman -S netcfg}}<br />
<br />
2) Create a netcfg configuration file {{ic|/etc/network.d/venet}}<br />
<br />
{{bc|1=CONNECTION='ethernet'<br />
DESCRIPTION='VPS venet connection'<br />
INTERFACE='venet0'<br />
IP='static'<br />
IPCFG=(<br />
#default<br />
'addr add 127.0.0.1/32 broadcast 0.0.0.0 dev venet0'<br />
#IPv4 address<br />
'addr add xxx.xxx.xxx.xxx/32 broadcast 0.0.0.0 dev venet0'<br />
#IPv4 route<br />
'route add default dev venet0'<br />
#IPv6 address<br />
'addr add xxxx:xx:xx::x/128 dev venet0'<br />
#IPv6 route<br />
'-6 route add default dev venet0'<br />
)<br />
DNS=('xxx.xxx.xxx.xxx' 'xxx.xxx.xxx.xxx')}}<br />
<br />
3) Edit your netcfg main conf file {{ic|/etc/conf.d/netcfg}}<br />
<br />
{{bc|1=NETWORKS=(venet)<br />
WIRED_INTERFACE="venet0"}}<br />
<br />
4) Try your new setup<br />
<br />
{{bc|rc.d stop network && ip addr flush venet0 && netcfg venet}}<br />
<br />
Your VPS should still be connected and have its IP addresses set correctly. (Check with {{ic|ip a}})<br />
<br />
DO NOT proceed to next step if this isn't the case.<br />
<br />
5) Make your new setup survive reboots<br />
<br />
In the {{ic|DAEMONS}} array in {{ic|/etc/rc.conf}}, replace {{ic|network}} with {{ic|net-profiles}}.<br />
<br />
Remove all networking information that is in {{ic|/etc/rc.conf}}.<br />
{{bc|reboot}}<br />
<br />
====Moving your VPS from initscripts to systemd====<br />
<br />
{{Warning|This has been known to work with OpenVZ on the 2.6.32 kernel, but systemd may not work on older kernels.}}<br />
<br />
This is very similar to a regular arch system, except you probably don't have access to your kernel line.<br />
<br />
1) Move from network in rc.conf to netcfg (see above).<br />
<br />
2) Install systemd<br />
<br />
{{bc|pacman -S systemd}}<br />
<br />
2 bonus for OpenVZ) Remove kernel core dump pattern since this is blocked by OpenVZ and causes errors<br />
<br />
Edit {{ic|/usr/lib/sysctl.d/coredump.conf}}, comment out the following line:<br />
{{bc|#kernel.core_pattern&#61;&#124;/usr/lib/systemd/systemd-coredump %p %u %g %s %t %e}}<br />
<br />
3) Move all configuration from {{ic|/etc/rc.conf}} (except the {{ic|DAEMONS}} array) to its appropriate location.<br />
<br />
See [https://wiki.archlinux.org/index.php/Systemd#Native_configuration Native configuration] and [https://wiki.archlinux.org/index.php/Rc.conf rc.conf] for details.<br />
<br />
Now your {{ic|/etc/rc.conf}} should only contain the {{ic|DAEMONS}} array.<br />
<br />
4) Install systemd-sysvcompat<br />
<br />
{{bc|pacman -S systemd-sysvcompat}}<br />
<br />
It will ask to replace sysvinit, say yes.<br />
<br />
{{bc|reboot}}<br />
<br />
5) Move daemons from the {{ic|DAEMONS}} array in {{ic|/etc/rc.conf}} to {{ic|systemd}}<br />
<br />
See the [https://wiki.archlinux.org/index.php/Systemd#Moving_away_from_the_DAEMONS_array guide] and the [https://wiki.archlinux.org/index.php/Daemons_List daemons list].<br />
<br />
If your {{ic|DAEMONS}} array is now empty, skip next step.<br />
<br />
6) Moving rc.d daemons with no systemd support, example: {{ic|vzquota}}<br />
<br />
Create a custom systemd service file for vzquota: {{ic|/etc/systemd/system/newvzquota.service}}:<br />
{{bc|1=[Unit]<br />
Description=Setup vzquota on VPS<br />
ConditionFileIsExecutable=/etc/rc.d/vzquota<br />
<br />
[Service]<br />
Type=oneshot<br />
ExecStart=/etc/rc.d/vzquota start<br />
ExecStop=/etc/rc.d/vzquota stop<br />
TimeoutSec=0<br />
StandardInput=tty<br />
RemainAfterExit=yes<br />
<br />
[Install]<br />
WantedBy=multi-user.target}}<br />
<br />
{{Note|It is recommended to choose a .service file name that is different from the name of the daemon, because systemd might try to call the LEGACY scripts with the old name.}}<br />
<br />
Enable this service:<br />
{{bc|systemctl enable newvzquota.service}}<br />
<br />
Remove {{ic|vzquota}} from the {{ic|DAEMONS}} array in {{ic|/etc/rc.conf}}<br />
<br />
Repeat this step to remove all daemons from {{ic|/etc/rc.conf}}.<br />
<br />
7) Removing {{ic|/etc/rc.local}} and {{ic|/etc/rc.local.shutdown}}<br />
<br />
Write [https://wiki.archlinux.org/index.php/Systemd#Writing_custom_.service_files custom .service files] to replace functionality in {{ic|/etc/rc.local}} and {{ic|/etc/rc.local.shutdown}}. You can take a look at {{ic|/usr/lib/systemd/system/rc-local.service}} and {{ic|/usr/lib/systemd/system/rc-local-shutdown.service}} for inspiration.<br />
<br />
8) Removing {{ic|initscripts}}<br />
<br />
Your {{ic|/etc/rc.conf}} file must look like this:<br />
{{bc|1=DAEMONS=()}}<br />
and {{ic|/etc/rc.local}} and {{ic|/etc/rc.local.shutdown}} must now be empty.<br />
<br />
Uninstall {{ic|initscripts}}<br />
{{bc|pacman -R initscripts}}<br />
<br />
{{bc|reboot}}<br />
<br />
===Xen===<br />
{{Expansion|Are there instructions specific to VPSes?}}<br />
See [[Xen#Arch as Xen guest (PVHVM mode)]] and/or [[Xen#Arch as Xen guest (PV mode)]].<br />
<br />
===Converting OpenStack and Xen components to systemd===<br />
There are three components that need to be enabled in systemd when using a VPS based on OpenStack/Xen, such as Rackspace NextGen Cloud. The current version of {{Pkg|xe-guest-utilities}} contains two of these: xe-linux-distribution and xe-daemon.<br />
<br />
You will need to create a custom service file for the OpenStack nova-agent, as the current version 0.0.1.37 only comes with a sysvinit start-up script.<br />
<br />
{{hc|1=/etc/systemd/system/nova-agent.service|2=<br />
[Unit]<br />
Description=nova-agent service<br />
After=xe-daemon.service<br />
<br />
[Service]<br />
Environment=LD_LIBRARY_PATH=/usr/share/nova-agent/0.0.1.37/lib<br />
ExecStart=usr/bin/nova-agent -n -l info /usr/share/nova-agent/nova-agent.py<br />
<br />
[Install]<br />
WantedBy=multi-user.target}}<br />
Once these steps are done, you can continue with converting the server from sysvinit to systemd.<br />
<br />
Make sure to enable the following services:<br />
# systemctl enable xe-linux-distribution<br />
# systemctl enable xe-daemon<br />
# systemctl enable nova-agent<br />
<br />
==Troubleshooting==<br />
===OpenVZ: kernel too old for glibc===<br />
Are you on a virtual private server (VPS) with an old kernel & broke your system? Are you using OpenVZ?<br />
<br />
Check your kernel version with:<br />
<br />
{{bc|uname -r}}<br />
<br />
If your kernel is older than 2.6.32, you will need a custom version of glibc ([https://www.archlinux.org/news/minimum-kernel-requirement-2632/ because of dependencies in glibc]).<br />
<br />
Arch Template Used: https://dev.archlinux.org/~ibiru/openvz/2010.05/arch-2010.05-i686-minimal.tar.gz<br />
<br />
{{Note|For installs that have not been updated to glibc-2.16, it will save you lots of time and prevent major breakage to do:<br />
pacman -U https://dev.archlinux.org/~ibiru/openvz/glibc-vps/i686/glibc-2.16.0-101-i686.pkg.tar.xz<br />
or<br />
pacman -U https://dev.archlinux.org/~ibiru/openvz/glibc-vps/x86_64/glibc-2.16.0-101-x86_64.pkg.tar.xz<br />
Add a single {{ic|-d}} if needed. ''The instructions below assume that this has been done.''<br />
}}<br />
<br />
Following similar instructions from [[DeveloperWiki:usrlib]].<br />
<br />
Try doing the following to fix it:<br />
<br />
1) Edit {{ic|/etc/pacman.conf}} and add the following repository '''ABOVE''' {{ic|[core]}}:<br />
<br />
For 32-bit:<br />
{{bc|<nowiki>[glibc-vps]<br />
Server = https://dev.archlinux.org/~ibiru/openvz/glibc-vps/i686</nowiki>}}<br />
<br />
For 64-bit:<br />
{{bc|<nowiki>[glibc-vps]<br />
Server = https://dev.archlinux.org/~ibiru/openvz/glibc-vps/x86_64</nowiki>}}<br />
<br />
2) Then run {{ic|pacman -Syy}} followed by {{ic|pacman -Syu}}. You will be notified to upgrade pacman first.<br />
<br />
3) Upgrade the [[pacman]] database by running {{ic|pacman-db-upgrade}} as root.<br />
<br />
4) Edit {{ic|/etc/pacman.conf.pacnew}} (new pacman config file) and add the following repository '''ABOVE''' {{ic|[core]}}:<br />
{{bc|<nowiki>[glibc-vps]<br />
Server = https://dev.archlinux.org/~ibiru/openvz/glibc-vps/$arch</nowiki>}}<br />
<br />
5) Replace {{ic|/etc/pacman.conf}} with {{ic|/etc/pacman.conf.pacnew}} (run as root):<br />
# mv /etc/pacman.conf.pacnew /etc/pacman.conf<br />
<br />
6) Upgrade your whole system with new packages again {{ic|pacman -Syu}}<br />
<br />
If you get the following error or a similar error:<br />
{{bc|initscripts: /etc/profile.d/locale.sh exists in filesystem}}<br />
<br />
Simply delete that file (e.g., {{ic|rm -f /etc/profile.d/locale.sh}}), then run {{ic|pacman -Syu}} again.<br />
<br />
If you get the following error or a similar error:<br />
{{bc|filesystem: /etc/mtab exists in filesystem}}<br />
<br />
Run {{ic|pacman -S filesystem --force}}<br />
<br />
If you get the following error or a similar error:<br />
{{bc|libusb-compat: /usr/bin/libusb-config exists in filesystem}}<br />
<br />
Run {{ic|pacman -S libusb}} and then {{ic|pacman -S libusb-compat}}<br />
<br />
7) Before rebooting, you need to [[pacman|install]] the {{Pkg|makedev}} package from the [[official repositories]] by running {{ic|pacman -S makedev}}.<br />
<br />
8) Add MAKEDEV to {{ic|/etc/rc.local}}:<br />
/usr/sbin/MAKEDEV tty<br />
/usr/sbin/MAKEDEV pty<br />
<br />
9) Edit {{ic|/etc/inittab}}, comment out the following lines; otherwise, you will see errors in {{ic|/var/log/errors.log}}):<br />
#c1:2345:respawn:/sbin/agetty -8 -s 38400 tty1 linux<br />
#c2:2345:respawn:/sbin/agetty -8 -s 38400 tty2 linux<br />
#c3:2345:respawn:/sbin/agetty -8 -s 38400 tty3 linux<br />
#c4:2345:respawn:/sbin/agetty -8 -s 38400 tty4 linux<br />
#c5:2345:respawn:/sbin/agetty -8 -s 38400 tty5 linux<br />
#c6:2345:respawn:/sbin/agetty -8 -s 38400 tty6 linux<br />
<br />
10) To enable use of the {{ic|hostname}} command, [[pacman|install]] the package {{Pkg|inetutils}} from the [[official repositories]]. <br />
<br />
11) Remove disabling of the SysRq key and setup of core dump pattern because this is blocked by OpenVZ and causes errors.<br />
<br />
Because sysctl does not use {{ic|/etc/sysctl.conf}} any more[https://www.archlinux.org/news/deprecation-of-etcsysctlconf/], you must transfer all settings to {{ic|/etc/sysctl.d/99-sysctl.conf}} (or any other file in {{ic|/etc/sysctl.d/}}; however, do not transfer the following line:<br />
{{bc|1=kernel.sysrq = 0}}<br />
<br />
Edit {{ic|/usr/lib/sysctl.d/coredump.conf}} and comment out the following line:<br />
{{bc|1=#kernel.core_pattern=&#124;/usr/lib/systemd/systemd-coredump %p %u %g %s %t %e}}<br />
<br />
12) Save and reboot.<br />
<br />
Enjoy & thank ioni if you happen to be in #archlinux<br />
<br />
===SSH fails: PTY allocation request failed on channel 0===<br />
<br />
Some VPSes have an outdated {{ic|/etc/rc.sysinit}}. You may be able to log in via serial console or with the following command:<br />
$ ssh root@broken.server '/bin/bash -i'<br />
<br />
Then run the following:<br />
# mv /etc/rc.sysinit.pacnew /etc/rc.sysinit<br />
# reboot<br />
<br />
Once it is working, you should be able to comment out the {{ic|udevd_modprobe}} line in {{ic|/etc/rc.sysinit}} to save a bit of RAM the next time you reboot.<br />
<br />
If the above does not work, take a look at [http://fsk141.com/fix-pty-allocation-request-failed-on-channel-0 this guide].</div>Justin8https://wiki.archlinux.org/index.php?title=Network_configuration&diff=303297Network configuration2014-03-06T08:47:01Z<p>Justin8: Updated from 80-net-name-slot.rules to 80-net-setup-link.rules due to naming change in systemd v210</p>
<hr />
<div>[[Category:Networking]]<br />
[[Category:Getting and installing Arch]]<br />
[[cs:Configuring Network]]<br />
[[el:Configuring Network]]<br />
[[es:Configuring Network]]<br />
[[fr:Connexions reseau]]<br />
[[it:Configuring Network]]<br />
[[ja:Network Configuration]]<br />
[[nl:Configuring Network]]<br />
[[pt:Configuring Network]]<br />
[[ro:Configurare retea]]<br />
[[ru:Configuring Network]]<br />
[[sk:Configuring Network]]<br />
[[tr:Ağ_Yapılandırması]]<br />
[[zh-CN:Network Configuration]]<br />
[[zh-TW:Network Configuration]]<br />
{{Related articles start}}<br />
{{Related|Jumbo Frames}}<br />
{{Related|Firewalls}}<br />
{{Related|Wireless network configuration}}<br />
{{Related|List of Applications#Network Managers}}<br />
{{Related articles end}}<br />
<br />
This page explains how to set up a '''wired''' connection to a network. If you need to set up '''wireless''' networking see the [[Wireless network configuration]] page.<br />
<br />
== Check the connection ==<br />
<br />
{{Note|If you receive an error like {{ic|ping: icmp open socket: Operation not permitted}} when executing ping, try to re-install the {{ic|iputils}} package.}} <br />
<br />
Many times, the basic installation procedure has created a working network configuration. To check if this is so, use the following command:<br />
<br />
{{Note|The {{ic|-c 3}} option calls it three times. See {{ic|man ping}} for more information.}}<br />
<br />
{{hc|$ ping -c 3 www.google.com|<nowiki><br />
PING www.l.google.com (74.125.224.146) 56(84) bytes of data.<br />
64 bytes from 74.125.224.146: icmp_req=1 ttl=50 time=437 ms<br />
64 bytes from 74.125.224.146: icmp_req=2 ttl=50 time=385 ms<br />
64 bytes from 74.125.224.146: icmp_req=3 ttl=50 time=298 ms<br />
<br />
--- www.l.google.com ping statistics ---<br />
3 packets transmitted, 3 received, 0% packet loss, time 1999ms<br />
rtt min/avg/max/mdev = 298.107/373.642/437.202/57.415 ms<br />
</nowiki>}}<br />
<br />
If it works, then you may only wish to personalize your settings from the options below.<br />
<br />
If the previous command complains about unknown hosts, it means that your machine was unable to resolve this domain name. It might be related to your service provider or your router/gateway. You can try pinging a static IP address to prove that your machine has access to the Internet.<br />
<br />
{{hc|$ ping -c 3 8.8.8.8|<nowiki><br />
PING 8.8.8.8 (8.8.8.8) 56(84) bytes of data.<br />
64 bytes from 8.8.8.8: icmp_req=1 ttl=53 time=52.9 ms<br />
64 bytes from 8.8.8.8: icmp_req=2 ttl=53 time=72.5 ms<br />
64 bytes from 8.8.8.8: icmp_req=3 ttl=53 time=70.6 ms<br />
<br />
--- 8.8.8.8 ping statistics ---<br />
3 packets transmitted, 3 received, 0% packet loss, time 2002ms<br />
rtt min/avg/max/mdev = 52.975/65.375/72.543/8.803 ms<br />
</nowiki>}}<br />
<br />
{{Note|{{ic|8.8.8.8}} is a static address that is easy to remember. It is the address of Google's primary DNS server, therefore it can be considered reliable, and is generally not blocked by content filtering systems and proxies.}}<br />
<br />
If you are able to ping {{ic|8.8.8.8}} but not {{ic|www.google.com}}, check your DNS configuration. See [[resolv.conf]] for details.<br />
<br />
== Set the hostname ==<br />
<br />
A [[Wikipedia:Hostname|hostname]] is a unique name created to identify a machine on a network: it is configured in {{ic|/etc/hostname}}. The file can contain the system's domain name, if any. To set the hostname, do:<br />
<br />
# hostnamectl set-hostname ''myhostname''<br />
<br />
This will put {{ic|''myhostname''}} into {{ic|/etc/hostname}}.<br />
<br />
See {{ic|man 5 hostname}} and {{ic|man 1 hostnamectl}} for details.<br />
<br />
{{Note|<br />
* {{ic|hostnamectl}} supports FQDNs<br />
* You no longer need to edit {{ic|/etc/hosts}}, {{pkg|systemd}} will provide host name resolution, and is installed on all systems by default.<br />
}}<br />
<br />
To set the hostname temporarily (until a reboot), use ''hostname'' from {{Pkg|inetutils}}:<br />
<br />
# hostname ''myhostname''<br />
<br />
== Device Driver ==<br />
<br />
=== Check the driver status ===<br />
<br />
[[udev]] should detect your network interface card ([[Wikipedia:Network_interface_controller|NIC]]) and automatically load the necessary module at start up. Check the "Ethernet controller" entry (or similar) from the {{ic|lspci -v}} output. It should tell you which kernel module contains the driver for your network device. For example:<br />
<br />
{{hc|$ lspci -v|<br />
02:00.0 Ethernet controller: Attansic Technology Corp. L1 Gigabit Ethernet Adapter (rev b0)<br />
...<br />
Kernel driver in use: atl1<br />
Kernel modules: atl1<br />
}}<br />
<br />
Next, check that the driver was loaded via {{ic|dmesg <nowiki>|</nowiki> grep ''module_name''}}. For example:<br />
<br />
$ dmesg | grep atl1<br />
...<br />
atl1 0000:02:00.0: eth0 link is up 100 Mbps full duplex<br />
<br />
Skip the next section if the driver was loaded successfully. Otherwise, you will need to know which module is needed for your particular model.<br />
<br />
=== Load the device module ===<br />
<br />
Google for the right module/driver for the chipset. Some common modules are {{ic|8139too}} for cards with a Realtek chipset, or {{ic|sis900}} for cards with a SiS chipset. Once you know which module to use, try to [[Kernel modules#Manual module handling|load it manually]]. If you get an error saying that the module was not found, it's possible that the driver is not included in Arch kernel. You may search the [[AUR]] for the module name.<br />
<br />
If udev is not detecting and loading the proper module automatically during bootup, see [[Kernel modules#Loading]].<br />
<br />
== Network Interfaces ==<br />
<br />
=== Device names ===<br />
<br />
For computers with multiple NICs, it is important to have fixed device name. Many configuration problems are caused by interface name changing.<br />
<br />
[[udev]] is responsible for which device gets which name. Systemd v197 introduced [http://www.freedesktop.org/wiki/Software/systemd/PredictableNetworkInterfaceNames Predictable Network Interface Names], which automatically assigns static names to network devices. Interfaces are now prefixed with {{ic|en}} (ethernet), {{ic|wl}} (WLAN), or {{ic|ww}} (WWAN) followed by an automatically generated identifier, creating an entry such as {{ic|enp0s25}}. <br />
<br />
This behavior may be disabled by adding a symlink:<br />
<br />
# ln -s /dev/null /etc/udev/rules.d/80-net-setup-link.rules<br />
<br />
Users upgrading from an earlier systemd version will have a blank rules file created automatically. So if you want to use persistent device names, just delete the file.<br />
<br />
{{Tip|You can run {{ic|ip link}} or {{ic|ls /sys/class/net}} to list all available interfaces.}}<br />
<br />
{{Note|When changing the interface naming scheme, do not forget to update all network-related configuration files and custom systemd unit files to reflect the change. Specifically, if you have [[netctl#Basic method|netctl static profiles]] enabled, run {{ic|netctl reenable ''profile''}} to update the generated service file.}}<br />
<br />
==== Change device name ====<br />
<br />
You can change the device name by defining the name manually with an udev-rule. For example:<br />
<br />
{{hc|/etc/udev/rules.d/10-network.rules|<nowiki><br />
SUBSYSTEM=="net", ACTION=="add", ATTR{address}=="aa:bb:cc:dd:ee:ff", NAME="net1"<br />
SUBSYSTEM=="net", ACTION=="add", ATTR{address}=="ff:ee:dd:cc:bb:aa", NAME="net0"<br />
</nowiki>}}<br />
<br />
A couple things to note:<br />
<br />
* To get the MAC address of each card, use this command: {{ic|cat /sys/class/net/''device_name''/address}}<br />
* Make sure to use the lower-case hex values in your udev rules. It doesn't like upper-case.<br />
<br />
If the network card has a dynamic MAC, you can use DEVPATH for example<br />
<br />
{{hc|/etc/udev/rules.d/10-network.rules|<nowiki><br />
SUBSYSTEM=="net", DEVPATH=="/devices/platform/wemac.*", NAME="int"<br />
</nowiki>}}<br />
<br />
{{Note|When choosing the static names '''it should be avoided to use names in the format of "eth''X''" and "wlan''X''"''', because this may lead to race conditions between the kernel and udev during boot. Instead, it is better to use interface names that are not used by the kernel as default, e.g.: {{ic|net0}}, {{ic|net1}}, {{ic|wifi0}}, {{ic|wifi1}}. For further details please see the [http://www.freedesktop.org/wiki/Software/systemd/PredictableNetworkInterfaceNames systemd] documentation.}}<br />
<br />
=== Set device MTU and queue Length ===<br />
<br />
You can change the device MTU and queue length by defining manually with an udev-rule. For example:<br />
<br />
{{hc|/etc/udev/rules.d/10-network.rules|<nowiki><br />
ACTION=="add", SUBSYSTEM=="net", KERNEL=="wl*", ATTR{mtu}="1480", ATTR{tx_queue_len}="2000"<br />
</nowiki>}}<br />
<br />
=== Get current device names ===<br />
<br />
Current NIC names can be found via sysfs<br />
<br />
{{hc|$ ls /sys/class/net|<br />
lo eth0 eth1 firewire0<br />
}}<br />
<br />
=== Enabling and disabling network interfaces ===<br />
<br />
You can activate or deactivate network interfaces using:<br />
<br />
# ip link set eth0 up<br />
# ip link set eth0 down<br />
<br />
To check the result:<br />
<br />
{{hc|$ ip link show dev eth0|<br />
2: eth0: <BROADCAST,MULTICAST,PROMISC,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast master br0 state UP mode DEFAULT qlen 1000<br />
...<br />
}}<br />
<br />
== Configure the IP address ==<br />
<br />
You have two options: a dynamically assigned address using [[Wikipedia:Dynamic Host Configuration Protocol|DHCP]], or an unchanging "static" address.<br />
<br />
=== Dynamic IP address ===<br />
<br />
The easiest is to use [[dhcpcd]], which is included in the {{Grp|base}} group. Either use the provided service file {{ic|dhcpcd@.service}}, passing the interface name as an argument, or start it manually by running {{ic|dhcpcd ''interface''}}.<br />
<br />
=== Static IP address ===<br />
<br />
There are various reasons why you may wish to assign static IP addresses on your network. For instance, one may gain a certain degree of predictability with unchanging addresses, or you may not have a DHCP server available.<br />
<br />
{{Note|If you share your Internet connection from a Windows machine without a router, be sure to use static IP addresses on both computers to avoid LAN problems.}}<br />
<br />
You need:<br />
<br />
* Static IP address<br />
* [[Wikipedia:Subnetwork|Subnet mask]]<br />
* [[Wikipedia:Broadcast_address|Broadcast address]]<br />
* [[Wikipedia:Default_gateway|Gateway]]'s IP address<br />
<br />
If you are running a private network, it is safe to use IP addresses in 192.168.*.* for your IP addresses, with a subnet mask of 255.255.255.0 and a broadcast address of 192.168.*.255. The gateway is usually 192.168.*.1 or 192.168.*.254.<br />
<br />
{{Tip|You may need to manually set the DNS servers, see [[resolv.conf]] for details.}}<br />
<br />
==== Manual assignment ====<br />
<br />
You can assign a static IP address in the console:<br />
<br />
# ip addr add ''IP_address''/''subnet_mask'' broadcast ''broadcast_address'' dev ''interface''<br />
<br />
For example:<br />
<br />
# ip addr add 192.168.1.2/24 broadcast 192.168.1.255 dev eth0<br />
<br />
{{Note|The subnet mask was specified using [[Wikipedia:CIDR_notation|CIDR notation]].}}<br />
<br />
For more options, see {{ic|man ip}}.<br />
<br />
Add your gateway IP address like so:<br />
<br />
# ip route add default via ''default_gateway''<br />
<br />
For example:<br />
<br />
# ip route add default via 192.168.1.1<br />
<br />
If you the get the error "No such process", it means you have to run {{ic|ip link set dev eth0 up}} as root.<br />
<br />
==== Manual connection at boot using systemd ====<br />
<br />
First create a configuration file for the [[systemd]] service, replace {{ic|''interface''}} with the proper network interface name:<br />
<br />
{{hc|/etc/conf.d/network@''interface''|<nowiki><br />
address=192.168.0.15<br />
netmask=24<br />
broadcast=192.168.0.255<br />
gateway=192.168.0.1<br />
</nowiki>}}<br />
<br />
Create a systemd unit file:<br />
<br />
{{hc|/etc/systemd/system/network@.service|<nowiki><br />
[Unit]<br />
Description=Network connectivity (%i)<br />
Wants=network.target<br />
Before=network.target<br />
BindsTo=sys-subsystem-net-devices-%i.device<br />
After=sys-subsystem-net-devices-%i.device<br />
<br />
[Service]<br />
Type=oneshot<br />
RemainAfterExit=yes<br />
EnvironmentFile=/etc/conf.d/network@%i<br />
<br />
ExecStart=/usr/bin/ip link set dev %i up<br />
ExecStart=/usr/bin/ip addr add ${address}/${netmask} broadcast ${broadcast} dev %i<br />
ExecStart=/usr/bin/sh -c 'test -n ${gateway} && /usr/bin/ip route add default via ${gateway}'<br />
<br />
ExecStop=/usr/bin/ip addr flush dev %i<br />
ExecStop=/usr/bin/ip link set dev %i down<br />
<br />
[Install]<br />
WantedBy=multi-user.target<br />
</nowiki>}}<br />
<br />
Enable the unit and start it, passing the name of the interface:<br />
<br />
# systemctl enable network@''interface''.service<br />
# systemctl start network@''interface''.service<br />
<br />
With [[systemd]] version >= 209, it's possible to manage network connections with the integrated systemd-networkd service (which is disabled by default with version 210). With this approach, only a configuration file per interface is needed, for example:<br />
<br />
{{hc|/etc/systemd/network/eth0.network|<nowiki><br />
[Match]<br />
Name=eth0<br />
<br />
[Network]<br />
Address=192.168.0.15/24<br />
Gateway=192.168.0.1<br />
</nowiki>}}<br />
<br />
Then start/enable {{ic|systemd-networkd.service}}.<br />
<br />
Further examples: https://coreos.com/blog/intro-to-systemd-networkd/<br />
<br />
==== Calculating addresses ====<br />
<br />
You can use {{ic|ipcalc}} provided by the {{Pkg|ipcalc}} package to calculate IP broadcast, network, netmask, and host ranges for more advanced configurations. For example, I use ethernet over firewire to connect a windows machine to arch. For security and network organization, I placed them on their own network and configured the netmask and broadcast so that they are the only 2 machines on it. To figure out the netmask and broadcast addresses for this, I used ipcalc, providing it with the IP of the arch firewire nic 10.66.66.1, and specifying ipcalc should create a network of only 2 hosts.<br />
<br />
{{hc|$ ipcalc -nb 10.66.66.1 -s 1|<nowiki><br />
Address: 10.66.66.1<br />
<br />
Netmask: 255.255.255.252 = 30<br />
Network: 10.66.66.0/30<br />
HostMin: 10.66.66.1<br />
HostMax: 10.66.66.2<br />
Broadcast: 10.66.66.3<br />
Hosts/Net: 2 Class A, Private Internet<br />
</nowiki>}}<br />
<br />
== Load configuration ==<br />
<br />
To test your settings either reboot the computer or reload the relevant systemd services. Then try pinging your gateway, DNS server, ISP provider and other Internet sites, in that order, to detect any connection problems along the way, as in this example:<br />
<br />
$ ping -c 3 www.google.com<br />
<br />
== Additional settings ==<br />
<br />
=== ifplugd for laptops ===<br />
<br />
{{Tip|[[dhcpcd]] provides the same feature out of the box.}}<br />
<br />
{{Pkg|ifplugd}} in [[official repositories]] is a daemon which will automatically configure your Ethernet device when a cable is plugged in and automatically unconfigure it if the cable is pulled. This is useful on laptops with onboard network adapters, since it will only configure the interface when a cable is really connected. Another use is when you just need to restart the network but do not want to restart the computer or do it from the shell.<br />
<br />
By default it is configured to work for the {{ic|eth0}} device. This and other settings like delays can be configured in {{ic|/etc/ifplugd/ifplugd.conf}}.<br />
<br />
{{Note|[[Netctl]] package includes {{ic|netctl-ifplugd@.service}}, otherwise you can use {{ic|ifplugd@.service}} from {{Pkg|ifplugd}} package. Use for example {{ic|systemctl enable ifplugd@eth0.service}}.}}<br />
<br />
=== Bonding or LAG ===<br />
<br />
See [[netctl#Bonding]].<br />
<br />
=== IP address aliasing ===<br />
<br />
{{Expansion|Manual method using [[Core utilities#ip|ip]] should be added; then move current example using ''netctl'' into [[netctl]].}}<br />
<br />
IP aliasing is the process of adding more than one IP address to a network interface. With this, one node on a network can have multiple connections to a network, each serving a different purpose. Typical uses are virtual hosting of Web and FTP servers, or reorganizing servers without having to update any other machines (this is especially useful for nameservers).<br />
<br />
==== Example ====<br />
<br />
You will need {{Pkg|netctl}} from the [[official repositories]].<br />
<br />
Prepare the configuration:<br />
<br />
{{hc|/etc/netctl/mynetwork|<nowiki><br />
Connection='ethernet'<br />
Description='Five different addresses on the same NIC.'<br />
Interface='eth0'<br />
IP='static'<br />
Address=('192.168.1.10' '192.168.178.11' '192.168.1.12' '192.168.1.13' '192.168.1.14' '192.168.1.15')<br />
Gateway='192.168.1.1'<br />
DNS=('192.168.1.1')<br />
</nowiki>}}<br />
<br />
Then simply execute: <br />
<br />
$ netctl start mynetwork<br />
<br />
=== Change MAC/hardware address ===<br />
<br />
See [[MAC Address Spoofing]].<br />
<br />
=== Internet Sharing ===<br />
<br />
See [[Internet sharing]].<br />
<br />
=== Router Configuration ===<br />
<br />
See [[Router]].<br />
<br />
=== Local network hostname resolution ===<br />
<br />
The pre-requisite is to [[#Set the hostname]] after which hostname resolution works on the local system itself {{hc|$ ping hostname|<br />
PING hostname <nowiki>(192.168.1.2) 56(84) bytes of data.<br />
64 bytes from hostname (192.168.1.2): icmp_seq=1 ttl=64 time=0.043 ms</nowiki>}}<br />
To enable other machines to address the host by name, either a manual configuration of the respective {{ic|/etc/hosts}} files or a service to propagate/resolve the name is required. <br />
<br />
When setting up a DNS server such as [[BIND]] or [[Unbound]] is overkill, manually editing your {{ic|/etc/hosts}} is too cumbersome, or when you want more flexibility with dynamic leaving and joining of hosts to the network, it is possible to handle hostname resolution on your local network using zero-configuration networking. There are two options available:<br />
<br />
*[[Samba]] provides hostname resolution via Microsoft's '''NetBIOS'''. It only requires installation of {{Pkg|samba}} and enabling of the {{ic|nmbd.service}} service. Computers running Windows, OS X, or Linux with {{ic|nmbd}} running, will be able to find your machine.<br />
<br />
*[[Avahi]] provides hostname resolution via '''zeroconf''', also known as Avahi or Bonjour. It requires slightly more complex configuration than Samba: see [[Avahi#Hostname resolution]] for details. Computers running OS X, or Linux with an Avahi daemon running, will be able to find your machine. Windows does not have an built-in Avahi client or daemon.<br />
<br />
== Troubleshooting ==<br />
<br />
=== Swapping computers on the cable modem ===<br />
<br />
Some cable ISPs (videotron for example) have the cable modem configured to recognize only one client PC, by the MAC address of its network interface. Once the cable modem has learned the MAC address of the first PC or equipment that talks to it, it will not respond to another MAC address in any way. Thus if you swap one PC for another (or for a router), the new PC (or router) will not work with the cable modem, because the new PC (or router) has a MAC address different from the old one. To reset the cable modem so that it will recognise the new PC, you must power the cable modem off and on again. Once the cable modem has rebooted and gone fully online again (indicator lights settled down), reboot the newly connected PC so that it makes a DHCP request, or manually make it request a new DHCP lease.<br />
<br />
If this method does not work, you will need to clone the MAC address of the original machine. See also [[Configuring Network#Change MAC/hardware address|Change MAC/hardware address]].<br />
<br />
=== The TCP window scaling problem ===<br />
<br />
TCP packets contain a "window" value in their headers indicating how much data the other host may send in return. This value is represented with only 16 bits, hence the window size is at most 64Kb. TCP packets are cached for a while (they have to be reordered), and as memory is (or used to be) limited, one host could easily run out of it.<br />
<br />
Back in 1992, as more and more memory became available, [http://www.faqs.org/rfcs/rfc1323.html RFC 1323] was written to improve the situation: Window Scaling. The "window" value, provided in all packets, will be modified by a Scale Factor defined once, at the very beginning of the connection.<br />
<br />
That 8-bit Scale Factor allows the Window to be up to 32 times higher than the initial 64Kb.<br />
<br />
It appears that some broken routers and firewalls on the Internet are rewriting the Scale Factor to 0 which causes misunderstandings between hosts.<br />
<br />
The Linux kernel 2.6.17 introduced a new calculation scheme generating higher Scale Factors, virtually making the aftermaths of the broken routers and firewalls more visible.<br />
<br />
The resulting connection is at best very slow or broken.<br />
<br />
==== How to diagnose the problem ====<br />
<br />
First of all, let's make it clear: this problem is odd. In some cases, you will not be able to use TCP connections (HTTP, FTP, ...) at all and in others, you will be able to communicate with some hosts (very few).<br />
<br />
When you have this problem, the {{ic|dmesg}}'s output is OK, logs are clean and {{ic|ip addr}} will report normal status... and actually everything appears normal.<br />
<br />
If you cannot browse any website, but you can ping some random hosts, chances are great that you're experiencing this problem: ping uses ICMP and is not affected by TCP problems.<br />
<br />
You can try to use Wireshark. You might see successful UDP and ICMP communications but unsuccessful TCP communications (only to foreign hosts).<br />
<br />
==== How to fix it (The bad way) ====<br />
<br />
To fix it the bad way, you can change the tcp_rmem value, on which Scale Factor calculation is based. Although it should work for most hosts, it is not guaranteed, especially for very distant ones.<br />
<br />
# echo "4096 87380 174760" > /proc/sys/net/ipv4/tcp_rmem<br />
<br />
==== How to fix it (The good way) ====<br />
<br />
Simply disable Window Scaling. Since Window Scaling is a nice TCP feature, it may be uncomfortable to disable it, especially if you cannot fix the broken router. There are several ways to disable Window Scaling, and it seems that the most bulletproof way (which will work with most kernels) is to add the following line to {{ic|/etc/sysctl.d/99-disable_window_scaling.conf}} (see also [[sysctl]])<br />
<br />
net.ipv4.tcp_window_scaling = 0<br />
<br />
==== How to fix it (The best way) ====<br />
<br />
This problem is caused by broken routers/firewalls, so let's change them. Some users have reported that the broken router was their very own DSL router.<br />
<br />
==== More about it ====<br />
<br />
This section is based on the LWN article [http://lwn.net/Articles/92727/ TCP window scaling and broken routers] and a Kernel Trap article: [http://kerneltrap.org/node/6723 Window Scaling on the Internet].<br />
<br />
There are also several relevant threads on the LKML.<br />
<br />
=== Realtek no link / WOL problem ===<br />
<br />
Users with Realtek 8168 8169 8101 8111(C) based NICs (cards / and on-board) may notice a problem where the NIC seems to be disabled on boot and has no Link light. This can usually be found on a dual boot system where Windows is also installed. It seems that using the offical Realtek drivers (dated anything after May 2007) under Windows is the cause. These newer drivers disable the Wake-On-LAN feature by disabling the NIC at Windows shutdown time, where it will remain disabled until the next time Windows boots. You will be able to notice if this problem is affecting you if the Link light remains off until Windows boots up; during Windows shutdown the Link light will switch off. Normal operation should be that the link light is always on as long as the system is on, even during POST. This problem will also affect other operative systems without newer drivers (eg. Live CDs). Here are a few fixes for this problem:<br />
<br />
==== Method 1 - Enable the NIC directly in Linux ====<br />
<br />
Get the ethernet NIC name from the output of<br />
<br />
$ ip a<br />
<br />
Bring up the device as root using the NIC name:<br />
<br />
# ip link set dev <NIC_name> up<br />
<br />
For ex, if <NIC_name> is enp7s0<br />
<br />
# ip link set dev enp7s0 up<br />
<br />
If it worked and the card is powered on, a new interface should appear in the output of {{ic|ifconfig}}<br />
<br />
==== Method 2 - Rollback/change Windows driver ====<br />
<br />
You can roll back your Windows NIC driver to the Microsoft provided one (if available), or roll back/install an official Realtek driver pre-dating May 2007 (may be on the CD that came with your hardware).<br />
<br />
==== Method 3 - Enable WOL in Windows driver ====<br />
<br />
Probably the best and the fastest fix is to change this setting in the Windows driver. This way it should be fixed system-wide and not only under Arch (eg. live CDs, other operative systems). In Windows, under Device Manager, find your Realtek network adapter and double-click it. Under the Advanced tab, change "Wake-on-LAN after shutdown" to Enable.<br />
<br />
In Windows XP (example)<br />
Right click my computer<br />
--> Hardware tab<br />
--> Device Manager<br />
--> Network Adapters<br />
--> "double click" Realtek ...<br />
--> Advanced tab<br />
--> Wake-On-Lan After Shutdown<br />
--> Enable<br />
<br />
{{Note|Newer Realtek Windows drivers (tested with ''Realtek 8111/8169 LAN Driver v5.708.1030.2008'', dated 2009/01/22, available from GIGABYTE) may refer to this option slightly differently, like ''Shutdown Wake-On-LAN --> Enable''. It seems that switching it to {{ic|Disable}} has no effect (you will notice the Link light still turns off upon Windows shutdown). One rather dirty workaround is to boot to Windows and just reset the system (perform an ungraceful restart/shutdown) thus not giving the Windows driver a chance to disable LAN. The Link light will remain on and the LAN adapter will remain accessible after POST - that is until you boot back to Windows and shut it down properly again.}}<br />
<br />
==== Method 4 - Newer Realtek Linux driver ====<br />
<br />
Any newer driver for these Realtek cards can be found for Linux on the realtek site. (untested but believed to also solve the problem).<br />
<br />
==== Method 5 - Enable ''LAN Boot ROM'' in BIOS/CMOS ====<br />
<br />
It appears that setting ''Integrated Peripherals --> Onboard LAN Boot ROM --> Enabled'' in BIOS/CMOS reactivates the Realtek LAN chip on system boot-up, despite the Windows driver disabling it on OS shutdown.<br />
<br />
{{Note|This was tested successfully multiple times with GIGABYTE system board GA-G31M-ES2L with BIOS version F8 released on 2009/02/05. YMMV.}}<br />
<br />
=== No eth0 with Atheros AR9485 ===<br />
<br />
The ethernet (eth0) for Atheros AR9485 are not working out-of-the-box (with installation media of February 2014). The working solution for this is to install the package {{AUR|backports-patched}} from AUR.<br />
<br />
=== Broadcom BCM57780 ===<br />
<br />
This Broadcom chipset sometimes does not behave well unless you specify the order of the modules to be loaded. The modules are {{ic|broadcom}} and {{ic|tg3}}, the former needing to be loaded first.<br />
<br />
These steps should help if your computer has this chipset:<br />
$ lspci | grep Ethernet<br />
02:00.0 Ethernet controller: Broadcom Corporation NetLink BCM57780 Gigabit Ethernet PCIe (rev 01)<br />
<br />
If your wired networking is not functioning in some way or another, try unplugging your cable then doing the following (as root):<br />
# modprobe -r tg3<br />
# modprobe broadcom<br />
# modprobe tg3<br />
<br />
Now plug you network cable in. If this solves your problems you can make this permanent by adding {{ic|broadcom}} and {{ic|tg3}} (in this order) to the {{ic|MODULES}} array in {{ic|/etc/mkinitcpio.conf}}:<br />
<br />
MODULES=".. broadcom tg3 .."<br />
<br />
Then rebuild the initramfs:<br />
<br />
# mkinitcpio -p linux<br />
<br />
{{Note|These methods may work for other chipsets, such as BCM57760.}}</div>Justin8https://wiki.archlinux.org/index.php?title=Bcache&diff=294814Bcache2014-01-28T23:09:20Z<p>Justin8: Removed embelishment as it is unnecessary to describe enhanceio (particularly in a subjective manner) in an article on bcache outside of the reference to it.</p>
<hr />
<div>[[Category:File systems]]<br />
== Introduction ==<br />
<br />
Bcache allows one to use an SSD as a read/write cache (in writeback mode) or read cache (writethrough or writearound) for another blockdevice (generally a rotating HDD or array). This article will show how to install arch using Bcache as the root partition. For an intro to bcache itself, see [http://bcache.evilpiepirate.org/ the bcache homepage]. Be sure to read and reference [http://atlas.evilpiepirate.org/git/linux-bcache.git/tree/Documentation/bcache.txt?h=bcache the bcache manual]. Bcache is in the mainline kernel since 3.10. The kernel on the arch install disk includes the bcache module since 2013.08.01.<br />
<br />
An alternative to Bcache is Facebook's [[Flashcache]] and its offspring [[Enhanceio]].<br />
<br />
Bcache needs the backing device to be formatted as a bcache block device. In most cases, [https://github.com/g2p/blocks blocks to-bcache] can do an in-place conversion.<br />
{{Warning|Be sure you back up any important data first.}}<br />
<br />
== Setting up a bcache device on an existing system ==<br />
<br />
1. Install the {{AUR|bcache-tools}} package from [[AUR]].<br />
<br />
2. Create a backing device (This will typically be your mechanical drive). The backing device can be a whole device, a partition or any other standard block device. This will create /dev/bcache0<br />
# make-bcache -B /dev/sdx1<br />
<br />
3. Create a cache device (This will typically be your SSD). The cache device can be a whole device, a partition or any other standard block device<br />
# make-bcache -C /dev/sdx1<br />
<br />
4. Register the cache device against your backing device. We need to find the UUID of your cache device and then add it to the bcache device initially. Udev rules will take care of this on reboot and will only need to be done once.<br />
# cd /sys/fs/bcache<br />
# ls<br />
# echo "UUID" > /sys/block/bcache0/bcache/attach<br />
<br />
5. Change your cache mode (if you want to cache writes as well as reads):<br />
# echo writeback > /sys/block/bcache0/bcache/cache_mode<br />
<br />
6. If you want to have this partition available during the initcpio (i.e. you require it at some point in the boot process) you need to add 'bcache' to your modules array in /etc/mkinitcpio.conf as well as adding the 'bcache' hook in your list between block and filesystem.<br />
<br />
<br />
== Installation to a bcache device ==<br />
<br />
1. Boot on the install disk (2013.08.01 minimum).<br />
<br />
2. Install the {{AUR|bcache-tools}} package from [[AUR]].<br />
<br />
3. Partition your hdd<br />
<br />
{{Note|While it may be true that Grub2 does not offer support for bcache as noted below, it does, however, fully support UEFI. It follows then, that so long as the necessary modules for the linux kernel to properly handle your boot device are either compiled into the kernel or are included in an initramfs, and you can include these files on it, the separate boot partition described below may be omitted in favor of the FAT EFI system partition. See [[GRUB]] and/or [[UEFI]] for more.}}<br />
grub can't handle bcache, so you'll need at least 2 partitions (boot and one for the bcache backing device). If you're doing UEFI, you'll need an EFI System Partition (ESP) as well. E.g.:<br />
1 2048 22527 10.0 MiB EF00 EFI System<br />
2 22528 432127 200.0 MiB 8300 arch_boot<br />
3 432128 625142414 297.9 GiB 8300 bcache_backing<br />
<br />
{{Note|This example has no swapfile/partition. For a swap partition on the cache, use LVM in step 7. For a swap partition outside the cache, be sure to make a swap partition now.}}<br />
<br />
4. Configure your HDD as a bcache backing device.<br />
<br />
# make-bcache -B /dev/sda3<br />
<br />
{{Note|<br />
* When preparing any boot disk it is important to know the ramifications of any decision you may make. Please review and review again the documentation for your chosen boot-loader/-manager and consider seriously how it might relate to bcache.<br />
* If all associated disks are partitioned at once as below bcache will automatically attach "-B backing stores" to the "-C ssd cache" and step 6 is unnecessary.<br />
}}<br />
<br />
# make-bcache -B /dev/sd? /dev/sd? -C /dev/sd?<br />
<br />
5. Register any bcache devices with the kernel (this needs to done every bootup if you do not use the bcache hook in your mkinitcpio, unless you attach the bcache devices after boot has completed)<br />
<br />
# for i in /dev/sd*; do echo $i; echo $i > /sys/fs/bcache/register_quiet; done<br />
<br />
You now have a {{ic|/dev/bcache0}} device.<br />
<br />
{{Note|the bcache user manual says to do {{ic|echo /dev/sd* > /sys/fs/bcache/register_quiet}}, but this did not work for me.}}<br />
<br />
6. Configure your SSD<br />
<br />
Format the SSD as a caching device and link it to the backing device<br />
<br />
# make-bcache -C /dev/sdb<br />
# echo /dev/sdb > /sys/fs/bcache/register <br />
# echo ''UUID__from_previous_command'' > /sys/block/bcache0/bcache/attach<br />
<br />
{{Note|If the UUID is forgotten, it can be found with {{ic|ls /sys/fs/bcache/}} after the cache device has been registered.}}<br />
<br />
7. Format the bcache device. Use LVM or btrfs subvolumes if you want to divide up the {{ic|/dev/bcache0}} device how you like (ex for seperate {{ic|/}}, {{ic|/home}}, {{ic|/var}}, etc):<br />
<br />
# mkfs.btrfs /dev/bcache0<br />
# mount /dev/bcache0 /mnt/<br />
# btrfs subvolume create /mnt/root<br />
# btrfs subvolume create /mnt/home<br />
# umount /mnt<br />
<br />
8. Prepare the installation mount point:<br />
<br />
# mkfs.ext4 /dev/sda2<br />
# mkfs.msdos /dev/sda1 (if your ESP is at least 500MB, use mkfs.vfat to make a FAT32 partition instead)<br />
# pacman -S arch-install-scripts<br />
# mount /dev/bcache0 -o subvol=root,compress=lzo /mnt/<br />
# mkdir /mnt/boot /mnt/home<br />
# mount /dev/bcache0 -o subvol=home,compress=lzo /mnt/home<br />
# mount /dev/sda2 /mnt/boot<br />
# mkdir /boot/efi<br />
# mount /dev/sda1 /mnt/boot/efi/<br />
<br />
9. Install the system as per the [[Installation_Guide#Connect_to_the_internet|Installation Guide]] as normal except this:<br />
<br />
Before you edit {{ic|/etc/mkinitcpio.conf}} and run {{ic|mkinitcpio -p linux}}:<br />
<br />
* install {{AUR|bcache-tools}} package from the [[AUR]].<br />
* Edit {{ic|/etc/mkinitcpio.conf}}:<br />
** add the "bcache" module<br />
** add the "bcache" hook between block and filesystem hooks<br />
<br />
== Configuring ==<br />
<br />
There are many options that can be configured (such as cache mode, cache flush interval, sequential write heuristic, etc.) This is currently done by writing to files in {{ic|/sys}}. See the [http://atlas.evilpiepirate.org/git/linux-bcache.git/tree/Documentation/bcache.txt bcache user documentation].<br />
<br />
Changing the cache mode is done by echoing one of 'writethrough', 'writeback', 'writearound' or 'none' to /sys/block/bcache[0-9]/bcache/cache_mode.<br />
<br />
== Advanced operations ==<br />
<br />
=== Resize backing device ===<br />
<br />
It's possible to resize the backing device so long as you don't move the partition start. This process is described on [http://comments.gmane.org/gmane.linux.kernel.bcache.devel/249 the mailing list]. Here is an example using btrfs volume directly on bcache0. For LVM containers or for other filesystems, procedure will differ.<br />
<br />
==== Example of growing ====<br />
<br />
In this example, I grow the filesystem by 4GB. <br />
<br />
1. Reboot to a live CD/USB Drive (need not be bcache enabled) and use fdisk, gdisk, parted, or your other favorite tool to delete the backing partition and recreate it with the same start and a total size 4G larger. <br />
<br />
{{Warning|Do not use a tool like GParted that might perform filesystem operations! It will not recognize the bcache partition and might overwrite part of it!!}}<br />
<br />
2. Reboot to your normal install. Your filesystem will be currently mounted. That is fine. Issue the command to resize the partition to its maximum. For btrfs, that is<br />
<br />
# btrfs filesystem resize max /<br />
<br />
For ext3/4, that is:<br />
<br />
# resize2fs /dev/bcache0<br />
<br />
==== Example of shrinking ====<br />
<br />
In this example, I shrink the filesystem by 4GB.<br />
<br />
1. Disable writeback cache (switch to writethrough cache) and wait for the disk to flush.<br />
<br />
# echo writethrough > /sys/block/bcache0/bcache/cache_mode<br />
$ watch cat /sys/block/bcache0/bcache/state<br />
<br />
wait until state reports "clean". This might take a while.<br />
<br />
2. Shrink the mounted filesystem by something more than the desired amount, to ensure we don't accidentally clip it later. For btrfs, that is:<br />
<br />
# btrfs filesystem resize -5G /<br />
<br />
For ext3/4 you can use ''resize2fs'', but only if the partition is unmounted<br />
<br />
$ df -h /home<br />
/dev/bcache0 290G 20G 270G 1% /home<br />
# umount /home<br />
# resize2fs /dev/bcache0 283G<br />
<br />
3. Reboot to a LiveCD/USB drive (doesn't need to support bcache) and use fdisk, gdisk, parted, or your other favorite tool to delete the backing partition and recreate it with the same start and a total size 4G smaller.<br />
<br />
{{Warning|Do not use a tool like GParted that might perform filesystem operations! It won't recognize the bcache partition and might overwrite part of it!!}}<br />
<br />
4. Reboot to your normal install. Your filesystem will be currently mounted. That is fine. Issue the command to resize the partition to its maximum (that is, the size we shrunk the actual partition to in step 3). For btrfs, that is:<br />
<br />
# btrfs filesystem resize max /<br />
<br />
For ext3/4, that is:<br />
<br />
# resize2fs /dev/bcache0<br />
<br />
5. Re-enable writeback cache if you want that enabled:<br />
<br />
# echo writeback > /sys/block/bcache0/bcache/cache_mode<br />
<br />
{{Note|If you're very careful you can shrink the filesystem to the exact size in step 2 and avoid step 4. Be careful, though, many partition tools don't do exactly what you want, but instead adjust the requested partition start/end points to end on sector boundaries. This may be difficult to calculate ahead of time}}<br />
<br />
== Troubleshooting ==<br />
<br />
=== /dev/bcache device doesn't exist on bootup ===<br />
<br />
If you are sent to a busy box shell with an error:<br />
<br />
{{bc|1=<br />
ERROR: Unable to find root device 'UUID=b6b2d82b-f87e-44d5-bbc5-c51dd7aace15'.<br />
You are being dropped to a recovery shell<br />
Type 'exit' to try and continue booting<br />
}}<br />
<br />
This might happen if the backing device is configured for "writeback" mode (default is writearound). When in "writeback" mode, the /dev/bcache0 device is not started until the cache device is both registered and attached. Registering is something that needs to happen every bootup, but attaching should only have to be done once. I've sometime had to re-attach.<br />
<br />
To continue booting, try one of the following:<br />
<br />
* Register both the backing device and the cacheing device<br />
<br />
# echo /dev/sda3 > /sys/fs/bcache/register<br />
# echo /dev/sdb > /sys/fs/bcache/register<br />
<br />
If the /dev/bcache0 device now exists, type exit and continue booting. You will need to fix your initcpio to ensure devices are registered before mounting the root device.<br />
<br />
{{Note|<br />
* An error of "sh: echo: write error: Invalid argument" means the device was already registered or is not recognized as either a bcache backing device or cache. If using the udev rule on boot it should only attempt to register a device if it finds a bcache superblock<br />
* This can also happen if using udev's 69-bcache.rules in Installation's step 7 and blkid and bcache-probe "disagree" due to rogue superblocks. See [http://bcache.evilpiepirate.org/#index6h1 bcache's wiki] for a possible explanation/resolution.<br />
}}<br />
<br />
* Re-attach the cache to the backing device:<br />
<br />
If the cache device was registered, a folder with the UUID of the cache should exist in {{ic|/sys/fs/bcache}}. Use that UUID when following the example below:<br />
<br />
# ls /sys/fs/bcache/<br />
b6b2d82b-f87e-44d5-bbc5-c51dd7aace15 register register_quiet<br />
# echo b6b2d82b-f87e-44d5-bbc5-c51dd7aace15 > /sys/block/sda/sda3/bcache/attach<br />
<br />
If the {{ic|/dev/bcache0}} device now exists, type exit and continue booting. You should not have to do this again. If it persists, ask on the bcache mailing list.<br />
<br />
{{Note|An error of {{ic|sh: echo: write error: Invalid argument}} means the device was already attached. An error of {{ic|sh: echo: write error: No such file or directory}} means the UUID is not a valid cache (make sure you typed it correctly).}}<br />
<br />
* Invalidate the cache and force the backing device to run without it. You might want to check some stats, such as "dirty_data" so you have some idea of how much data will be lost.<br />
<br />
# cat /sys/block/sda/sda3/bcache/dirty_data<br />
-3.9M<br />
<br />
dirty data is data in the cache that has not been written to the backing device. If you force the backing device to run, this data will be lost, even if you later re-attach the cache.<br />
<br />
# cat /sys/block/sda/sda3/bcache/running<br />
0<br />
# echo 1 > /sys/block/sda/sda3/bcache/running<br />
<br />
The {{ic|/dev/bcache0}} device will now exist. Type exit and continue booting. You might want to unregister the cache device and run make-bcache again. An fsck on {{ic|/dev/bcache0}} would also be wise. See the [http://atlas.evilpiepirate.org/git/linux-bcache.git/tree/Documentation/bcache.txt?h=bcache bcache user documentation].<br />
<br />
{{Warning|Only invalidate the cache if one of the two options above did not work.}}<br />
<br />
=== /sys/fs/bcache/ doesn't exist ===<br />
<br />
The kernel you booted is not bcache enabled.</div>Justin8https://wiki.archlinux.org/index.php?title=Bcache&diff=291940Bcache2014-01-07T22:21:16Z<p>Justin8: Removed reference to the now defunct bcache_udev hook as v1.0.5 is on AUR</p>
<hr />
<div>[[Category:File systems]]<br />
== Introduction ==<br />
<br />
Bcache allows one to use an SSD as a read/write cache (in writeback mode) or read cache (writethrough or writearound) for another blockdevice (generally a rotating HDD or array). This article will show how to install arch using Bcache as the root partition. For an intro to bcache itself, see [http://bcache.evilpiepirate.org/ the bcache homepage]. Be sure to read and reference [http://atlas.evilpiepirate.org/git/linux-bcache.git/tree/Documentation/bcache.txt?h=bcache the bcache manual]. Bcache is in the mainline kernel since 3.10. The kernel on the arch install disk includes the bcache module since 2013.08.01.<br />
<br />
An alternative to Bcache is Facebook's [[Flashcache]].<br />
<br />
Bcache needs the backing device to be formatted as a bcache block device. In most cases, [https://github.com/g2p/blocks blocks to-bcache] can do an in-place conversion.<br />
{{Warning|Be sure you back up any important data first.}}<br />
<br />
== Setting up a bcache device on an existing system ==<br />
<br />
1. Install the {{AUR|bcache-tools}} package from [[AUR]].<br />
<br />
2. Create a backing device (This will typically be your mechanical drive). The backing device can be a whole device, a partition or any other standard block device. This will create /dev/bcache0<br />
# make-bcache -B /dev/sdx1<br />
<br />
3. Create a cache device (This will typically be your SSD). The cache device can be a whole device, a partition or any other standard block device<br />
# make-bcache -C /dev/sdx1<br />
<br />
4. Register the cache device against your backing device. We need to find the UUID of your cache device and then add it to the bcache device initially. Udev rules will take care of this on reboot and will only need to be done once.<br />
# cd /sys/fs/bcache<br />
# ls<br />
# echo "UUID" > /sys/block/bcache0/bcache/attach<br />
<br />
5. Change your cache mode (if you want to cache writes as well as reads):<br />
# echo writeback > /sys/block/bcache0/bcache/cache_mode<br />
<br />
6. If you want to have this partition available during the initcpio (i.e. you require it at some point in the boot process) you need to add 'bcache' to your modules array in /etc/mkinitcpio.conf as well as adding the 'bcache' hook in your list between block and filesystem.<br />
<br />
<br />
== Installation to a bcache device ==<br />
<br />
1. Boot on the install disk (2013.08.01 minimum).<br />
<br />
2. Install the {{AUR|bcache-tools}} package from [[AUR]].<br />
<br />
3. Partition your hdd<br />
<br />
{{Note|While it may be true that Grub2 does not offer support for bcache as noted below, it does, however, fully support UEFI. It follows then, that so long as the necessary modules for the linux kernel to properly handle your boot device are either compiled into the kernel or are included in an initramfs, and you can include these files on it, the separate boot partition described below may be omitted in favor of the FAT EFI system partition. See [[GRUB]] and/or [[UEFI]] for more.}}<br />
grub can't handle bcache, so you'll need at least 2 partitions (boot and one for the bcache backing device). If you're doing UEFI, you'll need an EFI System Partition (ESP) as well. E.g.:<br />
1 2048 22527 10.0 MiB EF00 EFI System<br />
2 22528 432127 200.0 MiB 8300 arch_boot<br />
3 432128 625142414 297.9 GiB 8300 bcache_backing<br />
<br />
{{Note|This example has no swapfile/partition. For a swap partition on the cache, use LVM in step 7. For a swap partition outside the cache, be sure to make a swap partition now.}}<br />
<br />
4. Configure your HDD as a bcache backing device.<br />
<br />
# make-bcache -B /dev/sda3<br />
<br />
{{Note|<br />
* When preparing any boot disk it is important to know the ramifications of any decision you may make. Please review and review again the documentation for your chosen boot-loader/-manager and consider seriously how it might relate to bcache.<br />
* If all associated disks are partitioned at once as below bcache will automatically attach "-B backing stores" to the "-C ssd cache" and step 6 is unnecessary.<br />
}}<br />
<br />
# make-bcache -B /dev/sd? /dev/sd? -C /dev/sd?<br />
<br />
5. Register any bcache devices with the kernel (this needs to done every bootup if you do not use the bcache hook in your mkinitcpio, unless you attach the bcache devices after boot has completed)<br />
<br />
# for i in /dev/sd*; do echo $i; echo $i > /sys/fs/bcache/register_quiet; done<br />
<br />
You now have a {{ic|/dev/bcache0}} device.<br />
<br />
{{Note|the bcache user manual says to do {{ic|echo /dev/sd* > /sys/fs/bcache/register_quiet}}, but this did not work for me.}}<br />
<br />
6. Configure your SSD<br />
<br />
Format the SSD as a caching device and link it to the backing device<br />
<br />
# make-bcache -C /dev/sdb<br />
# echo /dev/sdb > /sys/fs/bcache/register <br />
# echo ''UUID__from_previous_command'' > /sys/block/bcache0/bcache/attach<br />
<br />
{{Note|If the UUID is forgotten, it can be found with {{ic|ls /sys/fs/bcache/}} after the cache device has been registered.}}<br />
<br />
7. Format the bcache device. Use LVM or btrfs subvolumes if you want to divide up the {{ic|/dev/bcache0}} device how you like (ex for seperate {{ic|/}}, {{ic|/home}}, {{ic|/var}}, etc):<br />
<br />
# mkfs.btrfs /dev/bcache0<br />
# mount /dev/bcache0 /mnt/<br />
# btrfs subvolume create /mnt/root<br />
# btrfs subvolume create /mnt/home<br />
# umount /mnt<br />
<br />
8. Prepare the installation mount point:<br />
<br />
# mkfs.ext4 /dev/sda2<br />
# mkfs.msdos /dev/sda1 (if your ESP is at least 500MB, use mkfs.vfat to make a FAT32 partition instead)<br />
# pacman -S arch-install-scripts<br />
# mount /dev/bcache0 -o subvol=root,compress=lzo /mnt/<br />
# mkdir /mnt/boot /mnt/home<br />
# mount /dev/bcache0 -o subvol=home,compress=lzo /mnt/home<br />
# mount /dev/sda2 /mnt/boot<br />
# mkdir /boot/efi<br />
# mount /dev/sda1 /mnt/boot/efi/<br />
<br />
9. Install the system as per the [[Installation_Guide#Connect_to_the_internet|Installation Guide]] as normal except this:<br />
<br />
Before you edit {{ic|/etc/mkinitcpio.conf}} and run {{ic|mkinitcpio -p linux}}:<br />
<br />
* install {{AUR|bcache-tools}} package from the [[AUR]].<br />
* Edit {{ic|/etc/mkinitcpio.conf}}:<br />
** add the "bcache" module<br />
** add the "bcache" hook between block and filesystem hooks<br />
<br />
== Configuring ==<br />
<br />
There are many options that can be configured (such as cache mode, cache flush interval, sequential write heuristic, etc.) This is currently done by writing to files in {{ic|/sys}}. See the [http://atlas.evilpiepirate.org/git/linux-bcache.git/tree/Documentation/bcache.txt bcache user documentation].<br />
<br />
Changing the cache mode is done by echoing one of 'writethrough', 'writeback', 'writearound' or 'none' to /sys/block/bcache[0-9]/bcache/cache_mode.<br />
<br />
== Advanced operations ==<br />
<br />
=== Resize backing device ===<br />
<br />
It's possible to resize the backing device so long as you don't move the partition start. This process is described on [http://comments.gmane.org/gmane.linux.kernel.bcache.devel/249 the mailing list]. Here is an example using btrfs volume directly on bcache0. For LVM containers or for other filesystems, procedure will differ.<br />
<br />
==== Example of growing ====<br />
<br />
In this example, I grow the filesystem by 4GB. <br />
<br />
1. Reboot to a live CD/USB Drive (need not be bcache enabled) and use fdisk, gdisk, parted, or your other favorite tool to delete the backing partition and recreate it with the same start and a total size 4G larger. <br />
<br />
{{Warning|Do not use a tool like GParted that might perform filesystem operations! It will not recognize the bcache partition and might overwrite part of it!!}}<br />
<br />
2. Reboot to your normal install. Your filesystem will be currently mounted. That is fine. Issue the command to resize the partition to its maximum. For btrfs, that is<br />
<br />
# btrfs filesystem resize max /<br />
<br />
For ext3/4, that is:<br />
<br />
# resize2fs /dev/bcache0<br />
<br />
==== Example of shrinking ====<br />
<br />
In this example, I shrink the filesystem by 4GB.<br />
<br />
1. Disable writeback cache (switch to writethrough cache) and wait for the disk to flush.<br />
<br />
# echo writethrough > /sys/block/bcache0/bcache/cache_mode<br />
$ watch cat /sys/block/bcache0/bcache/state<br />
<br />
wait until state reports "clean". This might take a while.<br />
<br />
2. Shrink the mounted filesystem by something more than the desired amount, to ensure we don't accidentally clip it later. For btrfs, that is:<br />
<br />
# btrfs filesystem resize -5G /<br />
<br />
For ext3/4 you can use ''resize2fs'', but only if the partition is unmounted<br />
<br />
$ df -h /home<br />
/dev/bcache0 290G 20G 270G 1% /home<br />
# umount /home<br />
# resize2fs /dev/bcache0 283G<br />
<br />
3. Reboot to a LiveCD/USB drive (doesn't need to support bcache) and use fdisk, gdisk, parted, or your other favorite tool to delete the backing partition and recreate it with the same start and a total size 4G smaller.<br />
<br />
{{Warning|Do not use a tool like GParted that might perform filesystem operations! It won't recognize the bcache partition and might overwrite part of it!!}}<br />
<br />
4. Reboot to your normal install. Your filesystem will be currently mounted. That is fine. Issue the command to resize the partition to its maximum (that is, the size we shrunk the actual partition to in step 3). For btrfs, that is:<br />
<br />
# btrfs filesystem resize max /<br />
<br />
For ext3/4, that is:<br />
<br />
# resize2fs /dev/bcache0<br />
<br />
5. Re-enable writeback cache if you want that enabled:<br />
<br />
# echo writeback > /sys/block/bcache0/bcache/cache_mode<br />
<br />
{{Note|If you're very careful you can shrink the filesystem to the exact size in step 2 and avoid step 4. Be careful, though, many partition tools don't do exactly what you want, but instead adjust the requested partition start/end points to end on sector boundaries. This may be difficult to calculate ahead of time}}<br />
<br />
== Troubleshooting ==<br />
<br />
=== /dev/bcache device doesn't exist on bootup ===<br />
<br />
If you are sent to a busy box shell with an error:<br />
<br />
{{bc|1=<br />
ERROR: Unable to find root device 'UUID=b6b2d82b-f87e-44d5-bbc5-c51dd7aace15'.<br />
You are being dropped to a recovery shell<br />
Type 'exit' to try and continue booting<br />
}}<br />
<br />
This might happen if the backing device is configured for "writeback" mode (default is writearound). When in "writeback" mode, the /dev/bcache0 device is not started until the cache device is both registered and attached. Registering is something that needs to happen every bootup, but attaching should only have to be done once. I've sometime had to re-attach.<br />
<br />
To continue booting, try one of the following:<br />
<br />
* Register both the backing device and the cacheing device<br />
<br />
# echo /dev/sda3 > /sys/fs/bcache/register<br />
# echo /dev/sdb > /sys/fs/bcache/register<br />
<br />
If the /dev/bcache0 device now exists, type exit and continue booting. You will need to fix your initcpio to ensure devices are registered before mounting the root device.<br />
<br />
{{Note|<br />
* An error of "sh: echo: write error: Invalid argument" means the device was already registered or is not recognized as either a bcache backing device or cache. If using the udev rule on boot it should only attempt to register a device if it finds a bcache superblock<br />
* This can also happen if using udev's 69-bcache.rules in Installation's step 7 and blkid and bcache-probe "disagree" due to rogue superblocks. See [http://bcache.evilpiepirate.org/#index6h1 bcache's wiki] for a possible explanation/resolution.<br />
}}<br />
<br />
* Re-attach the cache to the backing device:<br />
<br />
If the cache device was registered, a folder with the UUID of the cache should exist in {{ic|/sys/fs/bcache}}. Use that UUID when following the example below:<br />
<br />
# ls /sys/fs/bcache/<br />
b6b2d82b-f87e-44d5-bbc5-c51dd7aace15 register register_quiet<br />
# echo b6b2d82b-f87e-44d5-bbc5-c51dd7aace15 > /sys/block/sda/sda3/bcache/attach<br />
<br />
If the {{ic|/dev/bcache0}} device now exists, type exit and continue booting. You should not have to do this again. If it persists, ask on the bcache mailing list.<br />
<br />
{{Note|An error of {{ic|sh: echo: write error: Invalid argument}} means the device was already attached. An error of {{ic|sh: echo: write error: No such file or directory}} means the UUID is not a valid cache (make sure you typed it correctly).}}<br />
<br />
* Invalidate the cache and force the backing device to run without it. You might want to check some stats, such as "dirty_data" so you have some idea of how much data will be lost.<br />
<br />
# cat /sys/block/sda/sda3/bcache/dirty_data<br />
-3.9M<br />
<br />
dirty data is data in the cache that has not been written to the backing device. If you force the backing device to run, this data will be lost, even if you later re-attach the cache.<br />
<br />
# cat /sys/block/sda/sda3/bcache/running<br />
0<br />
# echo 1 > /sys/block/sda/sda3/bcache/running<br />
<br />
The {{ic|/dev/bcache0}} device will now exist. Type exit and continue booting. You might want to unregister the cache device and run make-bcache again. An fsck on {{ic|/dev/bcache0}} would also be wise. See the [http://atlas.evilpiepirate.org/git/linux-bcache.git/tree/Documentation/bcache.txt?h=bcache bcache user documentation].<br />
<br />
{{Warning|Only invalidate the cache if one of the two options above did not work.}}<br />
<br />
=== /sys/fs/bcache/ doesn't exist ===<br />
<br />
The kernel you booted is not bcache enabled.</div>Justin8https://wiki.archlinux.org/index.php?title=Talk:Bcache&diff=291862Talk:Bcache2014-01-07T04:29:13Z<p>Justin8: </p>
<hr />
<div>==Example installation==<br />
Here is mine Bcache installation. Maybe you can use some idea's. :)<br />
<br />
=== Prepare the storage drive ===<br />
<br />
====Bcache-tools installation====<br />
fakeroot and binutils are only needed. but it will fail in a error if you only install them.<br />
wget is from a failed archbang instal :(<br />
<br />
pacman -Syy<br />
pacman -S base-devel fakeroot binutils wget git<br />
<br />
cd ~/<br />
wget https://aur.archlinux.org/packages/bc/bcache-tools-git/bcache-tools-git.tar.gz<br />
tar -xzvf bcache-tools-git.tar.gz<br />
cd bcache-tools-git<br />
makepkg -si --asroot<br />
<br />
==== Making the partitions ====<br />
For /boot i use the slower HDD because i had troubles that mine SSD loaded the kernel while mine HDD was still spinning up :P<br />
<br />
Starting with the faster SSD drive<br />
<br />
fdisk /dev/sda<br />
<br />
o<br />
y<br />
n<br />
Partition type: p<br />
Partition-number: 1<br />
First-sector: [enter]<br />
Last-sector: +90G<br />
<br />
n<br />
Partition type: p<br />
Partition-number: 3<br />
First-sector: [enter]<br />
Last-sector: [enter]<br />
<br />
write and exit<br />
<br />
Now the slower 2 TB Harddrive<br />
<br />
fdisk /dev/sdb<br />
<br />
o<br />
y<br />
n<br />
Partition type: p<br />
Partition-number: 1<br />
First-sector: [enter]<br />
Last-sector: +1G<br />
a<br />
<br />
n<br />
Partition type: p<br />
Partition-number: 2<br />
First-sector: [enter]<br />
Last-sector: +16G [enter]<br />
t<br />
2<br />
82<br />
<br />
n<br />
Partition type: p<br />
Partition-number: 3<br />
First-sector: [enter]<br />
Last-sector: [enter]<br />
<br />
write and exit<br />
<br />
==== Making Bcache Device ====<br />
The --wipe-bcache is to remove a error :)<br />
and the dd commands is or cleaning out old partition data.<br />
<br />
make-bcache --wipe-bcache -B /dev/sdb3 -C /dev/sda2<br />
dd if=/dev/zero count=1 bs=1024 seek=1 of=/dev/sda2<br />
dd if=/dev/zero count=1 bs=1024 seek=1 of=/dev/sdb3<br />
<br />
If next commands give a error: invalid argument<br />
Then the device is already registered.<br />
<br />
echo /dev/sda2 > /sys/fs/bcache/register<br />
echo /dev/sdb3 > /sys/fs/bcache/register<br />
<br />
==== Formating the drives ====<br />
<br />
<br />
mkfs.ext4 -E discard /dev/sda1<br />
mkfs.ext4 -E discard /dev/sdb1<br />
mkfs.ext4 /dev/bcache0<br />
mkswap /dev/sdb2<br />
swapon /dev/sdb2<br />
<br />
==== Mount the partitions ====<br />
<br />
cd ~/<br />
mkdir -p /mnt/install<br />
mount /dev/sda1 /mnt/install<br />
mkdir -p /mnt/install/{boot,home,srv,data,var,mnt/.hdd}<br />
mount /dev/sdb1 /mnt/install/boot<br />
mount /dev/bcache0 /mnt/install/mnt/.hdd<br />
mkdir -p /mnt/install/mnt/.hdd/{home,srv,data,var}<br />
mount -o bind /mnt/install/mnt/.hdd/home /mnt/install/home<br />
mount -o bind /mnt/install/mnt/.hdd/srv /mnt/install/srv<br />
mount -o bind /mnt/install/mnt/.hdd/data /mnt/install/data<br />
mount -o bind /mnt/install/mnt/.hdd/var /mnt/install/var<br />
<br />
==== Bcache Check After mounting ====<br />
Reason for /boot on the HDD is because of boot up failure because of HDD spin-up.<br />
<br />
lsblk<br />
<br />
NAME MAJ:MIN RM SIZE RO TYPE MOUNTPOINT<br />
sda 8:0 0 167.7G 0 disk <br />
|-sda1 8:1 0 90G 0 part /mnt/install<br />
`-sda2 8:2 0 77.7G 0 part <br />
`-bcache0 253:0 0 1.8T 0 disk /mnt/install/mnt/.hdd<br />
sdb 8:16 0 1.8T 0 disk <br />
|-sdb1 8:17 0 1G 0 part /mnt/install/boot<br />
|-sdb2 8:18 0 16G 0 part [SWAP]<br />
`-sdb3 8:19 0 1.8T 0 part <br />
`-bcache0 253:0 0 1.8T 0 disk /mnt/install/mnt/.hdd<br />
<br />
==== Generate an fstab ====<br />
<br />
Generate an fstab file with the following command. UUIDs will be used because they have certain advantages.<br />
<br />
genfstab -U -p /mnt/install >> /mnt/install/etc/fstab<br />
sed -i '/dev\/sda/ s/rw,relatime,data=ordered/defaults,noatime,discard/g' /mnt/install/etc/fstab<br />
sed -i '/mnt\/install/ {s/\/mnt\/install//g; s/rw,relatime,data=ordered/defaults/g; s/none/bind/g}' /mnt/install/etc/fstab<br />
more /mnt/install/etc/fstab<br />
<br />
=== Install and configure kernel and bootloader ===<br />
==== Adding Bcache module and hook ====<br />
<br />
Edit /etc/mkinitcpio.conf as needed and re-generate the initramfs image with:<br />
adding the "bcache" module<br />
adding the "bcache_udev" hook between block and filesystems<br />
<br />
sed -i '/^MODULES=/ s/"$/ bcache"/' /etc/mkinitcpio.conf<br />
sed -i '/^HOOKS=/ s/block filesystems/block bcache_udev filesystems/' /etc/mkinitcpio.conf<br />
more /etc/mkinitcpio.conf<br />
<br />
==== Registering and attaching Bcache ====<br />
This part is a pain in the A.. <br />
The problem is that Bcache wil give a error at startup if this is not done.<br />
<br />
ERROR: Unable to find root device 'UUID=b6b2d82b-f87e-44d5-bbc5-c51dd7aace15'.<br />
You are being dropped to a recovery shell<br />
Type 'exit' to try and continue booting<br />
<br />
First get the correct "SET UUID" with this ls command. <br />
And use the UUID like the example underneath.<br />
Also check for the use of the correct devices.<br />
echo: write error: Invalid argument <<< This error is good !!! means that your value was already stored.<br />
<br />
ls /sys/fs/bcache/<br />
2300f944-0eb5-4a9e-b052-8d5ea63cbb8f register register_quiet<br />
<br />
sudo echo /dev/sda2 > /sys/fs/bcache/register<br />
sudo echo /dev/sdb3 > /sys/fs/bcache/register<br />
sudo echo 2300f944-0eb5-4a9e-b052-8d5ea63cbb8f > /sys/block/sdb/sdb3/bcache/attach<br />
<br />
mv /usr/lib/initcpio/hooks/bcache ~/bcache.old<br />
cat >> /usr/lib/initcpio/hooks/bcache << EOF<br />
#!/usr/bin/ash<br />
run_hook() {<br />
echo /dev/sda2 > /sys/fs/bcache/register<br />
echo /dev/sdb3 > /sys/fs/bcache/register<br />
}<br />
EOF<br />
<br />
==== Installing the Kernel ==== <br />
mkinitcpio -p linux<br />
<br />
I hope some stuff is helpfull :P<br />
17:58, 11 November 2013 (UTC)<br />
:I'm curious about two things. First, though I forget what the '-E' switch signifies, it seems you're formatting the raw, underlayer partitions themselves. I don't understand why.<br />
<br />
:Also, the "pain in the a?" So udev is not assembling your array automatically as the devices are discovered? That is unexpected behavior, and I suspect it has to do with the formatting you performed and udev's superblock discovery. Check the very bottom of the bcache wiki on their site. At least try lsblk to verify "bcache" partition types are not registering as "ext4" before assembling the array.<br />
<br />
:Oh. And please sign talk pages.<br />
<br />
[[User:Mikeserv|Mikeserv]] ([[User talk:Mikeserv|talk]]) 03:35, 12 November 2013 (UTC)<br />
<br />
<br />
<br />
Well this is a experiment with Bcache with mine own Wiki page and 100x testing the install, i am not a linux expert just a newbie who was reading a lot of howto's. Just see it as a compilation of different web pages and some logical thinking. :)<br />
<br />
The -E is just a copy and past ... not sure if it was needed.I just thought it was needed to get the discard flag while formating.<br />
<br />
The pain in the A.... this was before Udev ... i was busy getting Bcache to work 3 months ago ... 75% of boot up failed when booting up cold. Later i found out that the SSD was too fast with booting up and the normal HDD was still spinning up.... After i splitted up the SSD for root directory and bcache and moved the /boot to the HDD since then i can bootup 100% of the time without any error... This was before the udev thingy (wich i dont understand and need to read up) :)<br />
<br />
So please understand that i am just a linux noob trying to contribute :)<br />
<br />
[[User:Emesix|Emesix]] ([[User talk:Emesix|talk]])[[User:Emesix|Emesix]] ([[User talk:Emesix|talk]]) 00:02, 13 November 2013 (UTC)<br />
<br />
:No, I get it, and I didn't mean to insinuate that you were doing anything wrong, exactly, just that, as designed, it could operate better.<br />
:Basically, and you should check the man-pages and the wiki page here to be sure just in case I'm not entirely correct (which does happen), udev is the program that runs during boot (and pretty much all of the rest of the time, too) to detect your hardware and load drivers as necessary. Udev is a relatively recent addition to the Linux boot process and is unique in that it discovers and handles hardware as it shows up, can handle multiple devices simultaneously with non-blocking parallell operations, and, as a result, is significantly faster than the methods it has replaced.<br />
:I was the one who added the udev rules and step 7b to this wiki, and I was the one who wrote the "Rogue Superblock" section of "Troubleshooting" at the bottom of bcache's wiki. I only gained the knowledge to write either after spending several frustrating days with an issue that appears very similar to your "pain in the a."<br />
:As step 7a shows, the prevailing Arch Linux method for reassembling a bcache array at boot used to look something like:<br />
: 1) Wait 5 secs. <br />
: 2) Check to see if udev has discovered and added {disk UUIDS}. If yes goto 3), no, repeat 1) and 2).<br />
: 3) echo {pain in the a stuff} /sys/bla/bla/bla<br />
:This does work, of course, but you're adding an unnecessary wait loop to your boot process and you're performing a task with brittle shell script that depends on specific UUIDs that could be performed flexibly and at discovery time.<br />
:That's what the udev rules do. Basically, the rules instruct udev to treat as part of your bcache array any partition it finds that reports itself as a "bcache" partition type. It builds the bcache array from disk partitions at every boot only as soon as those partitions are ready, which would of course resolve the race condition you mentioned and allow you to boot from SSD if you liked.<br />
:And here's where our experience might converge: I added a partition to my bcache array after previously formatting it as ext4. If you don't know about superblocks (or magic blocks or whatever they're called) then you're just like I was a few months ago. In short the filesystem has to have a way to report on itself, so it sets aside a small marker on disk that says, "Hey, OS, I'm a disk partition of type EXT4 and I start at block offset whatever and end at block offset whatever. Also, I enjoy long walks on the beach and prefer red wine to white. See you around."<br />
:The problem I experienced was that by default ext4 tended to begin at an earlier offset than bcache, and so even though I formatted over the previously ext4 partition with bcache as instructed, bcache never overwrote ext4's first superblock. When udev attempted to build my array, it would always miss my first partition because it would read the first superblock, identify the partition as ext4, then skip it. Echoing the add instruction to /sys/ after boot was my fix for a couple of days, but eventually I figured it all out and wrote that short bit on superblocks in bcache's wiki.<br />
:Maybe that helps?<br />
:EDIT: Just looked back at the wiki proper and somebody has performed some major changes recently. While it does appear cleaner, it no longer makes sense. There are references to non-existent steps throughout, and the actual why's that were included at least to some degree seem to have been occluded in favor of dubious, though possibly more efficient how's. Anyway, step 7b no longer exists, but it used to include the bcache_udev mkinitcpio hook I (very barely) adapted from mdadm_udev when trying to fix Arch's bcache boot problems. It's now included in the AUR package the wiki recommends I guess, despite the wiki also telling us we must "echo ... /sys/... at every bootup."<br />
[[User:Mikeserv|Mikeserv]] ([[User talk:Mikeserv|talk]]) 07:08, 14 November 2013 (UTC)<br />
<br />
Sorry about that, I missed a couple of references back to the sections I changed. Hopefully it makes more sense now. [[User:Mikeserv|Mikeserv's]] udev rule is crucial to having a working bcache device on boot, as [[User:Emesix|Emesix]] had discovered, it will often result in a failed boot. The echo in to /sys/bcache* step was missed when I updated it to mention the udev rule as the 'default' method.<br />
<br />
--[[User:Justin8|Justin8]] ([[User talk:Justin8|talk]]) 10:45, 14 November 2013 (UTC)<br />
:For clarity's sake, the udev rule was never mine. The used rule has been included in the AUR package's git pull from at least the time I first installed bcache some months ago. The package maintainer at that time opted not to use it, I guess, and instead installed the legacy-type shell-scripted for-loop mkinitcpio hook I outlined above, which was, to be fair, also the process recommended by every other source of information I was able to dig up at the time to include bcache's own wiki. Frustrated with having to wait for such things which seemed to me to defeat the whole purpose of buying and configuring an SSD boot device, I eventually stumbled upon the mdadm_udev hook in /use/share/initcpio and (only slightly) modified it to apply the 69-bcache-rules instead.<br />
:Justin, thanks for your recent change. NEVERMIND FOLLOWING: I'm curious why you specify "unless assembling after boot?" Why include anything to do with bcache in the initramfs at all if you're building the array after the boot sequence? MAKES PERFECT SENSE OF COURSE. GOT A 1 TRACK MIND SOMETIMES I GUESS. SORRY.<br />
[[User:Mikeserv|Mikeserv]] ([[User talk:Mikeserv|talk]]) 11:11, 14 November 2013 (UTC)<br />
<br />
:Emesix, just looked closer at your instructions, and you actually do the thing I had to do and outlined in the bcache wiki to overwrite the rogue superblock here:<br />
:{{ic|<nowiki>dd if=/dev/zero count=1 bs=1024 seek=1 of=/dev/sd{a2,b3}</nowiki>}}<br />
:And the ext4 formatting you do after that I earlier incorrectly assumed to be for the "raw, underlayer partitions" is revealed upon further inspection to be for non-bcache partitions and so isn't even relevant; I apologize for jumping to conclusions before reviewing it fully, the quick check I did a couple of days ago just struck me as really familiar.<br />
:The udev stuff above should still be relevant, and it might be worth noting that the particular {{ic|dd}} command above is by no means a cure-all. It should only be used as written if you have previously identified a superblock located at byte offset 1024 that needs overwriting, though the actual superblock location can vary by filesystem. I've since learned that a much more flexible and useful tool for this is {{ic|wipefs}}, which is much less dangerous than it sounds.<br />
: [[User:Mikeserv|Mikeserv]] ([[User talk:Mikeserv|talk]]) 14:28, 14 November 2013 (UTC)<br />
<br />
Thanks for the explanation of the super-block [[User:Mikeserv|Mikeserv's]] was very insightful. But that Udev stuff is still a layer to high for me :P I don't know if its possible to do a little differently to avoid a unwanted wait loop:<br />
: 1) Check to see if udev has discovered and added {disk UUIDS}. If yes goto 3)<br />
: 2) Wait 5 secs and goto 1.<br />
: 3) echo {pain in the a stuff} /sys/bla/bla/bla<br />
The dd command is just a safety step. :) And because formatting the /dev/bcache0 was soon to follow, it didn't hurt if i accidentally killed the file system on the bcache :) I got the feeling that resizing the bcache partition was also giving problems wich the dd command fixed..... BTW i like the piece added with this command set:<br />
# cd /sys/fs/bcache<br />
# echo "UUID" > /sys/block/bcache0/bcache/attach<br />
p.s. I don't get why the wiki page is full of btrfs stuff. Is this not a combination of commands? And shouldn't a more conventional partition/file system scheme be more useful for novice users (like me) ??<br />
: [[User:Emesix|Emesix]] ([[User talk:Emesix|talk]]) 22:47, 14 November 2013 (UTC)<br />
<br />
::Regarding btrfs and its usefulness for new users... I don't know. With btrfs for instance, there are many normally high-level file-system concepts rendered nearly as mundane as directory maintenance like snapshotting and subvolumes, and still others that require only a single edit to a boot config like {{ic|autodefrag}} and {{ic|<nowiki>compress=lzo</nowiki>}} delivering inline filesystem compression and cleanup. In practice, you really don't have to understand the hows of these things at all to get incredible use out of them. <br />
::Consider just sub volumes: they can deliver, at the very least, a separated partition structure for the various segments of the Linux filesystem as is often recommended people set at installation with syntax and simplicity like unto {{ic|cd}} and {{ic|mkdir}}, only the {{ic|btrfs --help}} implementation is probably a little friendlier. With traditional filesystems this same setup required knowledge of disk geometry, extent counts and all manner of other arcane things to configure in any sanely optimized way. Btrfs will create you a RAID in less than 30 seconds. <br />
::I guess what I'm saying is the research vs reward ratio is pretty significantly in your favor with a btrfs disk. Of course, should the btrfs filesystem crash, well, you should probably call someone else is all I'll say about that.<br />
::In this wiki, I definitely see your point, especially considering the added complexity bcache brings. When I was trying to set mine up I spent a good deal of time in both the btrfs and bcache IRC dev channels and both sides were less than optimistic about friendly cooperation between the two. Still, it works and has done for months. I know, don't call you, right?<br />
::Now about udev, I'll try again. So, your script implements a wait loop to check occasionally if udev has done a thing (bring up the disks and pull in basic drivers like SCSI for block devices and ext4 for their component filesystems) so it can do a thing (register the disks as small parts of a larger whole and pull in the bcache driver). The udev rules eschew the notion of a secondary monitor script such as yours and instead have udev do it all (any disk it brings up as a "bcache" type is registered as a small part of a larger whole and the bcache driver is pulled in). <br />
::So you couldn't boot from SSD reliably before because you couldn't reliably script the order in which udev would bring up the disks (a consequence of its faster parallel operations), but udev doesn't have to script its order, udev just udevs as udev udevs best.<br />
::See? You will always boot reliably from your chosen device because udev won't misinterpret its own report on your chosen device's readiness. And that's all - there's nothing wrong with the shell script exactly, except that it's just plain unnecessary and added complication.<br />
[[User:Mikeserv|Mikeserv]] ([[User talk:Mikeserv|talk]]) 23:38, 14 November 2013 (UTC)<br />
<br />
:: I have to say, I want to try out using btrfs instead of LVM on my desktop sometime, but the cache article is not the place for it. Mentioning it as a good alternative on the installation and beginners guide and having details in the btrfs article is going to be more productive and logical. I added the section near the top of the page for configuring a simple cache setup with no regards to file systems and partition layouts as they should be beyond the scope of this article. <br />
<br />
--[[User:Justin8|Justin8]] ([[User talk:Justin8|talk]]) 02:13, 15 November 2013 (UTC)<br />
::Agreed. There's a lot of it, too. I guess that sometimes happens on less visited pages. Probably it will level out as the module is more established. I compromised on some of the grub stuff in a minor edit yesterday after you rightly pointed out how much unrelated stuff is in there. Never thought about it before, but I've only ever written 7b and 2 notes here and a short section on UEFI kernel update syncing.<br />
::Above is just about everything I know relatable to bcache by now, I guess. I wish there was some more in the wiki here on benchmarking. The bcache wiki itself reads like a stereo repair manual on the subject. I never have managed to wrap my mind round it before my eyes would shut of their own accord. <br />
[[User:Mikeserv|Mikeserv]] ([[User talk:Mikeserv|talk]]) 03:00, 15 November 2013 (UTC)<br />
<br />
:: Ha, that is the perfect description for their wiki; it's all failure modes and fixes but no reasoning. From what I could see, benchmarking of bcache is really only useful to benchmark writes; reads will vary exponentially depending on whether or not you get a cache hit. On top of that, sequential reads/writes bypass the cache. the only real way to benchmark it would be in 'real-world' style tests instead of synthetic benchmarks. I.e. how long it takes to boot your system. how long it takes to compile X, Y or Z. I put a little bit up on my blog when testing out bcache, mostly focussing on IOps performance changes since everything else is relatively the same with or without bcache (other than the smart re-ordering of metadata changes) I was going to just paste the graphs, but they are all js objects; you can see the results I got here: https://www.dray.be/?p=7<br />
[[User:Justin8|Justin8]] ([[User talk:Justin8|talk]]) 04:05, 15 November 2013 (UTC)<br />
<br />
:::Justin: so you're the AUR bcache-tools maintainer now? I guess you dropped enough hints. That's cool. Reading through your blog (pretty colors) I was reminded of another little bcache fact that may not be immediately apparent to all. It seems most tend to assume that the cache:backing-store must be on a 1:1 ratio. This is not the case. A single cache can serve as many storage devices as you might like. Conversely, there's nothing preventing one from configuring several mechanical discs in a RAID and then backing it on a 1:1 ratio. Regarding a cached RAID, though, I do recall reading a bcache IRC conversation in which a dev mentioned hardware configuration of the device would be desirable as bcache is designed to interface with a block layer and dropping a filesystem below it can have unexpected results. <br />
:::I only bring this up because your blog mentions you will switch from a 1tb to a 3tb soon. Personally, I use a 120GB Kingston SSD to cache 2 3tb WD Greens which are then formatted as a single btrfs RAID1 metadata/RAID0 data. The data is mostly downloaded video for a home theater server and as such is largely expendable in the event of catastrophe. Probably you know this, but just in case, I thought I'd point out that there's nothing preventing you from merely adding the the 3tb to the 1tb as you please.<br />
:::Lastly, you can format and configure an entire array with basically one line like: {{ic|<nowiki>make-bcache -B /dev/{1tb} /dev/{new_3tb} -C /dev/{60GB_SSD}</nowiki>}}. Then just reboot (or even reload udev) if you've got the udev rules loaded - udev will register the devices for you based on the auto-attach data recorded in the partition's superblocks as a result of the all-in-one add command. You wouldn't even need the bcache_udev hook or anything else in initramfs at this point, because obviously you shouldn't expect to boot to the bcache array yet as you haven't had a chance to format it with a usable filesystem or to copy any files to it, but it can simplify things.<br />
:::[[User:Mikeserv|Mikeserv]] ([[User talk:Mikeserv|talk]]) 07:04, 15 November 2013 (UTC)<br />
<br />
::::I did notice a reference to that from the previous maintainer in the aur comments. I currently have a 9x2tb RAID6 mdadm array that I really don't want to have to backup and restore, so I did the tests as it would be in my server (just on a smaller scale). I don't believe it would be a simple task to remove the device, bcache-ify it and put it back in. The blocks utility says that it supports it for any block device; but mdadm stores data about the offset in the superblocks instead of using the partition layout (which I discovered the hard way in the past and had to restore 10TB of backups from off-site :( ) It should be fine though doing it to the MDADM device itself. I wouldn't mind knowing the performance difference, so once I get an SSD for my server I might test it with both methods to do a bit of a comparison.<br />
::::I didn't realize you could specify the backing and cache devices all in one line like that. I didn't see it in the wiki at all when I did my first lot of edits; seems useful. Does it automatically attach the cache to the backing devices as well?<br />
::::Although I reboot fairly often for things like kernel updates/making sure pacman hasn't broken things, I am somewhat averse to rebooting when I don't have to. I migrated from a root device on an SSD and /home and /opt on a 3TB disk, on to a spare 1TB disk, did the tests for my blog on a different computer and then created the bcache device with the 3TB disk and 60GB SSD and migrated back all without a reboot. Sometimes rebooting can be easier, but it's more 'fun' not to :)<br />
::::[[User:Justin8|Justin8]] ([[User talk:Justin8|talk]]) 08:28, 15 November 2013 (UTC)<br />
:::::Well, as I mentioned, you could opt to restart udev rather than reboot, or alternatively you could choose to throw a {{ic|udevadm}} trigger, or to have some fun with {{ic|kexec}}, or merely to {{ic|echo}} the block-device register commands to {{ic|/sys/}} as per usual. You have correctly surmised that the one-liner auto-attaches the backing-stores to the cache, but it does not perform the registration, which is necessary anyway every time an array is re/assembled as it is.<br />
::::: I added it to the bcache Arch wiki page myself after discovering it quite by accident between nods while browsing bcache's own documentation a few months ago. The command is represented in the wiki even now, if rather humbly as a side note to the initial backing-store formatting process. I mentioned it on your talk page as well, because you had apparently previously edited it not to fix its then anachronistic reference to no longer needing a now completely different step 4, but to note that it would no longer be necessary at boot. At first I was bewildered, but I've since corrected it to note that it now obviates step 6.<br />
:::::By the way, I might know of another thing worth saying, of which {{ic|kexec}}'s mention has reminded me: because of the way SystemD performs its unmounts at shutdown, it can be very slightly dangerous to run a multi-device write-back cache outside of its purview, as you must do if you boot to a bcache array but have not incorporated SystemD into your initramfs. If this is your situation currently, then likely you will be able to catch an occasional error message printed to the console about a forced unmount in the second or two just before your screen blanks its last. Of course bcache is supposed to be able to handle this and much more, and it never caused me any issues (though I run in write-through mode), but I did switch to the new SystemD hook just as soon as I noticed it. Specifically the SystemD {{ic|kexec}} function should almost certainly be avoided whenever possible when operating a write-back cache without a true PID1 SystemD configuration.<br />
:::::[[User:Mikeserv|Mikeserv]] ([[User talk:Mikeserv|talk]]) 09:13, 15 November 2013 (UTC)<br />
<br />
::::::That is interesting. (I assume you're talking about the systemd hook for mkinitcpio that is intended to replace udev/usr/base/etc) How does using that new systemd hook fix it unmounting drives too soon? Also the wiki notes that it is incompatible with lvm2 currently; which just so happens to be what my root filesystem is using... Maybe it is yet another reason to change to btrfs finally. I am using writeback on my bcache device as well. But so long as the cache device is available on a reboot it shouldn't affect anything.<br />
::::::[[User:Justin8|Justin8]] ([[User talk:Justin8|talk]]) 07:58, 16 November 2013 (UTC)<br />
:::::::Well, depending on your use-case, you may find btrfs+bcache to be a less than ideal combination. As I understand it, btrfs is designed to concatenate write operations in memory at least until it has compiled enough of them to deliver a sequentially written block to disk. Obviously any write operations given bcache will come from btrfs when the two are combined, and because bcache is designed to pass-through sequential writes, most disk writes should bypass your ssd cache entirely. This doesn't concern me overmuch because the majority of my disk space is consumed by large video files which, when first written, are sequential anyway, and the slower write speed is mostly mitigated with the two-disk btrfs RAID I configured. It is very nice, however, to have the most used applications and most watched videos dynamically located on the ssd, especially since the one system's disks serve several other machine's multimedia requests.<br />
:::::::It is my theory that bcache can assist a btrfs file-system in avoiding some of its infamous disk-thrashing tendencies associated with its relatively huge metadata structures by caching and consolidating even these write operations into sequential blocks, which, if true, is probably especially useful in my particular configuration of striped data and mirrored metadata. Unfortunately, as you might have gleaned from a previous statement I've made, I can hardly back this up with any hard data as proof. I also theorize that the btrfs kernel parameter autodefrag, which in a conventional configuration can severely affect write-speeds, becomes a much more sensible option with the addition of an ssd configured with bcache, but I can't offer any evidence to support that either.<br />
:::::::As I said when first replying to E, though a repeat is probably long overdue, I am sometimes wrong (some may have cause even to sneer at the "sometimes"), and before I go on I should say again that anything I write is likely to serve one best if considered in combination with other sources of information. If it seems to anyone that what I say makes no sense or is contradictory in some way then it's probably best to verify that doubt one way or another before proceeding with any operation that might affect valued data, or even that might just result in an unnecessarily wasted day spent reconfiguring twice what used to be a computer system that would boot with the press of a button. Also, if anyone catches something wrong, please correct me.<br />
:::::::Sorry, it just occurred to me that others might venture this way eventually and act on what they read so I figured another disclaimer was called for. By the way, if you ever wind up as bewildered by the Arch Wiki LXC page as I was, definitely read through its talk page before giving up and moving on.<br />
:::::::Ok, so about systemd, we have to start here: <br />
:::::::I've noticed a lot of people, as I used to do, tend to ascribe unwarranted mystery to initramfs. Probably, at least in Arch's case, this is an unavoidable consequence to the level of automated excellence provided by its mkinitcpio script; it is because we so rarely have any cause to understand it that so few of us do. Certainly I didn't anyway, until very recently.<br />
:::::::The initramfs is a file containing an image of the first filesystem mounted by the kernel after it is called. This image-file can be compiled into the kernel itself at build-time, but, because it can be more convenient to swap in other images at will, it is conventional instead to feed its path to the kernel as a parameter. The kernel expects the image's final archived format to be "cpio" (CoPy In/Out archives can handle items many others can't like /dev/, /sys/ and similar), but it can decompress that from any format you can compile into it (like lzma, lz4, gz, xz, etc). In its current iteration (as of kernel 2.6.something), when loaded the kernel mounts initramfs as "rootfs," which is basically the contents of this image mounted to / as a tmpfs.<br />
:::::::As far as the kernel is concerned its primary boot job is to find and call /init, and all the rest is up to user-space. And because the linux kernel can potentially handle more / disk configurations than can practically be accounted for in a single compile without growing exponentially every year, it was 2.6.something when initramfs became mandatory. For all intents and purposes, you can still compile in any modules required to mount your root filesystem, run /init from there, and skip both compiling as built-in and parameterizing an initramfs image at load-time, but the kernel still contains and mounts a (basically) empty / image no matter what you do. <br />
:::::::mkinitcpio's "base" hook is basically just busybox and a couple of shell scripts, one of which is /init. Arch's default initramfs behavior is pretty similar to most others; it packs in as little as necessary to find and mount your filesystem at /newroot then uses "switchroot" (the only really out-of-the-ordinary part of the whole process) to simultaneously dismount the initramfs /, mount /newroot in its place and call systemd.<br />
:::::::Ok, sorry for the lecture, but there's already enough here to warrant my coming back when I've forgotten any of it, so I figured i'd keep my future self on the right track if I could manage.<br />
:::::::Anyway, so have a look at the lsblk printouts at the bottom of bcache's wiki here: http://bcache.evilpiepirate.org/#index6h1<br />
:::::::Notice that after the array is built successfully (shown in the second lsblk output) the /dev/bcache{0,1} partitions are represented twice; they're children of both the physical disks on which they actually reside and the cache disk that buffers their i/o. The first lsblk, pulled before the bcache device to which they're attached is registered, is representative of a more typical configuration in which each partition is descended only from the physical device on which it resides.<br />
:::::::That added complexity is, as near as I can figure, what trips systemd's shutdown unmounts up. The bcache module, its user-space tools, and your / array are all called by the initramfs udev which is killed at "switchroot" when systemd loads so it can load and properly manage its own child udev process. <br />
:::::::systemd is designed from the ground up to be PID1, /init, the daemons' daemon. Through /sys/, it manages a pid tree with itself as the root reliably tracking and killing even forked child processes when the original parent processes quit. Asking it to undo complicated mount structures built before ever it can hook into /sys/, however, is probably asking for trouble.<br />
:::::::Still, as you say, dirty or not the data is still there. bcache is a disk cache after all, and it will persist between reboots in the cache on the ssd even if the array is not cleanly disassembled before the disks are unmounted. And bcache is specifically designed to track in its own btrees the cached data's final disk targets in order to reliably handle exactly these kinds of situations. I qualified my earlier description of the scenario as only "very slightly" unsafe because the redundancy is reduced. There is also the outside possibility that the cached data could be orphaned if the original backing store > cache attachment relationship written to the array members' bcache partition superblocks is somehow broken, which, at least when I configured my disks only a few months ago, was noted in the bcache docs as rare but possible. This last, I should say, I have never experienced, but, and only for reasons I can't exactly put my finger on, I have a hunch it could be related to the whole systemd dirty unmount scenario described above.<br />
:::::::I specifically cautioned against kexec before because it's sort of a "switchkernel" version of the "switchroot" process I referenced above. kexec first writes an OS kernel to system memory, then it simultaneously writes out the current kernel's mapped region and writes the to-be-loaded kernel over it, before finally, in its last gasp, as it were, calling the new kernel to execute. It seems to me that if the daemons' daemon has difficulty handling preexisting conditions then testing the OS kernel in such circumstances is probably not something I'd like to do.<br />
:::::::Geez. That's a lot. But there's more.<br />
:::::::While I can't say for certain because I haven't looked into it, I suspect that an lvm2 / is only contra-indicated for the systemd mkinitcpio hook because the particular combination has not yet been scripted to the same level of automatic detection and reliable configuration as the rest. There's a huge difference between reliably configuring a thing and reliably automating a configuration of course, and as I'm sure Arch's core maintainers are painfully aware. It is probably for precisely this reason that Arch enables the bcache kernel module in its default kernel compile but provides no supported means of making use of it. Still, as far as I know, systemd is not inherently incompatible with lvm2, and because the initramfs is just a / doing (mostly) plain old linuxy / things, there's no reason you can't set it up any way you please. Maybe get to know lsinitcpio a little better if you're interested.<br />
:::::::And here's one of perhaps special import to the AUR's bcache-tools package maintainer; while putting systemd in the initramfs is the one way I've handled the shutdown problem, I doubt very seriously it is the only way. Two other possibilities come immediately to mind: either hooking into or emulating in some way the mkinitcpio shutdown (or is it suspend?) hook which I expect must have some means of handling an end event or two; or, and probably the way i'd go, systemd's own shutdown target, which definitely does and is probably specifically designed to handle just such a thing. Probably there are many others as well, but none spring to mind.<br />
:::::::You can wake up now. I'm through.<br />
:::::::[[User:Mikeserv|Mikeserv]] ([[User talk:Mikeserv|talk]]) 05:56, 17 November 2013 (UTC)<br />
<br />
::::::::That was long! Perhaps we should take this conversation off the talk page so that we don't fill it up too much. Do you use IRC/gtalk/skype/facebook at all? I'd like to discuess some more in regards to the possiblity of a shutdown hook for bcache.<br />
:::::::::Yes, long, but I think its relevant. I was kind of hoping some kind-hearted soul would happen by and, perhaps having learned something, convert some of my pedantic stream-of-consciousness into something more usefully accessible. Wanna know the worst of it? It was written almost entirely on my Nexus 4 at various times throughout the day. I guess I'm something of a masochist.<br />
:::::::::Anyway, anyone can edit it out, of course. I saved a copy.<br />
::::::::[[User:Mikeserv|Mikeserv]] ([[User talk:Mikeserv|talk]]) 06:22, 17 November 2013 (UTC)<br />
Dont leave me out :P I am learning alott from the info given here :) And i have 2 systems with Bcache on which i can test or benchmark :)<br />
--[[User:Emesix|Emesix]] ([[User talk:Emesix|talk]]) 00:38, 18 November 2013 (UTC)<br />
:Ok, E. I'm pretty sure a simple unregister call is what's needed in systemd's unmounts.target, but before we get there can you verify you've got the udev rules working as intended? <br />
:[[User:Mikeserv|Mikeserv]] ([[User talk:Mikeserv|talk]]) 10:14, 19 November 2013 (UTC)<br />
<br />
I'm at work currently, but I've made a pkgbuild for v1.0.5 tools that use the upstream mkinitcpio hook (it's almost identical to yours Mikeserv!) I want to test it a bit before I push it out. The changes are now in the bcache-tools-git package however as otherwise it wouldn't really build against the latest upstream patch. I should hopefully have the new version up in the next 10 or so hours.</div>Justin8https://wiki.archlinux.org/index.php?title=Bcache&diff=291861Bcache2014-01-07T03:59:30Z<p>Justin8: Changed to recommending bcache-tools package instead of bcache-tools-git; people who want the git version can still use it, but the recommendation should be for the stable version.</p>
<hr />
<div>[[Category:File systems]]<br />
== Introduction ==<br />
<br />
Bcache allows one to use an SSD as a read/write cache (in writeback mode) or read cache (writethrough or writearound) for another blockdevice (generally a rotating HDD or array). This article will show how to install arch using Bcache as the root partition. For an intro to bcache itself, see [http://bcache.evilpiepirate.org/ the bcache homepage]. Be sure to read and reference [http://atlas.evilpiepirate.org/git/linux-bcache.git/tree/Documentation/bcache.txt?h=bcache the bcache manual]. Bcache is in the mainline kernel since 3.10. The kernel on the arch install disk includes the bcache module since 2013.08.01.<br />
<br />
An alternative to Bcache is Facebook's [[Flashcache]].<br />
<br />
Bcache needs the backing device to be formatted as a bcache block device. In most cases, [https://github.com/g2p/blocks blocks to-bcache] can do an in-place conversion.<br />
{{Warning|Be sure you back up any important data first.}}<br />
<br />
== Setting up a bcache device on an existing system ==<br />
<br />
1. Install the {{AUR|bcache-tools}} package from [[AUR]].<br />
<br />
2. Create a backing device (This will typically be your mechanical drive). The backing device can be a whole device, a partition or any other standard block device. This will create /dev/bcache0<br />
# make-bcache -B /dev/sdx1<br />
<br />
3. Create a cache device (This will typically be your SSD). The cache device can be a whole device, a partition or any other standard block device<br />
# make-bcache -C /dev/sdx1<br />
<br />
4. Register the cache device against your backing device. We need to find the UUID of your cache device and then add it to the bcache device initially. Udev rules will take care of this on reboot and will only need to be done once.<br />
# cd /sys/fs/bcache<br />
# ls<br />
# echo "UUID" > /sys/block/bcache0/bcache/attach<br />
<br />
5. Change your cache mode (if you want to cache writes as well as reads):<br />
# echo writeback > /sys/block/bcache0/bcache/cache_mode<br />
<br />
6. If you want to have this partition available during the initcpio (i.e. you require it at some point in the boot process) you need to add 'bcache' to your modules array in /etc/mkinitcpio.conf as well as adding the 'bcache_udev' hook in your list between block and filesystem.<br />
<br />
<br />
== Installation to a bcache device ==<br />
<br />
1. Boot on the install disk (2013.08.01 minimum).<br />
<br />
2. Install the {{AUR|bcache-tools}} package from [[AUR]].<br />
<br />
3. Partition your hdd<br />
<br />
{{Note|While it may be true that Grub2 does not offer support for bcache as noted below, it does, however, fully support UEFI. It follows then, that so long as the necessary modules for the linux kernel to properly handle your boot device are either compiled into the kernel or are included in an initramfs, and you can include these files on it, the separate boot partition described below may be omitted in favor of the FAT EFI system partition. See [[GRUB]] and/or [[UEFI]] for more.}}<br />
grub can't handle bcache, so you'll need at least 2 partitions (boot and one for the bcache backing device). If you're doing UEFI, you'll need an EFI System Partition (ESP) as well. E.g.:<br />
1 2048 22527 10.0 MiB EF00 EFI System<br />
2 22528 432127 200.0 MiB 8300 arch_boot<br />
3 432128 625142414 297.9 GiB 8300 bcache_backing<br />
<br />
{{Note|This example has no swapfile/partition. For a swap partition on the cache, use LVM in step 7. For a swap partition outside the cache, be sure to make a swap partition now.}}<br />
<br />
4. Configure your HDD as a bcache backing device.<br />
<br />
# make-bcache -B /dev/sda3<br />
<br />
{{Note|<br />
* When preparing any boot disk it is important to know the ramifications of any decision you may make. Please review and review again the documentation for your chosen boot-loader/-manager and consider seriously how it might relate to bcache.<br />
* If all associated disks are partitioned at once as below bcache will automatically attach "-B backing stores" to the "-C ssd cache" and step 6 is unnecessary.<br />
}}<br />
<br />
# make-bcache -B /dev/sd? /dev/sd? -C /dev/sd?<br />
<br />
5. Register any bcache devices with the kernel (this needs to done every bootup if you do not use the bcache_udev hook in your mkinitcpio, unless you attach the bcache devices after boot has completed)<br />
<br />
# for i in /dev/sd*; do echo $i; echo $i > /sys/fs/bcache/register_quiet; done<br />
<br />
You now have a {{ic|/dev/bcache0}} device.<br />
<br />
{{Note|the bcache user manual says to do {{ic|echo /dev/sd* > /sys/fs/bcache/register_quiet}}, but this did not work for me.}}<br />
<br />
6. Configure your SSD<br />
<br />
Format the SSD as a caching device and link it to the backing device<br />
<br />
# make-bcache -C /dev/sdb<br />
# echo /dev/sdb > /sys/fs/bcache/register <br />
# echo ''UUID__from_previous_command'' > /sys/block/bcache0/bcache/attach<br />
<br />
{{Note|If the UUID is forgotten, it can be found with {{ic|ls /sys/fs/bcache/}} after the cache device has been registered.}}<br />
<br />
7. Format the bcache device. Use LVM or btrfs subvolumes if you want to divide up the {{ic|/dev/bcache0}} device how you like (ex for seperate {{ic|/}}, {{ic|/home}}, {{ic|/var}}, etc):<br />
<br />
# mkfs.btrfs /dev/bcache0<br />
# mount /dev/bcache0 /mnt/<br />
# btrfs subvolume create /mnt/root<br />
# btrfs subvolume create /mnt/home<br />
# umount /mnt<br />
<br />
8. Prepare the installation mount point:<br />
<br />
# mkfs.ext4 /dev/sda2<br />
# mkfs.msdos /dev/sda1 (if your ESP is at least 500MB, use mkfs.vfat to make a FAT32 partition instead)<br />
# pacman -S arch-install-scripts<br />
# mount /dev/bcache0 -o subvol=root,compress=lzo /mnt/<br />
# mkdir /mnt/boot /mnt/home<br />
# mount /dev/bcache0 -o subvol=home,compress=lzo /mnt/home<br />
# mount /dev/sda2 /mnt/boot<br />
# mkdir /boot/efi<br />
# mount /dev/sda1 /mnt/boot/efi/<br />
<br />
9. Install the system as per the [[Installation_Guide#Connect_to_the_internet|Installation Guide]] as normal except this:<br />
<br />
Before you edit {{ic|/etc/mkinitcpio.conf}} and run {{ic|mkinitcpio -p linux}}:<br />
<br />
* install {{AUR|bcache-tools}} package from the [[AUR]].<br />
* Edit {{ic|/etc/mkinitcpio.conf}}:<br />
** add the "bcache" module<br />
** add the "bcache_udev" hook between block and filesystem hooks<br />
<br />
== Configuring ==<br />
<br />
There are many options that can be configured (such as cache mode, cache flush interval, sequential write heuristic, etc.) This is currently done by writing to files in {{ic|/sys}}. See the [http://atlas.evilpiepirate.org/git/linux-bcache.git/tree/Documentation/bcache.txt bcache user documentation].<br />
<br />
Changing the cache mode is done by echoing one of 'writethrough', 'writeback', 'writearound' or 'none' to /sys/block/bcache[0-9]/bcache/cache_mode.<br />
<br />
== Advanced operations ==<br />
<br />
=== Resize backing device ===<br />
<br />
It's possible to resize the backing device so long as you don't move the partition start. This process is described on [http://comments.gmane.org/gmane.linux.kernel.bcache.devel/249 the mailing list]. Here is an example using btrfs volume directly on bcache0. For LVM containers or for other filesystems, procedure will differ.<br />
<br />
==== Example of growing ====<br />
<br />
In this example, I grow the filesystem by 4GB. <br />
<br />
1. Reboot to a live CD/USB Drive (need not be bcache enabled) and use fdisk, gdisk, parted, or your other favorite tool to delete the backing partition and recreate it with the same start and a total size 4G larger. <br />
<br />
{{Warning|Do not use a tool like GParted that might perform filesystem operations! It will not recognize the bcache partition and might overwrite part of it!!}}<br />
<br />
2. Reboot to your normal install. Your filesystem will be currently mounted. That is fine. Issue the command to resize the partition to its maximum. For btrfs, that is<br />
<br />
# btrfs filesystem resize max /<br />
<br />
For ext3/4, that is:<br />
<br />
# resize2fs /dev/bcache0<br />
<br />
==== Example of shrinking ====<br />
<br />
In this example, I shrink the filesystem by 4GB.<br />
<br />
1. Disable writeback cache (switch to writethrough cache) and wait for the disk to flush.<br />
<br />
# echo writethrough > /sys/block/bcache0/bcache/cache_mode<br />
$ watch cat /sys/block/bcache0/bcache/state<br />
<br />
wait until state reports "clean". This might take a while.<br />
<br />
2. Shrink the mounted filesystem by something more than the desired amount, to ensure we don't accidentally clip it later. For btrfs, that is:<br />
<br />
# btrfs filesystem resize -5G /<br />
<br />
For ext3/4 you can use ''resize2fs'', but only if the partition is unmounted<br />
<br />
$ df -h /home<br />
/dev/bcache0 290G 20G 270G 1% /home<br />
# umount /home<br />
# resize2fs /dev/bcache0 283G<br />
<br />
3. Reboot to a LiveCD/USB drive (doesn't need to support bcache) and use fdisk, gdisk, parted, or your other favorite tool to delete the backing partition and recreate it with the same start and a total size 4G smaller.<br />
<br />
{{Warning|Do not use a tool like GParted that might perform filesystem operations! It won't recognize the bcache partition and might overwrite part of it!!}}<br />
<br />
4. Reboot to your normal install. Your filesystem will be currently mounted. That is fine. Issue the command to resize the partition to its maximum (that is, the size we shrunk the actual partition to in step 3). For btrfs, that is:<br />
<br />
# btrfs filesystem resize max /<br />
<br />
For ext3/4, that is:<br />
<br />
# resize2fs /dev/bcache0<br />
<br />
5. Re-enable writeback cache if you want that enabled:<br />
<br />
# echo writeback > /sys/block/bcache0/bcache/cache_mode<br />
<br />
{{Note|If you're very careful you can shrink the filesystem to the exact size in step 2 and avoid step 4. Be careful, though, many partition tools don't do exactly what you want, but instead adjust the requested partition start/end points to end on sector boundaries. This may be difficult to calculate ahead of time}}<br />
<br />
== Troubleshooting ==<br />
<br />
=== /dev/bcache device doesn't exist on bootup ===<br />
<br />
If you are sent to a busy box shell with an error:<br />
<br />
{{bc|1=<br />
ERROR: Unable to find root device 'UUID=b6b2d82b-f87e-44d5-bbc5-c51dd7aace15'.<br />
You are being dropped to a recovery shell<br />
Type 'exit' to try and continue booting<br />
}}<br />
<br />
This might happen if the backing device is configured for "writeback" mode (default is writearound). When in "writeback" mode, the /dev/bcache0 device is not started until the cache device is both registered and attached. Registering is something that needs to happen every bootup, but attaching should only have to be done once. I've sometime had to re-attach.<br />
<br />
To continue booting, try one of the following:<br />
<br />
* Register both the backing device and the cacheing device<br />
<br />
# echo /dev/sda3 > /sys/fs/bcache/register<br />
# echo /dev/sdb > /sys/fs/bcache/register<br />
<br />
If the /dev/bcache0 device now exists, type exit and continue booting. You will need to fix your initcpio to ensure devices are registered before mounting the root device.<br />
<br />
{{Note|<br />
* An error of "sh: echo: write error: Invalid argument" means the device was already registered or is not recognized as either a bcache backing device or cache. If using the udev rule on boot it should only attempt to register a device if it finds a bcache superblock<br />
* This can also happen if using udev's 69-bcache.rules in Installation's step 7 and blkid and bcache-probe "disagree" due to rogue superblocks. See [http://bcache.evilpiepirate.org/#index6h1 bcache's wiki] for a possible explanation/resolution.<br />
}}<br />
<br />
* Re-attach the cache to the backing device:<br />
<br />
If the cache device was registered, a folder with the UUID of the cache should exist in {{ic|/sys/fs/bcache}}. Use that UUID when following the example below:<br />
<br />
# ls /sys/fs/bcache/<br />
b6b2d82b-f87e-44d5-bbc5-c51dd7aace15 register register_quiet<br />
# echo b6b2d82b-f87e-44d5-bbc5-c51dd7aace15 > /sys/block/sda/sda3/bcache/attach<br />
<br />
If the {{ic|/dev/bcache0}} device now exists, type exit and continue booting. You should not have to do this again. If it persists, ask on the bcache mailing list.<br />
<br />
{{Note|An error of {{ic|sh: echo: write error: Invalid argument}} means the device was already attached. An error of {{ic|sh: echo: write error: No such file or directory}} means the UUID is not a valid cache (make sure you typed it correctly).}}<br />
<br />
* Invalidate the cache and force the backing device to run without it. You might want to check some stats, such as "dirty_data" so you have some idea of how much data will be lost.<br />
<br />
# cat /sys/block/sda/sda3/bcache/dirty_data<br />
-3.9M<br />
<br />
dirty data is data in the cache that has not been written to the backing device. If you force the backing device to run, this data will be lost, even if you later re-attach the cache.<br />
<br />
# cat /sys/block/sda/sda3/bcache/running<br />
0<br />
# echo 1 > /sys/block/sda/sda3/bcache/running<br />
<br />
The {{ic|/dev/bcache0}} device will now exist. Type exit and continue booting. You might want to unregister the cache device and run make-bcache again. An fsck on {{ic|/dev/bcache0}} would also be wise. See the [http://atlas.evilpiepirate.org/git/linux-bcache.git/tree/Documentation/bcache.txt?h=bcache bcache user documentation].<br />
<br />
{{Warning|Only invalidate the cache if one of the two options above did not work.}}<br />
<br />
=== /sys/fs/bcache/ doesn't exist ===<br />
<br />
The kernel you booted is not bcache enabled.</div>Justin8https://wiki.archlinux.org/index.php?title=Btrfs&diff=287622Btrfs2013-12-11T09:01:54Z<p>Justin8: /* RAID features */</p>
<hr />
<div>[[Category:File systems]]<br />
[[ja:Btrfs]]<br />
[[zh-CN:Btrfs]]<br />
{{Related articles start}}<br />
{{Related|File Systems}}<br />
{{Related|Btrfs - Tips and tricks}}<br />
{{Related|Mkinitcpio-btrfs}}<br />
{{Related articles end}}<br />
From [[Wikipedia:Btrfs]]<br />
<br />
:''Btrfs (B-tree file system, variously pronounced: "Butter F S", "Better F S", "B-tree F S", or simply "Bee Tee Arr Eff Ess") is a GPL-licensed experimental copy-on-write file system for Linux. Development began at Oracle Corporation in 2007.''<br />
<br />
From [https://btrfs.wiki.kernel.org/index.php/Main_Page Btrfs Wiki]<br />
<br />
:''Btrfs is a new copy on write (CoW) filesystem for Linux aimed at implementing advanced features while focusing on fault tolerance, repair and easy administration. Jointly developed at Oracle, Red Hat, Fujitsu, Intel, SUSE, STRATO and many others, Btrfs is licensed under the GPL and open for contribution from anyone.''<br />
{{Warning|Btrfs is still considered experimental. See the Btrfs Wiki's [https://btrfs.wiki.kernel.org/index.php/Main_Page#Stability_status Stability status,] [https://btrfs.wiki.kernel.org/index.php/FAQ#Is_btrfs_stable.3F Is Btrfs stable,] and [https://btrfs.wiki.kernel.org/index.php/Getting_started Getting started] for more detailed information.}}<br />
== Installation ==<br />
<br />
Btrfs is included in the default kernel and its tools ({{Pkg|btrfs-progs}}) are available in the official repositories. [[GRUB]], [[mkinitcpio]], and [[Syslinux]] have support for Btrfs and require no additional configuration.<br />
<br />
=== Additional packages ===<br />
<br />
* {{Pkg|btrfs-progs}} includes ''btrfsck'', a tool that can fix errors on Btrfs filesystems.<br />
* {{AUR|mkinitcpio-btrfs}} enables roll-back abilities.<br />
* {{AUR|btrfs-progs-git}} for nightly<br />
<br />
{{Tip|See [https://btrfs.wiki.kernel.org/index.php/Getting_started Btrfs Wiki Getting Started] for suggestions regarding running Btrfs effectively.}}<br />
<br />
== General administration of Btrfs ==<br />
<br />
=== Creating a new file system ===<br />
<br />
A Btrfs file system can either be newly created or have one converted.<br />
<br />
To format a partition do:<br />
<br />
# mkfs.btrfs -L ''mylabel'' /dev/''partition''<br />
<br />
{{Note|1=As of [https://git.kernel.org/cgit/linux/kernel/git/mason/btrfs-progs.git/commit/?id=c652e4efb8e2dd76ef1627d8cd649c6af5905902 this] commit (November 2013), Btrfs default blocksize is 16KB.}}<br />
<br />
To use a larger blocksize for data/meta data, specify a value for the leafsize via the {{ic|-l}} switch as shown in this example using 16KB blocks:<br />
<br />
# mkfs.btrfs -L ''mylabel'' -l 16k /dev/''partition''<br />
<br />
Multiple devices can be entered to create a RAID. Supported RAID levels include RAID 0, RAID 1, RAID 10, RAID 5 and RAID 6. By default the metadata is mirrored and data is striped.<br />
<br />
# mkfs.btrfs [''options''] /dev/''part1'' /dev/''part2''<br />
<br />
=== Convert from Ext3/4 ===<br />
<br />
Boot from an install CD, then convert by doing:<br />
<br />
# btrfs-convert /dev/''partition''<br />
<br />
Mount the partion and test the conversion by checking the files. Be sure to change the {{ic|/etc/fstab}} to reflect the change ('''type''' to btrfs and '''fs_passno''' [the last field] to 0 as Btrfs does not do a file system check on boot). Also note that the UUID of the partition will have changed, so update fstab accordingly when using UUIDs. {{ic|chroot}} into the system and rebuild the GRUB menu list (see [[Install from Existing Linux]] and [[GRUB]] articles).<br />
<br />
To complete, delete the saved image, delete the sub-volume that image is on, then balance the drive to reclaim the space.<br />
<br />
# rm /ext2_saved/*<br />
# btrfs subvolume delete /ext2_saved<br />
<br />
=== Mount options ===<br />
<br />
See [https://btrfs.wiki.kernel.org/index.php/Mount_options Btrfs Wiki Mount options].<br />
<br />
In addition to configurations that can be made during or after file system creation, the various mount options for Btrfs can drastically change its performance characteristics.<br />
<br />
{{Warning|Specific mount options can disable safety features and increase the risk of complete file system corruption in the event of a power failure. (See link above.)}}<br />
<br />
As this is a file system that is in active development. Changes and regressions should be expected. See links in the "See also" section for some benchmarks.<br />
<br />
Users may need to add the {{ic|btrfs}} hook to [[mkinitcpio]] if adding a btrfs partition to {{ic|/etc/fstab}}.<br />
<br />
==== Examples ====<br />
{{Note|1=For questions regarding the use of {{ic|autodefrag}} on SSDs see [https://btrfs.wiki.kernel.org/index.php/Gotchas Btrfs Wiki Gotchas], [https://btrfs.wiki.kernel.org/index.php/Mount_options Btrfs Wiki Mount options], and [http://www.phoronix.com/scan.php?page=article&item=linux_btrfs_311&num=1 Phoronix 3.11 Benchmarking].}}<br />
<br />
* '''Linux 3.12'''<br />
** Btrfs on a SSD for system installation and an emphasis on maximizing performance.<br />
** {{bc|1=rw,noatime,compress=lzo,ssd,discard,space_cache,autodefrag,inode_cache}}<br />
** Btrfs on a HDD for archival purposes with an emphasis on maximizing space.<br />
** {{bc|1=rw,relatime,compress-force=zlib,autodefrag}}<br />
<br />
=== Displaying used/free space ===<br />
<br />
General linux userspace tools such as {{ic|/usr/bin/df}} will inaccurately report free space on a Btrfs partition since it does not take into account space allocated for and used by the metadata. It is recommended to use {{ic|/usr/bin/btrfs}} to query a btrfs partition. Below is an illustration of this effect, first querying using df, and then using btrfs fi df:<br />
<br />
{{hc|# df -h /|<br />
Filesystem Size Used Avail Use% Mounted on<br />
/dev/sda3 119G 3.0G 116G 3% /<br />
}}<br />
<br />
{{hc|# btrfs fi df /|2=<br />
Data: total=3.01GB, used=2.73GB<br />
System: total=4.00MB, used=16.00KB<br />
Metadata: total=1.01GB, used=181.83MB<br />
}}<br />
<br />
Notice that {{ic|df -h}} reports 3.0GB used but {{ic|btrfs fi df}} reports 2.73GB for the data. This is due to the way BTRFS allocates space into the pool. The true disk usage is the sum of all three 'used' values which is inferior to 3.0GB as reported by {{ic|df -h}}.<br />
<br />
Another useful command to show a less verbose readout of used space is {{ic|btrfs fi show}}:<br />
<br />
{{hc|# btrfs fi show /dev/sda3|<br />
failed to open /dev/sr0: No medium found<br />
Label: 'arch64' uuid: 02ad2ea2-be12-2233-8765-9e0a48e9303a<br />
Total devices 1 FS bytes used 2.91GB<br />
devid 1 size 118.95GB used 4.02GB path /dev/sda2<br />
<br />
Btrfs v0.20-rc1-358-g194aa4a-dirty<br />
}}<br />
<br />
== Limitations ==<br />
<br />
A few limitations should be known before trying.<br />
<br />
=== Encryption ===<br />
<br />
Btrfs has no built-in encryption support (this may come in future); users can encrypt the partition before running {{ic|mkfs.btrfs}}. See [[Dm-crypt with LUKS]]. <br />
<br />
Existing Btrfs file system, can use something like [[EncFS]] or [[TrueCrypt]], though perhaps without some of Btrfs' features.<br />
<br />
=== Swap file ===<br />
<br />
Btrfs does not support swap files. This is due to swap files requiring a function that Btrfs doesn't have for possibility of corruptions.<sup>[https://btrfs.wiki.kernel.org/index.php/FAQ#Does_btrfs_support_swap_files.3F link]</sup> A swap file can be mounted on a loop device with poorer performance but will not be able to hibernate. A ''systemd'' service file is available {{AUR|systemd-loop-swapfile}}.<br />
<br />
=== GRUB ===<br />
==== Partition offset ====<br />
{{Accuracy|This may no longer be needed. Need to verify with a direct installation of Btrfs without a partition table.}}<br />
<br />
[[GRUB]] can boot Btrfs partitions however the module may be larger than other [[File Systems]] and the {{ic|core.img}} file made by ''grub-install'' may not fit in the first 63 KB of the drive between the MBR and the first partition. Up-to-date partitions tools such as fdisk and gdisk avoid this issue by offsetting the first partition by 2 MB.<br />
<br />
==== Missing root ==== <br />
Users experiencing the following: {{ic|1=error no such device: root}} when booting from a RAID style setup then edit /usr/share/grub/grub-mkconfig_lib and remove both quotes from the line {{ic|1=echo " search --no-floppy --fs-uuid --set=root ${hints} ${fs_uuid}"}}. Regenerate the config for grub and the system should boot without an error.<br />
<br />
== Features ==<br />
<br />
Various features are available and can be adjusted.<br />
<br />
=== Copy-On-Write (CoW) ===<br />
<br />
The resolution at which data are written to the filesystem is dictated by Btrfs itself and by system-wide settings. Btrfs defaults to a 30 seconds checkpoint interval in which new data are committed to the filesystem. As of Btrfs v0.20-rc1-253-g7854c8b users cannot tweak this without recompiling a [http://www.mail-archive.com/linux-btrfs@vger.kernel.org/msg26090.html patched version of fs/btrfs/disk-io.c]. On 01-Aug-2013, David Sterba submitted [http://www.mail-archive.com/linux-btrfs@vger.kernel.org/msg26116.html this patch] to make this a formal mount time tuneable.<br />
<br />
System-wide settings also affect commit intervals. They include the files under {{ic|/proc/sys/vm/*}} and are out-of-scope of this wiki article. The kernel documentation for them resides in {{ic|Documentation/sysctl/vm.txt}}.<br />
<br />
CoW comes with some advantages, but can negatively affect performance with large files that have small random writes. It is recommended to disable CoW for database files and virtual machine images. <br />
<br />
One can disable CoW for the entire block device by mounting it with {{ic|nodatacow}} option. However, this will disable CoW for the entire file system.<br />
<br />
To disable CoW for single files/directories do:<br />
<br />
# chattr +C ''/dir/file''<br />
<br />
Note, from chattr man page: For btrfs, the 'C' flag should be set on new or empty files. If it is set on a file which already has data blocks, it is undefined when the blocks assigned to the file will be fully stable. If the 'C' flag is set on a directory, it will have no effect on the directory, but new files created in that directory will have the No_COW attribute.<br />
<br />
Likewise, to save space by forcing CoW when copying files use:<br />
<br />
# cp --reflink source dest <br />
<br />
As dest file is changed, only those blocks that are changed from source will be written to the disk. One might consider aliasing aliasing ''cp'' to {{ic|1=cp --reflink=auto}}.<br />
<br />
=== Multi-device filesystem and RAID feature ===<br />
<br />
==== Multi-device filesystem ====<br />
<br />
When creating a ''btrfs'' filesystem, one can pass many partitions or disk devices to ''mkfs.btrfs''. The filesystem will be created across these devices. One can '''"'''pool'''"''' this way, multiple partitions or devices to get a big ''btrfs'' filesystem.<br />
<br />
One can also add or remove device from an existing btrfs filesystem (caution is mandatory).<br />
<br />
A multi-device ''btrfs'' filesystem (also called a btrfs volume) is not recognized until <br />
# btrfs device scan<br />
has been run. This is the purpose of the ''btrfs'' mkinitcpio hook.<br />
<br />
==== RAID features ====<br />
<br />
When creating multi-device filesystem, one can also specify to use RAID0, RAID1, RAID10, RAID5 or RAID6 across the devices comprising the filesystem. RAID levels can be applied independently to data and meta data. By default, meta data is duplicated on single volumes or RAID1 on multi-disk sets.<br />
<br />
btrfs works in block-pairs for raid0, raid1, and raid10. This means:<br />
<br />
raid0 - block-pair stripped across 2 devices<br />
<br />
raid1 - block-pair written to 2 devices<br />
<br />
For 2 disk sets, this matches raid levels as defined in md-raid (mdadm). For 3+ disk-sets, the result is entirely different than md-raid. <br />
<br />
For example:<br />
<br />
3 1TB disks in an md based raid1 yields a {{ic|/dev/md0}} with 1 TB free space and the ability to safely loose 2 disks without losing data.<br />
3 1TB disks in a Btrfs volume with data=raid1 will allow the storage of approximately 1.5 TB of data before reporting full. Only 1 disk can safely be lost without losing data.<br />
<br />
Btrfs uses a round-robin scheme to decide how block-pairs are spread among disks. As of Linux 3.0, a quasi-round-robin scheme is used which prefers larger disks when distributing block pairs. This allows raid0 and raid1 to take advantage of most (and sometimes all) space in a disk set made of multiple disks. For example, a set consisting of a 1TB disk and 2 500GB disks with data=raid1 will place a copy of every block on the 1TB disk and alternate (round-robin) placing blocks on each of the 500GB disks. Full space utilization will be made. A set made from a 1TB disk, a 750GB disk, and a 500GB disk will work the same, but the filesystem will report full with 250GB unusable on the 750GB disk. To always take advantage of the full space (even in the last example), use data=single. (data=single is akin to JBOD defined by some raid controllers) See [https://btrfs.wiki.kernel.org/index.php/FAQ#How_much_space_do_I_get_with_unequal_devices_in_RAID-1_mode.3F the BTRFS FAQ] for more info.<br />
<br />
=== Sub-volumes ===<br />
<br />
See the following links for more details:<br />
*[https://btrfs.wiki.kernel.org/index.php/SysadminGuide#Subvolumes Btrfs Wiki SysadminGuide#Subvolumes]<br />
* [https://btrfs.wiki.kernel.org/index.php/Getting_started#Basic_Filesystem_Commands Btrfs Wiki Getting started#Basic Filessystem Commands]<br />
* [https://btrfs.wiki.kernel.org/index.php/Trees Btrfs Wiki Trees] <br />
<br />
==== Creating sub-volumes ====<br />
<br />
To create a sub-volume:<br />
<br />
# btrfs subvolume create ''subvolume-name''<br />
<br />
==== Listing sub-volumes ====<br />
<br />
To see a list of current sub-volumes:<br />
<br />
# btrfs subvolume list -p .<br />
<br />
==== Setting a default sub-volume ====<br />
<br />
{{Warning|1=''Changing the default subvolume with btrfs subvolume default will make the top level of the filesystem inaccessible, except by use of the subvolid=0 mount option.'' From, [https://btrfs.wiki.kernel.org/index.php/SysadminGuide Btrfs Wiki Sysadmin Guide].}}<br />
<br />
The default sub-volume is mounted if no {{ic|1=subvol=}} mount option is provided.<br />
<br />
{{bc|# btrfs subvolume set-default ''subvolume-id'' /.}}<br />
<br />
'''Example:'''<br />
<br />
{{hc|# btrfs subvolume list .|<br />
ID 258 gen 9512 top level 5 path root_subvolume<br />
ID 259 gen 9512 top level 258 path home<br />
ID 260 gen 9512 top level 258 path var<br />
ID 261 gen 9512 top level 258 path usr<br />
}}<br />
<br />
# btrfs subvolume set-default 258 .<br />
<br />
'''Reset:'''<br />
<br />
{{bc|# btrfs subvolume set-default 0 .}}<br />
<br />
==== Tips and tricks ====<br />
<br />
* For increased flexibility, install the system into a dedicated sub-volume, and, in the kernel boot parameters, use: {{ic|1=rootflags=subvol=''subvolume-name''}}.<br />
* If using a subvolume for the root partition, it is advisable to add '''crc32c''' (or '''crc32c-intel''' for Intel machines) to the modules array in {{ic|/etc/mkinitcpio.conf}}<br />
<br />
{{Note|Child sub-volumes are automatically mounted by Btrfs when the parent sub-volume is mounted.}}<br />
<br />
=== Snapshots ===<br />
<br />
To create a snapshot:<br />
<br />
# btrfs subvolume snapshot ''source'' [''dest''/]''name''<br />
<br />
Snapshots are not recursive. Every sub-volume inside sub-volume will be an empty directory inside the snapshot.<br />
<br />
=== Defragmentation ===<br />
<br />
Btrfs supports online defragmentation. To defragment the metadata of the root folder do:<br />
<br />
# btrfs filesystem defragment /<br />
<br />
This ''will not'' defragment the entire system. For more information read [https://btrfs.wiki.kernel.org/index.php/Problem_FAQ#Defragmenting_a_directory_doesn.27t_work this page] on the btrfs wiki.<br />
<br />
To defragment the entire system verbosely do:<br />
<br />
# find / -xdev -type f -print -exec btrfs filesystem defrag '{}' \;<br />
<br />
=== Compression ===<br />
<br />
Btrfs supports transparent compression, which means every file on the partition is automatically compressed. This does not only reduce the size of those files, but also [http://www.phoronix.com/scan.php?page=article&item=btrfs_compress_2635&num=1 improves performance], in particular if using the [http://www.phoronix.com/scan.php?page=article&item=btrfs_lzo_2638&num=1 lzo algorithm], in some specific use cases (e.g. single tread with heavy file IO), while obviously harming performance on other cases (e.g. multithreaded and/or cpu intensive tasks with large file IO). Compression is enabled using the {{ic|1=compress=gzip}} or {{ic|1=compress=lzo}} mount options. Only files created or modified after the mount option is added will be compressed, so to fully benefit from compression it should be enabled during installation. <br />
<br />
However, it can quite easily be applied to a subvolume using the defragment -czlib (or whichever algorithm chosen) command (the same command above could be used, by adding the -czlib and such, to recursively apply). Also keep in mind that for future files to be compressed, a simple {{ic|chattr +c}} should be applied to some directories, so as to automatically compress new files as they come.<br />
<br />
After [[Beginners%27_Guide#Prepare_the_storage_drive|preparing the storage drive]], simply switch to another terminal ({{ic|Ctrl+Alt+number}}), and run the following command:<br />
<br />
# mount -o remount,compress=lzo /dev/sd''XY'' /mnt/target<br />
<br />
After the installation is finished, add {{ic|1=compress=lzo}} to the mount options of the root filesystem in {{ic|/etc/[[fstab]]}}.<br />
<br />
=== Checkpoint Interval ===<br />
<br />
Starting with Linux 3.12, users are able to change the checkpoint interval from the default 30 s to any value by appending the '''commit''' mount flag in {{ic|/etc/fstab}} for the btrfs partition.<br />
<br />
LABEL=arch64 / btrfs defaults,noatime,ssd,compress=lzo,commit=120 0 0<br />
<br />
=== Partitioning ===<br />
<br />
Btrfs can occupy an entire data storage device and replace the [[MBR]] or [[GPT]] partitioning schemes. One can use [[Btrfs#Sub-volumes|subvolumes]] to simulate partitions. There are some limitations to this approach in single disk setups:<br />
<br />
* Cannot use different [[File_Systems|file systems]] for different [[fstab|mount points]].<br />
* Cannot use [[Swap|swap area]] as Btrfs does not support [[Swap#Swap_file|swap files]] and there is no place to create [[Swap#Swap_partition|swap partition]].<br />
* Cannot use [[UEFI]] to boot.<br />
<br />
To overwrite the existing partition table with Btrfs, run the following command:<br />
# mkfs.btrfs /dev/sd''X''<br />
Do not specify {{ic|/dev/sda''X''}} or it will format an existing partition instead of replacing the entire partitioning scheme.<br />
<br />
Install the [[Bootloaders|boot loader]] in a like fashion to installing it for a data storage device with a [[MBR|Master Boot Record]]. For example:<br />
# grub-install --recheck /dev/sd''X''<br />
for [[Grub#Install_to_440-byte_MBR_boot_code_region|GRUB]].<br />
<br />
{{Warning|Using the {{ic|btrfs filesystem set-default}} command to change the default sub-volume from anything other than the top level (ID 0) may break Grub. See [[Btrfs#Setting a default sub-volume]] to reset.}}<br />
<br />
=== Scrub ===<br />
<br />
See [https://btrfs.wiki.kernel.org/index.php/Glossary Btrfs Wiki Glossary].<br />
<br />
# btrfs scrub start /<br />
# btrfs scrub status /<br />
<br />
=== Balance ===<br />
<br />
See [https://btrfs.wiki.kernel.org/index.php/Glossary Btrfs Wiki Glossary].<br />
<br />
# btrfs balance /<br />
<br />
== See also ==<br />
<br />
* '''Official site'''<br />
** [https://btrfs.wiki.kernel.org/ Btrfs Wiki]<br />
** [https://btrfs.wiki.kernel.org/index.php/Glossary Btrfs Wiki Glossary]<br />
* '''Official FAQs'''<br />
** [https://btrfs.wiki.kernel.org/index.php/FAQ Btrfs Wiki FAQ]<br />
** [https://btrfs.wiki.kernel.org/index.php/Problem_FAQ Btrfs Wiki Problem FAQ]<br />
* '''Btrfs pull requests'''<br />
** [http://lkml.indiana.edu/hypermail/linux/kernel/1311.1/03526.html 3.13]<br />
** [http://lkml.indiana.edu/hypermail/linux/kernel/1309.1/02981.html 3.12]<br />
** [http://lkml.indiana.edu/hypermail/linux/kernel/1305.1/01064.html 3.11]<br />
* '''Performance related'''<br />
** [http://superuser.com/questions/432188/should-i-put-my-multi-device-btrfs-filesystem-on-disk-partitions-or-raw-devices Btrfs on raw disks?]<br />
** [http://comments.gmane.org/gmane.comp.file-systems.btrfs/19440 Varying leafsize and nodesize in Btrfs]<br />
** [http://comments.gmane.org/gmane.comp.file-systems.btrfs/15646 Btrfs support for efficient SSD operation (data blocks alignment)]<br />
** [https://btrfs.wiki.kernel.org/index.php/FAQ#Is_Btrfs_optimized_for_SSD.3F Is Btrfs optimized for SSDs?]<br />
** '''Phoronix mount option benchmarking'''<br />
*** [http://www.phoronix.com/scan.php?page=article&item=linux_btrfs_311&num=1 Linux 3.11]<br />
*** [http://www.phoronix.com/scan.php?page=news_item&px=MTM0OTU Linux 3.9]<br />
*** [http://www.phoronix.com/scan.php?page=article&item=btrfs_linux37_mounts&num=1 Linux 3.7]<br />
*** [http://www.phoronix.com/scan.php?page=article&item=linux_btrfs_options&num=1 Linux 3.2]<br />
** [http://blog.erdemagaoglu.com/post/4605524309/lzo-vs-snappy-vs-lzf-vs-zlib-a-comparison-of Lzo vs. zLib]<br />
* '''Miscellaneous'''<br />
** [http://www.funtoo.org/wiki/BTRFS_Fun Funtoo Wiki Btrfs Fun]<br />
** [http://www.phoronix.com/scan.php?page=news_item&px=MTA0ODU Avi Miller presenting Btrfs] at SCALE 10x, January 2012.<br />
** [http://www.phoronix.com/scan.php?page=news_item&px=MTA4Mzc Summary of Chris Mason's talk] from LFCS 2012<br />
** [http://git.kernel.org/?p&#61;linux/kernel/git/torvalds/linux-2.6.git;a&#61;commit;h&#61;35054394c4b3cecd52577c2662c84da1f3e73525 Btrfs: stop providing a bmap operation to avoid swapfile corruptions] 2009-01-21</div>Justin8https://wiki.archlinux.org/index.php?title=Btrfs&diff=287619Btrfs2013-12-11T08:52:22Z<p>Justin8: /* Creating a new file system */</p>
<hr />
<div>[[Category:File systems]]<br />
[[ja:Btrfs]]<br />
[[zh-CN:Btrfs]]<br />
{{Related articles start}}<br />
{{Related|File Systems}}<br />
{{Related|Btrfs - Tips and tricks}}<br />
{{Related|Mkinitcpio-btrfs}}<br />
{{Related articles end}}<br />
From [[Wikipedia:Btrfs]]<br />
<br />
:''Btrfs (B-tree file system, variously pronounced: "Butter F S", "Better F S", "B-tree F S", or simply "Bee Tee Arr Eff Ess") is a GPL-licensed experimental copy-on-write file system for Linux. Development began at Oracle Corporation in 2007.''<br />
<br />
From [https://btrfs.wiki.kernel.org/index.php/Main_Page Btrfs Wiki]<br />
<br />
:''Btrfs is a new copy on write (CoW) filesystem for Linux aimed at implementing advanced features while focusing on fault tolerance, repair and easy administration. Jointly developed at Oracle, Red Hat, Fujitsu, Intel, SUSE, STRATO and many others, Btrfs is licensed under the GPL and open for contribution from anyone.''<br />
{{Warning|Btrfs is still considered experimental. See the Btrfs Wiki's [https://btrfs.wiki.kernel.org/index.php/Main_Page#Stability_status Stability status,] [https://btrfs.wiki.kernel.org/index.php/FAQ#Is_btrfs_stable.3F Is Btrfs stable,] and [https://btrfs.wiki.kernel.org/index.php/Getting_started Getting started] for more detailed information.}}<br />
== Installation ==<br />
<br />
Btrfs is included in the default kernel and its tools ({{Pkg|btrfs-progs}}) are available in the official repositories. [[GRUB]], [[mkinitcpio]], and [[Syslinux]] have support for Btrfs and require no additional configuration.<br />
<br />
=== Additional packages ===<br />
<br />
* {{Pkg|btrfs-progs}} includes ''btrfsck'', a tool that can fix errors on Btrfs filesystems.<br />
* {{AUR|mkinitcpio-btrfs}} enables roll-back abilities.<br />
* {{AUR|btrfs-progs-git}} for nightly<br />
<br />
{{Tip|See [https://btrfs.wiki.kernel.org/index.php/Getting_started Btrfs Wiki Getting Started] for suggestions regarding running Btrfs effectively.}}<br />
<br />
== General administration of Btrfs ==<br />
<br />
=== Creating a new file system ===<br />
<br />
A Btrfs file system can either be newly created or have one converted.<br />
<br />
To format a partition do:<br />
<br />
# mkfs.btrfs -L ''mylabel'' /dev/''partition''<br />
<br />
{{Note|1=As of [https://git.kernel.org/cgit/linux/kernel/git/mason/btrfs-progs.git/commit/?id=c652e4efb8e2dd76ef1627d8cd649c6af5905902 this] commit (November 2013), Btrfs default blocksize is 16KB.}}<br />
<br />
To use a larger blocksize for data/meta data, specify a value for the leafsize via the {{ic|-l}} switch as shown in this example using 16KB blocks:<br />
<br />
# mkfs.btrfs -L ''mylabel'' -l 16k /dev/''partition''<br />
<br />
Multiple devices can be entered to create a RAID. Supported RAID levels include RAID 0, RAID 1, RAID 10, RAID 5 and RAID 6. By default the metadata is mirrored and data is striped.<br />
<br />
# mkfs.btrfs [''options''] /dev/''part1'' /dev/''part2''<br />
<br />
=== Convert from Ext3/4 ===<br />
<br />
Boot from an install CD, then convert by doing:<br />
<br />
# btrfs-convert /dev/''partition''<br />
<br />
Mount the partion and test the conversion by checking the files. Be sure to change the {{ic|/etc/fstab}} to reflect the change ('''type''' to btrfs and '''fs_passno''' [the last field] to 0 as Btrfs does not do a file system check on boot). Also note that the UUID of the partition will have changed, so update fstab accordingly when using UUIDs. {{ic|chroot}} into the system and rebuild the GRUB menu list (see [[Install from Existing Linux]] and [[GRUB]] articles).<br />
<br />
To complete, delete the saved image, delete the sub-volume that image is on, then balance the drive to reclaim the space.<br />
<br />
# rm /ext2_saved/*<br />
# btrfs subvolume delete /ext2_saved<br />
<br />
=== Mount options ===<br />
<br />
See [https://btrfs.wiki.kernel.org/index.php/Mount_options Btrfs Wiki Mount options].<br />
<br />
In addition to configurations that can be made during or after file system creation, the various mount options for Btrfs can drastically change its performance characteristics.<br />
<br />
{{Warning|Specific mount options can disable safety features and increase the risk of complete file system corruption in the event of a power failure. (See link above.)}}<br />
<br />
As this is a file system that is in active development. Changes and regressions should be expected. See links in the "See also" section for some benchmarks.<br />
<br />
Users may need to add the {{ic|btrfs}} hook to [[mkinitcpio]] if adding a btrfs partition to {{ic|/etc/fstab}}.<br />
<br />
==== Examples ====<br />
{{Note|1=For questions regarding the use of {{ic|autodefrag}} on SSDs see [https://btrfs.wiki.kernel.org/index.php/Gotchas Btrfs Wiki Gotchas], [https://btrfs.wiki.kernel.org/index.php/Mount_options Btrfs Wiki Mount options], and [http://www.phoronix.com/scan.php?page=article&item=linux_btrfs_311&num=1 Phoronix 3.11 Benchmarking].}}<br />
<br />
* '''Linux 3.12'''<br />
** Btrfs on a SSD for system installation and an emphasis on maximizing performance.<br />
** {{bc|1=rw,noatime,compress=lzo,ssd,discard,space_cache,autodefrag,inode_cache}}<br />
** Btrfs on a HDD for archival purposes with an emphasis on maximizing space.<br />
** {{bc|1=rw,relatime,compress-force=zlib,autodefrag}}<br />
<br />
=== Displaying used/free space ===<br />
<br />
General linux userspace tools such as {{ic|/usr/bin/df}} will inaccurately report free space on a Btrfs partition since it does not take into account space allocated for and used by the metadata. It is recommended to use {{ic|/usr/bin/btrfs}} to query a btrfs partition. Below is an illustration of this effect, first querying using df, and then using btrfs fi df:<br />
<br />
{{hc|# df -h /|<br />
Filesystem Size Used Avail Use% Mounted on<br />
/dev/sda3 119G 3.0G 116G 3% /<br />
}}<br />
<br />
{{hc|# btrfs fi df /|2=<br />
Data: total=3.01GB, used=2.73GB<br />
System: total=4.00MB, used=16.00KB<br />
Metadata: total=1.01GB, used=181.83MB<br />
}}<br />
<br />
Notice that {{ic|df -h}} reports 3.0GB used but {{ic|btrfs fi df}} reports 2.73GB for the data. This is due to the way BTRFS allocates space into the pool. The true disk usage is the sum of all three 'used' values which is inferior to 3.0GB as reported by {{ic|df -h}}.<br />
<br />
Another useful command to show a less verbose readout of used space is {{ic|btrfs fi show}}:<br />
<br />
{{hc|# btrfs fi show /dev/sda3|<br />
failed to open /dev/sr0: No medium found<br />
Label: 'arch64' uuid: 02ad2ea2-be12-2233-8765-9e0a48e9303a<br />
Total devices 1 FS bytes used 2.91GB<br />
devid 1 size 118.95GB used 4.02GB path /dev/sda2<br />
<br />
Btrfs v0.20-rc1-358-g194aa4a-dirty<br />
}}<br />
<br />
== Limitations ==<br />
<br />
A few limitations should be known before trying.<br />
<br />
=== Encryption ===<br />
<br />
Btrfs has no built-in encryption support (this may come in future); users can encrypt the partition before running {{ic|mkfs.btrfs}}. See [[Dm-crypt with LUKS]]. <br />
<br />
Existing Btrfs file system, can use something like [[EncFS]] or [[TrueCrypt]], though perhaps without some of Btrfs' features.<br />
<br />
=== Swap file ===<br />
<br />
Btrfs does not support swap files. This is due to swap files requiring a function that Btrfs doesn't have for possibility of corruptions.<sup>[https://btrfs.wiki.kernel.org/index.php/FAQ#Does_btrfs_support_swap_files.3F link]</sup> A swap file can be mounted on a loop device with poorer performance but will not be able to hibernate. A ''systemd'' service file is available {{AUR|systemd-loop-swapfile}}.<br />
<br />
=== GRUB ===<br />
==== Partition offset ====<br />
{{Accuracy|This may no longer be needed. Need to verify with a direct installation of Btrfs without a partition table.}}<br />
<br />
[[GRUB]] can boot Btrfs partitions however the module may be larger than other [[File Systems]] and the {{ic|core.img}} file made by ''grub-install'' may not fit in the first 63 KB of the drive between the MBR and the first partition. Up-to-date partitions tools such as fdisk and gdisk avoid this issue by offsetting the first partition by 2 MB.<br />
<br />
==== Missing root ==== <br />
Users experiencing the following: {{ic|1=error no such device: root}} when booting from a RAID style setup then edit /usr/share/grub/grub-mkconfig_lib and remove both quotes from the line {{ic|1=echo " search --no-floppy --fs-uuid --set=root ${hints} ${fs_uuid}"}}. Regenerate the config for grub and the system should boot without an error.<br />
<br />
== Features ==<br />
<br />
Various features are available and can be adjusted.<br />
<br />
=== Copy-On-Write (CoW) ===<br />
<br />
The resolution at which data are written to the filesystem is dictated by Btrfs itself and by system-wide settings. Btrfs defaults to a 30 seconds checkpoint interval in which new data are committed to the filesystem. As of Btrfs v0.20-rc1-253-g7854c8b users cannot tweak this without recompiling a [http://www.mail-archive.com/linux-btrfs@vger.kernel.org/msg26090.html patched version of fs/btrfs/disk-io.c]. On 01-Aug-2013, David Sterba submitted [http://www.mail-archive.com/linux-btrfs@vger.kernel.org/msg26116.html this patch] to make this a formal mount time tuneable.<br />
<br />
System-wide settings also affect commit intervals. They include the files under {{ic|/proc/sys/vm/*}} and are out-of-scope of this wiki article. The kernel documentation for them resides in {{ic|Documentation/sysctl/vm.txt}}.<br />
<br />
CoW comes with some advantages, but can negatively affect performance with large files that have small random writes. It is recommended to disable CoW for database files and virtual machine images. <br />
<br />
One can disable CoW for the entire block device by mounting it with {{ic|nodatacow}} option. However, this will disable CoW for the entire file system.<br />
<br />
To disable CoW for single files/directories do:<br />
<br />
# chattr +C ''/dir/file''<br />
<br />
Note, from chattr man page: For btrfs, the 'C' flag should be set on new or empty files. If it is set on a file which already has data blocks, it is undefined when the blocks assigned to the file will be fully stable. If the 'C' flag is set on a directory, it will have no effect on the directory, but new files created in that directory will have the No_COW attribute.<br />
<br />
Likewise, to save space by forcing CoW when copying files use:<br />
<br />
# cp --reflink source dest <br />
<br />
As dest file is changed, only those blocks that are changed from source will be written to the disk. One might consider aliasing aliasing ''cp'' to {{ic|1=cp --reflink=auto}}.<br />
<br />
=== Multi-device filesystem and RAID feature ===<br />
<br />
==== Multi-device filesystem ====<br />
<br />
When creating a ''btrfs'' filesystem, one can pass many partitions or disk devices to ''mkfs.btrfs''. The filesystem will be created across these devices. One can '''"'''pool'''"''' this way, multiple partitions or devices to get a big ''btrfs'' filesystem.<br />
<br />
One can also add or remove device from an existing btrfs filesystem (caution is mandatory).<br />
<br />
A multi-device ''btrfs'' filesystem (also called a btrfs volume) is not recognized until <br />
# btrfs device scan<br />
has been run. This is the purpose of the ''btrfs'' mkinitcpio hook.<br />
<br />
==== RAID features ====<br />
<br />
When creating multi-device filesystem, one can also specify to use RAID0, RAID1 or RAID10 across the devices comprising the filesystem. RAID levels can be applied independently to data and meta data. By default, meta data is duplicated on single volumes or RAID1 on multi-disk sets.<br />
<br />
btrfs works in block-pairs for raid0, raid1, and raid10. This means:<br />
<br />
raid0 - block-pair stripped across 2 devices<br />
<br />
raid1 - block-pair written to 2 devices<br />
<br />
For 2 disk sets, this matches raid levels as defined in md-raid (mdadm). For 3+ disk-sets, the result is entirely different than md-raid. <br />
<br />
For example:<br />
<br />
3 1TB disks in an md based raid1 yields a {{ic|/dev/md0}} with 1 TB free space and the ability to safely loose 2 disks without losing data.<br />
3 1TB disks in a Btrfs volume with data=raid1 will allow the storage of approximately 1.5 TB of data before reporting full. Only 1 disk can safely be lost without losing data.<br />
<br />
Btrfs uses a round-robin scheme to decide how block-pairs are spread among disks. As of Linux 3.0, a quasi-round-robin scheme is used which prefers larger disks when distributing block pairs. This allows raid0 and raid1 to take advantage of most (and sometimes all) space in a disk set made of multiple disks. For example, a set consisting of a 1TB disk and 2 500GB disks with data=raid1 will place a copy of every block on the 1TB disk and alternate (round-robin) placing blocks on each of the 500GB disks. Full space utilization will be made. A set made from a 1TB disk, a 750GB disk, and a 500GB disk will work the same, but the filesystem will report full with 250GB unusable on the 750GB disk. To always take advantage of the full space (even in the last example), use data=single. (data=single is akin to JBOD defined by some raid controllers) See [https://btrfs.wiki.kernel.org/index.php/FAQ#How_much_space_do_I_get_with_unequal_devices_in_RAID-1_mode.3F the BTRFS FAQ] for more info.<br />
<br />
=== Sub-volumes ===<br />
<br />
See the following links for more details:<br />
*[https://btrfs.wiki.kernel.org/index.php/SysadminGuide#Subvolumes Btrfs Wiki SysadminGuide#Subvolumes]<br />
* [https://btrfs.wiki.kernel.org/index.php/Getting_started#Basic_Filesystem_Commands Btrfs Wiki Getting started#Basic Filessystem Commands]<br />
* [https://btrfs.wiki.kernel.org/index.php/Trees Btrfs Wiki Trees] <br />
<br />
==== Creating sub-volumes ====<br />
<br />
To create a sub-volume:<br />
<br />
# btrfs subvolume create ''subvolume-name''<br />
<br />
==== Listing sub-volumes ====<br />
<br />
To see a list of current sub-volumes:<br />
<br />
# btrfs subvolume list -p .<br />
<br />
==== Setting a default sub-volume ====<br />
<br />
{{Warning|1=''Changing the default subvolume with btrfs subvolume default will make the top level of the filesystem inaccessible, except by use of the subvolid=0 mount option.'' From, [https://btrfs.wiki.kernel.org/index.php/SysadminGuide Btrfs Wiki Sysadmin Guide].}}<br />
<br />
The default sub-volume is mounted if no {{ic|1=subvol=}} mount option is provided.<br />
<br />
{{bc|# btrfs subvolume set-default ''subvolume-id'' /.}}<br />
<br />
'''Example:'''<br />
<br />
{{hc|# btrfs subvolume list .|<br />
ID 258 gen 9512 top level 5 path root_subvolume<br />
ID 259 gen 9512 top level 258 path home<br />
ID 260 gen 9512 top level 258 path var<br />
ID 261 gen 9512 top level 258 path usr<br />
}}<br />
<br />
# btrfs subvolume set-default 258 .<br />
<br />
'''Reset:'''<br />
<br />
{{bc|# btrfs subvolume set-default 0 .}}<br />
<br />
==== Tips and tricks ====<br />
<br />
* For increased flexibility, install the system into a dedicated sub-volume, and, in the kernel boot parameters, use: {{ic|1=rootflags=subvol=''subvolume-name''}}.<br />
* If using a subvolume for the root partition, it is advisable to add '''crc32c''' (or '''crc32c-intel''' for Intel machines) to the modules array in {{ic|/etc/mkinitcpio.conf}}<br />
<br />
{{Note|Child sub-volumes are automatically mounted by Btrfs when the parent sub-volume is mounted.}}<br />
<br />
=== Snapshots ===<br />
<br />
To create a snapshot:<br />
<br />
# btrfs subvolume snapshot ''source'' [''dest''/]''name''<br />
<br />
Snapshots are not recursive. Every sub-volume inside sub-volume will be an empty directory inside the snapshot.<br />
<br />
=== Defragmentation ===<br />
<br />
Btrfs supports online defragmentation. To defragment the metadata of the root folder do:<br />
<br />
# btrfs filesystem defragment /<br />
<br />
This ''will not'' defragment the entire system. For more information read [https://btrfs.wiki.kernel.org/index.php/Problem_FAQ#Defragmenting_a_directory_doesn.27t_work this page] on the btrfs wiki.<br />
<br />
To defragment the entire system verbosely do:<br />
<br />
# find / -xdev -type f -print -exec btrfs filesystem defrag '{}' \;<br />
<br />
=== Compression ===<br />
<br />
Btrfs supports transparent compression, which means every file on the partition is automatically compressed. This does not only reduce the size of those files, but also [http://www.phoronix.com/scan.php?page=article&item=btrfs_compress_2635&num=1 improves performance], in particular if using the [http://www.phoronix.com/scan.php?page=article&item=btrfs_lzo_2638&num=1 lzo algorithm], in some specific use cases (e.g. single tread with heavy file IO), while obviously harming performance on other cases (e.g. multithreaded and/or cpu intensive tasks with large file IO). Compression is enabled using the {{ic|1=compress=gzip}} or {{ic|1=compress=lzo}} mount options. Only files created or modified after the mount option is added will be compressed, so to fully benefit from compression it should be enabled during installation. <br />
<br />
However, it can quite easily be applied to a subvolume using the defragment -czlib (or whichever algorithm chosen) command (the same command above could be used, by adding the -czlib and such, to recursively apply). Also keep in mind that for future files to be compressed, a simple {{ic|chattr +c}} should be applied to some directories, so as to automatically compress new files as they come.<br />
<br />
After [[Beginners%27_Guide#Prepare_the_storage_drive|preparing the storage drive]], simply switch to another terminal ({{ic|Ctrl+Alt+number}}), and run the following command:<br />
<br />
# mount -o remount,compress=lzo /dev/sd''XY'' /mnt/target<br />
<br />
After the installation is finished, add {{ic|1=compress=lzo}} to the mount options of the root filesystem in {{ic|/etc/[[fstab]]}}.<br />
<br />
=== Checkpoint Interval ===<br />
<br />
Starting with Linux 3.12, users are able to change the checkpoint interval from the default 30 s to any value by appending the '''commit''' mount flag in {{ic|/etc/fstab}} for the btrfs partition.<br />
<br />
LABEL=arch64 / btrfs defaults,noatime,ssd,compress=lzo,commit=120 0 0<br />
<br />
=== Partitioning ===<br />
<br />
Btrfs can occupy an entire data storage device and replace the [[MBR]] or [[GPT]] partitioning schemes. One can use [[Btrfs#Sub-volumes|subvolumes]] to simulate partitions. There are some limitations to this approach in single disk setups:<br />
<br />
* Cannot use different [[File_Systems|file systems]] for different [[fstab|mount points]].<br />
* Cannot use [[Swap|swap area]] as Btrfs does not support [[Swap#Swap_file|swap files]] and there is no place to create [[Swap#Swap_partition|swap partition]].<br />
* Cannot use [[UEFI]] to boot.<br />
<br />
To overwrite the existing partition table with Btrfs, run the following command:<br />
# mkfs.btrfs /dev/sd''X''<br />
Do not specify {{ic|/dev/sda''X''}} or it will format an existing partition instead of replacing the entire partitioning scheme.<br />
<br />
Install the [[Bootloaders|boot loader]] in a like fashion to installing it for a data storage device with a [[MBR|Master Boot Record]]. For example:<br />
# grub-install --recheck /dev/sd''X''<br />
for [[Grub#Install_to_440-byte_MBR_boot_code_region|GRUB]].<br />
<br />
{{Warning|Using the {{ic|btrfs filesystem set-default}} command to change the default sub-volume from anything other than the top level (ID 0) may break Grub. See [[Btrfs#Setting a default sub-volume]] to reset.}}<br />
<br />
=== Scrub ===<br />
<br />
See [https://btrfs.wiki.kernel.org/index.php/Glossary Btrfs Wiki Glossary].<br />
<br />
# btrfs scrub start /<br />
# btrfs scrub status /<br />
<br />
=== Balance ===<br />
<br />
See [https://btrfs.wiki.kernel.org/index.php/Glossary Btrfs Wiki Glossary].<br />
<br />
# btrfs balance /<br />
<br />
== See also ==<br />
<br />
* '''Official site'''<br />
** [https://btrfs.wiki.kernel.org/ Btrfs Wiki]<br />
** [https://btrfs.wiki.kernel.org/index.php/Glossary Btrfs Wiki Glossary]<br />
* '''Official FAQs'''<br />
** [https://btrfs.wiki.kernel.org/index.php/FAQ Btrfs Wiki FAQ]<br />
** [https://btrfs.wiki.kernel.org/index.php/Problem_FAQ Btrfs Wiki Problem FAQ]<br />
* '''Btrfs pull requests'''<br />
** [http://lkml.indiana.edu/hypermail/linux/kernel/1311.1/03526.html 3.13]<br />
** [http://lkml.indiana.edu/hypermail/linux/kernel/1309.1/02981.html 3.12]<br />
** [http://lkml.indiana.edu/hypermail/linux/kernel/1305.1/01064.html 3.11]<br />
* '''Performance related'''<br />
** [http://superuser.com/questions/432188/should-i-put-my-multi-device-btrfs-filesystem-on-disk-partitions-or-raw-devices Btrfs on raw disks?]<br />
** [http://comments.gmane.org/gmane.comp.file-systems.btrfs/19440 Varying leafsize and nodesize in Btrfs]<br />
** [http://comments.gmane.org/gmane.comp.file-systems.btrfs/15646 Btrfs support for efficient SSD operation (data blocks alignment)]<br />
** [https://btrfs.wiki.kernel.org/index.php/FAQ#Is_Btrfs_optimized_for_SSD.3F Is Btrfs optimized for SSDs?]<br />
** '''Phoronix mount option benchmarking'''<br />
*** [http://www.phoronix.com/scan.php?page=article&item=linux_btrfs_311&num=1 Linux 3.11]<br />
*** [http://www.phoronix.com/scan.php?page=news_item&px=MTM0OTU Linux 3.9]<br />
*** [http://www.phoronix.com/scan.php?page=article&item=btrfs_linux37_mounts&num=1 Linux 3.7]<br />
*** [http://www.phoronix.com/scan.php?page=article&item=linux_btrfs_options&num=1 Linux 3.2]<br />
** [http://blog.erdemagaoglu.com/post/4605524309/lzo-vs-snappy-vs-lzf-vs-zlib-a-comparison-of Lzo vs. zLib]<br />
* '''Miscellaneous'''<br />
** [http://www.funtoo.org/wiki/BTRFS_Fun Funtoo Wiki Btrfs Fun]<br />
** [http://www.phoronix.com/scan.php?page=news_item&px=MTA0ODU Avi Miller presenting Btrfs] at SCALE 10x, January 2012.<br />
** [http://www.phoronix.com/scan.php?page=news_item&px=MTA4Mzc Summary of Chris Mason's talk] from LFCS 2012<br />
** [http://git.kernel.org/?p&#61;linux/kernel/git/torvalds/linux-2.6.git;a&#61;commit;h&#61;35054394c4b3cecd52577c2662c84da1f3e73525 Btrfs: stop providing a bmap operation to avoid swapfile corruptions] 2009-01-21</div>Justin8https://wiki.archlinux.org/index.php?title=Btrfs&diff=287618Btrfs2013-12-11T08:51:13Z<p>Justin8: /* Creating a new file system */</p>
<hr />
<div>[[Category:File systems]]<br />
[[ja:Btrfs]]<br />
[[zh-CN:Btrfs]]<br />
{{Related articles start}}<br />
{{Related|File Systems}}<br />
{{Related|Btrfs - Tips and tricks}}<br />
{{Related|Mkinitcpio-btrfs}}<br />
{{Related articles end}}<br />
From [[Wikipedia:Btrfs]]<br />
<br />
:''Btrfs (B-tree file system, variously pronounced: "Butter F S", "Better F S", "B-tree F S", or simply "Bee Tee Arr Eff Ess") is a GPL-licensed experimental copy-on-write file system for Linux. Development began at Oracle Corporation in 2007.''<br />
<br />
From [https://btrfs.wiki.kernel.org/index.php/Main_Page Btrfs Wiki]<br />
<br />
:''Btrfs is a new copy on write (CoW) filesystem for Linux aimed at implementing advanced features while focusing on fault tolerance, repair and easy administration. Jointly developed at Oracle, Red Hat, Fujitsu, Intel, SUSE, STRATO and many others, Btrfs is licensed under the GPL and open for contribution from anyone.''<br />
{{Warning|Btrfs is still considered experimental. See the Btrfs Wiki's [https://btrfs.wiki.kernel.org/index.php/Main_Page#Stability_status Stability status,] [https://btrfs.wiki.kernel.org/index.php/FAQ#Is_btrfs_stable.3F Is Btrfs stable,] and [https://btrfs.wiki.kernel.org/index.php/Getting_started Getting started] for more detailed information.}}<br />
== Installation ==<br />
<br />
Btrfs is included in the default kernel and its tools ({{Pkg|btrfs-progs}}) are available in the official repositories. [[GRUB]], [[mkinitcpio]], and [[Syslinux]] have support for Btrfs and require no additional configuration.<br />
<br />
=== Additional packages ===<br />
<br />
* {{Pkg|btrfs-progs}} includes ''btrfsck'', a tool that can fix errors on Btrfs filesystems.<br />
* {{AUR|mkinitcpio-btrfs}} enables roll-back abilities.<br />
* {{AUR|btrfs-progs-git}} for nightly<br />
<br />
{{Tip|See [https://btrfs.wiki.kernel.org/index.php/Getting_started Btrfs Wiki Getting Started] for suggestions regarding running Btrfs effectively.}}<br />
<br />
== General administration of Btrfs ==<br />
<br />
=== Creating a new file system ===<br />
<br />
A Btrfs file system can either be newly created or have one converted.<br />
<br />
To format a partition do:<br />
<br />
# mkfs.btrfs -L ''mylabel'' /dev/''partition''<br />
<br />
{{Note|1=As of [https://git.kernel.org/cgit/linux/kernel/git/mason/btrfs-progs.git/commit/?id=c652e4efb8e2dd76ef1627d8cd649c6af5905902 this] commit (November 2013), Btrfs default blocksize is 16KB.}}<br />
<br />
To use a larger blocksize for data/meta data, specify a value for the leafsize via the {{ic|-l}} switch as shown in this example using 16KB blocks:<br />
<br />
# mkfs.btrfs -L ''mylabel'' -l 16k /dev/''partition''<br />
<br />
Multiple devices can be entered to create a RAID. Supported RAID levels include RAID 0, RAID 1 and RAID 10. By default the metadata is mirrored and data is striped.<br />
<br />
# mkfs.btrfs [''options''] /dev/''part1'' /dev/''part2''<br />
<br />
=== Convert from Ext3/4 ===<br />
<br />
Boot from an install CD, then convert by doing:<br />
<br />
# btrfs-convert /dev/''partition''<br />
<br />
Mount the partion and test the conversion by checking the files. Be sure to change the {{ic|/etc/fstab}} to reflect the change ('''type''' to btrfs and '''fs_passno''' [the last field] to 0 as Btrfs does not do a file system check on boot). Also note that the UUID of the partition will have changed, so update fstab accordingly when using UUIDs. {{ic|chroot}} into the system and rebuild the GRUB menu list (see [[Install from Existing Linux]] and [[GRUB]] articles).<br />
<br />
To complete, delete the saved image, delete the sub-volume that image is on, then balance the drive to reclaim the space.<br />
<br />
# rm /ext2_saved/*<br />
# btrfs subvolume delete /ext2_saved<br />
<br />
=== Mount options ===<br />
<br />
See [https://btrfs.wiki.kernel.org/index.php/Mount_options Btrfs Wiki Mount options].<br />
<br />
In addition to configurations that can be made during or after file system creation, the various mount options for Btrfs can drastically change its performance characteristics.<br />
<br />
{{Warning|Specific mount options can disable safety features and increase the risk of complete file system corruption in the event of a power failure. (See link above.)}}<br />
<br />
As this is a file system that is in active development. Changes and regressions should be expected. See links in the "See also" section for some benchmarks.<br />
<br />
Users may need to add the {{ic|btrfs}} hook to [[mkinitcpio]] if adding a btrfs partition to {{ic|/etc/fstab}}.<br />
<br />
==== Examples ====<br />
{{Note|1=For questions regarding the use of {{ic|autodefrag}} on SSDs see [https://btrfs.wiki.kernel.org/index.php/Gotchas Btrfs Wiki Gotchas], [https://btrfs.wiki.kernel.org/index.php/Mount_options Btrfs Wiki Mount options], and [http://www.phoronix.com/scan.php?page=article&item=linux_btrfs_311&num=1 Phoronix 3.11 Benchmarking].}}<br />
<br />
* '''Linux 3.12'''<br />
** Btrfs on a SSD for system installation and an emphasis on maximizing performance.<br />
** {{bc|1=rw,noatime,compress=lzo,ssd,discard,space_cache,autodefrag,inode_cache}}<br />
** Btrfs on a HDD for archival purposes with an emphasis on maximizing space.<br />
** {{bc|1=rw,relatime,compress-force=zlib,autodefrag}}<br />
<br />
=== Displaying used/free space ===<br />
<br />
General linux userspace tools such as {{ic|/usr/bin/df}} will inaccurately report free space on a Btrfs partition since it does not take into account space allocated for and used by the metadata. It is recommended to use {{ic|/usr/bin/btrfs}} to query a btrfs partition. Below is an illustration of this effect, first querying using df, and then using btrfs fi df:<br />
<br />
{{hc|# df -h /|<br />
Filesystem Size Used Avail Use% Mounted on<br />
/dev/sda3 119G 3.0G 116G 3% /<br />
}}<br />
<br />
{{hc|# btrfs fi df /|2=<br />
Data: total=3.01GB, used=2.73GB<br />
System: total=4.00MB, used=16.00KB<br />
Metadata: total=1.01GB, used=181.83MB<br />
}}<br />
<br />
Notice that {{ic|df -h}} reports 3.0GB used but {{ic|btrfs fi df}} reports 2.73GB for the data. This is due to the way BTRFS allocates space into the pool. The true disk usage is the sum of all three 'used' values which is inferior to 3.0GB as reported by {{ic|df -h}}.<br />
<br />
Another useful command to show a less verbose readout of used space is {{ic|btrfs fi show}}:<br />
<br />
{{hc|# btrfs fi show /dev/sda3|<br />
failed to open /dev/sr0: No medium found<br />
Label: 'arch64' uuid: 02ad2ea2-be12-2233-8765-9e0a48e9303a<br />
Total devices 1 FS bytes used 2.91GB<br />
devid 1 size 118.95GB used 4.02GB path /dev/sda2<br />
<br />
Btrfs v0.20-rc1-358-g194aa4a-dirty<br />
}}<br />
<br />
== Limitations ==<br />
<br />
A few limitations should be known before trying.<br />
<br />
=== Encryption ===<br />
<br />
Btrfs has no built-in encryption support (this may come in future); users can encrypt the partition before running {{ic|mkfs.btrfs}}. See [[Dm-crypt with LUKS]]. <br />
<br />
Existing Btrfs file system, can use something like [[EncFS]] or [[TrueCrypt]], though perhaps without some of Btrfs' features.<br />
<br />
=== Swap file ===<br />
<br />
Btrfs does not support swap files. This is due to swap files requiring a function that Btrfs doesn't have for possibility of corruptions.<sup>[https://btrfs.wiki.kernel.org/index.php/FAQ#Does_btrfs_support_swap_files.3F link]</sup> A swap file can be mounted on a loop device with poorer performance but will not be able to hibernate. A ''systemd'' service file is available {{AUR|systemd-loop-swapfile}}.<br />
<br />
=== GRUB ===<br />
==== Partition offset ====<br />
{{Accuracy|This may no longer be needed. Need to verify with a direct installation of Btrfs without a partition table.}}<br />
<br />
[[GRUB]] can boot Btrfs partitions however the module may be larger than other [[File Systems]] and the {{ic|core.img}} file made by ''grub-install'' may not fit in the first 63 KB of the drive between the MBR and the first partition. Up-to-date partitions tools such as fdisk and gdisk avoid this issue by offsetting the first partition by 2 MB.<br />
<br />
==== Missing root ==== <br />
Users experiencing the following: {{ic|1=error no such device: root}} when booting from a RAID style setup then edit /usr/share/grub/grub-mkconfig_lib and remove both quotes from the line {{ic|1=echo " search --no-floppy --fs-uuid --set=root ${hints} ${fs_uuid}"}}. Regenerate the config for grub and the system should boot without an error.<br />
<br />
== Features ==<br />
<br />
Various features are available and can be adjusted.<br />
<br />
=== Copy-On-Write (CoW) ===<br />
<br />
The resolution at which data are written to the filesystem is dictated by Btrfs itself and by system-wide settings. Btrfs defaults to a 30 seconds checkpoint interval in which new data are committed to the filesystem. As of Btrfs v0.20-rc1-253-g7854c8b users cannot tweak this without recompiling a [http://www.mail-archive.com/linux-btrfs@vger.kernel.org/msg26090.html patched version of fs/btrfs/disk-io.c]. On 01-Aug-2013, David Sterba submitted [http://www.mail-archive.com/linux-btrfs@vger.kernel.org/msg26116.html this patch] to make this a formal mount time tuneable.<br />
<br />
System-wide settings also affect commit intervals. They include the files under {{ic|/proc/sys/vm/*}} and are out-of-scope of this wiki article. The kernel documentation for them resides in {{ic|Documentation/sysctl/vm.txt}}.<br />
<br />
CoW comes with some advantages, but can negatively affect performance with large files that have small random writes. It is recommended to disable CoW for database files and virtual machine images. <br />
<br />
One can disable CoW for the entire block device by mounting it with {{ic|nodatacow}} option. However, this will disable CoW for the entire file system.<br />
<br />
To disable CoW for single files/directories do:<br />
<br />
# chattr +C ''/dir/file''<br />
<br />
Note, from chattr man page: For btrfs, the 'C' flag should be set on new or empty files. If it is set on a file which already has data blocks, it is undefined when the blocks assigned to the file will be fully stable. If the 'C' flag is set on a directory, it will have no effect on the directory, but new files created in that directory will have the No_COW attribute.<br />
<br />
Likewise, to save space by forcing CoW when copying files use:<br />
<br />
# cp --reflink source dest <br />
<br />
As dest file is changed, only those blocks that are changed from source will be written to the disk. One might consider aliasing aliasing ''cp'' to {{ic|1=cp --reflink=auto}}.<br />
<br />
=== Multi-device filesystem and RAID feature ===<br />
<br />
==== Multi-device filesystem ====<br />
<br />
When creating a ''btrfs'' filesystem, one can pass many partitions or disk devices to ''mkfs.btrfs''. The filesystem will be created across these devices. One can '''"'''pool'''"''' this way, multiple partitions or devices to get a big ''btrfs'' filesystem.<br />
<br />
One can also add or remove device from an existing btrfs filesystem (caution is mandatory).<br />
<br />
A multi-device ''btrfs'' filesystem (also called a btrfs volume) is not recognized until <br />
# btrfs device scan<br />
has been run. This is the purpose of the ''btrfs'' mkinitcpio hook.<br />
<br />
==== RAID features ====<br />
<br />
When creating multi-device filesystem, one can also specify to use RAID0, RAID1 or RAID10 across the devices comprising the filesystem. RAID levels can be applied independently to data and meta data. By default, meta data is duplicated on single volumes or RAID1 on multi-disk sets.<br />
<br />
btrfs works in block-pairs for raid0, raid1, and raid10. This means:<br />
<br />
raid0 - block-pair stripped across 2 devices<br />
<br />
raid1 - block-pair written to 2 devices<br />
<br />
For 2 disk sets, this matches raid levels as defined in md-raid (mdadm). For 3+ disk-sets, the result is entirely different than md-raid. <br />
<br />
For example:<br />
<br />
3 1TB disks in an md based raid1 yields a {{ic|/dev/md0}} with 1 TB free space and the ability to safely loose 2 disks without losing data.<br />
3 1TB disks in a Btrfs volume with data=raid1 will allow the storage of approximately 1.5 TB of data before reporting full. Only 1 disk can safely be lost without losing data.<br />
<br />
Btrfs uses a round-robin scheme to decide how block-pairs are spread among disks. As of Linux 3.0, a quasi-round-robin scheme is used which prefers larger disks when distributing block pairs. This allows raid0 and raid1 to take advantage of most (and sometimes all) space in a disk set made of multiple disks. For example, a set consisting of a 1TB disk and 2 500GB disks with data=raid1 will place a copy of every block on the 1TB disk and alternate (round-robin) placing blocks on each of the 500GB disks. Full space utilization will be made. A set made from a 1TB disk, a 750GB disk, and a 500GB disk will work the same, but the filesystem will report full with 250GB unusable on the 750GB disk. To always take advantage of the full space (even in the last example), use data=single. (data=single is akin to JBOD defined by some raid controllers) See [https://btrfs.wiki.kernel.org/index.php/FAQ#How_much_space_do_I_get_with_unequal_devices_in_RAID-1_mode.3F the BTRFS FAQ] for more info.<br />
<br />
=== Sub-volumes ===<br />
<br />
See the following links for more details:<br />
*[https://btrfs.wiki.kernel.org/index.php/SysadminGuide#Subvolumes Btrfs Wiki SysadminGuide#Subvolumes]<br />
* [https://btrfs.wiki.kernel.org/index.php/Getting_started#Basic_Filesystem_Commands Btrfs Wiki Getting started#Basic Filessystem Commands]<br />
* [https://btrfs.wiki.kernel.org/index.php/Trees Btrfs Wiki Trees] <br />
<br />
==== Creating sub-volumes ====<br />
<br />
To create a sub-volume:<br />
<br />
# btrfs subvolume create ''subvolume-name''<br />
<br />
==== Listing sub-volumes ====<br />
<br />
To see a list of current sub-volumes:<br />
<br />
# btrfs subvolume list -p .<br />
<br />
==== Setting a default sub-volume ====<br />
<br />
{{Warning|1=''Changing the default subvolume with btrfs subvolume default will make the top level of the filesystem inaccessible, except by use of the subvolid=0 mount option.'' From, [https://btrfs.wiki.kernel.org/index.php/SysadminGuide Btrfs Wiki Sysadmin Guide].}}<br />
<br />
The default sub-volume is mounted if no {{ic|1=subvol=}} mount option is provided.<br />
<br />
{{bc|# btrfs subvolume set-default ''subvolume-id'' /.}}<br />
<br />
'''Example:'''<br />
<br />
{{hc|# btrfs subvolume list .|<br />
ID 258 gen 9512 top level 5 path root_subvolume<br />
ID 259 gen 9512 top level 258 path home<br />
ID 260 gen 9512 top level 258 path var<br />
ID 261 gen 9512 top level 258 path usr<br />
}}<br />
<br />
# btrfs subvolume set-default 258 .<br />
<br />
'''Reset:'''<br />
<br />
{{bc|# btrfs subvolume set-default 0 .}}<br />
<br />
==== Tips and tricks ====<br />
<br />
* For increased flexibility, install the system into a dedicated sub-volume, and, in the kernel boot parameters, use: {{ic|1=rootflags=subvol=''subvolume-name''}}.<br />
* If using a subvolume for the root partition, it is advisable to add '''crc32c''' (or '''crc32c-intel''' for Intel machines) to the modules array in {{ic|/etc/mkinitcpio.conf}}<br />
<br />
{{Note|Child sub-volumes are automatically mounted by Btrfs when the parent sub-volume is mounted.}}<br />
<br />
=== Snapshots ===<br />
<br />
To create a snapshot:<br />
<br />
# btrfs subvolume snapshot ''source'' [''dest''/]''name''<br />
<br />
Snapshots are not recursive. Every sub-volume inside sub-volume will be an empty directory inside the snapshot.<br />
<br />
=== Defragmentation ===<br />
<br />
Btrfs supports online defragmentation. To defragment the metadata of the root folder do:<br />
<br />
# btrfs filesystem defragment /<br />
<br />
This ''will not'' defragment the entire system. For more information read [https://btrfs.wiki.kernel.org/index.php/Problem_FAQ#Defragmenting_a_directory_doesn.27t_work this page] on the btrfs wiki.<br />
<br />
To defragment the entire system verbosely do:<br />
<br />
# find / -xdev -type f -print -exec btrfs filesystem defrag '{}' \;<br />
<br />
=== Compression ===<br />
<br />
Btrfs supports transparent compression, which means every file on the partition is automatically compressed. This does not only reduce the size of those files, but also [http://www.phoronix.com/scan.php?page=article&item=btrfs_compress_2635&num=1 improves performance], in particular if using the [http://www.phoronix.com/scan.php?page=article&item=btrfs_lzo_2638&num=1 lzo algorithm], in some specific use cases (e.g. single tread with heavy file IO), while obviously harming performance on other cases (e.g. multithreaded and/or cpu intensive tasks with large file IO). Compression is enabled using the {{ic|1=compress=gzip}} or {{ic|1=compress=lzo}} mount options. Only files created or modified after the mount option is added will be compressed, so to fully benefit from compression it should be enabled during installation. <br />
<br />
However, it can quite easily be applied to a subvolume using the defragment -czlib (or whichever algorithm chosen) command (the same command above could be used, by adding the -czlib and such, to recursively apply). Also keep in mind that for future files to be compressed, a simple {{ic|chattr +c}} should be applied to some directories, so as to automatically compress new files as they come.<br />
<br />
After [[Beginners%27_Guide#Prepare_the_storage_drive|preparing the storage drive]], simply switch to another terminal ({{ic|Ctrl+Alt+number}}), and run the following command:<br />
<br />
# mount -o remount,compress=lzo /dev/sd''XY'' /mnt/target<br />
<br />
After the installation is finished, add {{ic|1=compress=lzo}} to the mount options of the root filesystem in {{ic|/etc/[[fstab]]}}.<br />
<br />
=== Checkpoint Interval ===<br />
<br />
Starting with Linux 3.12, users are able to change the checkpoint interval from the default 30 s to any value by appending the '''commit''' mount flag in {{ic|/etc/fstab}} for the btrfs partition.<br />
<br />
LABEL=arch64 / btrfs defaults,noatime,ssd,compress=lzo,commit=120 0 0<br />
<br />
=== Partitioning ===<br />
<br />
Btrfs can occupy an entire data storage device and replace the [[MBR]] or [[GPT]] partitioning schemes. One can use [[Btrfs#Sub-volumes|subvolumes]] to simulate partitions. There are some limitations to this approach in single disk setups:<br />
<br />
* Cannot use different [[File_Systems|file systems]] for different [[fstab|mount points]].<br />
* Cannot use [[Swap|swap area]] as Btrfs does not support [[Swap#Swap_file|swap files]] and there is no place to create [[Swap#Swap_partition|swap partition]].<br />
* Cannot use [[UEFI]] to boot.<br />
<br />
To overwrite the existing partition table with Btrfs, run the following command:<br />
# mkfs.btrfs /dev/sd''X''<br />
Do not specify {{ic|/dev/sda''X''}} or it will format an existing partition instead of replacing the entire partitioning scheme.<br />
<br />
Install the [[Bootloaders|boot loader]] in a like fashion to installing it for a data storage device with a [[MBR|Master Boot Record]]. For example:<br />
# grub-install --recheck /dev/sd''X''<br />
for [[Grub#Install_to_440-byte_MBR_boot_code_region|GRUB]].<br />
<br />
{{Warning|Using the {{ic|btrfs filesystem set-default}} command to change the default sub-volume from anything other than the top level (ID 0) may break Grub. See [[Btrfs#Setting a default sub-volume]] to reset.}}<br />
<br />
=== Scrub ===<br />
<br />
See [https://btrfs.wiki.kernel.org/index.php/Glossary Btrfs Wiki Glossary].<br />
<br />
# btrfs scrub start /<br />
# btrfs scrub status /<br />
<br />
=== Balance ===<br />
<br />
See [https://btrfs.wiki.kernel.org/index.php/Glossary Btrfs Wiki Glossary].<br />
<br />
# btrfs balance /<br />
<br />
== See also ==<br />
<br />
* '''Official site'''<br />
** [https://btrfs.wiki.kernel.org/ Btrfs Wiki]<br />
** [https://btrfs.wiki.kernel.org/index.php/Glossary Btrfs Wiki Glossary]<br />
* '''Official FAQs'''<br />
** [https://btrfs.wiki.kernel.org/index.php/FAQ Btrfs Wiki FAQ]<br />
** [https://btrfs.wiki.kernel.org/index.php/Problem_FAQ Btrfs Wiki Problem FAQ]<br />
* '''Btrfs pull requests'''<br />
** [http://lkml.indiana.edu/hypermail/linux/kernel/1311.1/03526.html 3.13]<br />
** [http://lkml.indiana.edu/hypermail/linux/kernel/1309.1/02981.html 3.12]<br />
** [http://lkml.indiana.edu/hypermail/linux/kernel/1305.1/01064.html 3.11]<br />
* '''Performance related'''<br />
** [http://superuser.com/questions/432188/should-i-put-my-multi-device-btrfs-filesystem-on-disk-partitions-or-raw-devices Btrfs on raw disks?]<br />
** [http://comments.gmane.org/gmane.comp.file-systems.btrfs/19440 Varying leafsize and nodesize in Btrfs]<br />
** [http://comments.gmane.org/gmane.comp.file-systems.btrfs/15646 Btrfs support for efficient SSD operation (data blocks alignment)]<br />
** [https://btrfs.wiki.kernel.org/index.php/FAQ#Is_Btrfs_optimized_for_SSD.3F Is Btrfs optimized for SSDs?]<br />
** '''Phoronix mount option benchmarking'''<br />
*** [http://www.phoronix.com/scan.php?page=article&item=linux_btrfs_311&num=1 Linux 3.11]<br />
*** [http://www.phoronix.com/scan.php?page=news_item&px=MTM0OTU Linux 3.9]<br />
*** [http://www.phoronix.com/scan.php?page=article&item=btrfs_linux37_mounts&num=1 Linux 3.7]<br />
*** [http://www.phoronix.com/scan.php?page=article&item=linux_btrfs_options&num=1 Linux 3.2]<br />
** [http://blog.erdemagaoglu.com/post/4605524309/lzo-vs-snappy-vs-lzf-vs-zlib-a-comparison-of Lzo vs. zLib]<br />
* '''Miscellaneous'''<br />
** [http://www.funtoo.org/wiki/BTRFS_Fun Funtoo Wiki Btrfs Fun]<br />
** [http://www.phoronix.com/scan.php?page=news_item&px=MTA0ODU Avi Miller presenting Btrfs] at SCALE 10x, January 2012.<br />
** [http://www.phoronix.com/scan.php?page=news_item&px=MTA4Mzc Summary of Chris Mason's talk] from LFCS 2012<br />
** [http://git.kernel.org/?p&#61;linux/kernel/git/torvalds/linux-2.6.git;a&#61;commit;h&#61;35054394c4b3cecd52577c2662c84da1f3e73525 Btrfs: stop providing a bmap operation to avoid swapfile corruptions] 2009-01-21</div>Justin8https://wiki.archlinux.org/index.php?title=Talk:Bcache&diff=283301Talk:Bcache2013-11-17T06:25:22Z<p>Justin8: </p>
<hr />
<div>==Example installation==<br />
Here is mine Bcache installation. Maybe you can use some idea's. :)<br />
<br />
=== Prepare the storage drive ===<br />
<br />
====Bcache-tools installation====<br />
fakeroot and binutils are only needed. but it will fail in a error if you only install them.<br />
wget is from a failed archbang instal :(<br />
<br />
pacman -Syy<br />
pacman -S base-devel fakeroot binutils wget git<br />
<br />
cd ~/<br />
wget https://aur.archlinux.org/packages/bc/bcache-tools-git/bcache-tools-git.tar.gz<br />
tar -xzvf bcache-tools-git.tar.gz<br />
cd bcache-tools-git<br />
makepkg -si --asroot<br />
<br />
==== Making the partitions ====<br />
For /boot i use the slower HDD because i had troubles that mine SSD loaded the kernel while mine HDD was still spinning up :P<br />
<br />
Starting with the faster SSD drive<br />
<br />
fdisk /dev/sda<br />
<br />
o<br />
y<br />
n<br />
Partition type: p<br />
Partition-number: 1<br />
First-sector: [enter]<br />
Last-sector: +90G<br />
<br />
n<br />
Partition type: p<br />
Partition-number: 3<br />
First-sector: [enter]<br />
Last-sector: [enter]<br />
<br />
write and exit<br />
<br />
Now the slower 2 TB Harddrive<br />
<br />
fdisk /dev/sdb<br />
<br />
o<br />
y<br />
n<br />
Partition type: p<br />
Partition-number: 1<br />
First-sector: [enter]<br />
Last-sector: +1G<br />
a<br />
<br />
n<br />
Partition type: p<br />
Partition-number: 2<br />
First-sector: [enter]<br />
Last-sector: +16G [enter]<br />
t<br />
2<br />
82<br />
<br />
n<br />
Partition type: p<br />
Partition-number: 3<br />
First-sector: [enter]<br />
Last-sector: [enter]<br />
<br />
write and exit<br />
<br />
==== Making Bcache Device ====<br />
The --wipe-bcache is to remove a error :)<br />
and the dd commands is or cleaning out old partition data.<br />
<br />
make-bcache --wipe-bcache -B /dev/sdb3 -C /dev/sda2<br />
dd if=/dev/zero count=1 bs=1024 seek=1 of=/dev/sda2<br />
dd if=/dev/zero count=1 bs=1024 seek=1 of=/dev/sdb3<br />
<br />
If next commands give a error: invalid argument<br />
Then the device is already registered.<br />
<br />
echo /dev/sda2 > /sys/fs/bcache/register<br />
echo /dev/sdb3 > /sys/fs/bcache/register<br />
<br />
==== Formating the drives ====<br />
<br />
<br />
mkfs.ext4 -E discard /dev/sda1<br />
mkfs.ext4 -E discard /dev/sdb1<br />
mkfs.ext4 /dev/bcache0<br />
mkswap /dev/sdb2<br />
swapon /dev/sdb2<br />
<br />
==== Mount the partitions ====<br />
<br />
cd ~/<br />
mkdir -p /mnt/install<br />
mount /dev/sda1 /mnt/install<br />
mkdir -p /mnt/install/{boot,home,srv,data,var,mnt/.hdd}<br />
mount /dev/sdb1 /mnt/install/boot<br />
mount /dev/bcache0 /mnt/install/mnt/.hdd<br />
mkdir -p /mnt/install/mnt/.hdd/{home,srv,data,var}<br />
mount -o bind /mnt/install/mnt/.hdd/home /mnt/install/home<br />
mount -o bind /mnt/install/mnt/.hdd/srv /mnt/install/srv<br />
mount -o bind /mnt/install/mnt/.hdd/data /mnt/install/data<br />
mount -o bind /mnt/install/mnt/.hdd/var /mnt/install/var<br />
<br />
==== Bcache Check After mounting ====<br />
Reason for /boot on the HDD is because of boot up failure because of HDD spin-up.<br />
<br />
lsblk<br />
<br />
NAME MAJ:MIN RM SIZE RO TYPE MOUNTPOINT<br />
sda 8:0 0 167.7G 0 disk <br />
|-sda1 8:1 0 90G 0 part /mnt/install<br />
`-sda2 8:2 0 77.7G 0 part <br />
`-bcache0 253:0 0 1.8T 0 disk /mnt/install/mnt/.hdd<br />
sdb 8:16 0 1.8T 0 disk <br />
|-sdb1 8:17 0 1G 0 part /mnt/install/boot<br />
|-sdb2 8:18 0 16G 0 part [SWAP]<br />
`-sdb3 8:19 0 1.8T 0 part <br />
`-bcache0 253:0 0 1.8T 0 disk /mnt/install/mnt/.hdd<br />
<br />
==== Generate an fstab ====<br />
<br />
Generate an fstab file with the following command. UUIDs will be used because they have certain advantages.<br />
<br />
genfstab -U -p /mnt/install >> /mnt/install/etc/fstab<br />
sed -i '/dev\/sda/ s/rw,relatime,data=ordered/defaults,noatime,discard/g' /mnt/install/etc/fstab<br />
sed -i '/mnt\/install/ {s/\/mnt\/install//g; s/rw,relatime,data=ordered/defaults/g; s/none/bind/g}' /mnt/install/etc/fstab<br />
more /mnt/install/etc/fstab<br />
<br />
=== Install and configure kernel and bootloader ===<br />
==== Adding Bcache module and hook ====<br />
<br />
Edit /etc/mkinitcpio.conf as needed and re-generate the initramfs image with:<br />
adding the "bcache" module<br />
adding the "bcache_udev" hook between block and filesystems<br />
<br />
sed -i '/^MODULES=/ s/"$/ bcache"/' /etc/mkinitcpio.conf<br />
sed -i '/^HOOKS=/ s/block filesystems/block bcache_udev filesystems/' /etc/mkinitcpio.conf<br />
more /etc/mkinitcpio.conf<br />
<br />
==== Registering and attaching Bcache ====<br />
This part is a pain in the A.. <br />
The problem is that Bcache wil give a error at startup if this is not done.<br />
<br />
ERROR: Unable to find root device 'UUID=b6b2d82b-f87e-44d5-bbc5-c51dd7aace15'.<br />
You are being dropped to a recovery shell<br />
Type 'exit' to try and continue booting<br />
<br />
First get the correct "SET UUID" with this ls command. <br />
And use the UUID like the example underneath.<br />
Also check for the use of the correct devices.<br />
echo: write error: Invalid argument <<< This error is good !!! means that your value was already stored.<br />
<br />
ls /sys/fs/bcache/<br />
2300f944-0eb5-4a9e-b052-8d5ea63cbb8f register register_quiet<br />
<br />
sudo echo /dev/sda2 > /sys/fs/bcache/register<br />
sudo echo /dev/sdb3 > /sys/fs/bcache/register<br />
sudo echo 2300f944-0eb5-4a9e-b052-8d5ea63cbb8f > /sys/block/sdb/sdb3/bcache/attach<br />
<br />
mv /usr/lib/initcpio/hooks/bcache ~/bcache.old<br />
cat >> /usr/lib/initcpio/hooks/bcache << EOF<br />
#!/usr/bin/ash<br />
run_hook() {<br />
echo /dev/sda2 > /sys/fs/bcache/register<br />
echo /dev/sdb3 > /sys/fs/bcache/register<br />
}<br />
EOF<br />
<br />
==== Installing the Kernel ==== <br />
mkinitcpio -p linux<br />
<br />
I hope some stuff is helpfull :P<br />
17:58, 11 November 2013 (UTC)<br />
:I'm curious about two things. First, though I forget what the '-E' switch signifies, it seems you're formatting the raw, underlayer partitions themselves. I don't understand why.<br />
<br />
:Also, the "pain in the a?" So udev is not assembling your array automatically as the devices are discovered? That is unexpected behavior, and I suspect it has to do with the formatting you performed and udev's superblock discovery. Check the very bottom of the bcache wiki on their site. At least try lsblk to verify "bcache" partition types are not registering as "ext4" before assembling the array.<br />
<br />
:Oh. And please sign talk pages.<br />
<br />
[[User:Mikeserv|Mikeserv]] ([[User talk:Mikeserv|talk]]) 03:35, 12 November 2013 (UTC)<br />
<br />
<br />
<br />
Well this is a experiment with Bcache with mine own Wiki page and 100x testing the install, i am not a linux expert just a newbie who was reading a lot of howto's. Just see it as a compilation of different web pages and some logical thinking. :)<br />
<br />
The -E is just a copy and past ... not sure if it was needed.I just thought it was needed to get the discard flag while formating.<br />
<br />
The pain in the A.... this was before Udev ... i was busy getting Bcache to work 3 months ago ... 75% of boot up failed when booting up cold. Later i found out that the SSD was too fast with booting up and the normal HDD was still spinning up.... After i splitted up the SSD for root directory and bcache and moved the /boot to the HDD since then i can bootup 100% of the time without any error... This was before the udev thingy (wich i dont understand and need to read up) :)<br />
<br />
So please understand that i am just a linux noob trying to contribute :)<br />
<br />
[[User:Emesix|Emesix]] ([[User talk:Emesix|talk]])[[User:Emesix|Emesix]] ([[User talk:Emesix|talk]]) 00:02, 13 November 2013 (UTC)<br />
<br />
:No, I get it, and I didn't mean to insinuate that you were doing anything wrong, exactly, just that, as designed, it could operate better.<br />
:Basically, and you should check the man-pages and the wiki page here to be sure just in case I'm not entirely correct (which does happen), udev is the program that runs during boot (and pretty much all of the rest of the time, too) to detect your hardware and load drivers as necessary. Udev is a relatively recent addition to the Linux boot process and is unique in that it discovers and handles hardware as it shows up, can handle multiple devices simultaneously with non-blocking parallell operations, and, as a result, is significantly faster than the methods it has replaced.<br />
:I was the one who added the udev rules and step 7b to this wiki, and I was the one who wrote the "Rogue Superblock" section of "Troubleshooting" at the bottom of bcache's wiki. I only gained the knowledge to write either after spending several frustrating days with an issue that appears very similar to your "pain in the a."<br />
:As step 7a shows, the prevailing Arch Linux method for reassembling a bcache array at boot used to look something like:<br />
: 1) Wait 5 secs. <br />
: 2) Check to see if udev has discovered and added {disk UUIDS}. If yes goto 3), no, repeat 1) and 2).<br />
: 3) echo {pain in the a stuff} /sys/bla/bla/bla<br />
:This does work, of course, but you're adding an unnecessary wait loop to your boot process and you're performing a task with brittle shell script that depends on specific UUIDs that could be performed flexibly and at discovery time.<br />
:That's what the udev rules do. Basically, the rules instruct udev to treat as part of your bcache array any partition it finds that reports itself as a "bcache" partition type. It builds the bcache array from disk partitions at every boot only as soon as those partitions are ready, which would of course resolve the race condition you mentioned and allow you to boot from SSD if you liked.<br />
:And here's where our experience might converge: I added a partition to my bcache array after previously formatting it as ext4. If you don't know about superblocks (or magic blocks or whatever they're called) then you're just like I was a few months ago. In short the filesystem has to have a way to report on itself, so it sets aside a small marker on disk that says, "Hey, OS, I'm a disk partition of type EXT4 and I start at block offset whatever and end at block offset whatever. Also, I enjoy long walks on the beach and prefer red wine to white. See you around."<br />
:The problem I experienced was that by default ext4 tended to begin at an earlier offset than bcache, and so even though I formatted over the previously ext4 partition with bcache as instructed, bcache never overwrote ext4's first superblock. When udev attempted to build my array, it would always miss my first partition because it would read the first superblock, identify the partition as ext4, then skip it. Echoing the add instruction to /sys/ after boot was my fix for a couple of days, but eventually I figured it all out and wrote that short bit on superblocks in bcache's wiki.<br />
:Maybe that helps?<br />
:EDIT: Just looked back at the wiki proper and somebody has performed some major changes recently. While it does appear cleaner, it no longer makes sense. There are references to non-existent steps throughout, and the actual why's that were included at least to some degree seem to have been occluded in favor of dubious, though possibly more efficient how's. Anyway, step 7b no longer exists, but it used to include the bcache_udev mkinitcpio hook I (very barely) adapted from mdadm_udev when trying to fix Arch's bcache boot problems. It's now included in the AUR package the wiki recommends I guess, despite the wiki also telling us we must "echo ... /sys/... at every bootup."<br />
[[User:Mikeserv|Mikeserv]] ([[User talk:Mikeserv|talk]]) 07:08, 14 November 2013 (UTC)<br />
<br />
Sorry about that, I missed a couple of references back to the sections I changed. Hopefully it makes more sense now. [[User:Mikeserv|Mikeserv's]] udev rule is crucial to having a working bcache device on boot, as [[User:Emesix|Emesix]] had discovered, it will often result in a failed boot. The echo in to /sys/bcache* step was missed when I updated it to mention the udev rule as the 'default' method.<br />
<br />
--[[User:Justin8|Justin8]] ([[User talk:Justin8|talk]]) 10:45, 14 November 2013 (UTC)<br />
:For clarity's sake, the udev rule was never mine. The used rule has been included in the AUR package's git pull from at least the time I first installed bcache some months ago. The package maintainer at that time opted not to use it, I guess, and instead installed the legacy-type shell-scripted for-loop mkinitcpio hook I outlined above, which was, to be fair, also the process recommended by every other source of information I was able to dig up at the time to include bcache's own wiki. Frustrated with having to wait for such things which seemed to me to defeat the whole purpose of buying and configuring an SSD boot device, I eventually stumbled upon the mdadm_udev hook in /use/share/initcpio and (only slightly) modified it to apply the 69-bcache-rules instead.<br />
:Justin, thanks for your recent change. NEVERMIND FOLLOWING: I'm curious why you specify "unless assembling after boot?" Why include anything to do with bcache in the initramfs at all if you're building the array after the boot sequence? MAKES PERFECT SENSE OF COURSE. GOT A 1 TRACK MIND SOMETIMES I GUESS. SORRY.<br />
[[User:Mikeserv|Mikeserv]] ([[User talk:Mikeserv|talk]]) 11:11, 14 November 2013 (UTC)<br />
<br />
:Emesix, just looked closer at your instructions, and you actually do the thing I had to do and outlined in the bcache wiki to overwrite the rogue superblock here:<br />
:{{ic|<nowiki>dd if=/dev/zero count=1 bs=1024 seek=1 of=/dev/sd{a2,b3}</nowiki>}}<br />
:And the ext4 formatting you do after that I earlier incorrectly assumed to be for the "raw, underlayer partitions" is revealed upon further inspection to be for non-bcache partitions and so isn't even relevant; I apologize for jumping to conclusions before reviewing it fully, the quick check I did a couple of days ago just struck me as really familiar.<br />
:The udev stuff above should still be relevant, and it might be worth noting that the particular {{ic|dd}} command above is by no means a cure-all. It should only be used as written if you have previously identified a superblock located at byte offset 1024 that needs overwriting, though the actual superblock location can vary by filesystem. I've since learned that a much more flexible and useful tool for this is {{ic|wipefs}}, which is much less dangerous than it sounds.<br />
: [[User:Mikeserv|Mikeserv]] ([[User talk:Mikeserv|talk]]) 14:28, 14 November 2013 (UTC)<br />
<br />
Thanks for the explanation of the super-block [[User:Mikeserv|Mikeserv's]] was very insightful. But that Udev stuff is still a layer to high for me :P I don't know if its possible to do a little differently to avoid a unwanted wait loop:<br />
: 1) Check to see if udev has discovered and added {disk UUIDS}. If yes goto 3)<br />
: 2) Wait 5 secs and goto 1.<br />
: 3) echo {pain in the a stuff} /sys/bla/bla/bla<br />
The dd command is just a safety step. :) And because formatting the /dev/bcache0 was soon to follow, it didn't hurt if i accidentally killed the file system on the bcache :) I got the feeling that resizing the bcache partition was also giving problems wich the dd command fixed..... BTW i like the piece added with this command set:<br />
# cd /sys/fs/bcache<br />
# echo "UUID" > /sys/block/bcache0/bcache/attach<br />
p.s. I don't get why the wiki page is full of btrfs stuff. Is this not a combination of commands? And shouldn't a more conventional partition/file system scheme be more useful for novice users (like me) ??<br />
: [[User:Emesix|Emesix]] ([[User talk:Emesix|talk]]) 22:47, 14 November 2013 (UTC)<br />
<br />
::Regarding btrfs and its usefulness for new users... I don't know. With btrfs for instance, there are many normally high-level file-system concepts rendered nearly as mundane as directory maintenance like snapshotting and subvolumes, and still others that require only a single edit to a boot config like {{ic|autodefrag}} and {{ic|<nowiki>compress=lzo</nowiki>}} delivering inline filesystem compression and cleanup. In practice, you really don't have to understand the hows of these things at all to get incredible use out of them. <br />
::Consider just sub volumes: they can deliver, at the very least, a separated partition structure for the various segments of the Linux filesystem as is often recommended people set at installation with syntax and simplicity like unto {{ic|cd}} and {{ic|mkdir}}, only the {{ic|btrfs --help}} implementation is probably a little friendlier. With traditional filesystems this same setup required knowledge of disk geometry, extent counts and all manner of other arcane things to configure in any sanely optimized way. Btrfs will create you a RAID in less than 30 seconds. <br />
::I guess what I'm saying is the research vs reward ratio is pretty significantly in your favor with a btrfs disk. Of course, should the btrfs filesystem crash, well, you should probably call someone else is all I'll say about that.<br />
::In this wiki, I definitely see your point, especially considering the added complexity bcache brings. When I was trying to set mine up I spent a good deal of time in both the btrfs and bcache IRC dev channels and both sides were less than optimistic about friendly cooperation between the two. Still, it works and has done for months. I know, don't call you, right?<br />
::Now about udev, I'll try again. So, your script implements a wait loop to check occasionally if udev has done a thing (bring up the disks and pull in basic drivers like SCSI for block devices and ext4 for their component filesystems) so it can do a thing (register the disks as small parts of a larger whole and pull in the bcache driver). The udev rules eschew the notion of a secondary monitor script such as yours and instead have udev do it all (any disk it brings up as a "bcache" type is registered as a small part of a larger whole and the bcache driver is pulled in). <br />
::So you couldn't boot from SSD reliably before because you couldn't reliably script the order in which udev would bring up the disks (a consequence of its faster parallel operations), but udev doesn't have to script its order, udev just udevs as udev udevs best.<br />
::See? You will always boot reliably from your chosen device because udev won't misinterpret its own report on your chosen device's readiness. And that's all - there's nothing wrong with the shell script exactly, except that it's just plain unnecessary and added complication.<br />
[[User:Mikeserv|Mikeserv]] ([[User talk:Mikeserv|talk]]) 23:38, 14 November 2013 (UTC)<br />
<br />
:: I have to say, I want to try out using btrfs instead of LVM on my desktop sometime, but the cache article is not the place for it. Mentioning it as a good alternative on the installation and beginners guide and having details in the btrfs article is going to be more productive and logical. I added the section near the top of the page for configuring a simple cache setup with no regards to file systems and partition layouts as they should be beyond the scope of this article. <br />
<br />
--[[User:Justin8|Justin8]] ([[User talk:Justin8|talk]]) 02:13, 15 November 2013 (UTC)<br />
::Agreed. There's a lot of it, too. I guess that sometimes happens on less visited pages. Probably it will level out as the module is more established. I compromised on some of the grub stuff in a minor edit yesterday after you rightly pointed out how much unrelated stuff is in there. Never thought about it before, but I've only ever written 7b and 2 notes here and a short section on UEFI kernel update syncing.<br />
::Above is just about everything I know relatable to bcache by now, I guess. I wish there was some more in the wiki here on benchmarking. The bcache wiki itself reads like a stereo repair manual on the subject. I never have managed to wrap my mind round it before my eyes would shut of their own accord. <br />
[[User:Mikeserv|Mikeserv]] ([[User talk:Mikeserv|talk]]) 03:00, 15 November 2013 (UTC)<br />
<br />
:: Ha, that is the perfect description for their wiki; it's all failure modes and fixes but no reasoning. From what I could see, benchmarking of bcache is really only useful to benchmark writes; reads will vary exponentially depending on whether or not you get a cache hit. On top of that, sequential reads/writes bypass the cache. the only real way to benchmark it would be in 'real-world' style tests instead of synthetic benchmarks. I.e. how long it takes to boot your system. how long it takes to compile X, Y or Z. I put a little bit up on my blog when testing out bcache, mostly focussing on IOps performance changes since everything else is relatively the same with or without bcache (other than the smart re-ordering of metadata changes) I was going to just paste the graphs, but they are all js objects; you can see the results I got here: https://www.dray.be/?p=7<br />
[[User:Justin8|Justin8]] ([[User talk:Justin8|talk]]) 04:05, 15 November 2013 (UTC)<br />
<br />
:::Justin: so you're the AUR bcache-tools maintainer now? I guess you dropped enough hints. That's cool. Reading through your blog (pretty colors) I was reminded of another little bcache fact that may not be immediately apparent to all. It seems most tend to assume that the cache:backing-store must be on a 1:1 ratio. This is not the case. A single cache can serve as many storage devices as you might like. Conversely, there's nothing preventing one from configuring several mechanical discs in a RAID and then backing it on a 1:1 ratio. Regarding a cached RAID, though, I do recall reading a bcache IRC conversation in which a dev mentioned hardware configuration of the device would be desirable as bcache is designed to interface with a block layer and dropping a filesystem below it can have unexpected results. <br />
:::I only bring this up because your blog mentions you will switch from a 1tb to a 3tb soon. Personally, I use a 120GB Kingston SSD to cache 2 3tb WD Greens which are then formatted as a single btrfs RAID1 metadata/RAID0 data. The data is mostly downloaded video for a home theater server and as such is largely expendable in the event of catastrophe. Probably you know this, but just in case, I thought I'd point out that there's nothing preventing you from merely adding the the 3tb to the 1tb as you please.<br />
:::Lastly, you can format and configure an entire array with basically one line like: {{ic|<nowiki>make-bcache -B /dev/{1tb} /dev/{new_3tb} -C /dev/{60GB_SSD}</nowiki>}}. Then just reboot (or even reload udev) if you've got the udev rules loaded - udev will register the devices for you based on the auto-attach data recorded in the partition's superblocks as a result of the all-in-one add command. You wouldn't even need the bcache_udev hook or anything else in initramfs at this point, because obviously you shouldn't expect to boot to the bcache array yet as you haven't had a chance to format it with a usable filesystem or to copy any files to it, but it can simplify things.<br />
:::[[User:Mikeserv|Mikeserv]] ([[User talk:Mikeserv|talk]]) 07:04, 15 November 2013 (UTC)<br />
<br />
::::I did notice a reference to that from the previous maintainer in the aur comments. I currently have a 9x2tb RAID6 mdadm array that I really don't want to have to backup and restore, so I did the tests as it would be in my server (just on a smaller scale). I don't believe it would be a simple task to remove the device, bcache-ify it and put it back in. The blocks utility says that it supports it for any block device; but mdadm stores data about the offset in the superblocks instead of using the partition layout (which I discovered the hard way in the past and had to restore 10TB of backups from off-site :( ) It should be fine though doing it to the MDADM device itself. I wouldn't mind knowing the performance difference, so once I get an SSD for my server I might test it with both methods to do a bit of a comparison.<br />
::::I didn't realize you could specify the backing and cache devices all in one line like that. I didn't see it in the wiki at all when I did my first lot of edits; seems useful. Does it automatically attach the cache to the backing devices as well?<br />
::::Although I reboot fairly often for things like kernel updates/making sure pacman hasn't broken things, I am somewhat averse to rebooting when I don't have to. I migrated from a root device on an SSD and /home and /opt on a 3TB disk, on to a spare 1TB disk, did the tests for my blog on a different computer and then created the bcache device with the 3TB disk and 60GB SSD and migrated back all without a reboot. Sometimes rebooting can be easier, but it's more 'fun' not to :)<br />
::::[[User:Justin8|Justin8]] ([[User talk:Justin8|talk]]) 08:28, 15 November 2013 (UTC)<br />
:::::Well, as I mentioned, you could opt to restart udev rather than reboot, or alternatively you could choose to throw a {{ic|udevadm}} trigger, or to have some fun with {{ic|kexec}}, or merely to {{ic|echo}} the block-device register commands to {{ic|/sys/}} as per usual. You have correctly surmised that the one-liner auto-attaches the backing-stores to the cache, but it does not perform the registration, which is necessary anyway every time an array is re/assembled as it is.<br />
::::: I added it to the bcache Arch wiki page myself after discovering it quite by accident between nods while browsing bcache's own documentation a few months ago. The command is represented in the wiki even now, if rather humbly as a side note to the initial backing-store formatting process. I mentioned it on your talk page as well, because you had apparently previously edited it not to fix its then anachronistic reference to no longer needing a now completely different step 4, but to note that it would no longer be necessary at boot. At first I was bewildered, but I've since corrected it to note that it now obviates step 6.<br />
:::::By the way, I might know of another thing worth saying, of which {{ic|kexec}}'s mention has reminded me: because of the way SystemD performs its unmounts at shutdown, it can be very slightly dangerous to run a multi-device write-back cache outside of its purview, as you must do if you boot to a bcache array but have not incorporated SystemD into your initramfs. If this is your situation currently, then likely you will be able to catch an occasional error message printed to the console about a forced unmount in the second or two just before your screen blanks its last. Of course bcache is supposed to be able to handle this and much more, and it never caused me any issues (though I run in write-through mode), but I did switch to the new SystemD hook just as soon as I noticed it. Specifically the SystemD {{ic|kexec}} function should almost certainly be avoided whenever possible when operating a write-back cache without a true PID1 SystemD configuration.<br />
:::::[[User:Mikeserv|Mikeserv]] ([[User talk:Mikeserv|talk]]) 09:13, 15 November 2013 (UTC)<br />
<br />
::::::That is interesting. (I assume you're talking about the systemd hook for mkinitcpio that is intended to replace udev/usr/base/etc) How does using that new systemd hook fix it unmounting drives too soon? Also the wiki notes that it is incompatible with lvm2 currently; which just so happens to be what my root filesystem is using... Maybe it is yet another reason to change to btrfs finally. I am using writeback on my bcache device as well. But so long as the cache device is available on a reboot it shouldn't affect anything.<br />
::::::[[User:Justin8|Justin8]] ([[User talk:Justin8|talk]]) 07:58, 16 November 2013 (UTC)<br />
:::::::Well, depending on your use-case, you may find btrfs+bcache to be a less than ideal combination. As I understand it, btrfs is designed to concatenate write operations in memory at least until it has compiled enough of them to deliver a sequentially written block to disk. Obviously any write operations given bcache will come from btrfs when the two are combined, and because bcache is designed to pass-through sequential writes, most disk writes should bypass your ssd cache entirely. This doesn't concern me overmuch because the majority of my disk space is consumed by large video files which, when first written, are sequential anyway, and the slower write speed is mostly mitigated with the two-disk btrfs RAID I configured. It is very nice, however, to have the most used applications and most watched videos dynamically located on the ssd, especially since the one system's disks serve several other machine's multimedia requests.<br />
:::::::It is my theory that bcache can assist a btrfs file-system in avoiding some of its infamous disk-thrashing tendencies associated with its relatively huge metadata structures by caching and consolidating even these write operations into sequential blocks, which, if true, is probably especially useful in my particular configuration of striped data and mirrored metadata. Unfortunately, as you might have gleaned from a previous statement I've made, I can hardly back this up with any hard data as proof. I also theorize that the btrfs kernel parameter autodefrag, which in a conventional configuration can severely affect write-speeds, becomes a much more sensible option with the addition of an ssd configured with bcache, but I can't offer any evidence to support that either.<br />
:::::::As I said when first replying to E, though a repeat is probably long overdue, I am sometimes wrong (some may have cause even to sneer at the "sometimes"), and before I go on I should say again that anything I write is likely to serve one best if considered in combination with other sources of information. If it seems to anyone that what I say makes no sense or is contradictory in some way then it's probably best to verify that doubt one way or another before proceeding with any operation that might affect valued data, or even that might just result in an unnecessarily wasted day spent reconfiguring twice what used to be a computer system that would boot with the press of a button. Also, if anyone catches something wrong, please correct me.<br />
:::::::Sorry, it just occurred to me that others might venture this way eventually and act on what they read so I figured another disclaimer was called for. By the way, if you ever wind up as bewildered by the Arch Wiki LXC page as I was, definitely read through its talk page before giving up and moving on.<br />
:::::::Ok, so about systemd, we have to start here: <br />
:::::::I've noticed a lot of people, as I used to do, tend to ascribe unwarranted mystery to initramfs. Probably, at least in Arch's case, this is an unavoidable consequence to the level of automated excellence provided by its mkinitcpio script; it is because we so rarely have any cause to understand it that so few of us do. Certainly I didn't anyway, until very recently.<br />
:::::::The initramfs is a file containing an image of the first filesystem mounted by the kernel after it is called. This image-file can be compiled into the kernel itself at build-time, but, because it can be more convenient to swap in other images at will, it is conventional instead to feed its path to the kernel as a parameter. The kernel expects the image's final archived format to be "cpio" (CoPy In/Out archives can handle items many others can't like /dev/, /sys/ and similar), but it can decompress that from any format you can compile into it (like lzma, lz4, gz, xz, etc). In its current iteration (as of kernel 2.6.something), when loaded the kernel mounts initramfs as "rootfs," which is basically the contents of this image mounted to / as a tmpfs.<br />
:::::::As far as the kernel is concerned its primary boot job is to find and call /init, and all the rest is up to user-space. And because the linux kernel can potentially handle more / disk configurations than can practically be accounted for in a single compile without growing exponentially every year, it was 2.6.something when initramfs became mandatory. For all intents and purposes, you can still compile in any modules required to mount your root filesystem, run /init from there, and skip both compiling as built-in and parameterizing an initramfs image at load-time, but the kernel still contains and mounts a (basically) empty / image no matter what you do. <br />
:::::::mkinitcpio's "base" hook is basically just busybox and a couple of shell scripts, one of which is /init. Arch's default initramfs behavior is pretty similar to most others; it packs in as little as necessary to find and mount your filesystem at /newroot then uses "switchroot" (the only really out-of-the-ordinary part of the whole process) to simultaneously dismount the initramfs /, mount /newroot in its place and call systemd.<br />
:::::::Ok, sorry for the lecture, but there's already enough here to warrant my coming back when I've forgotten any of it, so I figured i'd keep my future self on the right track if I could manage.<br />
:::::::Anyway, so have a look at the lsblk printouts at the bottom of bcache's wiki here: http://bcache.evilpiepirate.org/#index6h1<br />
:::::::Notice that after the array is built successfully (shown in the second lsblk output) the /dev/bcache{0,1} partitions are represented twice; they're children of both the physical disks on which they actually reside and the cache disk that buffers their i/o. The first lsblk, pulled before the bcache device to which they're attached is registered, is representative of a more typical configuration in which each partition is descended only from the physical device on which it resides.<br />
:::::::That added complexity is, as near as I can figure, what trips systemd's shutdown unmounts up. The bcache module, its user-space tools, and your / array are all called by the initramfs udev which is killed at "switchroot" when systemd loads so it can load and properly manage its own child udev process. <br />
:::::::systemd is designed from the ground up to be PID1, /init, the daemons' daemon. Through /sys/, it manages a pid tree with itself as the root reliably tracking and killing even forked child processes when the original parent processes quit. Asking it to undo complicated mount structures built before ever it can hook into /sys/, however, is probably asking for trouble.<br />
:::::::Still, as you say, dirty or not the data is still there. bcache is a disk cache after all, and it will persist between reboots in the cache on the ssd even if the array is not cleanly disassembled before the disks are unmounted. And bcache is specifically designed to track in its own btrees the cached data's final disk targets in order to reliably handle exactly these kinds of situations. I qualified my earlier description of the scenario as only "very slightly" unsafe because the redundancy is reduced. There is also the outside possibility that the cached data could be orphaned if the original backing store > cache attachment relationship written to the array members' bcache partition superblocks is somehow broken, which, at least when I configured my disks only a few months ago, was noted in the bcache docs as rare but possible. This last, I should say, I have never experienced, but, and only for reasons I can't exactly put my finger on, I have a hunch it could be related to the whole systemd dirty unmount scenario described above.<br />
:::::::I specifically cautioned against kexec before because it's sort of a "switchkernel" version of the "switchroot" process I referenced above. kexec first writes an OS kernel to system memory, then it simultaneously writes out the current kernel's mapped region and writes the to-be-loaded kernel over it, before finally, in its last gasp, as it were, calling the new kernel to execute. It seems to me that if the daemons' daemon has difficulty handling preexisting conditions then testing the OS kernel in such circumstances is probably not something I'd like to do.<br />
:::::::Geez. That's a lot. But there's more.<br />
:::::::While I can't say for certain because I haven't looked into it, I suspect that an lvm2 / is only contra-indicated for the systemd mkinitcpio hook because the particular combination has not yet been scripted to the same level of automatic detection and reliable configuration as the rest. There's a huge difference between reliably configuring a thing and reliably automating a configuration of course, and as I'm sure Arch's core maintainers are painfully aware. It is probably for precisely this reason that Arch enables the bcache kernel module in its default kernel compile but provides no supported means of making use of it. Still, as far as I know, systemd is not inherently incompatible with lvm2, and because the initramfs is just a / doing (mostly) plain old linuxy / things, there's no reason you can't set it up any way you please. Maybe get to know lsinitcpio a little better if you're interested.<br />
:::::::And here's one of perhaps special import to the AUR's bcache-tools package maintainer; while putting systemd in the initramfs is the one way I've handled the shutdown problem, I doubt very seriously it is the only way. Two other possibilities come immediately to mind: either hooking into or emulating in some way the mkinitcpio shutdown (or is it suspend?) hook which I expect must have some means of handling an end event or two; or, and probably the way i'd go, systemd's own shutdown target, which definitely does and is probably specifically designed to handle just such a thing. Probably there are many others as well, but none spring to mind.<br />
:::::::You can wake up now. I'm through.<br />
:::::::[[User:Mikeserv|Mikeserv]] ([[User talk:Mikeserv|talk]]) 05:56, 17 November 2013 (UTC)<br />
<br />
::::::::That was long! Perhaps we should take this conversation off the talk page so that we don't fill it up too much. Do you use IRC/gtalk/skype/facebook at all? I'd like to discuess some more in regards to the possiblity of a shutdown hook for bcache.<br />
:::::::::Yes, long, but I think its relevant. I was kind of hoping some kind-hearted soul would happen by and, perhaps having learned something, convert some of my pedantic stream-of-consciousness into something more usefully accessible. Wanna know the worst of it? It was written almost entirely on my Nexus 4 at various times throughout the day. I guess I'm something of a masochist.<br />
:::::::::Anyway, anyone can edit it out, of course. I saved a copy.<br />
::::::::[[User:Mikeserv|Mikeserv]] ([[User talk:Mikeserv|talk]]) 06:22, 17 November 2013 (UTC)</div>Justin8https://wiki.archlinux.org/index.php?title=Talk:Bcache&diff=283294Talk:Bcache2013-11-17T06:09:59Z<p>Justin8: </p>
<hr />
<div>==Example installation==<br />
Here is mine Bcache installation. Maybe you can use some idea's. :)<br />
<br />
=== Prepare the storage drive ===<br />
<br />
====Bcache-tools installation====<br />
fakeroot and binutils are only needed. but it will fail in a error if you only install them.<br />
wget is from a failed archbang instal :(<br />
<br />
pacman -Syy<br />
pacman -S base-devel fakeroot binutils wget git<br />
<br />
cd ~/<br />
wget https://aur.archlinux.org/packages/bc/bcache-tools-git/bcache-tools-git.tar.gz<br />
tar -xzvf bcache-tools-git.tar.gz<br />
cd bcache-tools-git<br />
makepkg -si --asroot<br />
<br />
==== Making the partitions ====<br />
For /boot i use the slower HDD because i had troubles that mine SSD loaded the kernel while mine HDD was still spinning up :P<br />
<br />
Starting with the faster SSD drive<br />
<br />
fdisk /dev/sda<br />
<br />
o<br />
y<br />
n<br />
Partition type: p<br />
Partition-number: 1<br />
First-sector: [enter]<br />
Last-sector: +90G<br />
<br />
n<br />
Partition type: p<br />
Partition-number: 3<br />
First-sector: [enter]<br />
Last-sector: [enter]<br />
<br />
write and exit<br />
<br />
Now the slower 2 TB Harddrive<br />
<br />
fdisk /dev/sdb<br />
<br />
o<br />
y<br />
n<br />
Partition type: p<br />
Partition-number: 1<br />
First-sector: [enter]<br />
Last-sector: +1G<br />
a<br />
<br />
n<br />
Partition type: p<br />
Partition-number: 2<br />
First-sector: [enter]<br />
Last-sector: +16G [enter]<br />
t<br />
2<br />
82<br />
<br />
n<br />
Partition type: p<br />
Partition-number: 3<br />
First-sector: [enter]<br />
Last-sector: [enter]<br />
<br />
write and exit<br />
<br />
==== Making Bcache Device ====<br />
The --wipe-bcache is to remove a error :)<br />
and the dd commands is or cleaning out old partition data.<br />
<br />
make-bcache --wipe-bcache -B /dev/sdb3 -C /dev/sda2<br />
dd if=/dev/zero count=1 bs=1024 seek=1 of=/dev/sda2<br />
dd if=/dev/zero count=1 bs=1024 seek=1 of=/dev/sdb3<br />
<br />
If next commands give a error: invalid argument<br />
Then the device is already registered.<br />
<br />
echo /dev/sda2 > /sys/fs/bcache/register<br />
echo /dev/sdb3 > /sys/fs/bcache/register<br />
<br />
==== Formating the drives ====<br />
<br />
<br />
mkfs.ext4 -E discard /dev/sda1<br />
mkfs.ext4 -E discard /dev/sdb1<br />
mkfs.ext4 /dev/bcache0<br />
mkswap /dev/sdb2<br />
swapon /dev/sdb2<br />
<br />
==== Mount the partitions ====<br />
<br />
cd ~/<br />
mkdir -p /mnt/install<br />
mount /dev/sda1 /mnt/install<br />
mkdir -p /mnt/install/{boot,home,srv,data,var,mnt/.hdd}<br />
mount /dev/sdb1 /mnt/install/boot<br />
mount /dev/bcache0 /mnt/install/mnt/.hdd<br />
mkdir -p /mnt/install/mnt/.hdd/{home,srv,data,var}<br />
mount -o bind /mnt/install/mnt/.hdd/home /mnt/install/home<br />
mount -o bind /mnt/install/mnt/.hdd/srv /mnt/install/srv<br />
mount -o bind /mnt/install/mnt/.hdd/data /mnt/install/data<br />
mount -o bind /mnt/install/mnt/.hdd/var /mnt/install/var<br />
<br />
==== Bcache Check After mounting ====<br />
Reason for /boot on the HDD is because of boot up failure because of HDD spin-up.<br />
<br />
lsblk<br />
<br />
NAME MAJ:MIN RM SIZE RO TYPE MOUNTPOINT<br />
sda 8:0 0 167.7G 0 disk <br />
|-sda1 8:1 0 90G 0 part /mnt/install<br />
`-sda2 8:2 0 77.7G 0 part <br />
`-bcache0 253:0 0 1.8T 0 disk /mnt/install/mnt/.hdd<br />
sdb 8:16 0 1.8T 0 disk <br />
|-sdb1 8:17 0 1G 0 part /mnt/install/boot<br />
|-sdb2 8:18 0 16G 0 part [SWAP]<br />
`-sdb3 8:19 0 1.8T 0 part <br />
`-bcache0 253:0 0 1.8T 0 disk /mnt/install/mnt/.hdd<br />
<br />
==== Generate an fstab ====<br />
<br />
Generate an fstab file with the following command. UUIDs will be used because they have certain advantages.<br />
<br />
genfstab -U -p /mnt/install >> /mnt/install/etc/fstab<br />
sed -i '/dev\/sda/ s/rw,relatime,data=ordered/defaults,noatime,discard/g' /mnt/install/etc/fstab<br />
sed -i '/mnt\/install/ {s/\/mnt\/install//g; s/rw,relatime,data=ordered/defaults/g; s/none/bind/g}' /mnt/install/etc/fstab<br />
more /mnt/install/etc/fstab<br />
<br />
=== Install and configure kernel and bootloader ===<br />
==== Adding Bcache module and hook ====<br />
<br />
Edit /etc/mkinitcpio.conf as needed and re-generate the initramfs image with:<br />
adding the "bcache" module<br />
adding the "bcache_udev" hook between block and filesystems<br />
<br />
sed -i '/^MODULES=/ s/"$/ bcache"/' /etc/mkinitcpio.conf<br />
sed -i '/^HOOKS=/ s/block filesystems/block bcache_udev filesystems/' /etc/mkinitcpio.conf<br />
more /etc/mkinitcpio.conf<br />
<br />
==== Registering and attaching Bcache ====<br />
This part is a pain in the A.. <br />
The problem is that Bcache wil give a error at startup if this is not done.<br />
<br />
ERROR: Unable to find root device 'UUID=b6b2d82b-f87e-44d5-bbc5-c51dd7aace15'.<br />
You are being dropped to a recovery shell<br />
Type 'exit' to try and continue booting<br />
<br />
First get the correct "SET UUID" with this ls command. <br />
And use the UUID like the example underneath.<br />
Also check for the use of the correct devices.<br />
echo: write error: Invalid argument <<< This error is good !!! means that your value was already stored.<br />
<br />
ls /sys/fs/bcache/<br />
2300f944-0eb5-4a9e-b052-8d5ea63cbb8f register register_quiet<br />
<br />
sudo echo /dev/sda2 > /sys/fs/bcache/register<br />
sudo echo /dev/sdb3 > /sys/fs/bcache/register<br />
sudo echo 2300f944-0eb5-4a9e-b052-8d5ea63cbb8f > /sys/block/sdb/sdb3/bcache/attach<br />
<br />
mv /usr/lib/initcpio/hooks/bcache ~/bcache.old<br />
cat >> /usr/lib/initcpio/hooks/bcache << EOF<br />
#!/usr/bin/ash<br />
run_hook() {<br />
echo /dev/sda2 > /sys/fs/bcache/register<br />
echo /dev/sdb3 > /sys/fs/bcache/register<br />
}<br />
EOF<br />
<br />
==== Installing the Kernel ==== <br />
mkinitcpio -p linux<br />
<br />
I hope some stuff is helpfull :P<br />
17:58, 11 November 2013 (UTC)<br />
:I'm curious about two things. First, though I forget what the '-E' switch signifies, it seems you're formatting the raw, underlayer partitions themselves. I don't understand why.<br />
<br />
:Also, the "pain in the a?" So udev is not assembling your array automatically as the devices are discovered? That is unexpected behavior, and I suspect it has to do with the formatting you performed and udev's superblock discovery. Check the very bottom of the bcache wiki on their site. At least try lsblk to verify "bcache" partition types are not registering as "ext4" before assembling the array.<br />
<br />
:Oh. And please sign talk pages.<br />
<br />
[[User:Mikeserv|Mikeserv]] ([[User talk:Mikeserv|talk]]) 03:35, 12 November 2013 (UTC)<br />
<br />
<br />
<br />
Well this is a experiment with Bcache with mine own Wiki page and 100x testing the install, i am not a linux expert just a newbie who was reading a lot of howto's. Just see it as a compilation of different web pages and some logical thinking. :)<br />
<br />
The -E is just a copy and past ... not sure if it was needed.I just thought it was needed to get the discard flag while formating.<br />
<br />
The pain in the A.... this was before Udev ... i was busy getting Bcache to work 3 months ago ... 75% of boot up failed when booting up cold. Later i found out that the SSD was too fast with booting up and the normal HDD was still spinning up.... After i splitted up the SSD for root directory and bcache and moved the /boot to the HDD since then i can bootup 100% of the time without any error... This was before the udev thingy (wich i dont understand and need to read up) :)<br />
<br />
So please understand that i am just a linux noob trying to contribute :)<br />
<br />
[[User:Emesix|Emesix]] ([[User talk:Emesix|talk]])[[User:Emesix|Emesix]] ([[User talk:Emesix|talk]]) 00:02, 13 November 2013 (UTC)<br />
<br />
:No, I get it, and I didn't mean to insinuate that you were doing anything wrong, exactly, just that, as designed, it could operate better.<br />
:Basically, and you should check the man-pages and the wiki page here to be sure just in case I'm not entirely correct (which does happen), udev is the program that runs during boot (and pretty much all of the rest of the time, too) to detect your hardware and load drivers as necessary. Udev is a relatively recent addition to the Linux boot process and is unique in that it discovers and handles hardware as it shows up, can handle multiple devices simultaneously with non-blocking parallell operations, and, as a result, is significantly faster than the methods it has replaced.<br />
:I was the one who added the udev rules and step 7b to this wiki, and I was the one who wrote the "Rogue Superblock" section of "Troubleshooting" at the bottom of bcache's wiki. I only gained the knowledge to write either after spending several frustrating days with an issue that appears very similar to your "pain in the a."<br />
:As step 7a shows, the prevailing Arch Linux method for reassembling a bcache array at boot used to look something like:<br />
: 1) Wait 5 secs. <br />
: 2) Check to see if udev has discovered and added {disk UUIDS}. If yes goto 3), no, repeat 1) and 2).<br />
: 3) echo {pain in the a stuff} /sys/bla/bla/bla<br />
:This does work, of course, but you're adding an unnecessary wait loop to your boot process and you're performing a task with brittle shell script that depends on specific UUIDs that could be performed flexibly and at discovery time.<br />
:That's what the udev rules do. Basically, the rules instruct udev to treat as part of your bcache array any partition it finds that reports itself as a "bcache" partition type. It builds the bcache array from disk partitions at every boot only as soon as those partitions are ready, which would of course resolve the race condition you mentioned and allow you to boot from SSD if you liked.<br />
:And here's where our experience might converge: I added a partition to my bcache array after previously formatting it as ext4. If you don't know about superblocks (or magic blocks or whatever they're called) then you're just like I was a few months ago. In short the filesystem has to have a way to report on itself, so it sets aside a small marker on disk that says, "Hey, OS, I'm a disk partition of type EXT4 and I start at block offset whatever and end at block offset whatever. Also, I enjoy long walks on the beach and prefer red wine to white. See you around."<br />
:The problem I experienced was that by default ext4 tended to begin at an earlier offset than bcache, and so even though I formatted over the previously ext4 partition with bcache as instructed, bcache never overwrote ext4's first superblock. When udev attempted to build my array, it would always miss my first partition because it would read the first superblock, identify the partition as ext4, then skip it. Echoing the add instruction to /sys/ after boot was my fix for a couple of days, but eventually I figured it all out and wrote that short bit on superblocks in bcache's wiki.<br />
:Maybe that helps?<br />
:EDIT: Just looked back at the wiki proper and somebody has performed some major changes recently. While it does appear cleaner, it no longer makes sense. There are references to non-existent steps throughout, and the actual why's that were included at least to some degree seem to have been occluded in favor of dubious, though possibly more efficient how's. Anyway, step 7b no longer exists, but it used to include the bcache_udev mkinitcpio hook I (very barely) adapted from mdadm_udev when trying to fix Arch's bcache boot problems. It's now included in the AUR package the wiki recommends I guess, despite the wiki also telling us we must "echo ... /sys/... at every bootup."<br />
[[User:Mikeserv|Mikeserv]] ([[User talk:Mikeserv|talk]]) 07:08, 14 November 2013 (UTC)<br />
<br />
Sorry about that, I missed a couple of references back to the sections I changed. Hopefully it makes more sense now. [[User:Mikeserv|Mikeserv's]] udev rule is crucial to having a working bcache device on boot, as [[User:Emesix|Emesix]] had discovered, it will often result in a failed boot. The echo in to /sys/bcache* step was missed when I updated it to mention the udev rule as the 'default' method.<br />
<br />
--[[User:Justin8|Justin8]] ([[User talk:Justin8|talk]]) 10:45, 14 November 2013 (UTC)<br />
:For clarity's sake, the udev rule was never mine. The used rule has been included in the AUR package's git pull from at least the time I first installed bcache some months ago. The package maintainer at that time opted not to use it, I guess, and instead installed the legacy-type shell-scripted for-loop mkinitcpio hook I outlined above, which was, to be fair, also the process recommended by every other source of information I was able to dig up at the time to include bcache's own wiki. Frustrated with having to wait for such things which seemed to me to defeat the whole purpose of buying and configuring an SSD boot device, I eventually stumbled upon the mdadm_udev hook in /use/share/initcpio and (only slightly) modified it to apply the 69-bcache-rules instead.<br />
:Justin, thanks for your recent change. NEVERMIND FOLLOWING: I'm curious why you specify "unless assembling after boot?" Why include anything to do with bcache in the initramfs at all if you're building the array after the boot sequence? MAKES PERFECT SENSE OF COURSE. GOT A 1 TRACK MIND SOMETIMES I GUESS. SORRY.<br />
[[User:Mikeserv|Mikeserv]] ([[User talk:Mikeserv|talk]]) 11:11, 14 November 2013 (UTC)<br />
<br />
:Emesix, just looked closer at your instructions, and you actually do the thing I had to do and outlined in the bcache wiki to overwrite the rogue superblock here:<br />
:{{ic|<nowiki>dd if=/dev/zero count=1 bs=1024 seek=1 of=/dev/sd{a2,b3}</nowiki>}}<br />
:And the ext4 formatting you do after that I earlier incorrectly assumed to be for the "raw, underlayer partitions" is revealed upon further inspection to be for non-bcache partitions and so isn't even relevant; I apologize for jumping to conclusions before reviewing it fully, the quick check I did a couple of days ago just struck me as really familiar.<br />
:The udev stuff above should still be relevant, and it might be worth noting that the particular {{ic|dd}} command above is by no means a cure-all. It should only be used as written if you have previously identified a superblock located at byte offset 1024 that needs overwriting, though the actual superblock location can vary by filesystem. I've since learned that a much more flexible and useful tool for this is {{ic|wipefs}}, which is much less dangerous than it sounds.<br />
: [[User:Mikeserv|Mikeserv]] ([[User talk:Mikeserv|talk]]) 14:28, 14 November 2013 (UTC)<br />
<br />
Thanks for the explanation of the super-block [[User:Mikeserv|Mikeserv's]] was very insightful. But that Udev stuff is still a layer to high for me :P I don't know if its possible to do a little differently to avoid a unwanted wait loop:<br />
: 1) Check to see if udev has discovered and added {disk UUIDS}. If yes goto 3)<br />
: 2) Wait 5 secs and goto 1.<br />
: 3) echo {pain in the a stuff} /sys/bla/bla/bla<br />
The dd command is just a safety step. :) And because formatting the /dev/bcache0 was soon to follow, it didn't hurt if i accidentally killed the file system on the bcache :) I got the feeling that resizing the bcache partition was also giving problems wich the dd command fixed..... BTW i like the piece added with this command set:<br />
# cd /sys/fs/bcache<br />
# echo "UUID" > /sys/block/bcache0/bcache/attach<br />
p.s. I don't get why the wiki page is full of btrfs stuff. Is this not a combination of commands? And shouldn't a more conventional partition/file system scheme be more useful for novice users (like me) ??<br />
: [[User:Emesix|Emesix]] ([[User talk:Emesix|talk]]) 22:47, 14 November 2013 (UTC)<br />
<br />
::Regarding btrfs and its usefulness for new users... I don't know. With btrfs for instance, there are many normally high-level file-system concepts rendered nearly as mundane as directory maintenance like snapshotting and subvolumes, and still others that require only a single edit to a boot config like {{ic|autodefrag}} and {{ic|<nowiki>compress=lzo</nowiki>}} delivering inline filesystem compression and cleanup. In practice, you really don't have to understand the hows of these things at all to get incredible use out of them. <br />
::Consider just sub volumes: they can deliver, at the very least, a separated partition structure for the various segments of the Linux filesystem as is often recommended people set at installation with syntax and simplicity like unto {{ic|cd}} and {{ic|mkdir}}, only the {{ic|btrfs --help}} implementation is probably a little friendlier. With traditional filesystems this same setup required knowledge of disk geometry, extent counts and all manner of other arcane things to configure in any sanely optimized way. Btrfs will create you a RAID in less than 30 seconds. <br />
::I guess what I'm saying is the research vs reward ratio is pretty significantly in your favor with a btrfs disk. Of course, should the btrfs filesystem crash, well, you should probably call someone else is all I'll say about that.<br />
::In this wiki, I definitely see your point, especially considering the added complexity bcache brings. When I was trying to set mine up I spent a good deal of time in both the btrfs and bcache IRC dev channels and both sides were less than optimistic about friendly cooperation between the two. Still, it works and has done for months. I know, don't call you, right?<br />
::Now about udev, I'll try again. So, your script implements a wait loop to check occasionally if udev has done a thing (bring up the disks and pull in basic drivers like SCSI for block devices and ext4 for their component filesystems) so it can do a thing (register the disks as small parts of a larger whole and pull in the bcache driver). The udev rules eschew the notion of a secondary monitor script such as yours and instead have udev do it all (any disk it brings up as a "bcache" type is registered as a small part of a larger whole and the bcache driver is pulled in). <br />
::So you couldn't boot from SSD reliably before because you couldn't reliably script the order in which udev would bring up the disks (a consequence of its faster parallel operations), but udev doesn't have to script its order, udev just udevs as udev udevs best.<br />
::See? You will always boot reliably from your chosen device because udev won't misinterpret its own report on your chosen device's readiness. And that's all - there's nothing wrong with the shell script exactly, except that it's just plain unnecessary and added complication.<br />
[[User:Mikeserv|Mikeserv]] ([[User talk:Mikeserv|talk]]) 23:38, 14 November 2013 (UTC)<br />
<br />
:: I have to say, I want to try out using btrfs instead of LVM on my desktop sometime, but the cache article is not the place for it. Mentioning it as a good alternative on the installation and beginners guide and having details in the btrfs article is going to be more productive and logical. I added the section near the top of the page for configuring a simple cache setup with no regards to file systems and partition layouts as they should be beyond the scope of this article. <br />
<br />
--[[User:Justin8|Justin8]] ([[User talk:Justin8|talk]]) 02:13, 15 November 2013 (UTC)<br />
::Agreed. There's a lot of it, too. I guess that sometimes happens on less visited pages. Probably it will level out as the module is more established. I compromised on some of the grub stuff in a minor edit yesterday after you rightly pointed out how much unrelated stuff is in there. Never thought about it before, but I've only ever written 7b and 2 notes here and a short section on UEFI kernel update syncing.<br />
::Above is just about everything I know relatable to bcache by now, I guess. I wish there was some more in the wiki here on benchmarking. The bcache wiki itself reads like a stereo repair manual on the subject. I never have managed to wrap my mind round it before my eyes would shut of their own accord. <br />
[[User:Mikeserv|Mikeserv]] ([[User talk:Mikeserv|talk]]) 03:00, 15 November 2013 (UTC)<br />
<br />
:: Ha, that is the perfect description for their wiki; it's all failure modes and fixes but no reasoning. From what I could see, benchmarking of bcache is really only useful to benchmark writes; reads will vary exponentially depending on whether or not you get a cache hit. On top of that, sequential reads/writes bypass the cache. the only real way to benchmark it would be in 'real-world' style tests instead of synthetic benchmarks. I.e. how long it takes to boot your system. how long it takes to compile X, Y or Z. I put a little bit up on my blog when testing out bcache, mostly focussing on IOps performance changes since everything else is relatively the same with or without bcache (other than the smart re-ordering of metadata changes) I was going to just paste the graphs, but they are all js objects; you can see the results I got here: https://www.dray.be/?p=7<br />
[[User:Justin8|Justin8]] ([[User talk:Justin8|talk]]) 04:05, 15 November 2013 (UTC)<br />
<br />
:::Justin: so you're the AUR bcache-tools maintainer now? I guess you dropped enough hints. That's cool. Reading through your blog (pretty colors) I was reminded of another little bcache fact that may not be immediately apparent to all. It seems most tend to assume that the cache:backing-store must be on a 1:1 ratio. This is not the case. A single cache can serve as many storage devices as you might like. Conversely, there's nothing preventing one from configuring several mechanical discs in a RAID and then backing it on a 1:1 ratio. Regarding a cached RAID, though, I do recall reading a bcache IRC conversation in which a dev mentioned hardware configuration of the device would be desirable as bcache is designed to interface with a block layer and dropping a filesystem below it can have unexpected results. <br />
:::I only bring this up because your blog mentions you will switch from a 1tb to a 3tb soon. Personally, I use a 120GB Kingston SSD to cache 2 3tb WD Greens which are then formatted as a single btrfs RAID1 metadata/RAID0 data. The data is mostly downloaded video for a home theater server and as such is largely expendable in the event of catastrophe. Probably you know this, but just in case, I thought I'd point out that there's nothing preventing you from merely adding the the 3tb to the 1tb as you please.<br />
:::Lastly, you can format and configure an entire array with basically one line like: {{ic|<nowiki>make-bcache -B /dev/{1tb} /dev/{new_3tb} -C /dev/{60GB_SSD}</nowiki>}}. Then just reboot (or even reload udev) if you've got the udev rules loaded - udev will register the devices for you based on the auto-attach data recorded in the partition's superblocks as a result of the all-in-one add command. You wouldn't even need the bcache_udev hook or anything else in initramfs at this point, because obviously you shouldn't expect to boot to the bcache array yet as you haven't had a chance to format it with a usable filesystem or to copy any files to it, but it can simplify things.<br />
:::[[User:Mikeserv|Mikeserv]] ([[User talk:Mikeserv|talk]]) 07:04, 15 November 2013 (UTC)<br />
<br />
::::I did notice a reference to that from the previous maintainer in the aur comments. I currently have a 9x2tb RAID6 mdadm array that I really don't want to have to backup and restore, so I did the tests as it would be in my server (just on a smaller scale). I don't believe it would be a simple task to remove the device, bcache-ify it and put it back in. The blocks utility says that it supports it for any block device; but mdadm stores data about the offset in the superblocks instead of using the partition layout (which I discovered the hard way in the past and had to restore 10TB of backups from off-site :( ) It should be fine though doing it to the MDADM device itself. I wouldn't mind knowing the performance difference, so once I get an SSD for my server I might test it with both methods to do a bit of a comparison.<br />
::::I didn't realize you could specify the backing and cache devices all in one line like that. I didn't see it in the wiki at all when I did my first lot of edits; seems useful. Does it automatically attach the cache to the backing devices as well?<br />
::::Although I reboot fairly often for things like kernel updates/making sure pacman hasn't broken things, I am somewhat averse to rebooting when I don't have to. I migrated from a root device on an SSD and /home and /opt on a 3TB disk, on to a spare 1TB disk, did the tests for my blog on a different computer and then created the bcache device with the 3TB disk and 60GB SSD and migrated back all without a reboot. Sometimes rebooting can be easier, but it's more 'fun' not to :)<br />
::::[[User:Justin8|Justin8]] ([[User talk:Justin8|talk]]) 08:28, 15 November 2013 (UTC)<br />
:::::Well, as I mentioned, you could opt to restart udev rather than reboot, or alternatively you could choose to throw a {{ic|udevadm}} trigger, or to have some fun with {{ic|kexec}}, or merely to {{ic|echo}} the block-device register commands to {{ic|/sys/}} as per usual. You have correctly surmised that the one-liner auto-attaches the backing-stores to the cache, but it does not perform the registration, which is necessary anyway every time an array is re/assembled as it is.<br />
::::: I added it to the bcache Arch wiki page myself after discovering it quite by accident between nods while browsing bcache's own documentation a few months ago. The command is represented in the wiki even now, if rather humbly as a side note to the initial backing-store formatting process. I mentioned it on your talk page as well, because you had apparently previously edited it not to fix its then anachronistic reference to no longer needing a now completely different step 4, but to note that it would no longer be necessary at boot. At first I was bewildered, but I've since corrected it to note that it now obviates step 6.<br />
:::::By the way, I might know of another thing worth saying, of which {{ic|kexec}}'s mention has reminded me: because of the way SystemD performs its unmounts at shutdown, it can be very slightly dangerous to run a multi-device write-back cache outside of its purview, as you must do if you boot to a bcache array but have not incorporated SystemD into your initramfs. If this is your situation currently, then likely you will be able to catch an occasional error message printed to the console about a forced unmount in the second or two just before your screen blanks its last. Of course bcache is supposed to be able to handle this and much more, and it never caused me any issues (though I run in write-through mode), but I did switch to the new SystemD hook just as soon as I noticed it. Specifically the SystemD {{ic|kexec}} function should almost certainly be avoided whenever possible when operating a write-back cache without a true PID1 SystemD configuration.<br />
:::::[[User:Mikeserv|Mikeserv]] ([[User talk:Mikeserv|talk]]) 09:13, 15 November 2013 (UTC)<br />
<br />
::::::That is interesting. (I assume you're talking about the systemd hook for mkinitcpio that is intended to replace udev/usr/base/etc) How does using that new systemd hook fix it unmounting drives too soon? Also the wiki notes that it is incompatible with lvm2 currently; which just so happens to be what my root filesystem is using... Maybe it is yet another reason to change to btrfs finally. I am using writeback on my bcache device as well. But so long as the cache device is available on a reboot it shouldn't affect anything.<br />
::::::[[User:Justin8|Justin8]] ([[User talk:Justin8|talk]]) 07:58, 16 November 2013 (UTC)<br />
:::::::Well, depending on your use-case, you may find btrfs+bcache to be a less than ideal combination. As I understand it, btrfs is designed to concatenate write operations in memory at least until it has compiled enough of them to deliver a sequentially written block to disk. Obviously any write operations given bcache will come from btrfs when the two are combined, and because bcache is designed to pass-through sequential writes, most disk writes should bypass your ssd cache entirely. This doesn't concern me overmuch because the majority of my disk space is consumed by large video files which, when first written, are sequential anyway, and the slower write speed is mostly mitigated with the two-disk btrfs RAID I configured. It is very nice, however, to have the most used applications and most watched videos dynamically located on the ssd, especially since the one system's disks serve several other machine's multimedia requests.<br />
:::::::It is my theory that bcache can assist a btrfs file-system in avoiding some of its infamous disk-thrashing tendencies associated with its relatively huge metadata structures by caching and consolidating even these write operations into sequential blocks, which, if true, is probably especially useful in my particular configuration of striped data and mirrored metadata. Unfortunately, as you might have gleaned from a previous statement I've made, I can hardly back this up with any hard data as proof. I also theorize that the btrfs kernel parameter autodefrag, which in a conventional configuration can severely affect write-speeds, becomes a much more sensible option with the addition of an ssd configured with bcache, but I can't offer any evidence to support that either.<br />
:::::::As I said when first replying to E, though a repeat is probably long overdue, I am sometimes wrong (some may have cause even to sneer at the "sometimes"), and before I go on I should say again that anything I write is likely to serve one best if considered in combination with other sources of information. If it seems to anyone that what I say makes no sense or is contradictory in some way then it's probably best to verify that doubt one way or another before proceeding with any operation that might affect valued data, or even that might just result in an unnecessarily wasted day spent reconfiguring twice what used to be a computer system that would boot with the press of a button. Also, if anyone catches something wrong, please correct me.<br />
:::::::Sorry, it just occurred to me that others might venture this way eventually and act on what they read so I figured another disclaimer was called for. By the way, if you ever wind up as bewildered by the Arch Wiki LXC page as I was, definitely read through its talk page before giving up and moving on.<br />
:::::::Ok, so about systemd, we have to start here: <br />
:::::::I've noticed a lot of people, as I used to do, tend to ascribe unwarranted mystery to initramfs. Probably, at least in Arch's case, this is an unavoidable consequence to the level of automated excellence provided by its mkinitcpio script; it is because we so rarely have any cause to understand it that so few of us do. Certainly I didn't anyway, until very recently.<br />
:::::::The initramfs is a file containing an image of the first filesystem mounted by the kernel after it is called. This image-file can be compiled into the kernel itself at build-time, but, because it can be more convenient to swap in other images at will, it is conventional instead to feed its path to the kernel as a parameter. The kernel expects the image's final archived format to be "cpio" (CoPy In/Out archives can handle items many others can't like /dev/, /sys/ and similar), but it can decompress that from any format you can compile into it (like lzma, lz4, gz, xz, etc). In its current iteration (as of kernel 2.6.something), when loaded the kernel mounts initramfs as "rootfs," which is basically the contents of this image mounted to / as a tmpfs.<br />
:::::::As far as the kernel is concerned its primary boot job is to find and call /init, and all the rest is up to user-space. And because the linux kernel can potentially handle more / disk configurations than can practically be accounted for in a single compile without growing exponentially every year, it was 2.6.something when initramfs became mandatory. For all intents and purposes, you can still compile in any modules required to mount your root filesystem, run /init from there, and skip both compiling as built-in and parameterizing an initramfs image at load-time, but the kernel still contains and mounts a (basically) empty / image no matter what you do. <br />
:::::::mkinitcpio's "base" hook is basically just busybox and a couple of shell scripts, one of which is /init. Arch's default initramfs behavior is pretty similar to most others; it packs in as little as necessary to find and mount your filesystem at /newroot then uses "switchroot" (the only really out-of-the-ordinary part of the whole process) to simultaneously dismount the initramfs /, mount /newroot in its place and call systemd.<br />
:::::::Ok, sorry for the lecture, but there's already enough here to warrant my coming back when I've forgotten any of it, so I figured i'd keep my future self on the right track if I could manage.<br />
:::::::Anyway, so have a look at the lsblk printouts at the bottom of bcache's wiki here: http://bcache.evilpiepirate.org/#index6h1<br />
:::::::Notice that after the array is built successfully (shown in the second lsblk output) the /dev/bcache{0,1} partitions are represented twice; they're children of both the physical disks on which they actually reside and the cache disk that buffers their i/o. The first lsblk, pulled before the bcache device to which they're attached is registered, is representative of a more typical configuration in which each partition is descended only from the physical device on which it resides.<br />
:::::::That added complexity is, as near as I can figure, what trips systemd's shutdown unmounts up. The bcache module, its user-space tools, and your / array are all called by the initramfs udev which is killed at "switchroot" when systemd loads so it can load and properly manage its own child udev process. <br />
:::::::systemd is designed from the ground up to be PID1, /init, the daemons' daemon. Through /sys/, it manages a pid tree with itself as the root reliably tracking and killing even forked child processes when the original parent processes quit. Asking it to undo complicated mount structures built before ever it can hook into /sys/, however, is probably asking for trouble.<br />
:::::::Still, as you say, dirty or not the data is still there. bcache is a disk cache after all, and it will persist between reboots in the cache on the ssd even if the array is not cleanly disassembled before the disks are unmounted. And bcache is specifically designed to track in its own btrees the cached data's final disk targets in order to reliably handle exactly these kinds of situations. I qualified my earlier description of the scenario as only "very slightly" unsafe because the redundancy is reduced. There is also the outside possibility that the cached data could be orphaned if the original backing store > cache attachment relationship written to the array members' bcache partition superblocks is somehow broken, which, at least when I configured my disks only a few months ago, was noted in the bcache docs as rare but possible. This last, I should say, I have never experienced, but, and only for reasons I can't exactly put my finger on, I have a hunch it could be related to the whole systemd dirty unmount scenario described above.<br />
:::::::I specifically cautioned against kexec before because it's sort of a "switchkernel" version of the "switchroot" process I referenced above. kexec first writes an OS kernel to system memory, then it simultaneously writes out the current kernel's mapped region and writes the to-be-loaded kernel over it, before finally, in its last gasp, as it were, calling the new kernel to execute. It seems to me that if the daemons' daemon has difficulty handling preexisting conditions then testing the OS kernel in such circumstances is probably not something I'd like to do.<br />
:::::::Geez. That's a lot. But there's more.<br />
:::::::While I can't say for certain because I haven't looked into it, I suspect that an lvm2 / is only contra-indicated for the systemd mkinitcpio hook because the particular combination has not yet been scripted to the same level of automatic detection and reliable configuration as the rest. There's a huge difference between reliably configuring a thing and reliably automating a configuration of course, and as I'm sure Arch's core maintainers are painfully aware. It is probably for precisely this reason that Arch enables the bcache kernel module in its default kernel compile but provides no supported means of making use of it. Still, as far as I know, systemd is not inherently incompatible with lvm2, and because the initramfs is just a / doing (mostly) plain old linuxy / things, there's no reason you can't set it up any way you please. Maybe get to know lsinitcpio a little better if you're interested.<br />
:::::::And here's one of perhaps special import to the AUR's bcache-tools package maintainer; while putting systemd in the initramfs is the one way I've handled the shutdown problem, I doubt very seriously it is the only way. Two other possibilities come immediately to mind: either hooking into or emulating in some way the mkinitcpio shutdown (or is it suspend?) hook which I expect must have some means of handling an end event or two; or, and probably the way i'd go, systemd's own shutdown target, which definitely does and is probably specifically designed to handle just such a thing. Probably there are many others as well, but none spring to mind.<br />
:::::::You can wake up now. I'm through.<br />
:::::::[[User:Mikeserv|Mikeserv]] ([[User talk:Mikeserv|talk]]) 05:56, 17 November 2013 (UTC)<br />
<br />
::::::::That was long! Perhaps we should take this conversation off the talk page so that we don't fill it up too much. Do you use IRC/gtalk/skype/facebook at all? I'd like to discuess some more in regards to the possiblity of a shutdown hook for bcache.</div>Justin8https://wiki.archlinux.org/index.php?title=Talk:Bcache&diff=283123Talk:Bcache2013-11-16T07:58:44Z<p>Justin8: </p>
<hr />
<div>==Example installation==<br />
Here is mine Bcache installation. Maybe you can use some idea's. :)<br />
<br />
=== Prepare the storage drive ===<br />
<br />
====Bcache-tools installation====<br />
fakeroot and binutils are only needed. but it will fail in a error if you only install them.<br />
wget is from a failed archbang instal :(<br />
<br />
pacman -Syy<br />
pacman -S base-devel fakeroot binutils wget git<br />
<br />
cd ~/<br />
wget https://aur.archlinux.org/packages/bc/bcache-tools-git/bcache-tools-git.tar.gz<br />
tar -xzvf bcache-tools-git.tar.gz<br />
cd bcache-tools-git<br />
makepkg -si --asroot<br />
<br />
==== Making the partitions ====<br />
For /boot i use the slower HDD because i had troubles that mine SSD loaded the kernel while mine HDD was still spinning up :P<br />
<br />
Starting with the faster SSD drive<br />
<br />
fdisk /dev/sda<br />
<br />
o<br />
y<br />
n<br />
Partition type: p<br />
Partition-number: 1<br />
First-sector: [enter]<br />
Last-sector: +90G<br />
<br />
n<br />
Partition type: p<br />
Partition-number: 3<br />
First-sector: [enter]<br />
Last-sector: [enter]<br />
<br />
write and exit<br />
<br />
Now the slower 2 TB Harddrive<br />
<br />
fdisk /dev/sdb<br />
<br />
o<br />
y<br />
n<br />
Partition type: p<br />
Partition-number: 1<br />
First-sector: [enter]<br />
Last-sector: +1G<br />
a<br />
<br />
n<br />
Partition type: p<br />
Partition-number: 2<br />
First-sector: [enter]<br />
Last-sector: +16G [enter]<br />
t<br />
2<br />
82<br />
<br />
n<br />
Partition type: p<br />
Partition-number: 3<br />
First-sector: [enter]<br />
Last-sector: [enter]<br />
<br />
write and exit<br />
<br />
==== Making Bcache Device ====<br />
The --wipe-bcache is to remove a error :)<br />
and the dd commands is or cleaning out old partition data.<br />
<br />
make-bcache --wipe-bcache -B /dev/sdb3 -C /dev/sda2<br />
dd if=/dev/zero count=1 bs=1024 seek=1 of=/dev/sda2<br />
dd if=/dev/zero count=1 bs=1024 seek=1 of=/dev/sdb3<br />
<br />
If next commands give a error: invalid argument<br />
Then the device is already registered.<br />
<br />
echo /dev/sda2 > /sys/fs/bcache/register<br />
echo /dev/sdb3 > /sys/fs/bcache/register<br />
<br />
==== Formating the drives ====<br />
<br />
<br />
mkfs.ext4 -E discard /dev/sda1<br />
mkfs.ext4 -E discard /dev/sdb1<br />
mkfs.ext4 /dev/bcache0<br />
mkswap /dev/sdb2<br />
swapon /dev/sdb2<br />
<br />
==== Mount the partitions ====<br />
<br />
cd ~/<br />
mkdir -p /mnt/install<br />
mount /dev/sda1 /mnt/install<br />
mkdir -p /mnt/install/{boot,home,srv,data,var,mnt/.hdd}<br />
mount /dev/sdb1 /mnt/install/boot<br />
mount /dev/bcache0 /mnt/install/mnt/.hdd<br />
mkdir -p /mnt/install/mnt/.hdd/{home,srv,data,var}<br />
mount -o bind /mnt/install/mnt/.hdd/home /mnt/install/home<br />
mount -o bind /mnt/install/mnt/.hdd/srv /mnt/install/srv<br />
mount -o bind /mnt/install/mnt/.hdd/data /mnt/install/data<br />
mount -o bind /mnt/install/mnt/.hdd/var /mnt/install/var<br />
<br />
==== Bcache Check After mounting ====<br />
Reason for /boot on the HDD is because of boot up failure because of HDD spin-up.<br />
<br />
lsblk<br />
<br />
NAME MAJ:MIN RM SIZE RO TYPE MOUNTPOINT<br />
sda 8:0 0 167.7G 0 disk <br />
|-sda1 8:1 0 90G 0 part /mnt/install<br />
`-sda2 8:2 0 77.7G 0 part <br />
`-bcache0 253:0 0 1.8T 0 disk /mnt/install/mnt/.hdd<br />
sdb 8:16 0 1.8T 0 disk <br />
|-sdb1 8:17 0 1G 0 part /mnt/install/boot<br />
|-sdb2 8:18 0 16G 0 part [SWAP]<br />
`-sdb3 8:19 0 1.8T 0 part <br />
`-bcache0 253:0 0 1.8T 0 disk /mnt/install/mnt/.hdd<br />
<br />
==== Generate an fstab ====<br />
<br />
Generate an fstab file with the following command. UUIDs will be used because they have certain advantages.<br />
<br />
genfstab -U -p /mnt/install >> /mnt/install/etc/fstab<br />
sed -i '/dev\/sda/ s/rw,relatime,data=ordered/defaults,noatime,discard/g' /mnt/install/etc/fstab<br />
sed -i '/mnt\/install/ {s/\/mnt\/install//g; s/rw,relatime,data=ordered/defaults/g; s/none/bind/g}' /mnt/install/etc/fstab<br />
more /mnt/install/etc/fstab<br />
<br />
=== Install and configure kernel and bootloader ===<br />
==== Adding Bcache module and hook ====<br />
<br />
Edit /etc/mkinitcpio.conf as needed and re-generate the initramfs image with:<br />
adding the "bcache" module<br />
adding the "bcache_udev" hook between block and filesystems<br />
<br />
sed -i '/^MODULES=/ s/"$/ bcache"/' /etc/mkinitcpio.conf<br />
sed -i '/^HOOKS=/ s/block filesystems/block bcache_udev filesystems/' /etc/mkinitcpio.conf<br />
more /etc/mkinitcpio.conf<br />
<br />
==== Registering and attaching Bcache ====<br />
This part is a pain in the A.. <br />
The problem is that Bcache wil give a error at startup if this is not done.<br />
<br />
ERROR: Unable to find root device 'UUID=b6b2d82b-f87e-44d5-bbc5-c51dd7aace15'.<br />
You are being dropped to a recovery shell<br />
Type 'exit' to try and continue booting<br />
<br />
First get the correct "SET UUID" with this ls command. <br />
And use the UUID like the example underneath.<br />
Also check for the use of the correct devices.<br />
echo: write error: Invalid argument <<< This error is good !!! means that your value was already stored.<br />
<br />
ls /sys/fs/bcache/<br />
2300f944-0eb5-4a9e-b052-8d5ea63cbb8f register register_quiet<br />
<br />
sudo echo /dev/sda2 > /sys/fs/bcache/register<br />
sudo echo /dev/sdb3 > /sys/fs/bcache/register<br />
sudo echo 2300f944-0eb5-4a9e-b052-8d5ea63cbb8f > /sys/block/sdb/sdb3/bcache/attach<br />
<br />
mv /usr/lib/initcpio/hooks/bcache ~/bcache.old<br />
cat >> /usr/lib/initcpio/hooks/bcache << EOF<br />
#!/usr/bin/ash<br />
run_hook() {<br />
echo /dev/sda2 > /sys/fs/bcache/register<br />
echo /dev/sdb3 > /sys/fs/bcache/register<br />
}<br />
EOF<br />
<br />
==== Installing the Kernel ==== <br />
mkinitcpio -p linux<br />
<br />
I hope some stuff is helpfull :P<br />
17:58, 11 November 2013 (UTC)<br />
:I'm curious about two things. First, though I forget what the '-E' switch signifies, it seems you're formatting the raw, underlayer partitions themselves. I don't understand why.<br />
<br />
:Also, the "pain in the a?" So udev is not assembling your array automatically as the devices are discovered? That is unexpected behavior, and I suspect it has to do with the formatting you performed and udev's superblock discovery. Check the very bottom of the bcache wiki on their site. At least try lsblk to verify "bcache" partition types are not registering as "ext4" before assembling the array.<br />
<br />
:Oh. And please sign talk pages.<br />
<br />
[[User:Mikeserv|Mikeserv]] ([[User talk:Mikeserv|talk]]) 03:35, 12 November 2013 (UTC)<br />
<br />
<br />
<br />
Well this is a experiment with Bcache with mine own Wiki page and 100x testing the install, i am not a linux expert just a newbie who was reading a lot of howto's. Just see it as a compilation of different web pages and some logical thinking. :)<br />
<br />
The -E is just a copy and past ... not sure if it was needed.I just thought it was needed to get the discard flag while formating.<br />
<br />
The pain in the A.... this was before Udev ... i was busy getting Bcache to work 3 months ago ... 75% of boot up failed when booting up cold. Later i found out that the SSD was too fast with booting up and the normal HDD was still spinning up.... After i splitted up the SSD for root directory and bcache and moved the /boot to the HDD since then i can bootup 100% of the time without any error... This was before the udev thingy (wich i dont understand and need to read up) :)<br />
<br />
So please understand that i am just a linux noob trying to contribute :)<br />
<br />
[[User:Emesix|Emesix]] ([[User talk:Emesix|talk]])[[User:Emesix|Emesix]] ([[User talk:Emesix|talk]]) 00:02, 13 November 2013 (UTC)<br />
<br />
:No, I get it, and I didn't mean to insinuate that you were doing anything wrong, exactly, just that, as designed, it could operate better.<br />
:Basically, and you should check the man-pages and the wiki page here to be sure just in case I'm not entirely correct (which does happen), udev is the program that runs during boot (and pretty much all of the rest of the time, too) to detect your hardware and load drivers as necessary. Udev is a relatively recent addition to the Linux boot process and is unique in that it discovers and handles hardware as it shows up, can handle multiple devices simultaneously with non-blocking parallell operations, and, as a result, is significantly faster than the methods it has replaced.<br />
:I was the one who added the udev rules and step 7b to this wiki, and I was the one who wrote the "Rogue Superblock" section of "Troubleshooting" at the bottom of bcache's wiki. I only gained the knowledge to write either after spending several frustrating days with an issue that appears very similar to your "pain in the a."<br />
:As step 7a shows, the prevailing Arch Linux method for reassembling a bcache array at boot used to look something like:<br />
: 1) Wait 5 secs. <br />
: 2) Check to see if udev has discovered and added {disk UUIDS}. If yes goto 3), no, repeat 1) and 2).<br />
: 3) echo {pain in the a stuff} /sys/bla/bla/bla<br />
:This does work, of course, but you're adding an unnecessary wait loop to your boot process and you're performing a task with brittle shell script that depends on specific UUIDs that could be performed flexibly and at discovery time.<br />
:That's what the udev rules do. Basically, the rules instruct udev to treat as part of your bcache array any partition it finds that reports itself as a "bcache" partition type. It builds the bcache array from disk partitions at every boot only as soon as those partitions are ready, which would of course resolve the race condition you mentioned and allow you to boot from SSD if you liked.<br />
:And here's where our experience might converge: I added a partition to my bcache array after previously formatting it as ext4. If you don't know about superblocks (or magic blocks or whatever they're called) then you're just like I was a few months ago. In short the filesystem has to have a way to report on itself, so it sets aside a small marker on disk that says, "Hey, OS, I'm a disk partition of type EXT4 and I start at block offset whatever and end at block offset whatever. Also, I enjoy long walks on the beach and prefer red wine to white. See you around."<br />
:The problem I experienced was that by default ext4 tended to begin at an earlier offset than bcache, and so even though I formatted over the previously ext4 partition with bcache as instructed, bcache never overwrote ext4's first superblock. When udev attempted to build my array, it would always miss my first partition because it would read the first superblock, identify the partition as ext4, then skip it. Echoing the add instruction to /sys/ after boot was my fix for a couple of days, but eventually I figured it all out and wrote that short bit on superblocks in bcache's wiki.<br />
:Maybe that helps?<br />
:EDIT: Just looked back at the wiki proper and somebody has performed some major changes recently. While it does appear cleaner, it no longer makes sense. There are references to non-existent steps throughout, and the actual why's that were included at least to some degree seem to have been occluded in favor of dubious, though possibly more efficient how's. Anyway, step 7b no longer exists, but it used to include the bcache_udev mkinitcpio hook I (very barely) adapted from mdadm_udev when trying to fix Arch's bcache boot problems. It's now included in the AUR package the wiki recommends I guess, despite the wiki also telling us we must "echo ... /sys/... at every bootup."<br />
[[User:Mikeserv|Mikeserv]] ([[User talk:Mikeserv|talk]]) 07:08, 14 November 2013 (UTC)<br />
<br />
Sorry about that, I missed a couple of references back to the sections I changed. Hopefully it makes more sense now. [[User:Mikeserv|Mikeserv's]] udev rule is crucial to having a working bcache device on boot, as [[User:Emesix|Emesix]] had discovered, it will often result in a failed boot. The echo in to /sys/bcache* step was missed when I updated it to mention the udev rule as the 'default' method.<br />
<br />
--[[User:Justin8|Justin8]] ([[User talk:Justin8|talk]]) 10:45, 14 November 2013 (UTC)<br />
:For clarity's sake, the udev rule was never mine. The used rule has been included in the AUR package's git pull from at least the time I first installed bcache some months ago. The package maintainer at that time opted not to use it, I guess, and instead installed the legacy-type shell-scripted for-loop mkinitcpio hook I outlined above, which was, to be fair, also the process recommended by every other source of information I was able to dig up at the time to include bcache's own wiki. Frustrated with having to wait for such things which seemed to me to defeat the whole purpose of buying and configuring an SSD boot device, I eventually stumbled upon the mdadm_udev hook in /use/share/initcpio and (only slightly) modified it to apply the 69-bcache-rules instead.<br />
:Justin, thanks for your recent change. NEVERMIND FOLLOWING: I'm curious why you specify "unless assembling after boot?" Why include anything to do with bcache in the initramfs at all if you're building the array after the boot sequence? MAKES PERFECT SENSE OF COURSE. GOT A 1 TRACK MIND SOMETIMES I GUESS. SORRY.<br />
[[User:Mikeserv|Mikeserv]] ([[User talk:Mikeserv|talk]]) 11:11, 14 November 2013 (UTC)<br />
<br />
:Emesix, just looked closer at your instructions, and you actually do the thing I had to do and outlined in the bcache wiki to overwrite the rogue superblock here:<br />
:{{ic|<nowiki>dd if=/dev/zero count=1 bs=1024 seek=1 of=/dev/sd{a2,b3}</nowiki>}}<br />
:And the ext4 formatting you do after that I earlier incorrectly assumed to be for the "raw, underlayer partitions" is revealed upon further inspection to be for non-bcache partitions and so isn't even relevant; I apologize for jumping to conclusions before reviewing it fully, the quick check I did a couple of days ago just struck me as really familiar.<br />
:The udev stuff above should still be relevant, and it might be worth noting that the particular {{ic|dd}} command above is by no means a cure-all. It should only be used as written if you have previously identified a superblock located at byte offset 1024 that needs overwriting, though the actual superblock location can vary by filesystem. I've since learned that a much more flexible and useful tool for this is {{ic|wipefs}}, which is much less dangerous than it sounds.<br />
: [[User:Mikeserv|Mikeserv]] ([[User talk:Mikeserv|talk]]) 14:28, 14 November 2013 (UTC)<br />
<br />
Thanks for the explanation of the super-block [[User:Mikeserv|Mikeserv's]] was very insightful. But that Udev stuff is still a layer to high for me :P I don't know if its possible to do a little differently to avoid a unwanted wait loop:<br />
: 1) Check to see if udev has discovered and added {disk UUIDS}. If yes goto 3)<br />
: 2) Wait 5 secs and goto 1.<br />
: 3) echo {pain in the a stuff} /sys/bla/bla/bla<br />
The dd command is just a safety step. :) And because formatting the /dev/bcache0 was soon to follow, it didn't hurt if i accidentally killed the file system on the bcache :) I got the feeling that resizing the bcache partition was also giving problems wich the dd command fixed..... BTW i like the piece added with this command set:<br />
# cd /sys/fs/bcache<br />
# echo "UUID" > /sys/block/bcache0/bcache/attach<br />
p.s. I don't get why the wiki page is full of btrfs stuff. Is this not a combination of commands? And shouldn't a more conventional partition/file system scheme be more useful for novice users (like me) ??<br />
: [[User:Emesix|Emesix]] ([[User talk:Emesix|talk]]) 22:47, 14 November 2013 (UTC)<br />
<br />
::Regarding btrfs and its usefulness for new users... I don't know. With btrfs for instance, there are many normally high-level file-system concepts rendered nearly as mundane as directory maintenance like snapshotting and subvolumes, and still others that require only a single edit to a boot config like {{ic|autodefrag}} and {{ic|<nowiki>compress=lzo</nowiki>}} delivering inline filesystem compression and cleanup. In practice, you really don't have to understand the hows of these things at all to get incredible use out of them. <br />
::Consider just sub volumes: they can deliver, at the very least, a separated partition structure for the various segments of the Linux filesystem as is often recommended people set at installation with syntax and simplicity like unto {{ic|cd}} and {{ic|mkdir}}, only the {{ic|btrfs --help}} implementation is probably a little friendlier. With traditional filesystems this same setup required knowledge of disk geometry, extent counts and all manner of other arcane things to configure in any sanely optimized way. Btrfs will create you a RAID in less than 30 seconds. <br />
::I guess what I'm saying is the research vs reward ratio is pretty significantly in your favor with a btrfs disk. Of course, should the btrfs filesystem crash, well, you should probably call someone else is all I'll say about that.<br />
::In this wiki, I definitely see your point, especially considering the added complexity bcache brings. When I was trying to set mine up I spent a good deal of time in both the btrfs and bcache IRC dev channels and both sides were less than optimistic about friendly cooperation between the two. Still, it works and has done for months. I know, don't call you, right?<br />
::Now about udev, I'll try again. So, your script implements a wait loop to check occasionally if udev has done a thing (bring up the disks and pull in basic drivers like SCSI for block devices and ext4 for their component filesystems) so it can do a thing (register the disks as small parts of a larger whole and pull in the bcache driver). The udev rules eschew the notion of a secondary monitor script such as yours and instead have udev do it all (any disk it brings up as a "bcache" type is registered as a small part of a larger whole and the bcache driver is pulled in). <br />
::So you couldn't boot from SSD reliably before because you couldn't reliably script the order in which udev would bring up the disks (a consequence of its faster parallel operations), but udev doesn't have to script its order, udev just udevs as udev udevs best.<br />
::See? You will always boot reliably from your chosen device because udev won't misinterpret its own report on your chosen device's readiness. And that's all - there's nothing wrong with the shell script exactly, except that it's just plain unnecessary and added complication.<br />
[[User:Mikeserv|Mikeserv]] ([[User talk:Mikeserv|talk]]) 23:38, 14 November 2013 (UTC)<br />
<br />
:: I have to say, I want to try out using btrfs instead of LVM on my desktop sometime, but the cache article is not the place for it. Mentioning it as a good alternative on the installation and beginners guide and having details in the btrfs article is going to be more productive and logical. I added the section near the top of the page for configuring a simple cache setup with no regards to file systems and partition layouts as they should be beyond the scope of this article. <br />
<br />
--[[User:Justin8|Justin8]] ([[User talk:Justin8|talk]]) 02:13, 15 November 2013 (UTC)<br />
::Agreed. There's a lot of it, too. I guess that sometimes happens on less visited pages. Probably it will level out as the module is more established. I compromised on some of the grub stuff in a minor edit yesterday after you rightly pointed out how much unrelated stuff is in there. Never thought about it before, but I've only ever written 7b and 2 notes here and a short section on UEFI kernel update syncing.<br />
::Above is just about everything I know relatable to bcache by now, I guess. I wish there was some more in the wiki here on benchmarking. The bcache wiki itself reads like a stereo repair manual on the subject. I never have managed to wrap my mind round it before my eyes would shut of their own accord. <br />
[[User:Mikeserv|Mikeserv]] ([[User talk:Mikeserv|talk]]) 03:00, 15 November 2013 (UTC)<br />
<br />
:: Ha, that is the perfect description for their wiki; it's all failure modes and fixes but no reasoning. From what I could see, benchmarking of bcache is really only useful to benchmark writes; reads will vary exponentially depending on whether or not you get a cache hit. On top of that, sequential reads/writes bypass the cache. the only real way to benchmark it would be in 'real-world' style tests instead of synthetic benchmarks. I.e. how long it takes to boot your system. how long it takes to compile X, Y or Z. I put a little bit up on my blog when testing out bcache, mostly focussing on IOps performance changes since everything else is relatively the same with or without bcache (other than the smart re-ordering of metadata changes) I was going to just paste the graphs, but they are all js objects; you can see the results I got here: https://www.dray.be/?p=7<br />
[[User:Justin8|Justin8]] ([[User talk:Justin8|talk]]) 04:05, 15 November 2013 (UTC)<br />
<br />
:::Justin: so you're the AUR bcache-tools maintainer now? I guess you dropped enough hints. That's cool. Reading through your blog (pretty colors) I was reminded of another little bcache fact that may not be immediately apparent to all. It seems most tend to assume that the cache:backing-store must be on a 1:1 ratio. This is not the case. A single cache can serve as many storage devices as you might like. Conversely, there's nothing preventing one from configuring several mechanical discs in a RAID and then backing it on a 1:1 ratio. Regarding a cached RAID, though, I do recall reading a bcache IRC conversation in which a dev mentioned hardware configuration of the device would be desirable as bcache is designed to interface with a block layer and dropping a filesystem below it can have unexpected results. <br />
:::I only bring this up because your blog mentions you will switch from a 1tb to a 3tb soon. Personally, I use a 120GB Kingston SSD to cache 2 3tb WD Greens which are then formatted as a single btrfs RAID1 metadata/RAID0 data. The data is mostly downloaded video for a home theater server and as such is largely expendable in the event of catastrophe. Probably you know this, but just in case, I thought I'd point out that there's nothing preventing you from merely adding the the 3tb to the 1tb as you please.<br />
:::Lastly, you can format and configure an entire array with basically one line like: {{ic|<nowiki>make-bcache -B /dev/{1tb} /dev/{new_3tb} -C /dev/{60GB_SSD}</nowiki>}}. Then just reboot (or even reload udev) if you've got the udev rules loaded - udev will register the devices for you based on the auto-attach data recorded in the partition's superblocks as a result of the all-in-one add command. You wouldn't even need the bcache_udev hook or anything else in initramfs at this point, because obviously you shouldn't expect to boot to the bcache array yet as you haven't had a chance to format it with a usable filesystem or to copy any files to it, but it can simplify things.<br />
:::[[User:Mikeserv|Mikeserv]] ([[User talk:Mikeserv|talk]]) 07:04, 15 November 2013 (UTC)<br />
<br />
::::I did notice a reference to that from the previous maintainer in the aur comments. I currently have a 9x2tb RAID6 mdadm array that I really don't want to have to backup and restore, so I did the tests as it would be in my server (just on a smaller scale). I don't believe it would be a simple task to remove the device, bcache-ify it and put it back in. The blocks utility says that it supports it for any block device; but mdadm stores data about the offset in the superblocks instead of using the partition layout (which I discovered the hard way in the past and had to restore 10TB of backups from off-site :( ) It should be fine though doing it to the MDADM device itself. I wouldn't mind knowing the performance difference, so once I get an SSD for my server I might test it with both methods to do a bit of a comparison.<br />
::::I didn't realize you could specify the backing and cache devices all in one line like that. I didn't see it in the wiki at all when I did my first lot of edits; seems useful. Does it automatically attach the cache to the backing devices as well?<br />
::::Although I reboot fairly often for things like kernel updates/making sure pacman hasn't broken things, I am somewhat averse to rebooting when I don't have to. I migrated from a root device on an SSD and /home and /opt on a 3TB disk, on to a spare 1TB disk, did the tests for my blog on a different computer and then created the bcache device with the 3TB disk and 60GB SSD and migrated back all without a reboot. Sometimes rebooting can be easier, but it's more 'fun' not to :)<br />
::::[[User:Justin8|Justin8]] ([[User talk:Justin8|talk]]) 08:28, 15 November 2013 (UTC)<br />
:::::Well, as I mentioned, you could opt to restart udev rather than reboot, or alternatively you could choose to throw a {{ic|udevadm}} trigger, or to have some fun with {{ic|kexec}}, or merely to {{ic|echo}} the block-device register commands to {{ic|/sys/}} as per usual. You have correctly surmised that the one-liner auto-attaches the backing-stores to the cache, but it does not perform the registration, which is necessary anyway every time an array is re/assembled as it is.<br />
::::: I added it to the bcache Arch wiki page myself after discovering it quite by accident between nods while browsing bcache's own documentation a few months ago. The command is represented in the wiki even now, if rather humbly as a side note to the initial backing-store formatting process. I mentioned it on your talk page as well, because you had apparently previously edited it not to fix its then anachronistic reference to no longer needing a now completely different step 4, but to note that it would no longer be necessary at boot. At first I was bewildered, but I've since corrected it to note that it now obviates step 6.<br />
:::::By the way, I might know of another thing worth saying, of which {{ic|kexec}}'s mention has reminded me: because of the way SystemD performs its unmounts at shutdown, it can be very slightly dangerous to run a multi-device write-back cache outside of its purview, as you must do if you boot to a bcache array but have not incorporated SystemD into your initramfs. If this is your situation currently, then likely you will be able to catch an occasional error message printed to the console about a forced unmount in the second or two just before your screen blanks its last. Of course bcache is supposed to be able to handle this and much more, and it never caused me any issues (though I run in write-through mode), but I did switch to the new SystemD hook just as soon as I noticed it. Specifically the SystemD {{ic|kexec}} function should almost certainly be avoided whenever possible when operating a write-back cache without a true PID1 SystemD configuration.<br />
:::::[[User:Mikeserv|Mikeserv]] ([[User talk:Mikeserv|talk]]) 09:13, 15 November 2013 (UTC)<br />
<br />
::::::That is interesting. (I assume you're talking about the systemd hook for mkinitcpio that is intended to replace udev/usr/base/etc) How does using that new systemd hook fix it unmounting drives too soon? Also the wiki notes that it is incompatible with lvm2 currently; which just so happens to be what my root filesystem is using... Maybe it is yet another reason to change to btrfs finally. I am using writeback on my bcache device as well. But so long as the cache device is available on a reboot it shouldn't affect anything.<br />
::::::[[User:Justin8|Justin8]] ([[User talk:Justin8|talk]]) 07:58, 16 November 2013 (UTC)</div>Justin8https://wiki.archlinux.org/index.php?title=Talk:Bcache&diff=282930Talk:Bcache2013-11-15T08:28:49Z<p>Justin8: </p>
<hr />
<div>==Example installation==<br />
Here is mine Bcache installation. Maybe you can use some idea's. :)<br />
<br />
=== Prepare the storage drive ===<br />
<br />
====Bcache-tools installation====<br />
fakeroot and binutils are only needed. but it will fail in a error if you only install them.<br />
wget is from a failed archbang instal :(<br />
<br />
pacman -Syy<br />
pacman -S base-devel fakeroot binutils wget git<br />
<br />
cd ~/<br />
wget https://aur.archlinux.org/packages/bc/bcache-tools-git/bcache-tools-git.tar.gz<br />
tar -xzvf bcache-tools-git.tar.gz<br />
cd bcache-tools-git<br />
makepkg -si --asroot<br />
<br />
==== Making the partitions ====<br />
For /boot i use the slower HDD because i had troubles that mine SSD loaded the kernel while mine HDD was still spinning up :P<br />
<br />
Starting with the faster SSD drive<br />
<br />
fdisk /dev/sda<br />
<br />
o<br />
y<br />
n<br />
Partition type: p<br />
Partition-number: 1<br />
First-sector: [enter]<br />
Last-sector: +90G<br />
<br />
n<br />
Partition type: p<br />
Partition-number: 3<br />
First-sector: [enter]<br />
Last-sector: [enter]<br />
<br />
write and exit<br />
<br />
Now the slower 2 TB Harddrive<br />
<br />
fdisk /dev/sdb<br />
<br />
o<br />
y<br />
n<br />
Partition type: p<br />
Partition-number: 1<br />
First-sector: [enter]<br />
Last-sector: +1G<br />
a<br />
<br />
n<br />
Partition type: p<br />
Partition-number: 2<br />
First-sector: [enter]<br />
Last-sector: +16G [enter]<br />
t<br />
2<br />
82<br />
<br />
n<br />
Partition type: p<br />
Partition-number: 3<br />
First-sector: [enter]<br />
Last-sector: [enter]<br />
<br />
write and exit<br />
<br />
==== Making Bcache Device ====<br />
The --wipe-bcache is to remove a error :)<br />
and the dd commands is or cleaning out old partition data.<br />
<br />
make-bcache --wipe-bcache -B /dev/sdb3 -C /dev/sda2<br />
dd if=/dev/zero count=1 bs=1024 seek=1 of=/dev/sda2<br />
dd if=/dev/zero count=1 bs=1024 seek=1 of=/dev/sdb3<br />
<br />
If next commands give a error: invalid argument<br />
Then the device is already registered.<br />
<br />
echo /dev/sda2 > /sys/fs/bcache/register<br />
echo /dev/sdb3 > /sys/fs/bcache/register<br />
<br />
==== Formating the drives ====<br />
<br />
<br />
mkfs.ext4 -E discard /dev/sda1<br />
mkfs.ext4 -E discard /dev/sdb1<br />
mkfs.ext4 /dev/bcache0<br />
mkswap /dev/sdb2<br />
swapon /dev/sdb2<br />
<br />
==== Mount the partitions ====<br />
<br />
cd ~/<br />
mkdir -p /mnt/install<br />
mount /dev/sda1 /mnt/install<br />
mkdir -p /mnt/install/{boot,home,srv,data,var,mnt/.hdd}<br />
mount /dev/sdb1 /mnt/install/boot<br />
mount /dev/bcache0 /mnt/install/mnt/.hdd<br />
mkdir -p /mnt/install/mnt/.hdd/{home,srv,data,var}<br />
mount -o bind /mnt/install/mnt/.hdd/home /mnt/install/home<br />
mount -o bind /mnt/install/mnt/.hdd/srv /mnt/install/srv<br />
mount -o bind /mnt/install/mnt/.hdd/data /mnt/install/data<br />
mount -o bind /mnt/install/mnt/.hdd/var /mnt/install/var<br />
<br />
==== Bcache Check After mounting ====<br />
Reason for /boot on the HDD is because of boot up failure because of HDD spin-up.<br />
<br />
lsblk<br />
<br />
NAME MAJ:MIN RM SIZE RO TYPE MOUNTPOINT<br />
sda 8:0 0 167.7G 0 disk <br />
|-sda1 8:1 0 90G 0 part /mnt/install<br />
`-sda2 8:2 0 77.7G 0 part <br />
`-bcache0 253:0 0 1.8T 0 disk /mnt/install/mnt/.hdd<br />
sdb 8:16 0 1.8T 0 disk <br />
|-sdb1 8:17 0 1G 0 part /mnt/install/boot<br />
|-sdb2 8:18 0 16G 0 part [SWAP]<br />
`-sdb3 8:19 0 1.8T 0 part <br />
`-bcache0 253:0 0 1.8T 0 disk /mnt/install/mnt/.hdd<br />
<br />
==== Generate an fstab ====<br />
<br />
Generate an fstab file with the following command. UUIDs will be used because they have certain advantages.<br />
<br />
genfstab -U -p /mnt/install >> /mnt/install/etc/fstab<br />
sed -i '/dev\/sda/ s/rw,relatime,data=ordered/defaults,noatime,discard/g' /mnt/install/etc/fstab<br />
sed -i '/mnt\/install/ {s/\/mnt\/install//g; s/rw,relatime,data=ordered/defaults/g; s/none/bind/g}' /mnt/install/etc/fstab<br />
more /mnt/install/etc/fstab<br />
<br />
=== Install and configure kernel and bootloader ===<br />
==== Adding Bcache module and hook ====<br />
<br />
Edit /etc/mkinitcpio.conf as needed and re-generate the initramfs image with:<br />
adding the "bcache" module<br />
adding the "bcache_udev" hook between block and filesystems<br />
<br />
sed -i '/^MODULES=/ s/"$/ bcache"/' /etc/mkinitcpio.conf<br />
sed -i '/^HOOKS=/ s/block filesystems/block bcache_udev filesystems/' /etc/mkinitcpio.conf<br />
more /etc/mkinitcpio.conf<br />
<br />
==== Registering and attaching Bcache ====<br />
This part is a pain in the A.. <br />
The problem is that Bcache wil give a error at startup if this is not done.<br />
<br />
ERROR: Unable to find root device 'UUID=b6b2d82b-f87e-44d5-bbc5-c51dd7aace15'.<br />
You are being dropped to a recovery shell<br />
Type 'exit' to try and continue booting<br />
<br />
First get the correct "SET UUID" with this ls command. <br />
And use the UUID like the example underneath.<br />
Also check for the use of the correct devices.<br />
echo: write error: Invalid argument <<< This error is good !!! means that your value was already stored.<br />
<br />
ls /sys/fs/bcache/<br />
2300f944-0eb5-4a9e-b052-8d5ea63cbb8f register register_quiet<br />
<br />
sudo echo /dev/sda2 > /sys/fs/bcache/register<br />
sudo echo /dev/sdb3 > /sys/fs/bcache/register<br />
sudo echo 2300f944-0eb5-4a9e-b052-8d5ea63cbb8f > /sys/block/sdb/sdb3/bcache/attach<br />
<br />
mv /usr/lib/initcpio/hooks/bcache ~/bcache.old<br />
cat >> /usr/lib/initcpio/hooks/bcache << EOF<br />
#!/usr/bin/ash<br />
run_hook() {<br />
echo /dev/sda2 > /sys/fs/bcache/register<br />
echo /dev/sdb3 > /sys/fs/bcache/register<br />
}<br />
EOF<br />
<br />
==== Installing the Kernel ==== <br />
mkinitcpio -p linux<br />
<br />
I hope some stuff is helpfull :P<br />
17:58, 11 November 2013 (UTC)<br />
:I'm curious about two things. First, though I forget what the '-E' switch signifies, it seems you're formatting the raw, underlayer partitions themselves. I don't understand why.<br />
<br />
:Also, the "pain in the a?" So udev is not assembling your array automatically as the devices are discovered? That is unexpected behavior, and I suspect it has to do with the formatting you performed and udev's superblock discovery. Check the very bottom of the bcache wiki on their site. At least try lsblk to verify "bcache" partition types are not registering as "ext4" before assembling the array.<br />
<br />
:Oh. And please sign talk pages.<br />
<br />
[[User:Mikeserv|Mikeserv]] ([[User talk:Mikeserv|talk]]) 03:35, 12 November 2013 (UTC)<br />
<br />
<br />
<br />
Well this is a experiment with Bcache with mine own Wiki page and 100x testing the install, i am not a linux expert just a newbie who was reading a lot of howto's. Just see it as a compilation of different web pages and some logical thinking. :)<br />
<br />
The -E is just a copy and past ... not sure if it was needed.I just thought it was needed to get the discard flag while formating.<br />
<br />
The pain in the A.... this was before Udev ... i was busy getting Bcache to work 3 months ago ... 75% of boot up failed when booting up cold. Later i found out that the SSD was too fast with booting up and the normal HDD was still spinning up.... After i splitted up the SSD for root directory and bcache and moved the /boot to the HDD since then i can bootup 100% of the time without any error... This was before the udev thingy (wich i dont understand and need to read up) :)<br />
<br />
So please understand that i am just a linux noob trying to contribute :)<br />
<br />
[[User:Emesix|Emesix]] ([[User talk:Emesix|talk]])[[User:Emesix|Emesix]] ([[User talk:Emesix|talk]]) 00:02, 13 November 2013 (UTC)<br />
<br />
:No, I get it, and I didn't mean to insinuate that you were doing anything wrong, exactly, just that, as designed, it could operate better.<br />
:Basically, and you should check the man-pages and the wiki page here to be sure just in case I'm not entirely correct (which does happen), udev is the program that runs during boot (and pretty much all of the rest of the time, too) to detect your hardware and load drivers as necessary. Udev is a relatively recent addition to the Linux boot process and is unique in that it discovers and handles hardware as it shows up, can handle multiple devices simultaneously with non-blocking parallell operations, and, as a result, is significantly faster than the methods it has replaced.<br />
:I was the one who added the udev rules and step 7b to this wiki, and I was the one who wrote the "Rogue Superblock" section of "Troubleshooting" at the bottom of bcache's wiki. I only gained the knowledge to write either after spending several frustrating days with an issue that appears very similar to your "pain in the a."<br />
:As step 7a shows, the prevailing Arch Linux method for reassembling a bcache array at boot used to look something like:<br />
: 1) Wait 5 secs. <br />
: 2) Check to see if udev has discovered and added {disk UUIDS}. If yes goto 3), no, repeat 1) and 2).<br />
: 3) echo {pain in the a stuff} /sys/bla/bla/bla<br />
:This does work, of course, but you're adding an unnecessary wait loop to your boot process and you're performing a task with brittle shell script that depends on specific UUIDs that could be performed flexibly and at discovery time.<br />
:That's what the udev rules do. Basically, the rules instruct udev to treat as part of your bcache array any partition it finds that reports itself as a "bcache" partition type. It builds the bcache array from disk partitions at every boot only as soon as those partitions are ready, which would of course resolve the race condition you mentioned and allow you to boot from SSD if you liked.<br />
:And here's where our experience might converge: I added a partition to my bcache array after previously formatting it as ext4. If you don't know about superblocks (or magic blocks or whatever they're called) then you're just like I was a few months ago. In short the filesystem has to have a way to report on itself, so it sets aside a small marker on disk that says, "Hey, OS, I'm a disk partition of type EXT4 and I start at block offset whatever and end at block offset whatever. Also, I enjoy long walks on the beach and prefer red wine to white. See you around."<br />
:The problem I experienced was that by default ext4 tended to begin at an earlier offset than bcache, and so even though I formatted over the previously ext4 partition with bcache as instructed, bcache never overwrote ext4's first superblock. When udev attempted to build my array, it would always miss my first partition because it would read the first superblock, identify the partition as ext4, then skip it. Echoing the add instruction to /sys/ after boot was my fix for a couple of days, but eventually I figured it all out and wrote that short bit on superblocks in bcache's wiki.<br />
:Maybe that helps?<br />
:EDIT: Just looked back at the wiki proper and somebody has performed some major changes recently. While it does appear cleaner, it no longer makes sense. There are references to non-existent steps throughout, and the actual why's that were included at least to some degree seem to have been occluded in favor of dubious, though possibly more efficient how's. Anyway, step 7b no longer exists, but it used to include the bcache_udev mkinitcpio hook I (very barely) adapted from mdadm_udev when trying to fix Arch's bcache boot problems. It's now included in the AUR package the wiki recommends I guess, despite the wiki also telling us we must "echo ... /sys/... at every bootup."<br />
[[User:Mikeserv|Mikeserv]] ([[User talk:Mikeserv|talk]]) 07:08, 14 November 2013 (UTC)<br />
<br />
Sorry about that, I missed a couple of references back to the sections I changed. Hopefully it makes more sense now. [[User:Mikeserv|Mikeserv's]] udev rule is crucial to having a working bcache device on boot, as [[User:Emesix|Emesix]] had discovered, it will often result in a failed boot. The echo in to /sys/bcache* step was missed when I updated it to mention the udev rule as the 'default' method.<br />
<br />
--[[User:Justin8|Justin8]] ([[User talk:Justin8|talk]]) 10:45, 14 November 2013 (UTC)<br />
:For clarity's sake, the udev rule was never mine. The used rule has been included in the AUR package's git pull from at least the time I first installed bcache some months ago. The package maintainer at that time opted not to use it, I guess, and instead installed the legacy-type shell-scripted for-loop mkinitcpio hook I outlined above, which was, to be fair, also the process recommended by every other source of information I was able to dig up at the time to include bcache's own wiki. Frustrated with having to wait for such things which seemed to me to defeat the whole purpose of buying and configuring an SSD boot device, I eventually stumbled upon the mdadm_udev hook in /use/share/initcpio and (only slightly) modified it to apply the 69-bcache-rules instead.<br />
:Justin, thanks for your recent change. NEVERMIND FOLLOWING: I'm curious why you specify "unless assembling after boot?" Why include anything to do with bcache in the initramfs at all if you're building the array after the boot sequence? MAKES PERFECT SENSE OF COURSE. GOT A 1 TRACK MIND SOMETIMES I GUESS. SORRY.<br />
[[User:Mikeserv|Mikeserv]] ([[User talk:Mikeserv|talk]]) 11:11, 14 November 2013 (UTC)<br />
<br />
:Emesix, just looked closer at your instructions, and you actually do the thing I had to do and outlined in the bcache wiki to overwrite the rogue superblock here:<br />
:{{ic|<nowiki>dd if=/dev/zero count=1 bs=1024 seek=1 of=/dev/sd{a2,b3}</nowiki>}}<br />
:And the ext4 formatting you do after that I earlier incorrectly assumed to be for the "raw, underlayer partitions" is revealed upon further inspection to be for non-bcache partitions and so isn't even relevant; I apologize for jumping to conclusions before reviewing it fully, the quick check I did a couple of days ago just struck me as really familiar.<br />
:The udev stuff above should still be relevant, and it might be worth noting that the particular {{ic|dd}} command above is by no means a cure-all. It should only be used as written if you have previously identified a superblock located at byte offset 1024 that needs overwriting, though the actual superblock location can vary by filesystem. I've since learned that a much more flexible and useful tool for this is {{ic|wipefs}}, which is much less dangerous than it sounds.<br />
: [[User:Mikeserv|Mikeserv]] ([[User talk:Mikeserv|talk]]) 14:28, 14 November 2013 (UTC)<br />
<br />
Thanks for the explanation of the super-block [[User:Mikeserv|Mikeserv's]] was very insightful. But that Udev stuff is still a layer to high for me :P I don't know if its possible to do a little differently to avoid a unwanted wait loop:<br />
: 1) Check to see if udev has discovered and added {disk UUIDS}. If yes goto 3)<br />
: 2) Wait 5 secs and goto 1.<br />
: 3) echo {pain in the a stuff} /sys/bla/bla/bla<br />
The dd command is just a safety step. :) And because formatting the /dev/bcache0 was soon to follow, it didn't hurt if i accidentally killed the file system on the bcache :) I got the feeling that resizing the bcache partition was also giving problems wich the dd command fixed..... BTW i like the piece added with this command set:<br />
# cd /sys/fs/bcache<br />
# echo "UUID" > /sys/block/bcache0/bcache/attach<br />
p.s. I don't get why the wiki page is full of btrfs stuff. Is this not a combination of commands? And shouldn't a more conventional partition/file system scheme be more useful for novice users (like me) ??<br />
: [[User:Emesix|Emesix]] ([[User talk:Emesix|talk]]) 22:47, 14 November 2013 (UTC)<br />
<br />
::Regarding btrfs and its usefulness for new users... I don't know. With btrfs for instance, there are many normally high-level file-system concepts rendered nearly as mundane as directory maintenance like snapshotting and subvolumes, and still others that require only a single edit to a boot config like {{ic|autodefrag}} and {{ic|<nowiki>compress=lzo</nowiki>}} delivering inline filesystem compression and cleanup. In practice, you really don't have to understand the hows of these things at all to get incredible use out of them. <br />
::Consider just sub volumes: they can deliver, at the very least, a separated partition structure for the various segments of the Linux filesystem as is often recommended people set at installation with syntax and simplicity like unto {{ic|cd}} and {{ic|mkdir}}, only the {{ic|btrfs --help}} implementation is probably a little friendlier. With traditional filesystems this same setup required knowledge of disk geometry, extent counts and all manner of other arcane things to configure in any sanely optimized way. Btrfs will create you a RAID in less than 30 seconds. <br />
::I guess what I'm saying is the research vs reward ratio is pretty significantly in your favor with a btrfs disk. Of course, should the btrfs filesystem crash, well, you should probably call someone else is all I'll say about that.<br />
::In this wiki, I definitely see your point, especially considering the added complexity bcache brings. When I was trying to set mine up I spent a good deal of time in both the btrfs and bcache IRC dev channels and both sides were less than optimistic about friendly cooperation between the two. Still, it works and has done for months. I know, don't call you, right?<br />
::Now about udev, I'll try again. So, your script implements a wait loop to check occasionally if udev has done a thing (bring up the disks and pull in basic drivers like SCSI for block devices and ext4 for their component filesystems) so it can do a thing (register the disks as small parts of a larger whole and pull in the bcache driver). The udev rules eschew the notion of a secondary monitor script such as yours and instead have udev do it all (any disk it brings up as a "bcache" type is registered as a small part of a larger whole and the bcache driver is pulled in). <br />
::So you couldn't boot from SSD reliably before because you couldn't reliably script the order in which udev would bring up the disks (a consequence of its faster parallel operations), but udev doesn't have to script its order, udev just udevs as udev udevs best.<br />
::See? You will always boot reliably from your chosen device because udev won't misinterpret its own report on your chosen device's readiness. And that's all - there's nothing wrong with the shell script exactly, except that it's just plain unnecessary and added complication.<br />
[[User:Mikeserv|Mikeserv]] ([[User talk:Mikeserv|talk]]) 23:38, 14 November 2013 (UTC)<br />
<br />
:: I have to say, I want to try out using btrfs instead of LVM on my desktop sometime, but the cache article is not the place for it. Mentioning it as a good alternative on the installation and beginners guide and having details in the btrfs article is going to be more productive and logical. I added the section near the top of the page for configuring a simple cache setup with no regards to file systems and partition layouts as they should be beyond the scope of this article. <br />
<br />
--[[User:Justin8|Justin8]] ([[User talk:Justin8|talk]]) 02:13, 15 November 2013 (UTC)<br />
::Agreed. There's a lot of it, too. I guess that sometimes happens on less visited pages. Probably it will level out as the module is more established. I compromised on some of the grub stuff in a minor edit yesterday after you rightly pointed out how much unrelated stuff is in there. Never thought about it before, but I've only ever written 7b and 2 notes here and a short section on UEFI kernel update syncing.<br />
::Above is just about everything I know relatable to bcache by now, I guess. I wish there was some more in the wiki here on benchmarking. The bcache wiki itself reads like a stereo repair manual on the subject. I never have managed to wrap my mind round it before my eyes would shut of their own accord. <br />
[[User:Mikeserv|Mikeserv]] ([[User talk:Mikeserv|talk]]) 03:00, 15 November 2013 (UTC)<br />
<br />
:: Ha, that is the perfect description for their wiki; it's all failure modes and fixes but no reasoning. From what I could see, benchmarking of bcache is really only useful to benchmark writes; reads will vary exponentially depending on whether or not you get a cache hit. On top of that, sequential reads/writes bypass the cache. the only real way to benchmark it would be in 'real-world' style tests instead of synthetic benchmarks. I.e. how long it takes to boot your system. how long it takes to compile X, Y or Z. I put a little bit up on my blog when testing out bcache, mostly focussing on IOps performance changes since everything else is relatively the same with or without bcache (other than the smart re-ordering of metadata changes) I was going to just paste the graphs, but they are all js objects; you can see the results I got here: https://www.dray.be/?p=7<br />
[[User:Justin8|Justin8]] ([[User talk:Justin8|talk]]) 04:05, 15 November 2013 (UTC)<br />
<br />
:::Justin: so you're the AUR bcache-tools maintainer now? I guess you dropped enough hints. That's cool. Reading through your blog (pretty colors) I was reminded of another little bcache fact that may not be immediately apparent to all. It seems most tend to assume that the cache:backing-store must be on a 1:1 ratio. This is not the case. A single cache can serve as many storage devices as you might like. Conversely, there's nothing preventing one from configuring several mechanical discs in a RAID and then backing it on a 1:1 ratio. Regarding a cached RAID, though, I do recall reading a bcache IRC conversation in which a dev mentioned hardware configuration of the device would be desirable as bcache is designed to interface with a block layer and dropping a filesystem below it can have unexpected results. <br />
:::I only bring this up because your blog mentions you will switch from a 1tb to a 3tb soon. Personally, I use a 120GB Kingston SSD to cache 2 3tb WD Greens which are then formatted as a single btrfs RAID1 metadata/RAID0 data. The data is mostly downloaded video for a home theater server and as such is largely expendable in the event of catastrophe. Probably you know this, but just in case, I thought I'd point out that there's nothing preventing you from merely adding the the 3tb to the 1tb as you please.<br />
:::Lastly, you can format and configure an entire array with basically one line like: {{ic|<nowiki>make-bcache -B /dev/{1tb} /dev/{new_3tb} -C /dev/{60GB_SSD}</nowiki>}}. Then just reboot (or even reload udev) if you've got the udev rules loaded - udev will register the devices for you based on the auto-attach data recorded in the partition's superblocks as a result of the all-in-one add command. You wouldn't even need the bcache_udev hook or anything else in initramfs at this point, because obviously you shouldn't expect to boot to the bcache array yet as you haven't had a chance to format it with a usable filesystem or to copy any files to it, but it can simplify things.<br />
:::[[User:Mikeserv|Mikeserv]] ([[User talk:Mikeserv|talk]]) 07:04, 15 November 2013 (UTC)<br />
<br />
::::I did notice a reference to that from the previous maintainer in the aur comments. I currently have a 9x2tb RAID6 mdadm array that I really don't want to have to backup and restore, so I did the tests as it would be in my server (just on a smaller scale). I don't believe it would be a simple task to remove the device, bcache-ify it and put it back in. The blocks utility says that it supports it for any block device; but mdadm stores data about the offset in the superblocks instead of using the partition layout (which I discovered the hard way in the past and had to restore 10TB of backups from off-site :( ) It should be fine though doing it to the MDADM device itself. I wouldn't mind knowing the performance difference, so once I get an SSD for my server I might test it with both methods to do a bit of a comparison.<br />
::::I didn't realize you could specify the backing and cache devices all in one line like that. I didn't see it in the wiki at all when I did my first lot of edits; seems useful. Does it automatically attach the cache to the backing devices as well?<br />
::::Although I reboot fairly often for things like kernel updates/making sure pacman hasn't broken things, I am somewhat averse to rebooting when I don't have to. I migrated from a root device on an SSD and /home and /opt on a 3TB disk, on to a spare 1TB disk, did the tests for my blog on a different computer and then created the bcache device with the 3TB disk and 60GB SSD and migrated back all without a reboot. Sometimes rebooting can be easier, but it's more 'fun' not to :)<br />
::::[[User:Justin8|Justin8]] ([[User talk:Justin8|talk]]) 08:28, 15 November 2013 (UTC)</div>Justin8https://wiki.archlinux.org/index.php?title=Talk:Bcache&diff=282904Talk:Bcache2013-11-15T04:05:14Z<p>Justin8: </p>
<hr />
<div>==Example installation==<br />
Here is mine Bcache installation. Maybe you can use some idea's. :)<br />
<br />
=== Prepare the storage drive ===<br />
<br />
====Bcache-tools installation====<br />
fakeroot and binutils are only needed. but it will fail in a error if you only install them.<br />
wget is from a failed archbang instal :(<br />
<br />
pacman -Syy<br />
pacman -S base-devel fakeroot binutils wget git<br />
<br />
cd ~/<br />
wget https://aur.archlinux.org/packages/bc/bcache-tools-git/bcache-tools-git.tar.gz<br />
tar -xzvf bcache-tools-git.tar.gz<br />
cd bcache-tools-git<br />
makepkg -si --asroot<br />
<br />
==== Making the partitions ====<br />
For /boot i use the slower HDD because i had troubles that mine SSD loaded the kernel while mine HDD was still spinning up :P<br />
<br />
Starting with the faster SSD drive<br />
<br />
fdisk /dev/sda<br />
<br />
o<br />
y<br />
n<br />
Partition type: p<br />
Partition-number: 1<br />
First-sector: [enter]<br />
Last-sector: +90G<br />
<br />
n<br />
Partition type: p<br />
Partition-number: 3<br />
First-sector: [enter]<br />
Last-sector: [enter]<br />
<br />
write and exit<br />
<br />
Now the slower 2 TB Harddrive<br />
<br />
fdisk /dev/sdb<br />
<br />
o<br />
y<br />
n<br />
Partition type: p<br />
Partition-number: 1<br />
First-sector: [enter]<br />
Last-sector: +1G<br />
a<br />
<br />
n<br />
Partition type: p<br />
Partition-number: 2<br />
First-sector: [enter]<br />
Last-sector: +16G [enter]<br />
t<br />
2<br />
82<br />
<br />
n<br />
Partition type: p<br />
Partition-number: 3<br />
First-sector: [enter]<br />
Last-sector: [enter]<br />
<br />
write and exit<br />
<br />
==== Making Bcache Device ====<br />
The --wipe-bcache is to remove a error :)<br />
and the dd commands is or cleaning out old partition data.<br />
<br />
make-bcache --wipe-bcache -B /dev/sdb3 -C /dev/sda2<br />
dd if=/dev/zero count=1 bs=1024 seek=1 of=/dev/sda2<br />
dd if=/dev/zero count=1 bs=1024 seek=1 of=/dev/sdb3<br />
<br />
If next commands give a error: invalid argument<br />
Then the device is already registered.<br />
<br />
echo /dev/sda2 > /sys/fs/bcache/register<br />
echo /dev/sdb3 > /sys/fs/bcache/register<br />
<br />
==== Formating the drives ====<br />
<br />
<br />
mkfs.ext4 -E discard /dev/sda1<br />
mkfs.ext4 -E discard /dev/sdb1<br />
mkfs.ext4 /dev/bcache0<br />
mkswap /dev/sdb2<br />
swapon /dev/sdb2<br />
<br />
==== Mount the partitions ====<br />
<br />
cd ~/<br />
mkdir -p /mnt/install<br />
mount /dev/sda1 /mnt/install<br />
mkdir -p /mnt/install/{boot,home,srv,data,var,mnt/.hdd}<br />
mount /dev/sdb1 /mnt/install/boot<br />
mount /dev/bcache0 /mnt/install/mnt/.hdd<br />
mkdir -p /mnt/install/mnt/.hdd/{home,srv,data,var}<br />
mount -o bind /mnt/install/mnt/.hdd/home /mnt/install/home<br />
mount -o bind /mnt/install/mnt/.hdd/srv /mnt/install/srv<br />
mount -o bind /mnt/install/mnt/.hdd/data /mnt/install/data<br />
mount -o bind /mnt/install/mnt/.hdd/var /mnt/install/var<br />
<br />
==== Bcache Check After mounting ====<br />
Reason for /boot on the HDD is because of boot up failure because of HDD spin-up.<br />
<br />
lsblk<br />
<br />
NAME MAJ:MIN RM SIZE RO TYPE MOUNTPOINT<br />
sda 8:0 0 167.7G 0 disk <br />
|-sda1 8:1 0 90G 0 part /mnt/install<br />
`-sda2 8:2 0 77.7G 0 part <br />
`-bcache0 253:0 0 1.8T 0 disk /mnt/install/mnt/.hdd<br />
sdb 8:16 0 1.8T 0 disk <br />
|-sdb1 8:17 0 1G 0 part /mnt/install/boot<br />
|-sdb2 8:18 0 16G 0 part [SWAP]<br />
`-sdb3 8:19 0 1.8T 0 part <br />
`-bcache0 253:0 0 1.8T 0 disk /mnt/install/mnt/.hdd<br />
<br />
==== Generate an fstab ====<br />
<br />
Generate an fstab file with the following command. UUIDs will be used because they have certain advantages.<br />
<br />
genfstab -U -p /mnt/install >> /mnt/install/etc/fstab<br />
sed -i '/dev\/sda/ s/rw,relatime,data=ordered/defaults,noatime,discard/g' /mnt/install/etc/fstab<br />
sed -i '/mnt\/install/ {s/\/mnt\/install//g; s/rw,relatime,data=ordered/defaults/g; s/none/bind/g}' /mnt/install/etc/fstab<br />
more /mnt/install/etc/fstab<br />
<br />
=== Install and configure kernel and bootloader ===<br />
==== Adding Bcache module and hook ====<br />
<br />
Edit /etc/mkinitcpio.conf as needed and re-generate the initramfs image with:<br />
adding the "bcache" module<br />
adding the "bcache_udev" hook between block and filesystems<br />
<br />
sed -i '/^MODULES=/ s/"$/ bcache"/' /etc/mkinitcpio.conf<br />
sed -i '/^HOOKS=/ s/block filesystems/block bcache_udev filesystems/' /etc/mkinitcpio.conf<br />
more /etc/mkinitcpio.conf<br />
<br />
==== Registering and attaching Bcache ====<br />
This part is a pain in the A.. <br />
The problem is that Bcache wil give a error at startup if this is not done.<br />
<br />
ERROR: Unable to find root device 'UUID=b6b2d82b-f87e-44d5-bbc5-c51dd7aace15'.<br />
You are being dropped to a recovery shell<br />
Type 'exit' to try and continue booting<br />
<br />
First get the correct "SET UUID" with this ls command. <br />
And use the UUID like the example underneath.<br />
Also check for the use of the correct devices.<br />
echo: write error: Invalid argument <<< This error is good !!! means that your value was already stored.<br />
<br />
ls /sys/fs/bcache/<br />
2300f944-0eb5-4a9e-b052-8d5ea63cbb8f register register_quiet<br />
<br />
sudo echo /dev/sda2 > /sys/fs/bcache/register<br />
sudo echo /dev/sdb3 > /sys/fs/bcache/register<br />
sudo echo 2300f944-0eb5-4a9e-b052-8d5ea63cbb8f > /sys/block/sdb/sdb3/bcache/attach<br />
<br />
mv /usr/lib/initcpio/hooks/bcache ~/bcache.old<br />
cat >> /usr/lib/initcpio/hooks/bcache << EOF<br />
#!/usr/bin/ash<br />
run_hook() {<br />
echo /dev/sda2 > /sys/fs/bcache/register<br />
echo /dev/sdb3 > /sys/fs/bcache/register<br />
}<br />
EOF<br />
<br />
==== Installing the Kernel ==== <br />
mkinitcpio -p linux<br />
<br />
I hope some stuff is helpfull :P<br />
17:58, 11 November 2013 (UTC)<br />
:I'm curious about two things. First, though I forget what the '-E' switch signifies, it seems you're formatting the raw, underlayer partitions themselves. I don't understand why.<br />
<br />
:Also, the "pain in the a?" So udev is not assembling your array automatically as the devices are discovered? That is unexpected behavior, and I suspect it has to do with the formatting you performed and udev's superblock discovery. Check the very bottom of the bcache wiki on their site. At least try lsblk to verify "bcache" partition types are not registering as "ext4" before assembling the array.<br />
<br />
:Oh. And please sign talk pages.<br />
<br />
[[User:Mikeserv|Mikeserv]] ([[User talk:Mikeserv|talk]]) 03:35, 12 November 2013 (UTC)<br />
<br />
<br />
<br />
Well this is a experiment with Bcache with mine own Wiki page and 100x testing the install, i am not a linux expert just a newbie who was reading a lot of howto's. Just see it as a compilation of different web pages and some logical thinking. :)<br />
<br />
The -E is just a copy and past ... not sure if it was needed.I just thought it was needed to get the discard flag while formating.<br />
<br />
The pain in the A.... this was before Udev ... i was busy getting Bcache to work 3 months ago ... 75% of boot up failed when booting up cold. Later i found out that the SSD was too fast with booting up and the normal HDD was still spinning up.... After i splitted up the SSD for root directory and bcache and moved the /boot to the HDD since then i can bootup 100% of the time without any error... This was before the udev thingy (wich i dont understand and need to read up) :)<br />
<br />
So please understand that i am just a linux noob trying to contribute :)<br />
<br />
[[User:Emesix|Emesix]] ([[User talk:Emesix|talk]])[[User:Emesix|Emesix]] ([[User talk:Emesix|talk]]) 00:02, 13 November 2013 (UTC)<br />
<br />
:No, I get it, and I didn't mean to insinuate that you were doing anything wrong, exactly, just that, as designed, it could operate better.<br />
:Basically, and you should check the man-pages and the wiki page here to be sure just in case I'm not entirely correct (which does happen), udev is the program that runs during boot (and pretty much all of the rest of the time, too) to detect your hardware and load drivers as necessary. Udev is a relatively recent addition to the Linux boot process and is unique in that it discovers and handles hardware as it shows up, can handle multiple devices simultaneously with non-blocking parallell operations, and, as a result, is significantly faster than the methods it has replaced.<br />
:I was the one who added the udev rules and step 7b to this wiki, and I was the one who wrote the "Rogue Superblock" section of "Troubleshooting" at the bottom of bcache's wiki. I only gained the knowledge to write either after spending several frustrating days with an issue that appears very similar to your "pain in the a."<br />
:As step 7a shows, the prevailing Arch Linux method for reassembling a bcache array at boot used to look something like:<br />
: 1) Wait 5 secs. <br />
: 2) Check to see if udev has discovered and added {disk UUIDS}. If yes goto 3), no, repeat 1) and 2).<br />
: 3) echo {pain in the a stuff} /sys/bla/bla/bla<br />
:This does work, of course, but you're adding an unnecessary wait loop to your boot process and you're performing a task with brittle shell script that depends on specific UUIDs that could be performed flexibly and at discovery time.<br />
:That's what the udev rules do. Basically, the rules instruct udev to treat as part of your bcache array any partition it finds that reports itself as a "bcache" partition type. It builds the bcache array from disk partitions at every boot only as soon as those partitions are ready, which would of course resolve the race condition you mentioned and allow you to boot from SSD if you liked.<br />
:And here's where our experience might converge: I added a partition to my bcache array after previously formatting it as ext4. If you don't know about superblocks (or magic blocks or whatever they're called) then you're just like I was a few months ago. In short the filesystem has to have a way to report on itself, so it sets aside a small marker on disk that says, "Hey, OS, I'm a disk partition of type EXT4 and I start at block offset whatever and end at block offset whatever. Also, I enjoy long walks on the beach and prefer red wine to white. See you around."<br />
:The problem I experienced was that by default ext4 tended to begin at an earlier offset than bcache, and so even though I formatted over the previously ext4 partition with bcache as instructed, bcache never overwrote ext4's first superblock. When udev attempted to build my array, it would always miss my first partition because it would read the first superblock, identify the partition as ext4, then skip it. Echoing the add instruction to /sys/ after boot was my fix for a couple of days, but eventually I figured it all out and wrote that short bit on superblocks in bcache's wiki.<br />
:Maybe that helps?<br />
:EDIT: Just looked back at the wiki proper and somebody has performed some major changes recently. While it does appear cleaner, it no longer makes sense. There are references to non-existent steps throughout, and the actual why's that were included at least to some degree seem to have been occluded in favor of dubious, though possibly more efficient how's. Anyway, step 7b no longer exists, but it used to include the bcache_udev mkinitcpio hook I (very barely) adapted from mdadm_udev when trying to fix Arch's bcache boot problems. It's now included in the AUR package the wiki recommends I guess, despite the wiki also telling us we must "echo ... /sys/... at every bootup."<br />
[[User:Mikeserv|Mikeserv]] ([[User talk:Mikeserv|talk]]) 07:08, 14 November 2013 (UTC)<br />
<br />
Sorry about that, I missed a couple of references back to the sections I changed. Hopefully it makes more sense now. [[User:Mikeserv|Mikeserv's]] udev rule is crucial to having a working bcache device on boot, as [[User:Emesix|Emesix]] had discovered, it will often result in a failed boot. The echo in to /sys/bcache* step was missed when I updated it to mention the udev rule as the 'default' method.<br />
<br />
--[[User:Justin8|Justin8]] ([[User talk:Justin8|talk]]) 10:45, 14 November 2013 (UTC)<br />
:For clarity's sake, the udev rule was never mine. The used rule has been included in the AUR package's git pull from at least the time I first installed bcache some months ago. The package maintainer at that time opted not to use it, I guess, and instead installed the legacy-type shell-scripted for-loop mkinitcpio hook I outlined above, which was, to be fair, also the process recommended by every other source of information I was able to dig up at the time to include bcache's own wiki. Frustrated with having to wait for such things which seemed to me to defeat the whole purpose of buying and configuring an SSD boot device, I eventually stumbled upon the mdadm_udev hook in /use/share/initcpio and (only slightly) modified it to apply the 69-bcache-rules instead.<br />
:Justin, thanks for your recent change. NEVERMIND FOLLOWING: I'm curious why you specify "unless assembling after boot?" Why include anything to do with bcache in the initramfs at all if you're building the array after the boot sequence? MAKES PERFECT SENSE OF COURSE. GOT A 1 TRACK MIND SOMETIMES I GUESS. SORRY.<br />
[[User:Mikeserv|Mikeserv]] ([[User talk:Mikeserv|talk]]) 11:11, 14 November 2013 (UTC)<br />
<br />
:Emesix, just looked closer at your instructions, and you actually do the thing I had to do and outlined in the bcache wiki to overwrite the rogue superblock here:<br />
:{{ic|<nowiki>dd if=/dev/zero count=1 bs=1024 seek=1 of=/dev/sd{a2,b3}</nowiki>}}<br />
:And the ext4 formatting you do after that I earlier incorrectly assumed to be for the "raw, underlayer partitions" is revealed upon further inspection to be for non-bcache partitions and so isn't even relevant; I apologize for jumping to conclusions before reviewing it fully, the quick check I did a couple of days ago just struck me as really familiar.<br />
:The udev stuff above should still be relevant, and it might be worth noting that the particular {{ic|dd}} command above is by no means a cure-all. It should only be used as written if you have previously identified a superblock located at byte offset 1024 that needs overwriting, though the actual superblock location can vary by filesystem. I've since learned that a much more flexible and useful tool for this is {{ic|wipefs}}, which is much less dangerous than it sounds.<br />
: [[User:Mikeserv|Mikeserv]] ([[User talk:Mikeserv|talk]]) 14:28, 14 November 2013 (UTC)<br />
<br />
Thanks for the explanation of the super-block [[User:Mikeserv|Mikeserv's]] was very insightful. But that Udev stuff is still a layer to high for me :P I don't know if its possible to do a little differently to avoid a unwanted wait loop:<br />
: 1) Check to see if udev has discovered and added {disk UUIDS}. If yes goto 3)<br />
: 2) Wait 5 secs and goto 1.<br />
: 3) echo {pain in the a stuff} /sys/bla/bla/bla<br />
The dd command is just a safety step. :) And because formatting the /dev/bcache0 was soon to follow, it didn't hurt if i accidentally killed the file system on the bcache :) I got the feeling that resizing the bcache partition was also giving problems wich the dd command fixed..... BTW i like the piece added with this command set:<br />
# cd /sys/fs/bcache<br />
# echo "UUID" > /sys/block/bcache0/bcache/attach<br />
p.s. I don't get why the wiki page is full of btrfs stuff. Is this not a combination of commands? And shouldn't a more conventional partition/file system scheme be more useful for novice users (like me) ??<br />
: [[User:Emesix|Emesix]] ([[User talk:Emesix|talk]]) 22:47, 14 November 2013 (UTC)<br />
<br />
::Regarding btrfs and its usefulness for new users... I don't know. With btrfs for instance, there are many normally high-level file-system concepts rendered nearly as mundane as directory maintenance like snapshotting and subvolumes, and still others that require only a single edit to a boot config like {{ic|autodefrag}} and {{ic|<nowiki>compress=lzo</nowiki>}} delivering inline filesystem compression and cleanup. In practice, you really don't have to understand the hows of these things at all to get incredible use out of them. <br />
::Consider just sub volumes: they can deliver, at the very least, a separated partition structure for the various segments of the Linux filesystem as is often recommended people set at installation with syntax and simplicity like unto {{ic|cd}} and {{ic|mkdir}}, only the {{ic|btrfs --help}} implementation is probably a little friendlier. With traditional filesystems this same setup required knowledge of disk geometry, extent counts and all manner of other arcane things to configure in any sanely optimized way. Btrfs will create you a RAID in less than 30 seconds. <br />
::I guess what I'm saying is the research vs reward ratio is pretty significantly in your favor with a btrfs disk. Of course, should the btrfs filesystem crash, well, you should probably call someone else is all I'll say about that.<br />
::In this wiki, I definitely see your point, especially considering the added complexity bcache brings. When I was trying to set mine up I spent a good deal of time in both the btrfs and bcache IRC dev channels and both sides were less than optimistic about friendly cooperation between the two. Still, it works and has done for months. I know, don't call you, right?<br />
::Now about udev, I'll try again. So, your script implements a wait loop to check occasionally if udev has done a thing (bring up the disks and pull in basic drivers like SCSI for block devices and ext4 for their component filesystems) so it can do a thing (register the disks as small parts of a larger whole and pull in the bcache driver). The udev rules eschew the notion of a secondary monitor script such as yours and instead have udev do it all (any disk it brings up as a "bcache" type is registered as a small part of a larger whole and the bcache driver is pulled in). <br />
::So you couldn't boot from SSD reliably before because you couldn't reliably script the order in which udev would bring up the disks (a consequence of its faster parallel operations), but udev doesn't have to script its order, udev just udevs as udev udevs best.<br />
::See? You will always boot reliably from your chosen device because udev won't misinterpret its own report on your chosen device's readiness. And that's all - there's nothing wrong with the shell script exactly, except that it's just plain unnecessary and added complication.<br />
[[User:Mikeserv|Mikeserv]] ([[User talk:Mikeserv|talk]]) 23:38, 14 November 2013 (UTC)<br />
<br />
:: I have to say, I want to try out using btrfs instead of LVM on my desktop sometime, but the cache article is not the place for it. Mentioning it as a good alternative on the installation and beginners guide and having details in the btrfs article is going to be more productive and logical. I added the section near the top of the page for configuring a simple cache setup with no regards to file systems and partition layouts as they should be beyond the scope of this article. <br />
<br />
--[[User:Justin8|Justin8]] ([[User talk:Justin8|talk]]) 02:13, 15 November 2013 (UTC)<br />
::Agreed. There's a lot of it, too. I guess that sometimes happens on less visited pages. Probably it will level out as the module is more established. I compromised on some of the grub stuff in a minor edit yesterday after you rightly pointed out how much unrelated stuff is in there. Never thought about it before, but I've only ever written 7b and 2 notes here and a short section on UEFI kernel update syncing.<br />
::Above is just about everything I know relatable to bcache by now, I guess. I wish there was some more in the wiki here on benchmarking. The bcache wiki itself reads like a stereo repair manual on the subject. I never have managed to wrap my mind round it before my eyes would shut of their own accord. <br />
[[User:Mikeserv|Mikeserv]] ([[User talk:Mikeserv|talk]]) 03:00, 15 November 2013 (UTC)<br />
<br />
:: Ha, that is the perfect description for their wiki; it's all failure modes and fixes but no reasoning. From what I could see, benchmarking of bcache is really only useful to benchmark writes; reads will vary exponentially depending on whether or not you get a cache hit. On top of that, sequential reads/writes bypass the cache. the only real way to benchmark it would be in 'real-world' style tests instead of synthetic benchmarks. I.e. how long it takes to boot your system. how long it takes to compile X, Y or Z. I put a little bit up on my blog when testing out bcache, mostly focussing on IOps performance changes since everything else is relatively the same with or without bcache (other than the smart re-ordering of metadata changes) I was going to just paste the graphs, but they are all js objects; you can see the results I got here: https://www.dray.be/?p=7<br />
[[User:Justin8|Justin8]] ([[User talk:Justin8|talk]]) 04:05, 15 November 2013 (UTC)</div>Justin8https://wiki.archlinux.org/index.php?title=Talk:Bcache&diff=282880Talk:Bcache2013-11-15T02:13:11Z<p>Justin8: </p>
<hr />
<div>==Example installation==<br />
Here is mine Bcache installation. Maybe you can use some idea's. :)<br />
<br />
=== Prepare the storage drive ===<br />
<br />
====Bcache-tools installation====<br />
fakeroot and binutils are only needed. but it will fail in a error if you only install them.<br />
wget is from a failed archbang instal :(<br />
<br />
pacman -Syy<br />
pacman -S base-devel fakeroot binutils wget git<br />
<br />
cd ~/<br />
wget https://aur.archlinux.org/packages/bc/bcache-tools-git/bcache-tools-git.tar.gz<br />
tar -xzvf bcache-tools-git.tar.gz<br />
cd bcache-tools-git<br />
makepkg -si --asroot<br />
<br />
==== Making the partitions ====<br />
For /boot i use the slower HDD because i had troubles that mine SSD loaded the kernel while mine HDD was still spinning up :P<br />
<br />
Starting with the faster SSD drive<br />
<br />
fdisk /dev/sda<br />
<br />
o<br />
y<br />
n<br />
Partition type: p<br />
Partition-number: 1<br />
First-sector: [enter]<br />
Last-sector: +90G<br />
<br />
n<br />
Partition type: p<br />
Partition-number: 3<br />
First-sector: [enter]<br />
Last-sector: [enter]<br />
<br />
write and exit<br />
<br />
Now the slower 2 TB Harddrive<br />
<br />
fdisk /dev/sdb<br />
<br />
o<br />
y<br />
n<br />
Partition type: p<br />
Partition-number: 1<br />
First-sector: [enter]<br />
Last-sector: +1G<br />
a<br />
<br />
n<br />
Partition type: p<br />
Partition-number: 2<br />
First-sector: [enter]<br />
Last-sector: +16G [enter]<br />
t<br />
2<br />
82<br />
<br />
n<br />
Partition type: p<br />
Partition-number: 3<br />
First-sector: [enter]<br />
Last-sector: [enter]<br />
<br />
write and exit<br />
<br />
==== Making Bcache Device ====<br />
The --wipe-bcache is to remove a error :)<br />
and the dd commands is or cleaning out old partition data.<br />
<br />
make-bcache --wipe-bcache -B /dev/sdb3 -C /dev/sda2<br />
dd if=/dev/zero count=1 bs=1024 seek=1 of=/dev/sda2<br />
dd if=/dev/zero count=1 bs=1024 seek=1 of=/dev/sdb3<br />
<br />
If next commands give a error: invalid argument<br />
Then the device is already registered.<br />
<br />
echo /dev/sda2 > /sys/fs/bcache/register<br />
echo /dev/sdb3 > /sys/fs/bcache/register<br />
<br />
==== Formating the drives ====<br />
<br />
<br />
mkfs.ext4 -E discard /dev/sda1<br />
mkfs.ext4 -E discard /dev/sdb1<br />
mkfs.ext4 /dev/bcache0<br />
mkswap /dev/sdb2<br />
swapon /dev/sdb2<br />
<br />
==== Mount the partitions ====<br />
<br />
cd ~/<br />
mkdir -p /mnt/install<br />
mount /dev/sda1 /mnt/install<br />
mkdir -p /mnt/install/{boot,home,srv,data,var,mnt/.hdd}<br />
mount /dev/sdb1 /mnt/install/boot<br />
mount /dev/bcache0 /mnt/install/mnt/.hdd<br />
mkdir -p /mnt/install/mnt/.hdd/{home,srv,data,var}<br />
mount -o bind /mnt/install/mnt/.hdd/home /mnt/install/home<br />
mount -o bind /mnt/install/mnt/.hdd/srv /mnt/install/srv<br />
mount -o bind /mnt/install/mnt/.hdd/data /mnt/install/data<br />
mount -o bind /mnt/install/mnt/.hdd/var /mnt/install/var<br />
<br />
==== Bcache Check After mounting ====<br />
Reason for /boot on the HDD is because of boot up failure because of HDD spin-up.<br />
<br />
lsblk<br />
<br />
NAME MAJ:MIN RM SIZE RO TYPE MOUNTPOINT<br />
sda 8:0 0 167.7G 0 disk <br />
|-sda1 8:1 0 90G 0 part /mnt/install<br />
`-sda2 8:2 0 77.7G 0 part <br />
`-bcache0 253:0 0 1.8T 0 disk /mnt/install/mnt/.hdd<br />
sdb 8:16 0 1.8T 0 disk <br />
|-sdb1 8:17 0 1G 0 part /mnt/install/boot<br />
|-sdb2 8:18 0 16G 0 part [SWAP]<br />
`-sdb3 8:19 0 1.8T 0 part <br />
`-bcache0 253:0 0 1.8T 0 disk /mnt/install/mnt/.hdd<br />
<br />
==== Generate an fstab ====<br />
<br />
Generate an fstab file with the following command. UUIDs will be used because they have certain advantages.<br />
<br />
genfstab -U -p /mnt/install >> /mnt/install/etc/fstab<br />
sed -i '/dev\/sda/ s/rw,relatime,data=ordered/defaults,noatime,discard/g' /mnt/install/etc/fstab<br />
sed -i '/mnt\/install/ {s/\/mnt\/install//g; s/rw,relatime,data=ordered/defaults/g; s/none/bind/g}' /mnt/install/etc/fstab<br />
more /mnt/install/etc/fstab<br />
<br />
=== Install and configure kernel and bootloader ===<br />
==== Adding Bcache module and hook ====<br />
<br />
Edit /etc/mkinitcpio.conf as needed and re-generate the initramfs image with:<br />
adding the "bcache" module<br />
adding the "bcache_udev" hook between block and filesystems<br />
<br />
sed -i '/^MODULES=/ s/"$/ bcache"/' /etc/mkinitcpio.conf<br />
sed -i '/^HOOKS=/ s/block filesystems/block bcache_udev filesystems/' /etc/mkinitcpio.conf<br />
more /etc/mkinitcpio.conf<br />
<br />
==== Registering and attaching Bcache ====<br />
This part is a pain in the A.. <br />
The problem is that Bcache wil give a error at startup if this is not done.<br />
<br />
ERROR: Unable to find root device 'UUID=b6b2d82b-f87e-44d5-bbc5-c51dd7aace15'.<br />
You are being dropped to a recovery shell<br />
Type 'exit' to try and continue booting<br />
<br />
First get the correct "SET UUID" with this ls command. <br />
And use the UUID like the example underneath.<br />
Also check for the use of the correct devices.<br />
echo: write error: Invalid argument <<< This error is good !!! means that your value was already stored.<br />
<br />
ls /sys/fs/bcache/<br />
2300f944-0eb5-4a9e-b052-8d5ea63cbb8f register register_quiet<br />
<br />
sudo echo /dev/sda2 > /sys/fs/bcache/register<br />
sudo echo /dev/sdb3 > /sys/fs/bcache/register<br />
sudo echo 2300f944-0eb5-4a9e-b052-8d5ea63cbb8f > /sys/block/sdb/sdb3/bcache/attach<br />
<br />
mv /usr/lib/initcpio/hooks/bcache ~/bcache.old<br />
cat >> /usr/lib/initcpio/hooks/bcache << EOF<br />
#!/usr/bin/ash<br />
run_hook() {<br />
echo /dev/sda2 > /sys/fs/bcache/register<br />
echo /dev/sdb3 > /sys/fs/bcache/register<br />
}<br />
EOF<br />
<br />
==== Installing the Kernel ==== <br />
mkinitcpio -p linux<br />
<br />
I hope some stuff is helpfull :P<br />
17:58, 11 November 2013 (UTC)<br />
:I'm curious about two things. First, though I forget what the '-E' switch signifies, it seems you're formatting the raw, underlayer partitions themselves. I don't understand why.<br />
<br />
:Also, the "pain in the a?" So udev is not assembling your array automatically as the devices are discovered? That is unexpected behavior, and I suspect it has to do with the formatting you performed and udev's superblock discovery. Check the very bottom of the bcache wiki on their site. At least try lsblk to verify "bcache" partition types are not registering as "ext4" before assembling the array.<br />
<br />
:Oh. And please sign talk pages.<br />
<br />
[[User:Mikeserv|Mikeserv]] ([[User talk:Mikeserv|talk]]) 03:35, 12 November 2013 (UTC)<br />
<br />
<br />
<br />
Well this is a experiment with Bcache with mine own Wiki page and 100x testing the install, i am not a linux expert just a newbie who was reading a lot of howto's. Just see it as a compilation of different web pages and some logical thinking. :)<br />
<br />
The -E is just a copy and past ... not sure if it was needed.I just thought it was needed to get the discard flag while formating.<br />
<br />
The pain in the A.... this was before Udev ... i was busy getting Bcache to work 3 months ago ... 75% of boot up failed when booting up cold. Later i found out that the SSD was too fast with booting up and the normal HDD was still spinning up.... After i splitted up the SSD for root directory and bcache and moved the /boot to the HDD since then i can bootup 100% of the time without any error... This was before the udev thingy (wich i dont understand and need to read up) :)<br />
<br />
So please understand that i am just a linux noob trying to contribute :)<br />
<br />
[[User:Emesix|Emesix]] ([[User talk:Emesix|talk]])[[User:Emesix|Emesix]] ([[User talk:Emesix|talk]]) 00:02, 13 November 2013 (UTC)<br />
<br />
:No, I get it, and I didn't mean to insinuate that you were doing anything wrong, exactly, just that, as designed, it could operate better.<br />
:Basically, and you should check the man-pages and the wiki page here to be sure just in case I'm not entirely correct (which does happen), udev is the program that runs during boot (and pretty much all of the rest of the time, too) to detect your hardware and load drivers as necessary. Udev is a relatively recent addition to the Linux boot process and is unique in that it discovers and handles hardware as it shows up, can handle multiple devices simultaneously with non-blocking parallell operations, and, as a result, is significantly faster than the methods it has replaced.<br />
:I was the one who added the udev rules and step 7b to this wiki, and I was the one who wrote the "Rogue Superblock" section of "Troubleshooting" at the bottom of bcache's wiki. I only gained the knowledge to write either after spending several frustrating days with an issue that appears very similar to your "pain in the a."<br />
:As step 7a shows, the prevailing Arch Linux method for reassembling a bcache array at boot used to look something like:<br />
: 1) Wait 5 secs. <br />
: 2) Check to see if udev has discovered and added {disk UUIDS}. If yes goto 3), no, repeat 1) and 2).<br />
: 3) echo {pain in the a stuff} /sys/bla/bla/bla<br />
:This does work, of course, but you're adding an unnecessary wait loop to your boot process and you're performing a task with brittle shell script that depends on specific UUIDs that could be performed flexibly and at discovery time.<br />
:That's what the udev rules do. Basically, the rules instruct udev to treat as part of your bcache array any partition it finds that reports itself as a "bcache" partition type. It builds the bcache array from disk partitions at every boot only as soon as those partitions are ready, which would of course resolve the race condition you mentioned and allow you to boot from SSD if you liked.<br />
:And here's where our experience might converge: I added a partition to my bcache array after previously formatting it as ext4. If you don't know about superblocks (or magic blocks or whatever they're called) then you're just like I was a few months ago. In short the filesystem has to have a way to report on itself, so it sets aside a small marker on disk that says, "Hey, OS, I'm a disk partition of type EXT4 and I start at block offset whatever and end at block offset whatever. Also, I enjoy long walks on the beach and prefer red wine to white. See you around."<br />
:The problem I experienced was that by default ext4 tended to begin at an earlier offset than bcache, and so even though I formatted over the previously ext4 partition with bcache as instructed, bcache never overwrote ext4's first superblock. When udev attempted to build my array, it would always miss my first partition because it would read the first superblock, identify the partition as ext4, then skip it. Echoing the add instruction to /sys/ after boot was my fix for a couple of days, but eventually I figured it all out and wrote that short bit on superblocks in bcache's wiki.<br />
:Maybe that helps?<br />
:EDIT: Just looked back at the wiki proper and somebody has performed some major changes recently. While it does appear cleaner, it no longer makes sense. There are references to non-existent steps throughout, and the actual why's that were included at least to some degree seem to have been occluded in favor of dubious, though possibly more efficient how's. Anyway, step 7b no longer exists, but it used to include the bcache_udev mkinitcpio hook I (very barely) adapted from mdadm_udev when trying to fix Arch's bcache boot problems. It's now included in the AUR package the wiki recommends I guess, despite the wiki also telling us we must "echo ... /sys/... at every bootup."<br />
[[User:Mikeserv|Mikeserv]] ([[User talk:Mikeserv|talk]]) 07:08, 14 November 2013 (UTC)<br />
<br />
Sorry about that, I missed a couple of references back to the sections I changed. Hopefully it makes more sense now. [[User:Mikeserv|Mikeserv's]] udev rule is crucial to having a working bcache device on boot, as [[User:Emesix|Emesix]] had discovered, it will often result in a failed boot. The echo in to /sys/bcache* step was missed when I updated it to mention the udev rule as the 'default' method.<br />
<br />
--[[User:Justin8|Justin8]] ([[User talk:Justin8|talk]]) 10:45, 14 November 2013 (UTC)<br />
:For clarity's sake, the udev rule was never mine. The used rule has been included in the AUR package's git pull from at least the time I first installed bcache some months ago. The package maintainer at that time opted not to use it, I guess, and instead installed the legacy-type shell-scripted for-loop mkinitcpio hook I outlined above, which was, to be fair, also the process recommended by every other source of information I was able to dig up at the time to include bcache's own wiki. Frustrated with having to wait for such things which seemed to me to defeat the whole purpose of buying and configuring an SSD boot device, I eventually stumbled upon the mdadm_udev hook in /use/share/initcpio and (only slightly) modified it to apply the 69-bcache-rules instead.<br />
:Justin, thanks for your recent change. NEVERMIND FOLLOWING: I'm curious why you specify "unless assembling after boot?" Why include anything to do with bcache in the initramfs at all if you're building the array after the boot sequence? MAKES PERFECT SENSE OF COURSE. GOT A 1 TRACK MIND SOMETIMES I GUESS. SORRY.<br />
[[User:Mikeserv|Mikeserv]] ([[User talk:Mikeserv|talk]]) 11:11, 14 November 2013 (UTC)<br />
<br />
:Emesix, just looked closer at your instructions, and you actually do the thing I had to do and outlined in the bcache wiki to overwrite the rogue superblock here:<br />
:{{ic|<nowiki>dd if=/dev/zero count=1 bs=1024 seek=1 of=/dev/sd{a2,b3}</nowiki>}}<br />
:And the ext4 formatting you do after that I earlier incorrectly assumed to be for the "raw, underlayer partitions" is revealed upon further inspection to be for non-bcache partitions and so isn't even relevant; I apologize for jumping to conclusions before reviewing it fully, the quick check I did a couple of days ago just struck me as really familiar.<br />
:The udev stuff above should still be relevant, and it might be worth noting that the particular {{ic|dd}} command above is by no means a cure-all. It should only be used as written if you have previously identified a superblock located at byte offset 1024 that needs overwriting, though the actual superblock location can vary by filesystem. I've since learned that a much more flexible and useful tool for this is {{ic|wipefs}}, which is much less dangerous than it sounds.<br />
: [[User:Mikeserv|Mikeserv]] ([[User talk:Mikeserv|talk]]) 14:28, 14 November 2013 (UTC)<br />
<br />
Thanks for the explanation of the super-block [[User:Mikeserv|Mikeserv's]] was very insightful. But that Udev stuff is still a layer to high for me :P I don't know if its possible to do a little differently to avoid a unwanted wait loop:<br />
: 1) Check to see if udev has discovered and added {disk UUIDS}. If yes goto 3)<br />
: 2) Wait 5 secs and goto 1.<br />
: 3) echo {pain in the a stuff} /sys/bla/bla/bla<br />
The dd command is just a safety step. :) And because formatting the /dev/bcache0 was soon to follow, it didn't hurt if i accidentally killed the file system on the bcache :) I got the feeling that resizing the bcache partition was also giving problems wich the dd command fixed..... BTW i like the piece added with this command set:<br />
# cd /sys/fs/bcache<br />
# echo "UUID" > /sys/block/bcache0/bcache/attach<br />
p.s. I don't get why the wiki page is full of btrfs stuff. Is this not a combination of commands? And shouldn't a more conventional partition/file system scheme be more useful for novice users (like me) ??<br />
: [[User:Emesix|Emesix]] ([[User talk:Emesix|talk]]) 22:47, 14 November 2013 (UTC)<br />
<br />
::Regarding btrfs and its usefulness for new users... I don't know. With btrfs for instance, there are many normally high-level file-system concepts rendered nearly as mundane as directory maintenance like snapshotting and subvolumes, and still others that require only a single edit to a boot config like {{ic|autodefrag}} and {{ic|<nowiki>compress=lzo</nowiki>}} delivering inline filesystem compression and cleanup. In practice, you really don't have to understand the hows of these things at all to get incredible use out of them. <br />
::Consider just sub volumes: they can deliver, at the very least, a separated partition structure for the various segments of the Linux filesystem as is often recommended people set at installation with syntax and simplicity like unto {{ic|cd}} and {{ic|mkdir}}, only the {{ic|btrfs --help}} implementation is probably a little friendlier. With traditional filesystems this same setup required knowledge of disk geometry, extent counts and all manner of other arcane things to configure in any sanely optimized way. Btrfs will create you a RAID in less than 30 seconds. <br />
::I guess what I'm saying is the research vs reward ratio is pretty significantly in your favor with a btrfs disk. Of course, should the btrfs filesystem crash, well, you should probably call someone else is all I'll say about that.<br />
::In this wiki, I definitely see your point, especially considering the added complexity bcache brings. When I was trying to set mine up I spent a good deal of time in both the btrfs and bcache IRC dev channels and both sides were less than optimistic about friendly cooperation between the two. Still, it works and has done for months. I know, don't call you, right?<br />
::Now about udev, I'll try again. So, your script implements a wait loop to check occasionally if udev has done a thing (bring up the disks and pull in basic drivers like SCSI for block devices and ext4 for their component filesystems) so it can do a thing (register the disks as small parts of a larger whole and pull in the bcache driver). The udev rules eschew the notion of a secondary monitor script such as yours and instead have udev do it all (any disk it brings up as a "bcache" type is registered as a small part of a larger whole and the bcache driver is pulled in). <br />
::So you couldn't boot from SSD reliably before because you couldn't reliably script the order in which udev would bring up the disks (a consequence of its faster parallel operations), but udev doesn't have to script its order, udev just udevs as udev udevs best.<br />
::See? You will always boot reliably from your chosen device because udev won't misinterpret its own report on your chosen device's readiness. And that's all - there's nothing wrong with the shell script exactly, except that it's just plain unnecessary and added complication.<br />
[[User:Mikeserv|Mikeserv]] ([[User talk:Mikeserv|talk]]) 23:38, 14 November 2013 (UTC)<br />
<br />
:: I have to say, I want to try out using btrfs instead of LVM on my desktop sometime, but the cache article is not the place for it. Mentioning it as a good alternative on the installation and beginners guide and having details in the btrfs article is going to be more productive and logical. I added the section near the top of the page for configuring a simple cache setup with no regards to file systems and partition layouts as they should be beyond the scope of this article. <br />
<br />
--[[User:Justin8|Justin8]] ([[User talk:Justin8|talk]]) 02:13, 15 November 2013 (UTC)</div>Justin8https://wiki.archlinux.org/index.php?title=Talk:Bcache&diff=282775Talk:Bcache2013-11-14T10:45:38Z<p>Justin8: </p>
<hr />
<div>==Example installation==<br />
Here is mine Bcache installation. Maybe you can use some idea's. :)<br />
<br />
=== Prepare the storage drive ===<br />
<br />
====Bcache-tools installation====<br />
fakeroot and binutils are only needed. but it will fail in a error if you only install them.<br />
wget is from a failed archbang instal :(<br />
<br />
pacman -Syy<br />
pacman -S base-devel fakeroot binutils wget git<br />
<br />
cd ~/<br />
wget https://aur.archlinux.org/packages/bc/bcache-tools-git/bcache-tools-git.tar.gz<br />
tar -xzvf bcache-tools-git.tar.gz<br />
cd bcache-tools-git<br />
makepkg -si --asroot<br />
<br />
==== Making the partitions ====<br />
For /boot i use the slower HDD because i had troubles that mine SSD loaded the kernel while mine HDD was still spinning up :P<br />
<br />
Starting with the faster SSD drive<br />
<br />
fdisk /dev/sda<br />
<br />
o<br />
y<br />
n<br />
Partition type: p<br />
Partition-number: 1<br />
First-sector: [enter]<br />
Last-sector: +90G<br />
<br />
n<br />
Partition type: p<br />
Partition-number: 3<br />
First-sector: [enter]<br />
Last-sector: [enter]<br />
<br />
write and exit<br />
<br />
Now the slower 2 TB Harddrive<br />
<br />
fdisk /dev/sdb<br />
<br />
o<br />
y<br />
n<br />
Partition type: p<br />
Partition-number: 1<br />
First-sector: [enter]<br />
Last-sector: +1G<br />
a<br />
<br />
n<br />
Partition type: p<br />
Partition-number: 2<br />
First-sector: [enter]<br />
Last-sector: +16G [enter]<br />
t<br />
2<br />
82<br />
<br />
n<br />
Partition type: p<br />
Partition-number: 3<br />
First-sector: [enter]<br />
Last-sector: [enter]<br />
<br />
write and exit<br />
<br />
==== Making Bcache Device ====<br />
The --wipe-bcache is to remove a error :)<br />
and the dd commands is or cleaning out old partition data.<br />
<br />
make-bcache --wipe-bcache -B /dev/sdb3 -C /dev/sda2<br />
dd if=/dev/zero count=1 bs=1024 seek=1 of=/dev/sda2<br />
dd if=/dev/zero count=1 bs=1024 seek=1 of=/dev/sdb3<br />
<br />
If next commands give a error: invalid argument<br />
Then the device is already registered.<br />
<br />
echo /dev/sda2 > /sys/fs/bcache/register<br />
echo /dev/sdb3 > /sys/fs/bcache/register<br />
<br />
==== Formating the drives ====<br />
<br />
<br />
mkfs.ext4 -E discard /dev/sda1<br />
mkfs.ext4 -E discard /dev/sdb1<br />
mkfs.ext4 /dev/bcache0<br />
mkswap /dev/sdb2<br />
swapon /dev/sdb2<br />
<br />
==== Mount the partitions ====<br />
<br />
cd ~/<br />
mkdir -p /mnt/install<br />
mount /dev/sda1 /mnt/install<br />
mkdir -p /mnt/install/{boot,home,srv,data,var,mnt/.hdd}<br />
mount /dev/sdb1 /mnt/install/boot<br />
mount /dev/bcache0 /mnt/install/mnt/.hdd<br />
mkdir -p /mnt/install/mnt/.hdd/{home,srv,data,var}<br />
mount -o bind /mnt/install/mnt/.hdd/home /mnt/install/home<br />
mount -o bind /mnt/install/mnt/.hdd/srv /mnt/install/srv<br />
mount -o bind /mnt/install/mnt/.hdd/data /mnt/install/data<br />
mount -o bind /mnt/install/mnt/.hdd/var /mnt/install/var<br />
<br />
==== Bcache Check After mounting ====<br />
Reason for /boot on the HDD is because of boot up failure because of HDD spin-up.<br />
<br />
lsblk<br />
<br />
NAME MAJ:MIN RM SIZE RO TYPE MOUNTPOINT<br />
sda 8:0 0 167.7G 0 disk <br />
|-sda1 8:1 0 90G 0 part /mnt/install<br />
`-sda2 8:2 0 77.7G 0 part <br />
`-bcache0 253:0 0 1.8T 0 disk /mnt/install/mnt/.hdd<br />
sdb 8:16 0 1.8T 0 disk <br />
|-sdb1 8:17 0 1G 0 part /mnt/install/boot<br />
|-sdb2 8:18 0 16G 0 part [SWAP]<br />
`-sdb3 8:19 0 1.8T 0 part <br />
`-bcache0 253:0 0 1.8T 0 disk /mnt/install/mnt/.hdd<br />
<br />
==== Generate an fstab ====<br />
<br />
Generate an fstab file with the following command. UUIDs will be used because they have certain advantages.<br />
<br />
genfstab -U -p /mnt/install >> /mnt/install/etc/fstab<br />
sed -i '/dev\/sda/ s/rw,relatime,data=ordered/defaults,noatime,discard/g' /mnt/install/etc/fstab<br />
sed -i '/mnt\/install/ {s/\/mnt\/install//g; s/rw,relatime,data=ordered/defaults/g; s/none/bind/g}' /mnt/install/etc/fstab<br />
more /mnt/install/etc/fstab<br />
<br />
=== Install and configure kernel and bootloader ===<br />
==== Adding Bcache module and hook ====<br />
<br />
Edit /etc/mkinitcpio.conf as needed and re-generate the initramfs image with:<br />
adding the "bcache" module<br />
adding the "bcache_udev" hook between block and filesystems<br />
<br />
sed -i '/^MODULES=/ s/"$/ bcache"/' /etc/mkinitcpio.conf<br />
sed -i '/^HOOKS=/ s/block filesystems/block bcache_udev filesystems/' /etc/mkinitcpio.conf<br />
more /etc/mkinitcpio.conf<br />
<br />
==== Registering and attaching Bcache ====<br />
This part is a pain in the A.. <br />
The problem is that Bcache wil give a error at startup if this is not done.<br />
<br />
ERROR: Unable to find root device 'UUID=b6b2d82b-f87e-44d5-bbc5-c51dd7aace15'.<br />
You are being dropped to a recovery shell<br />
Type 'exit' to try and continue booting<br />
<br />
First get the correct "SET UUID" with this ls command. <br />
And use the UUID like the example underneath.<br />
Also check for the use of the correct devices.<br />
echo: write error: Invalid argument <<< This error is good !!! means that your value was already stored.<br />
<br />
ls /sys/fs/bcache/<br />
2300f944-0eb5-4a9e-b052-8d5ea63cbb8f register register_quiet<br />
<br />
sudo echo /dev/sda2 > /sys/fs/bcache/register<br />
sudo echo /dev/sdb3 > /sys/fs/bcache/register<br />
sudo echo 2300f944-0eb5-4a9e-b052-8d5ea63cbb8f > /sys/block/sdb/sdb3/bcache/attach<br />
<br />
mv /usr/lib/initcpio/hooks/bcache ~/bcache.old<br />
cat >> /usr/lib/initcpio/hooks/bcache << EOF<br />
#!/usr/bin/ash<br />
run_hook() {<br />
echo /dev/sda2 > /sys/fs/bcache/register<br />
echo /dev/sdb3 > /sys/fs/bcache/register<br />
}<br />
EOF<br />
<br />
==== Installing the Kernel ==== <br />
mkinitcpio -p linux<br />
<br />
I hope some stuff is helpfull :P<br />
17:58, 11 November 2013 (UTC)<br />
:I'm curious about two things. First, though I forget what the '-E' switch signifies, it seems you're formatting the raw, underlayer partitions themselves. I don't understand why.<br />
<br />
:Also, the "pain in the a?" So udev is not assembling your array automatically as the devices are discovered? That is unexpected behavior, and I suspect it has to do with the formatting you performed and udev's superblock discovery. Check the very bottom of the bcache wiki on their site. At least try lsblk to verify "bcache" partition types are not registering as "ext4" before assembling the array.<br />
<br />
:Oh. And please sign talk pages.<br />
<br />
[[User:Mikeserv|Mikeserv]] ([[User talk:Mikeserv|talk]]) 03:35, 12 November 2013 (UTC)<br />
<br />
<br />
<br />
Well this is a experiment with Bcache with mine own Wiki page and 100x testing the install, i am not a linux expert just a newbie who was reading a lot of howto's. Just see it as a compilation of different web pages and some logical thinking. :)<br />
<br />
The -E is just a copy and past ... not sure if it was needed.I just thought it was needed to get the discard flag while formating.<br />
<br />
The pain in the A.... this was before Udev ... i was busy getting Bcache to work 3 months ago ... 75% of boot up failed when booting up cold. Later i found out that the SSD was too fast with booting up and the normal HDD was still spinning up.... After i splitted up the SSD for root directory and bcache and moved the /boot to the HDD since then i can bootup 100% of the time without any error... This was before the udev thingy (wich i dont understand and need to read up) :)<br />
<br />
So please understand that i am just a linux noob trying to contribute :)<br />
<br />
[[User:Emesix|Emesix]] ([[User talk:Emesix|talk]])[[User:Emesix|Emesix]] ([[User talk:Emesix|talk]]) 00:02, 13 November 2013 (UTC)<br />
<br />
:No, I get it, and I didn't mean to insinuate that you were doing anything wrong, exactly, just that, as designed, it could operate better.<br />
:Basically, and you should check the man-pages and the wiki page here to be sure just in case I'm not entirely correct (which does happen), udev is the program that runs during boot (and pretty much all of the rest of the time, too) to detect your hardware and load drivers as necessary. Udev is a relatively recent addition to the Linux boot process and is unique in that it discovers and handles hardware as it shows up, can handle multiple devices simultaneously with non-blocking parallell operations, and, as a result, is significantly faster than the methods it has replaced.<br />
:I was the one who added the udev rules and step 7b to this wiki, and I was the one who wrote the "Rogue Superblock" section of "Troubleshooting" at the bottom of bcache's wiki. I only gained the knowledge to write either after spending several frustrating days with an issue that appears very similar to your "pain in the a."<br />
:As step 7a shows, the prevailing Arch Linux method for reassembling a bcache array at boot used to look something like:<br />
: 1) Wait 5 secs. <br />
: 2) Check to see if udev has discovered and added {disk UUIDS}. If yes goto 3), no, repeat 1) and 2).<br />
: 3) echo {pain in the a stuff} /sys/bla/bla/bla<br />
:This does work, of course, but you're adding an unnecessary wait loop to your boot process and you're performing a task with brittle shell script that depends on specific UUIDs that could be performed flexibly and at discovery time.<br />
:That's what the udev rules do. Basically, the rules instruct udev to treat as part of your bcache array any partition it finds that reports itself as a "bcache" partition type. It builds the bcache array from disk partitions at every boot only as soon as those partitions are ready, which would of course resolve the race condition you mentioned and allow you to boot from SSD if you liked.<br />
:And here's where our experience might converge: I added a partition to my bcache array after previously formatting it as ext4. If you don't know about superblocks (or magic blocks or whatever they're called) then you're just like I was a few months ago. In short the filesystem has to have a way to report on itself, so it sets aside a small marker on disk that says, "Hey, OS, I'm a disk partition of type EXT4 and I start at block offset whatever and end at block offset whatever. Also, I enjoy long walks on the beach and prefer red wine to white. See you around."<br />
:The problem I experienced was that by default ext4 tended to begin at an earlier offset than bcache, and so even though I formatted over the previously ext4 partition with bcache as instructed, bcache never overwrote ext4's first superblock. When udev attempted to build my array, it would always miss my first partition because it would read the first superblock, identify the partition as ext4, then skip it. Echoing the add instruction to /sys/ after boot was my fix for a couple of days, but eventually I figured it all out and wrote that short bit on superblocks in bcache's wiki.<br />
:Maybe that helps?<br />
:EDIT: Just looked back at the wiki proper and somebody has performed some major changes recently. While it does appear cleaner, it no longer makes sense. There are references to non-existent steps throughout, and the actual why's that were included at least to some degree seem to have been occluded in favor of dubious, though possibly more efficient how's. Anyway, step 7b no longer exists, but it used to include the bcache_udev mkinitcpio hook I (very barely) adapted from mdadm_udev when trying to fix Arch's bcache boot problems. It's now included in the AUR package the wiki recommends I guess, despite the wiki also telling us we must "echo ... /sys/... at every bootup."<br />
[[User:Mikeserv|Mikeserv]] ([[User talk:Mikeserv|talk]]) 07:08, 14 November 2013 (UTC)<br />
<br />
Sorry about that, I missed a couple of references back to the sections I changed. Hopefully it makes more sense now. [[User:Mikeserv|Mikeserv's]] udev rule is crucial to having a working bcache device on boot, as [[User:Emesix|Emesix]] had discovered, it will often result in a failed boot. The echo in to /sys/bcache* step was missed when I updated it to mention the udev rule as the 'default' method.<br />
<br />
--[[User:Justin8|Justin8]] ([[User talk:Justin8|talk]]) 10:45, 14 November 2013 (UTC)</div>Justin8https://wiki.archlinux.org/index.php?title=Bcache&diff=282772Bcache2013-11-14T10:39:13Z<p>Justin8: </p>
<hr />
<div>[[Category:File systems]]<br />
== Introduction ==<br />
<br />
Bcache allows one to use an SSD as a read/write cache (in writeback mode) or read cache (writethrough or writearound) for another blockdevice (generally a rotating HDD or array). This article will show how to install arch using Bcache as the root partition. For an intro to bcache itself, see [http://bcache.evilpiepirate.org/ the bcache homepage]. Be sure to read and reference [http://atlas.evilpiepirate.org/git/linux-bcache.git/tree/Documentation/bcache.txt?h=bcache the bcache manual]. Bcache is in the mainline kernel since 3.10. The kernel on the arch install disk includes the bcache module since 2013.08.01.<br />
<br />
An alternative to Bcache is Facebook's [[Flashcache]].<br />
<br />
Bcache needs the backing device to be formatted as a bcache block device. In most cases, [https://github.com/g2p/blocks blocks to-bcache] can do an in-place conversion.<br />
{{Warning|Be sure you back up any important data first.}}<br />
<br />
== Setting up a bcache device on an existing system ==<br />
<br />
1. Install the {{AUR|bcache-tools-git}} package from [[AUR]].<br />
<br />
2. Create a backing device (This will typically be your mechanical drive). The backing device can be a whole device, a partition or any other standard block device. This will create /dev/bcache0<br />
# make-bcache -B /dev/sdx1<br />
<br />
3. Create a cache device (This will typically be your SSD). The cache device can be a whole device, a partition or any other standard block device<br />
# make-bcache -C /dev/sdx1<br />
<br />
4. Register the cache device against your backing device. We need to find the UUID of your cache device and then add it to the bcache device initially. Udev rules will take care of this on reboot and will only need to be done once.<br />
# cd /sys/fs/bcache<br />
# ls<br />
# echo "UUID" > /sys/block/bcache0/bcache/attach<br />
<br />
5. Change your cache mode (if you want to cache writes as well as reads):<br />
# echo writeback > /sys/block/bcache0/bcache/cache_mode<br />
<br />
6. If you want to have this partition available during the initcpio (i.e. you require it at some point in the boot process) you need to add 'bcache' to your modules array in /etc/mkinitcpio.conf as well as adding the 'bcache_udev' hook in your list between block and filesystem.<br />
<br />
<br />
== Installation to a bcache device ==<br />
<br />
1. Boot on the install disk (2013.08.01 minimum).<br />
<br />
2. Install the {{AUR|bcache-tools-git}} package from [[AUR]].<br />
<br />
3. Partition your hdd<br />
<br />
grub can't handle bcache, so you'll need at least 2 partitions (boot and one for the bcache backing device). If you're doing UEFI, you'll need an EFI System Partition (ESP) as well. E.g.:<br />
1 2048 22527 10.0 MiB EF00 EFI System<br />
2 22528 432127 200.0 MiB 8300 arch_boot<br />
3 432128 625142414 297.9 GiB 8300 bcache_backing<br />
<br />
{{Note|This example has no swapfile/partition. For a swap partition on the cache, use LVM in step 7. For a swap partition outside the cache, be sure to make a swap partition now.}}<br />
<br />
4. Configure your HDD as a bcache backing device.<br />
<br />
# make-bcache -B /dev/sda3<br />
<br />
{{Note|<br />
* [[GRUB2]] either needs an EFI partition, "bios boot" partition, or an 'embedding area' of 2KB or so after the MBR that doesn't get overwritten. Also, since GRUB2 does not know about bcache, so you also need a {{ic|/boot}} partition that grub can read. Partitioning {{ic|/dev/sda}} is a must unless you're booting from a different device.<br />
* If all associated disks are partitioned at once as below bcache will automatically attach "-B backing stores" to the "-C ssd cache" and step 4 is unnecessary on reboots.<br />
}}<br />
<br />
# make-bcache -B /dev/sd? /dev/sd? -C /dev/sd?<br />
<br />
5. Register any bcache devices with the kernel (this needs to done every bootup if you do not use the bcache_udev hook in your mkinitcpio, unless you attach the bcache devices after boot has completed)<br />
<br />
# for i in /dev/sd*; do echo $i; echo $i > /sys/fs/bcache/register_quiet; done<br />
<br />
You now have a {{ic|/dev/bcache0}} device.<br />
<br />
{{Note|the bcache user manual says to do {{ic|echo /dev/sd* > /sys/fs/bcache/register_quiet}}, but this did not work for me.}}<br />
<br />
6. Configure your SSD<br />
<br />
Format the SSD as a caching device and link it to the backing device<br />
<br />
# make-bcache -C /dev/sdb<br />
# echo /dev/sdb > /sys/fs/bcache/register <br />
# echo ''UUID__from_previous_command'' > /sys/block/bcache0/bcache/attach<br />
<br />
{{Note|If the UUID is forgotten, it can be found with {{ic|ls /sys/fs/bcache/}} after the cache device has been registered.}}<br />
<br />
7. Format the bcache device. Use LVM or btrfs subvolumes if you want to divide up the {{ic|/dev/bcache0}} device how you like (ex for seperate {{ic|/}}, {{ic|/home}}, {{ic|/var}}, etc):<br />
<br />
# mkfs.btrfs /dev/bcache0<br />
# mount /dev/bcache0 /mnt/<br />
# btrfs subvolume create /mnt/root<br />
# btrfs subvolume create /mnt/home<br />
# umount /mnt<br />
<br />
8. Prepare the installation mount point:<br />
<br />
# mkfs.ext4 /dev/sda2<br />
# mkfs.msdos /dev/sda1 (if your ESP is at least 500MB, use mkfs.vfat to make a FAT32 partition instead)<br />
# pacman -S arch-install-scripts<br />
# mount /dev/bcache0 -o subvol=root,compress=lzo /mnt/<br />
# mkdir /mnt/boot /mnt/home<br />
# mount /dev/bcache0 -o subvol=home,compress=lzo /mnt/home<br />
# mount /dev/sda2 /mnt/boot<br />
# mkdir /boot/efi<br />
# mount /dev/sda1 /mnt/boot/efi/<br />
<br />
9. Install the system as per the [[Installation_Guide#Connect_to_the_internet|Installation Guide]] as normal except this:<br />
<br />
Before you edit {{ic|/etc/mkinitcpio.conf}} and run {{ic|mkinitcpio -p linux}}:<br />
<br />
* install {{AUR|bcache-tools-git}} package from the [[AUR]].<br />
* Edit {{ic|/etc/mkinitcpio.conf}}:<br />
** add the "bcache" module<br />
** add the "bcache_udev" hook between block and filesystem hooks<br />
<br />
== Configuring ==<br />
<br />
There are many options that can be configured (such as cache mode, cache flush interval, sequential write heuristic, etc.) This is currently done by writing to files in {{ic|/sys}}. See the [http://atlas.evilpiepirate.org/git/linux-bcache.git/tree/Documentation/bcache.txt bcache user documentation].<br />
<br />
Changing the cache mode is done by echoing one of 'writethrough', 'writeback', 'writearound' or 'none' to /sys/block/bcache[0-9]/bcache/cache_mode.<br />
<br />
== Advanced operations ==<br />
<br />
=== Resize backing device ===<br />
<br />
It's possible to resize the backing device so long as you don't move the partition start. This process is described on [http://comments.gmane.org/gmane.linux.kernel.bcache.devel/249 the mailing list]. Here is an example using btrfs volume directly on bcache0. For LVM containers or for other filesystems, procedure will differ.<br />
<br />
==== Example of growing ====<br />
<br />
In this example, I grow the filesystem by 4GB. <br />
<br />
1. Reboot to a live CD/USB Drive (need not be bcache enabled) and use fdisk, gdisk, parted, or your other favorite tool to delete the backing partition and recreate it with the same start and a total size 4G larger. <br />
<br />
{{Warning|Do not use a tool like GParted that might perform filesystem operations! It will not recognize the bcache partition and might overwrite part of it!!}}<br />
<br />
2. Reboot to your normal install. Your filesystem will be currently mounted. That is fine. Issue the command to resize the partition to its maximum. For btrfs, that is<br />
<br />
# btrfs filesystem resize max /<br />
<br />
For ext3/4, that is:<br />
<br />
# resize2fs /dev/bcache0<br />
<br />
==== Example of shrinking ====<br />
<br />
In this example, I shrink the filesystem by 4GB.<br />
<br />
1. Disable writeback cache (switch to writethrough cache) and wait for the disk to flush.<br />
<br />
# echo writethrough > /sys/block/bcache0/bcache/cache_mode<br />
$ watch cat /sys/block/bcache0/bcache/state<br />
<br />
wait until state reports "clean". This might take a while.<br />
<br />
2. Shrink the mounted filesystem by something more than the desired amount, to ensure we don't accidentally clip it later. For btrfs, that is:<br />
<br />
# btrfs filesystem resize -5G /<br />
<br />
For ext3/4 you can use ''resize2fs'', but only if the partition is unmounted<br />
<br />
$ df -h /home<br />
/dev/bcache0 290G 20G 270G 1% /home<br />
# umount /home<br />
# resize2fs /dev/bcache0 283G<br />
<br />
3. Reboot to a LiveCD/USB drive (doesn't need to support bcache) and use fdisk, gdisk, parted, or your other favorite tool to delete the backing partition and recreate it with the same start and a total size 4G smaller.<br />
<br />
{{Warning|Do not use a tool like GParted that might perform filesystem operations! It won't recognize the bcache partition and might overwrite part of it!!}}<br />
<br />
4. Reboot to your normal install. Your filesystem will be currently mounted. That is fine. Issue the command to resize the partition to its maximum (that is, the size we shrunk the actual partition to in step 3). For btrfs, that is:<br />
<br />
# btrfs filesystem resize max /<br />
<br />
For ext3/4, that is:<br />
<br />
# resize2fs /dev/bcache0<br />
<br />
5. Re-enable writeback cache if you want that enabled:<br />
<br />
# echo writeback > /sys/block/bcache0/bcache/cache_mode<br />
<br />
{{Note|If you're very careful you can shrink the filesystem to the exact size in step 2 and avoid step 4. Be careful, though, many partition tools don't do exactly what you want, but instead adjust the requested partition start/end points to end on sector boundaries. This may be difficult to calculate ahead of time}}<br />
<br />
== Troubleshooting ==<br />
<br />
=== /dev/bcache device doesn't exist on bootup ===<br />
<br />
If you are sent to a busy box shell with an error:<br />
<br />
{{bc|1=<br />
ERROR: Unable to find root device 'UUID=b6b2d82b-f87e-44d5-bbc5-c51dd7aace15'.<br />
You are being dropped to a recovery shell<br />
Type 'exit' to try and continue booting<br />
}}<br />
<br />
This might happen if the backing device is configured for "writeback" mode (default is writearound). When in "writeback" mode, the /dev/bcache0 device is not started until the cache device is both registered and attached. Registering is something that needs to happen every bootup, but attaching should only have to be done once. I've sometime had to re-attach.<br />
<br />
To continue booting, try one of the following:<br />
<br />
* Register both the backing device and the cacheing device<br />
<br />
# echo /dev/sda3 > /sys/fs/bcache/register<br />
# echo /dev/sdb > /sys/fs/bcache/register<br />
<br />
If the /dev/bcache0 device now exists, type exit and continue booting. You will need to fix your initcpio to ensure devices are registered before mounting the root device.<br />
<br />
{{Note|<br />
* An error of "sh: echo: write error: Invalid argument" means the device was already registered or is not recognized as either a bcache backing device or cache. If using the udev rule on boot it should only attempt to register a device if it finds a bcache superblock<br />
* This can also happen if using udev's 69-bcache.rules in Installation's step 7 and blkid and bcache-probe "disagree" due to rogue superblocks. See [http://bcache.evilpiepirate.org/#index6h1 bcache's wiki] for a possible explanation/resolution.<br />
}}<br />
<br />
* Re-attach the cache to the backing device:<br />
<br />
If the cache device was registered, a folder with the UUID of the cache should exist in {{ic|/sys/fs/bcache}}. Use that UUID when following the example below:<br />
<br />
# ls /sys/fs/bcache/<br />
b6b2d82b-f87e-44d5-bbc5-c51dd7aace15 register register_quiet<br />
# echo b6b2d82b-f87e-44d5-bbc5-c51dd7aace15 > /sys/block/sda/sda3/bcache/attach<br />
<br />
If the {{ic|/dev/bcache0}} device now exists, type exit and continue booting. You should not have to do this again. If it persists, ask on the bcache mailing list.<br />
<br />
{{Note|An error of {{ic|sh: echo: write error: Invalid argument}} means the device was already attached. An error of {{ic|sh: echo: write error: No such file or directory}} means the UUID is not a valid cache (make sure you typed it correctly).}}<br />
<br />
* Invalidate the cache and force the backing device to run without it. You might want to check some stats, such as "dirty_data" so you have some idea of how much data will be lost.<br />
<br />
# cat /sys/block/sda/sda3/bcache/dirty_data<br />
-3.9M<br />
<br />
dirty data is data in the cache that has not been written to the backing device. If you force the backing device to run, this data will be lost, even if you later re-attach the cache.<br />
<br />
# cat /sys/block/sda/sda3/bcache/running<br />
0<br />
# echo 1 > /sys/block/sda/sda3/bcache/running<br />
<br />
The {{ic|/dev/bcache0}} device will now exist. Type exit and continue booting. You might want to unregister the cache device and run make-bcache again. An fsck on {{ic|/dev/bcache0}} would also be wise. See the [http://atlas.evilpiepirate.org/git/linux-bcache.git/tree/Documentation/bcache.txt?h=bcache bcache user documentation].<br />
<br />
{{Warning|Only invalidate the cache if one of the two options above did not work.}}<br />
<br />
=== /sys/fs/bcache/ doesn't exist ===<br />
<br />
The kernel you booted is not bcache enabled.</div>Justin8https://wiki.archlinux.org/index.php?title=User_talk:Mikeserv&diff=282771User talk:Mikeserv2013-11-14T10:39:08Z<p>Justin8: Created page with "==bcache== Hi, in relation to the points you raised: 1. The udev hook information and how it works is not a place for a bcache article, that is a part of udev. If you check ..."</p>
<hr />
<div>==bcache==<br />
<br />
Hi, in relation to the points you raised:<br />
<br />
1. The udev hook information and how it works is not a place for a bcache article, that is a part of udev. If you check similar udev functionality, such as that of the mdadm_udev hook [[RAID#Add_to_Kernel_Image]] you will see that these articles don't also copy a udev rule in to the wiki for no reason. If you think it may be best to put this back in we can, but I disagree and think that it does not belong there when it is a part of the package now, and the primary method to use the bcache hook. The original one was merely the loop to register all devices that existed at the time it is run. It was a time-sensitive hook in the boot process and I could not make it work at all; It *HAS* to run before filesystem, but putting it between block and filesystem hooks resulted in it running before block devices had fully initialized. The only way to have a functioning bcache device on boot is to use the udev hook. Perhaps we should have the explanation of the udev hook in the talk page? It seems to be a better place than on the main wiki page.<br />
<br />
2. I added a section at the top for creating a bcache device as the old section was very long winded and targeted at a user following the installation guide but wanting a bcache root device. I left this in place and titled it as such (Installation to a bcache device) and created the section above it (Setting up a bcache device on an existing system) as these are the steps many users will want to follow if they already understand how to install the OS and just want the more diluted form. There is still a large amount of superfluous information in the installation section as it goes in to how to partition your drive and about making sure you have an EFI partition, and not to make a swap file on a btrfs partition; all of which are quite unrelated to creating a bcache device and are already covered in the installation guide. I didn't remove it as I didn't want to create a huge change. Most of the 1200 characters missing is from removing the udev rule and putting it in to the package.<br />
<br />
3. I've fixed the reference to 'at every bootup' to be clearer and say that it will be required *unless* using the bcache_udev hook as it negates the need for it entirely.<br />
<br />
4. I see on the discussion page that you were the one who wrote the udev rule that was included, I didn't see this the other day when I did a quick search for whoever wrote that section. I've added you as a contributor to the package. Thanks for writing it; it's the way it should have been done at the start in my opinion.<br />
<br />
5. I do understand the material, I just missed one or two references to sections that I had changed. Old and outdated material should be deleted if we want to maintain the wiki in a useful form (have you seen the ubuntu wiki? everything from the past 5-7 years is covered in their articles and it is very hard to find anything useful). I would just like to keep the Arch wiki as up to date and informative as it always has been.<br />
<br />
Let me know if the new changes are not up to scratch and I will try to make them better. I think the fixed references and that they now make sense.</div>Justin8https://wiki.archlinux.org/index.php?title=Dnsmasq&diff=282716Dnsmasq2013-11-13T23:13:12Z<p>Justin8: /* Other methods */ - Removed outdated info. The AUR package no longer exists and the specified usage of DHCPCD instead of dhclient was incorrect.</p>
<hr />
<div>[[Category:Domain Name System]]<br />
[[it:Dnsmasq]]<br />
[[ru:Dnsmasq]]<br />
[[zh-CN:Dnsmasq]]<br />
{{Lowercase_title}}<br />
<br />
'''dnsmasq''' provides services as a DNS cacher and a DHCP server. As a Domain Name Server (DNS), it can cache DNS queries to improve connection speed to previously visited sites. As a DHCP server, {{Pkg|dnsmasq}} can be used to provide internal IP addresses and routes to computers on a LAN. Either or both of these services can be implemented. dnsmasq is considered to be lightweight and easy to configure; it is designed for personal computer use or for use on a network with less than 50 computers. It also comes with a [[PXE]] server.<br />
<br />
== Installing ==<br />
<br />
[[pacman|Install]] {{Pkg|dnsmasq}} from the [[Official Repositories|official repositories]].<br />
<br />
== DNS Cache Setup ==<br />
<br />
To set up dnsmasq as a DNS caching daemon on a single computer edit {{ic|/etc/dnsmasq.conf}} and uncomment the localhost listening address:<br />
<br />
listen-address=127.0.0.1<br />
<br />
To use this computer to act as a default DNS specify the fixed IP address of the network:<br />
<br />
listen-address=<192.168.1.1> # Example IP<br />
<br />
=== DNS Addresses File ===<br />
<br />
{{Merge|resolv.conf|Same topic. Also note that most of this can be done also natively in {{ic|/etc/resolvconf.conf}} using the {{ic|name_servers}} and {{ic|name_servers_append}} options.}}<br />
<br />
After configuring dnsmasq the DHCP client will need to prepend the localhost address to the known DNS addresses in {{ic|/etc/resolv.conf}}. This causes all queries to be sent to dnsmasq before trying to resolve them with an external DNS. After the DHCP client is configured the network will need to be restarted for changes to take effect.<br />
<br />
==== resolv.conf ====<br />
<br />
One option is a pure {{ic|resolv.conf}} configuration. To do this, just make the first nameserver in {{ic|/etc/resolv.conf}} point point to localhost:<br />
<br />
{{hc|/etc/resolv.conf|<br />
nameserver 127.0.0.1<br />
# External nameservers<br />
...<br />
}}<br />
<br />
Now DNS queries will be resolved first with dnsmasq, only checking external servers if dnsmasq cannot resolve the query. {{Pkg|dhcpcd}}, unfortunately, tends to overwrite {{ic|/etc/resolv.conf}} by default, so if you use DHCP it is a good idea to protect {{ic|/etc/resolv.conf}}. To do this, append {{ic|nohook resolv.conf}} to the dhcpcd config file:<br />
<br />
{{hc|/etc/dhcpcd.conf|<br />
...<br />
nohook resolv.conf}}<br />
<br />
===== More than three nameservers =====<br />
<br />
A limitation the way Linux handles DNS queries is that there can only be a maximum of three nameservers used in {{ic|resolv.conf}}. As a workaround, you can make localhost the only nameserver in {{ic|resolv.conf}}, and then create a separate {{ic|resolv-file}} for your external nameservers. First, create a new resolv file for dnsmasq:<br />
<br />
{{hc|/etc/resolv.dnsmasq.conf|<br />
# Google's nameservers, for example<br />
nameserver 8.8.8.8<br />
nameserver 8.8.4.4<br />
}}<br />
<br />
And then edit {{ic|/etc/dnsmasq.conf}} to use your new resolv file:<br />
<br />
{{hc|/etc/dnsmasq.conf|<br />
...<br />
resolv-file&#61;/etc/resolv.dnsmasq.conf<br />
...<br />
}}<br />
<br />
==== dhcpcd ====<br />
<br />
{{Pkg|dhcpcd}} has the ability to prepend or append nameservers to {{ic|/etc/resolv.conf}} by creating (or editing) the {{ic|/etc/resolv.conf.head}} and {{ic|/etc/resolv.conf.tail}} files respectively:<br />
<br />
echo "nameserver 127.0.0.1" > /etc/resolv.conf.head<br />
<br />
==== dhclient ====<br />
<br />
For dhclient, uncomment in {{ic|/etc/dhclient.conf}}:<br />
<br />
prepend domain-name-servers 127.0.0.1;<br />
<br />
==== NetworkManager ====<br />
<br />
NetworkManager has the ability to start {{ic|dnsmasq}} from its configuration file. Add the option {{ic|1=dns=dnsmasq}} to {{ic|NetworkManager.conf}} in the {{ic|[main]}} section then disable {{ic|dnsmasq}} from loading as a daemon:<br />
<br />
{{hc|/etc/NetworkManager/NetworkManager.conf|<nowiki><br />
[main]<br />
plugins=keyfile<br />
dns=dnsmasq<br />
</nowiki>}}<br />
<br />
For permanent caching add a config directory for {{ic|dnsmasq}} and set the cache number of nameservers (default: 150?):<br />
<br />
mkdir /etc/NetworkManager/dnsmasq.d<br />
echo "cache-size=1000" | sudo tee /etc/NetworkManager/dnsmasq.d/cache<br />
<br />
===== Other methods =====<br />
<br />
Another option is in NetworkManagers' settings (usually by right-clicking the applet) and entering settings manually. Setting up will depending on the type of front-end used; the process usually involves right-clicking on the applet, editing (or creating) a profile, and then choosing DHCP type as 'Automatic (specify addresses).' The DNS addresses will need to be entered and are usually in this form: {{ic|127.0.0.1, DNS-server-one, ...}}.<br />
<br />
===== Custom Configuration =====<br />
<br />
As of NetworkManager 0.9.6, custom configurations can be created for {{Pkg|dnsmasq}} by creating configuration files in {{ic|/etc/NetworkManager/dnsmasq.d/}}<br />
<br />
{{Note|You have to add the following line to your NetworkManager configuration file {{ic|/etc/NetworkManager/NetworkManager.conf}} : {{ic|dns&#61;dnsmasq}} in order to enable custom configuration files.}}<br />
<br />
{{Tip|This method can allow you to enable custom DNS settings on particular domains. For instance : {{ic|server&#61;/example1.com/exemple2.com/xx.xxx.xxx.x}} change the first DNS address to {{ic|xx.xxx.xxx.xx}} while browsing only the following websites {{ic|example1.com, example2.com}}. This method is preferred to a global DNS configuration when using particular DNS nameservers which lack of speed, stability, privacy and security.}}<br />
<br />
== DHCP Server Setup ==<br />
<br />
By default dnsmasq has the DHCP functionality turned off, if you want to use it you must turn it on in ({{ic|/etc/dnsmasq.conf}}). Here are the important settings:<br />
<br />
{{bc|<nowiki><br />
# Only listen to routers' LAN NIC. Doing so opens up tcp/udp port 53 to<br />
# localhost and udp port 67 to world:<br />
interface=<LAN-NIC><br />
<br />
# dnsmasq will open tcp/udp port 53 and udp port 67 to world to help with<br />
# dynamic interfaces (assigning dynamic ips). Dnsmasq will discard world<br />
# requests to them, but the paranoid might like to close them and let the <br />
# kernel handle them:<br />
bind-interfaces<br />
<br />
# Dynamic range of IPs to make available to LAN pc<br />
dhcp-range=192.168.111.50,192.168.111.100,12h<br />
<br />
# If you’d like to have dnsmasq assign static IPs, bind the LAN computer's<br />
# NIC MAC address:<br />
dhcp-host=aa:bb:cc:dd:ee:ff,192.168.111.50<br />
</nowiki>}}<br />
<br />
== Start the daemon ==<br />
To have dnsmasq to load upon startup:<br />
<br />
{{bc|# systemctl enable dnsmasq}}<br />
<br />
To start dnsmasq immediately:<br />
<br />
{{bc|# systemctl start dnsmasq}}<br />
<br />
To see if dnsmasq started properly, check the log; dnsmasq sends its messages to {{ic|/var/log/messages.log}}. The network will also need to be restarted so the the DHCP client can create a new {{ic|/etc/resolv.conf}}.<br />
<br />
== Test ==<br />
=== DNS Caching ===<br />
<br />
To do a lookup speed test choose a website that has not been visited since dnsmasq has been started ({{ic|dig}} is part of the {{Pkg|dnsutils}} package):<br />
<br />
$ dig archlinux.org | grep "Query time"<br />
<br />
Running the command again will use the cached DNS IP and result in a faster lookup time if dnsmasq is setup correctly.<br />
<br />
=== DHCP Server ===<br />
<br />
From a computer that is connected to the one with dnsmasq on it, configure it to use DHCP for automatic IP address assignment, then attempt to log into the network normally.<br />
<br />
== Tips and tricks ==<br />
<br />
=== Prevent OpenDNS Redirecting Google Queries ===<br />
<br />
To prevent OpenDNS from redirecting all Google queries to their own search server, add to {{ic|/etc/dnsmasq.conf}}:<br />
{{bc|1=server=/www.google.com/<ISP DNS IP>}}<br />
<br />
=== View leases ===<br />
{{bc|$ cat /var/lib/misc/dnsmasq.leases}}</div>Justin8https://wiki.archlinux.org/index.php?title=NetworkManager&diff=282714NetworkManager2013-11-13T23:02:23Z<p>Justin8: /* Real AP */</p>
<hr />
<div>[[Category:Networking]]<br />
[[cs:NetworkManager]]<br />
[[de:Networkmanager]]<br />
[[es:NetworkManager]]<br />
[[fr:NetworkManager]]<br />
[[it:NetworkManager]]<br />
[[ja:NetworkManager]]<br />
[[pt:NetworkManager]]<br />
[[ru:NetworkManager]]<br />
[[tr:NetworkManager]]<br />
[[zh-CN:NetworkManager]]<br />
{{Article summary start}}<br />
{{Article summary text|Covers installation and configuration of NetworkManager &ndash; a set of co-operative tools that make networking simple and straightforward.}}<br />
{{Article summary heading|Overview}}<br />
{{Article summary text|{{Networking overview}}}}<br />
{{Article summary end}}<br />
<br />
[http://projects.gnome.org/NetworkManager/ NetworkManager] is a program for providing detection and configuration for systems to automatically connect to network. NetworkManager's functionality can be useful for both wireless and wired networks. For wireless networks, NetworkManager prefers known wireless networks and has the ability to switch to the most reliable network. NetworkManager-aware applications can switch from online and offline mode. NetworkManager also prefers wired connections over wireless ones, has support for modem connections and certain types of VPN. NetworkManager was originally developed by Red Hat and now is hosted by the [[GNOME]] project.<br />
<br />
== Base install ==<br />
<br />
NetworkManager can be installed with the package {{Pkg|networkmanager}}, available in the [[official repositories]].<br />
<br />
=== VPN support ===<br />
<br />
Network Manager VPN support is based on a plug-in system. If you need VPN support via network manager you have to install one of the following packages from the [[official repositories]]:<br />
<br />
* {{Pkg|networkmanager-openconnect}}<br />
* {{Pkg|networkmanager-openvpn}}<br />
* {{Pkg|networkmanager-pptp}}<br />
* {{Pkg|networkmanager-vpnc}}<br />
<br />
== Graphical front-ends ==<br />
<br />
To configure and have easy access to NetworkManager most people will want to install an applet. This GUI front-end usually resides in the system tray (or notification area) and allows network selection and configuration of NetworkManager. Various applets exist for different types of desktops.<br />
<br />
=== GNOME ===<br />
<br />
GNOME's {{Pkg|network-manager-applet}} is lightweight enough and works across all environments.<br />
<br />
If you want to store authentication details (Wireless/DSL) and enable global connection settings, i.e "available to all users" install and configure [[GNOME Keyring]].<br />
<br />
=== KDE ===<br />
<br />
The Plasma-nm front-end is a Plasma widget available in the official repositories as package {{Pkg|kdeplasma-applets-plasma-nm}}. The older KNetworkManager front-end Plasma widget is also available as package {{aur|kdeplasma-applets-networkmanagement}} from the aur.<br />
<br />
If you have both the Plasma widget and {{ic|nm-applet}} installed and do not want to start {{ic|nm-applet}} when using KDE, add the following line to {{ic|/etc/xdg/autostart/nm-applet.desktop}}:<br />
NotShowIn=KDE<br />
<br />
See [http://userbase.kde.org/NetworkManagement Userbase page] for more info.<br />
<br />
=== XFCE ===<br />
{{Pkg|network-manager-applet}} will work fine in XFCE, but in order to see notifications, ''including error messages'', {{ic|nm-applet}} needs an implementation of the Freedesktop desktop notifications specification (see the [http://www.galago-project.org/specs/notification/0.9/index.html Galapago Project]) to display them. To enable notifications install {{Pkg|xfce4-notifyd}}, a package that provides an implementation for the specification.<br />
<br />
Without such a notification daemon, {{ic|nm-applet}} outputs the following errors to stdout/stderr:<br />
<br />
(nm-applet:24209): libnotify-WARNING **: Failed to connect to proxy<br />
** (nm-applet:24209): WARNING **: get_all_cb: couldn't retrieve<br />
system settings properties: (25) Launch helper exited with unknown<br />
return code 1.<br />
** (nm-applet:24209): WARNING **: fetch_connections_done: error<br />
fetching connections: (25) Launch helper exited with unknown return<br />
code 1.<br />
** (nm-applet:24209): WARNING **: Failed to register as an agent:<br />
(25) Launch helper exited with unknown return code 1<br />
<br />
{{ic|nm-applet}} will still work fine, though, but without notifications.<br />
<br />
If nm-applet is not prompting for a password when connecting to new wifi networks, and is just disconnecting immediately, you probably need to install {{Pkg|gnome-keyring}}.<br />
<br />
=== Openbox ===<br />
<br />
To function properly in Openbox, the GNOME applet requires the {{Pkg|xfce4-notifyd}} notification daemon for the same reason as in XFCE and the {{Pkg|gnome-icon-theme}} package to be able to display the applet in the systray.<br />
<br />
If you want to store authentication details (Wireless/DSL) install and configure [[gnome-keyring]].<br />
<br />
{{ic|nm-applet}} installs the autostart file at {{ic|/etc/xdg/autostart/nm-applet.desktop}}. If you have issues with it (e.g. {{ic|nm-applet}} is started twice or is not started at all), see [[Openbox#Autostart directory]] or [https://bbs.archlinux.org/viewtopic.php?pid=993738] for solution.<br />
<br />
=== Other desktops and window managers ===<br />
<br />
In all other scenarios it is recommended to use the GNOME applet. You will also need to be sure that the {{Pkg|gnome-icon-theme}} package is installed to be able to display the applet.<br />
<br />
To store connection secrets install and configure [[gnome-keyring]].<br />
<br />
In order to run {{ic|nm-applet}} without a systray, you can use {{Pkg|trayer}} or {{Pkg|stalonetray}}. For example, you can add a script like this one in your path:<br />
{{hc|nmgui|<nowiki><br />
#!/bin/sh<br />
nm-applet > /dev/null 2>/dev/null &<br />
stalonetray > /dev/null 2>/dev/null<br />
killall nm-applet<br />
</nowiki>}}<br />
<br />
When you close the stalonetray window, it closes {{ic|nm-applet}} too, so no extra memory is used once you are done with network settings.<br />
<br />
=== Command line ===<br />
<br />
The {{Pkg|networkmanager}} package contains [http://manpages.ubuntu.com/manpages/maverick/man1/nmcli.1.html nmcli] since version 0.8.1.<br />
<br />
== Configuration ==<br />
<br />
NetworkManager will require some additional steps to be able run properly.<br />
<br />
Verify that your {{ic|/etc/hosts}} is correct before continuing. If you previously tried to connect before doing this step, NetworkManager may have altered it. An example hostname line in {{ic|/etc/hosts}}:<br />
<br />
{{hc|/etc/hosts|<br />
127.0.0.1 localhost<br />
::1 localhost<br />
}}<br />
<br />
In case you have nss-myhostname turned off, the line would look like:<br />
<br />
{{hc|/etc/hosts|<br />
127.0.0.1 my-laptop localhost<br />
::1 my-laptop localhost<br />
}}<br />
<br />
{{Note|It may be a good idea to use {{ic|1=systemctl --type=service}} to ensure that no other service is running that may want to configure the network. Multiple networking services will conflict.}}<br />
<br />
=== Enable NetworkManager ===<br />
<br />
Once the NetworkManager daemon is started, it will automatically connect to any available "system connections" that have already been configured. Any "user connections" or unconfigured connections will need {{ic|nmcli}} or an applet to configure and connect.<br />
<br />
You can enable NetworkManager at startup with the following command:<br />
<br />
{{bc|# systemctl enable NetworkManager}}<br />
<br />
You can start the NetworkManager daemon immediately with the following command:<br />
<br />
{{bc|# systemctl start NetworkManager}}<br />
<br />
==== Avoid warnings ====<br />
<br />
NetworkManager will print probably meaningless [https://bugs.archlinux.org/task/34971 warnings (Bug#34971)] to your system log, when [[Networkmanager#Network_services_with_NetworkManager_dispatcher|NetworkManager-dispatcher.service]] and [https://www.archlinux.org/packages/?name=modemmanager ModemManager.service] are not enabled. To keep the log clean and suppress this messages, enable both, even if they are not required by your environment:<br />
<br />
{{bc|# systemctl enable NetworkManager-dispatcher.service && systemctl enable ModemManager.service}}<br />
{{bc|# systemctl start NetworkManager-dispatcher.service && systemctl start ModemManager.service}}<br />
<br />
=== Enable NetworkManager Wait Online ===<br />
If you have services which fail if they are started before the network is up, you have to use {{ic|NetworkManager-wait-online.service}} in addition to the NetworkManager service. This is however hardly ever necessary since most network daemons start up fine, even if the network has not been configured yet.<br />
<br />
You can enable NetworkManager Wait Online at startup with the following command:<br />
<br />
{{bc|# systemctl enable NetworkManager-wait-online}}<br />
<br />
In some cases the service will still fail to start sucessfully on boot:<br />
<br />
NetworkManager-wait-online.service: main process exited, code=exited, status=1/FAILURE<br />
Failed to start Network Manager Wait Online<br />
Unit NetworkManger-wait-online.service entered failed state<br />
Starting Network.<br />
Reached target Network.<br />
<br />
This is due to the timeout setting in {{ic|/usr/lib/systemd/system/NetworkManager-wait-online.service}} being too short. Change the default timeout from 30 to a higher value.<br />
<br />
=== Set up PolicyKit permissions ===<br />
<br />
See [[General Troubleshooting#Session permissions]] for setting up a working session.<br />
<br />
With a working session, you have several options for granting the necessary privileges to NetworkManager:<br />
<br />
''Option 1.'' Run a [[PolicyKit]] authentication agent when you log in, such as {{ic|/usr/lib/polkit-gnome/polkit-gnome-authentication-agent-1}} (part of {{Pkg|polkit-gnome}}). You will be prompted for your password whenever you add or remove a network connection.<br />
<br />
''Option 2.'' Add yourself to the {{ic|wheel}} group. You will not have to enter your password, but your user account may be granted other permissions as well, such as the ability to use [[sudo]] without entering the root password.<br />
<br />
''Option 3.'' Add yourself to the {{ic|network}} group and create the following file:<br />
{{hc|/etc/polkit-1/rules.d/50-org.freedesktop.NetworkManager.rules|<nowiki><br />
polkit.addRule(function(action, subject) {<br />
if (action.id.indexOf("org.freedesktop.NetworkManager.") == 0 && subject.isInGroup("network")) {<br />
return polkit.Result.YES;<br />
}<br />
});</nowiki>}}<br />
All users in the {{ic|network}} group will be able to add and remove networks without a password. This will not work under systemd if you do not have an active session with [[Systemd#Using_systemd-logind|systemd-logind]].<br />
<br />
=== Network services with NetworkManager dispatcher===<br />
<br />
There are quite a few network services that you will not want running until NetworkManager brings up an interface. Good examples are [[NTPd]] and network filesystem mounts of various types (e.g. '''netfs'''). NetworkManager has the ability to start these services when you connect to a network and stop them when you disconnect. To activate the feature you need to enable/start the NetworkManager-dispatcher [[daemon|service]].<br />
<br />
Once the feature is active, scripts can be added to the {{ic|/etc/NetworkManager/dispatcher.d}} directory. These scripts will need to have executable, user permissions. For security, it is good practice to make them owned by '''root:root''' and writable only by the owner.<br />
<br />
The scripts will be run in alphabetical order at connection time, and in reverse alphabetical order at disconnect time. They receive two arguments: the name of the interface (e.g. ''eth0'') and the status (''up'' or ''down'' for interfaces and ''vpn-up'' or ''vpn-down'' for vpn connections). To ensure what order they come up in, it is common to use numerical characters prior to the name of the script (e.g. {{ic|10_portmap}} or {{ic|30_netfs}} (which ensures that the portmapper is up before NFS mounts are attempted).<br />
<br />
{{Warning|<br />
* For security reason. You should disable write access for group and other. For example use 755 mask. In other case it can refuse to execute script, with error message "nm-dispatcher.action: Script could not be executed: writable by group or other, or set-UID." in {{ic|/var/log/messages.log}}.<br />
* If you connect to foreign or public networks, be aware of what services you are starting and what servers you expect to be available for them to connect to. You could make a security hole by starting the wrong services while connected to a public network.<br />
}}<br />
<br />
==== Avoiding the three seconds timeout ====<br />
<br />
If the above is working, then this section is not relevant. However, there is a general problem related to running dispatcher scripts which take longer than 3 seconds to be executed. NetworkManager uses an internal timeout of 3 seconds (see the [https://bugzilla.redhat.com/show_bug.cgi?id=982734 Bugtracker] for more information) and automatically kills scripts that take longer than 3 seconds to finish. In this case, the dispatcher service file, located in {{ic|/usr/lib/systemd/system/NetworkManager-dispatcher.service}}, has to be modified to remain active after exist. Create the service file {{ic|/etc/systemd/system/NetworkManager-dispatcher.service}} with the following content:<br />
<br />
.include /usr/lib/systemd/system/NetworkManager-dispatcher.service<br />
[Service]<br />
RemainAfterExit=yes<br />
<br />
Now enable the modified {{ic|NetworkManager-dispatcher}} script.<br />
<br />
==== Start OpenNTPD ====<br />
<br />
The following example starts the OpenNTPD daemon when an interface is brought up. Save the file as {{ic|/etc/NetworkManager/dispatcher.d/20_openntpd}} and make it executable.<br />
<br />
{{bc|1=<br />
#!/bin/sh<br />
interface=$1 status=$2<br />
case $status in<br />
up)<br />
systemctl start openntpd<br />
;;<br />
down)<br />
if ! nm-tool | awk '/State:/{print $2}' | grep -qs ^connected; then<br />
systemctl stop openntpd<br />
fi<br />
;;<br />
esac<br />
}}<br />
<br />
==== Mount remote folder with sshfs ====<br />
<br />
As the script is run in a very restrictive environment, you have to export {{ic|SSH_AUTH_SOCK}} in order to connect to your SSH agent. There are different ways to accomplish this, see [https://bbs.archlinux.org/viewtopic.php?pid=1042030#p1042030 this link] for more information. The example below works with [[gnome-keyring]], and will ask you for the password if not unlocked already. In case NetworkManager connects automatically on login, it is likely gnome-keyring has not yet started and the export will fail (hence the sleep). The {{ic|UUID}} to match can be found with the command {{ic|nmcli con status}} or {{ic|nmcli con list}}. <br />
<br />
{{bc|<nowiki><br />
#!/bin/sh<br />
USER='username'<br />
REMOTE='user@host:/remote/path'<br />
LOCAL='/local/path'<br />
<br />
interface=$1 status=$2<br />
if [ "$CONNECTION_UUID" = "</nowiki>''uuid''<nowiki>" ]; then<br />
case $status in<br />
up)<br />
export SSH_AUTH_SOCK=$(find /tmp -maxdepth 1 -type s -user "$USER" -name 'ssh')<br />
su "$USER" -c "sshfs $REMOTE $LOCAL"<br />
;;<br />
down)<br />
fusermount -u "$LOCAL"<br />
;;<br />
esac<br />
fi<br />
</nowiki>}}<br />
<br />
==== Use dispatcher to connect to a VPN after a network-connection is established ====<br />
<br />
In this example we want to connect automatically to a previously defined VPN connection after connecting to a specific Wi-Fi network. First thing to do is to create the dispatcher script that defines what to do after we are connected to the network. <br />
<br />
:1. Create the dispatcher script:<br />
{{hc|/etc/NetworkManager/dispatcher.d/vpn-up|<nowiki><br />
#!/bin/sh<br />
VPN_NAME="name of VPN connection defined in NetworkManager"<br />
ESSID="wifi network ESSID (not connection name)"<br />
<br />
interface=$1 status=$2<br />
case $status in<br />
up|vpn-down)<br />
if iwgetid | grep -qs ":\"$ESSID\""; then<br />
nmcli con up id "$VPN_NAME"<br />
fi<br />
;;<br />
down)<br />
if iwgetid | grep -qs ":\"$ESSID\""; then<br />
if nmcli con status id "$VPN_NAME" | grep -qs activated; then<br />
nmcli con down id "$VPN_NAME"<br />
fi<br />
fi<br />
;;<br />
esac<br />
</nowiki>}}<br />
Remember to make it executable with {{ic|chmod +x}} and to make the VPN connection available to all users. <br />
<br />
Trying to connect using this setup will fail and NetworkManager will complain about 'no valid VPN secrets', because of [http://projects.gnome.org/NetworkManager/developers/migrating-to-09/secrets-flags.html the way VPN secrets are stored] which brings us to step 2:<br />
<br />
:2. Edit your VPN connection configuration file to make NetworkManager store the secrets by itself rather than inside a keyring [https://bugzilla.redhat.com/show_bug.cgi?id=710552 that will be inaccessible for root]: open up {{ic|/etc/NetworkManager/system-connections/''name of your VPN connection''}} and change the {{ic|password-flags}} and {{ic|secret-flags}} form {{ic|1}} to {{ic|0}}.<br />
<br />
{{Note|It may now be necessary to re-open the NetworkManager connection editor and re-enter the VPN passwords/secrets.}}<br />
<br />
=== Proxy settings ===<br />
<br />
NetworkManager does not directly handle proxy settings, but if you are using GNOME, you could use [http://marin.jb.free.fr/proxydriver/ proxydriver] wich handles proxy settings using NetworkManager's informations. You can find the package for {{AUR|proxydriver}} in the [[AUR]].<br />
<br />
In order for proxydriver to be able to change the proxy settings, you would need to execute this command, as part of the GNOME startup process (System -> Preferences -> Startup Applications):<br />
<br />
xhost +si:localuser:your_username<br />
<br />
See: [[Proxy settings]]<br />
<br />
== Testing ==<br />
<br />
NetworkManager applets are designed to load upon login so no further configuration should be necessary for most users. If you have already disabled your previous network settings and disconnected from your network, you can now test if NetworkManager will work. The first step is to [[Daemon|start]] the ''networkmanager'' daemon.<br />
<br />
Some applets will provide you with a {{ic|.desktop}} file so that the NetworkManager applet can be loaded through the application menu. If it does not, you are going to either have to discover the command to use or logout and login again to start the applet. Once the applet is started, it will likely begin polling network connections with for auto-configuration with a DHCP server.<br />
<br />
To start the GNOME applet in non-xdg-compliant window managers like [[Awesome]]:<br />
<br />
nm-applet --sm-disable &<br />
<br />
For static IPs you will have to configure NetworkManager to understand them. The process usually involves right-clicking the applet and selecting something like 'Edit Connections'.<br />
<br />
== Troubleshooting ==<br />
<br />
Some fixes to common problems.<br />
<br />
=== Using nm-applet, wifi networks don't prompt for password and just disconnect ===<br />
<br />
This happens when no keyring package is installed. An easy solution is to install {{Pkg|gnome-keyring}}.<br />
<br />
=== No traffic via PPTP tunnel ===<br />
<br />
PPTP connection logins successfully, you see ppp0 interface with correct VPN IP, but you cannot even ping remote IP. It is due to lack of MPPE (Microsoft Point-to-Point Encryption) support in stock Arch pppd. It is recommended to first try with the stock Arch {{Pkg|ppp}} as it may work as intended.<br />
<br />
To solve the problem it should be sufficient to install {{AUR|ppp-mppe}} from the [[AUR]].<br />
<br />
WPA2-Enterprise wireless networks demanding MSCHAPv2 type-2 authentication with PEAP sometimes require ppp-mppe rather than the stock ppp package. [[netctl]] seems to work out of the box without ppp-mppe, however. In either case, usage of MSCHAPv2 is discouraged as it is highly vulnerable, although using another method is usually not an option. See this [https://www.cloudcracker.com/blog/2012/07/29/cracking-ms-chap-v2/ blog post].<br />
<br />
=== Network management disabled ===<br />
<br />
Sometimes when NetworkManager shuts down but the pid (state) file does not get removed and you will get a 'Network management disabled' message. If this happens, you'll have to remove it manually:<br />
<br />
# rm /var/lib/NetworkManager/NetworkManager.state<br />
<br />
If this happens upon reboot, you can add an action to your {{ic|/etc/rc.local}} to have it removed upon bootup:<br />
<br />
{{bc|<nowiki>nmpid=/var/lib/NetworkManager/NetworkManager.state<br />
[ -f $nmpid ] && rm $nmpid</nowiki>}}<br />
<br />
=== Customizing resolv.conf ===<br />
<br />
See the main page: [[resolv.conf]]. Also make sure that NetworkManager uses {{Pkg|dhcpcd}} and not {{Pkg|dhclient}}. If you want to use {{Pkg|dhclient}}, you may try the {{AUR|networkmanager-dispatch-resolv}} package from the [[AUR]].<br />
<br />
=== DHCP problems ===<br />
<br />
{{Poor writing|solutions for {{Pkg|dhcpcd}} and {{Pkg|dhclient}} mixed together}}<br />
<br />
If you have problems with getting an IP via DHCP, try to add the following to your {{ic|/etc/dhclient.conf}}:<br />
interface "eth0" {<br />
send dhcp-client-identifier 01:aa:bb:cc:dd:ee:ff;<br />
}<br />
Where {{ic|aa:bb:cc:dd:ee:ff}} is the MAC address of this NIC. The MAC address can be found using the {{ic|ip link show eth0}} command from the {{Pkg|iproute2}} package.<br />
<br />
For some (incompliant) routers, you will not be able to connect properly unless you comment the line<br />
require dhcp_server_identifier<br />
in {{ic|/etc/dhcpcd.conf}} (note that this file is distinct from {{ic|dhcpd.conf}}). This should not cause issues unless you have multiple DHCP servers on your network (not typical); see [http://technet.microsoft.com/en-us/library/cc977442.aspx this page] for more information.<br />
<br />
=== Hostname problems ===<br />
<br />
{{Accuracy|no description of the problem|Talk:NetworkManager#Hostname_problems_Entry}}<br />
<br />
Add the following line to /etc/NetworkManager/NetworkManager.conf:<br />
dhcp=dhcpcd<br />
then restart.<br />
systemctl restart NetworkManager<br />
source https://bbs.archlinux.org/viewtopic.php?id=152376<br />
<br />
=== Missing default route ===<br />
<br />
On at least one KDE4 system, no default route was created when establishing wireless connections with NetworkManager. Changing the route settings of the wireless connection to remove the default selection "Use only for resources on this connection" solved the issue.<br />
<br />
=== 3G modem not detected ===<br />
<br />
See [[USB_3G_Modem#Network_Manager]].<br />
<br />
=== Switching off WLAN on laptops ===<br />
<br />
Sometimes NetworkManager will not work when you disable your Wi-Fi adapter with a switch on your laptop and try to enable it again afterwards. This is often a problem with {{ic|rfkill}}. Install {{Pkg|rfkill}} from the [[official repositories]] and use <br />
<br />
$ watch -n1 rfkill list all<br />
<br />
to check if the driver notifies {{ic|rfkill}} about the wireless adapter's status.<br />
If one identifier stays blocked after you switch on the adapter you could try to manually unblock it with (where X is the number of the identifier provided by the above output):<br />
<br />
# rfkill event unblock X<br />
<br />
=== Static IP settings revert to DHCP ===<br />
<br />
Due to an unresolved bug, when changing default connections to static IP, {{ic|nm-applet}} may not properly store the configuration change, and will revert to automatic DHCP.<br />
<br />
To work around this issue you have to edit the default connection (e.g. "Auto eth0") in {{ic|nm-applet}}, change the connection name (e.g. "my eth0"), uncheck the "Available to all users" checkbox, change your static IP settings as desired, and click '''Apply'''. This will save a new connection with the given name.<br />
<br />
Next, you will want to make the default connection not connect automatically. To do so, run {{ic|nm-connection-editor}} (''not'' as root). In the connection editor, edit the default connection (eg "Auto eth0") and uncheck "Connect automatically". Click '''Apply''' and close the connection editor.<br />
<br />
=== Cannot edit connections as normal user ===<br />
<br />
See [[#Set_up_PolicyKit_permissions]].<br />
<br />
=== Forget hidden wireless network ===<br />
<br />
Since hidden network are not displayed in the selection list of the Wireless view, they cannot be forgotten (removed) with the GUI. You can delete one with the following command:<br />
<br />
# rm /etc/NetworkManager/system-connections/[SSID]<br />
<br />
This works for any other connection.<br />
<br />
=== VPN not working in Gnome ===<br />
<br />
When setting up openconnect or vpnc connections in NetworkManager while using Gnome, you'll sometimes never see the dialog box pop up and the following error appears in /var/log/errors.log:<br />
<br />
localhost NetworkManager[399]: <error> [1361719690.10506] [nm-vpn-connection.c:1405] get_secrets_cb(): Failed to request VPN secrets #3: (6) No agents were available for this request.<br />
<br />
This is caused by the Gnome NM Applet expecting dialog scripts to be at /usr/lib/gnome-shell, when NetworkManager's packages put them in /usr/lib/networkmanager.<br />
As a "temporary" fix (this bug has been around for a while now), make the following symlink(s):<br />
<br />
# For OpenConnect<br />
ln -s /usr/lib/networkmanager/nm-openconnect-auth-dialog /usr/lib/gnome-shell/ <br />
<br />
# For VPNC (i.e. Cisco VPN)<br />
ln -s /usr/lib/networkmanager/nm-vpnc-auth-dialog /usr/lib/gnome-shell/<br />
<br />
This may need to be done for any other NM VPN plugins as well, but these are the two most common.<br />
<br />
== Tips and tricks ==<br />
<br />
=== Sharing internet connection over Wi-Fi ===<br />
<br />
You can share your internet connection (e.g.: 3G or wired) with a few clicks using nm. You will need a supported Wi-Fi card (Cards based on Atheros AR9xx or at least AR5xx are probably best choice)<br />
<br />
==== Ad-hoc ====<br />
<br />
* [[pacman|Install]] the {{Pkg|dnsmasq}} package to be able to actually share the connection.<br />
* Custom dnsmasq.conf may interfere with nm (not sure about this, but i think so).<br />
* Click on nm-applet -> Create new wireless network.<br />
* Follow wizard (if using WEP be sure to use 5 or 13 character long password, different lengths will fail).<br />
* Settings will remain stored for the next time you need it.<br />
<br />
==== Real AP ====<br />
<br />
Support of infrastructure mode (which is needed by Android phones as they intentionally do not support ad-hoc) is supported by NetworkManager as of late 2012.<br />
<br />
See: http://fedoraproject.org/wiki/Features/RealHotspot<br />
<br />
=== Checking if networking is up inside a cron job or script ===<br />
<br />
Some cron jobs require networking to be up to succeed. You may wish to avoid running these jobs when the network is down. To accomplish this, add an '''if''' test for networking that queries NetworkManager's {{ic|nm-tool}} and checks the state of networking. The test shown here succeeds if any interface is up, and fails if they are all down. This is convenient for laptops that might be hardwired, might be on wireless, or might be off the network.<br />
{{bc|<nowiki><br />
if [ $(nm-tool|grep State|cut -f2 -d' ') == "connected" ]; then<br />
#Whatever you want to do if the network is online<br />
else<br />
#Whatever you want to do if the network is offline - note, this and the else above are optional<br />
fi<br />
</nowiki>}}<br />
<br />
This useful for a {{ic|cron.hourly}} script that runs {{ic|fpupdate}} for the F-Prot virus scanner signature update, as an example. Another way it might be useful, with a little modification, is to differentiate between networks using various parts of the output from {{ic|nm-tool}}; for example, since the active wireless network is denoted with an asterisk, you could grep for the network name and then grep for a literal asterisk.<br />
<br />
=== Automatically unlock keyring after login ===<br />
<br />
==== GNOME ====<br />
<br />
# Right click on the {{ic|nm-applet}} icon in your panel and select Edit Connections and open the Wireless tab<br />
# Select the connection you want to work with and click the Edit button<br />
# Check the boxes “Connect Automatically” and “Available to all users”<br />
Log out and log back in to complete.<br />
<br />
{{Note|The following method is dated and known not to work on at least one machine!}}<br />
* In {{ic|/etc/pam.d/gdm}} (or your corresponding daemon in {{ic|/etc/pam.d}}), add these lines at the end of the "auth" and "session" blocks if they do not exist already: <br />
auth optional pam_gnome_keyring.so<br />
session optional pam_gnome_keyring.so auto_start<br />
<br />
* In {{ic|/etc/pam.d/passwd}}, use this line for the 'password' block:<br />
password optional pam_gnome_keyring.so<br />
<br />
:Next time you log in, you should be asked if you want the password to be unlocked automatically on login.<br />
<br />
==== KDE ====<br />
{{Note|See http://live.gnome.org/GnomeKeyring/Pam for reference, and if you are using KDE with KDM, you can use {{AUR|pam-keyring-tool}} from the [[AUR]].}}<br />
<br />
Put a script like the following in {{ic|~/.kde4/Autostart}}:<br />
#!/bin/sh<br />
echo PASSWORD | /usr/bin/pam-keyring-tool --unlock --keyring=default -s<br />
Similar should work with Openbox, LXDE, etc.<br />
<br />
==== SLiM login manager ====<br />
See [[Slim#SLiM and Gnome Keyring]].<br />
<br />
=== Ignore specific devices ===<br />
<br />
Sometimes it may be desired that NetworkManager ignores specific devices and does not try to configure addresses and routes for them.You can quickly and easily ignore devices by MAC by using the following in {{ic|/etc/NetworkManager/NetworkManager.conf}} :<br />
[keyfile]<br />
unmanaged-devices=mac:00:22:68:1c:59:b1;mac:00:1E:65:30:D1:C4<br />
After you have put this in, [[Daemon|restart]] NetworkManager, and you should be able to configure interfaces without NetworkManager altering what you have set.<br />
<br />
=== Connect faster ===<br />
<br />
{{Merge|Network_Configuration#Additional_settings|This section contains mostly generally applicable tips.}}<br />
<br />
==== Disabling IPv6 ====<br />
<br />
Slow connection or reconnection to the network may be due to superfluous IPv6 queries in NetworkManager. If there is no IPv6 support on the local network, connecting to a network may take longer than normal while NetworkManager tries to establish an IPv6 connection that eventually times out. The solution is to disable IPv6 within NetworkManager which will make network connection faster. This has to be done once for every network you connect to.<br />
<br />
* Right-click on the network status icon.<br />
* Click on "Edit Connections".<br />
* Go to the "Wired" or "Wireless" tab, as appropriate.<br />
* Select the name of the network.<br />
* Click on "Edit".<br />
* Go to the "IPv6 Settings" tab.<br />
* In the "Method" dropdown, choose "Ignore/Disabled".<br />
* Click on "Save".<br />
<br />
==== Speed up DHCP by disabling ARP probing in DHCPCD ====<br />
<br />
{{ic|dhcpcd}} contains an implementation of a recommendation of the DHCP standard ([http://www.ietf.org/rfc/rfc2131.txt RFC2131] section 2.2) to check via ARP if the assigned IP address is really not taken. This seems mostly useless in home networks, so you can save about 5 seconds on every connect by adding the following line to {{ic|/etc/dhcpcd.conf}}:<br />
<br />
noarp<br />
<br />
This is equivalent to passing {{ic|--noarp}} to {{ic|dhcpcd}}, and disables the described ARP probing, speeding up connections to networks with DHCP.<br />
<br />
==== Use OpenDNS servers ====<br />
<br />
Create {{ic|/etc/resolv.conf.opendns}} with the nameservers:<br />
<br />
nameserver 208.67.222.222<br />
nameserver 208.67.220.220<br />
<br />
or use Google DNS servers, because people have been getting ads via the OpenDNS servers lately <br />
<br />
nameserver 8.8.8.8<br />
nameserver 8.8.4.4<br />
<br />
And have the dispatcher replace the discovered DHCP servers with the OpenDNS ones:<br />
<br />
{{hc|/etc/NetworkManager/dispatcher.d/dns-servers-opendns|<nowiki><br />
#!/bin/bash<br />
# Use OpenDNS servers over DHCP discovered servers<br />
<br />
cp -f /etc/resolv.conf.opendns /etc/resolv.conf</nowiki>}}<br />
<br />
Make the script executable:<br />
<br />
# chmod +x /etc/NetworkManager/dispatcher.d/dns-servers-opendns<br />
<br />
=== Enable DNS Caching ===<br />
<br />
DNS requests can be sped up by caching previous requests locally for subsequent lookup. NetworkManager has a plugin to enable DNS caching using dnsmasq, but it is not enabled in the default configuration. It is, however, easy to enable using the following instructions.<br />
<br />
Start by [[pacman|installing]] {{Pkg|dnsmasq}}. Then, edit {{ic|/etc/NetworkManager/NetworkManager.conf}} and add the following line under the {{ic|[main]}} section:<br />
<br />
dns=dnsmasq<br />
<br />
Now restart NetworkManager or reboot. NetworkManager will automatically start dnsmasq and add 127.0.0.1 to {{ic|/etc/resolv.conf}}. The actual DNS servers can be found in {{ic|/var/run/NetworkManager/dnsmasq.conf}}. You can verify dnsmasq is being used by doing the same DNS lookup twice with dig and verifying the server and query times.</div>Justin8https://wiki.archlinux.org/index.php?title=NetworkManager&diff=282713NetworkManager2013-11-13T22:59:31Z<p>Justin8: /* Real AP */</p>
<hr />
<div>[[Category:Networking]]<br />
[[cs:NetworkManager]]<br />
[[de:Networkmanager]]<br />
[[es:NetworkManager]]<br />
[[fr:NetworkManager]]<br />
[[it:NetworkManager]]<br />
[[ja:NetworkManager]]<br />
[[pt:NetworkManager]]<br />
[[ru:NetworkManager]]<br />
[[tr:NetworkManager]]<br />
[[zh-CN:NetworkManager]]<br />
{{Article summary start}}<br />
{{Article summary text|Covers installation and configuration of NetworkManager &ndash; a set of co-operative tools that make networking simple and straightforward.}}<br />
{{Article summary heading|Overview}}<br />
{{Article summary text|{{Networking overview}}}}<br />
{{Article summary end}}<br />
<br />
[http://projects.gnome.org/NetworkManager/ NetworkManager] is a program for providing detection and configuration for systems to automatically connect to network. NetworkManager's functionality can be useful for both wireless and wired networks. For wireless networks, NetworkManager prefers known wireless networks and has the ability to switch to the most reliable network. NetworkManager-aware applications can switch from online and offline mode. NetworkManager also prefers wired connections over wireless ones, has support for modem connections and certain types of VPN. NetworkManager was originally developed by Red Hat and now is hosted by the [[GNOME]] project.<br />
<br />
== Base install ==<br />
<br />
NetworkManager can be installed with the package {{Pkg|networkmanager}}, available in the [[official repositories]].<br />
<br />
=== VPN support ===<br />
<br />
Network Manager VPN support is based on a plug-in system. If you need VPN support via network manager you have to install one of the following packages from the [[official repositories]]:<br />
<br />
* {{Pkg|networkmanager-openconnect}}<br />
* {{Pkg|networkmanager-openvpn}}<br />
* {{Pkg|networkmanager-pptp}}<br />
* {{Pkg|networkmanager-vpnc}}<br />
<br />
== Graphical front-ends ==<br />
<br />
To configure and have easy access to NetworkManager most people will want to install an applet. This GUI front-end usually resides in the system tray (or notification area) and allows network selection and configuration of NetworkManager. Various applets exist for different types of desktops.<br />
<br />
=== GNOME ===<br />
<br />
GNOME's {{Pkg|network-manager-applet}} is lightweight enough and works across all environments.<br />
<br />
If you want to store authentication details (Wireless/DSL) and enable global connection settings, i.e "available to all users" install and configure [[GNOME Keyring]].<br />
<br />
=== KDE ===<br />
<br />
The Plasma-nm front-end is a Plasma widget available in the official repositories as package {{Pkg|kdeplasma-applets-plasma-nm}}. The older KNetworkManager front-end Plasma widget is also available as package {{aur|kdeplasma-applets-networkmanagement}} from the aur.<br />
<br />
If you have both the Plasma widget and {{ic|nm-applet}} installed and do not want to start {{ic|nm-applet}} when using KDE, add the following line to {{ic|/etc/xdg/autostart/nm-applet.desktop}}:<br />
NotShowIn=KDE<br />
<br />
See [http://userbase.kde.org/NetworkManagement Userbase page] for more info.<br />
<br />
=== XFCE ===<br />
{{Pkg|network-manager-applet}} will work fine in XFCE, but in order to see notifications, ''including error messages'', {{ic|nm-applet}} needs an implementation of the Freedesktop desktop notifications specification (see the [http://www.galago-project.org/specs/notification/0.9/index.html Galapago Project]) to display them. To enable notifications install {{Pkg|xfce4-notifyd}}, a package that provides an implementation for the specification.<br />
<br />
Without such a notification daemon, {{ic|nm-applet}} outputs the following errors to stdout/stderr:<br />
<br />
(nm-applet:24209): libnotify-WARNING **: Failed to connect to proxy<br />
** (nm-applet:24209): WARNING **: get_all_cb: couldn't retrieve<br />
system settings properties: (25) Launch helper exited with unknown<br />
return code 1.<br />
** (nm-applet:24209): WARNING **: fetch_connections_done: error<br />
fetching connections: (25) Launch helper exited with unknown return<br />
code 1.<br />
** (nm-applet:24209): WARNING **: Failed to register as an agent:<br />
(25) Launch helper exited with unknown return code 1<br />
<br />
{{ic|nm-applet}} will still work fine, though, but without notifications.<br />
<br />
If nm-applet is not prompting for a password when connecting to new wifi networks, and is just disconnecting immediately, you probably need to install {{Pkg|gnome-keyring}}.<br />
<br />
=== Openbox ===<br />
<br />
To function properly in Openbox, the GNOME applet requires the {{Pkg|xfce4-notifyd}} notification daemon for the same reason as in XFCE and the {{Pkg|gnome-icon-theme}} package to be able to display the applet in the systray.<br />
<br />
If you want to store authentication details (Wireless/DSL) install and configure [[gnome-keyring]].<br />
<br />
{{ic|nm-applet}} installs the autostart file at {{ic|/etc/xdg/autostart/nm-applet.desktop}}. If you have issues with it (e.g. {{ic|nm-applet}} is started twice or is not started at all), see [[Openbox#Autostart directory]] or [https://bbs.archlinux.org/viewtopic.php?pid=993738] for solution.<br />
<br />
=== Other desktops and window managers ===<br />
<br />
In all other scenarios it is recommended to use the GNOME applet. You will also need to be sure that the {{Pkg|gnome-icon-theme}} package is installed to be able to display the applet.<br />
<br />
To store connection secrets install and configure [[gnome-keyring]].<br />
<br />
In order to run {{ic|nm-applet}} without a systray, you can use {{Pkg|trayer}} or {{Pkg|stalonetray}}. For example, you can add a script like this one in your path:<br />
{{hc|nmgui|<nowiki><br />
#!/bin/sh<br />
nm-applet > /dev/null 2>/dev/null &<br />
stalonetray > /dev/null 2>/dev/null<br />
killall nm-applet<br />
</nowiki>}}<br />
<br />
When you close the stalonetray window, it closes {{ic|nm-applet}} too, so no extra memory is used once you are done with network settings.<br />
<br />
=== Command line ===<br />
<br />
The {{Pkg|networkmanager}} package contains [http://manpages.ubuntu.com/manpages/maverick/man1/nmcli.1.html nmcli] since version 0.8.1.<br />
<br />
== Configuration ==<br />
<br />
NetworkManager will require some additional steps to be able run properly.<br />
<br />
Verify that your {{ic|/etc/hosts}} is correct before continuing. If you previously tried to connect before doing this step, NetworkManager may have altered it. An example hostname line in {{ic|/etc/hosts}}:<br />
<br />
{{hc|/etc/hosts|<br />
127.0.0.1 localhost<br />
::1 localhost<br />
}}<br />
<br />
In case you have nss-myhostname turned off, the line would look like:<br />
<br />
{{hc|/etc/hosts|<br />
127.0.0.1 my-laptop localhost<br />
::1 my-laptop localhost<br />
}}<br />
<br />
{{Note|It may be a good idea to use {{ic|1=systemctl --type=service}} to ensure that no other service is running that may want to configure the network. Multiple networking services will conflict.}}<br />
<br />
=== Enable NetworkManager ===<br />
<br />
Once the NetworkManager daemon is started, it will automatically connect to any available "system connections" that have already been configured. Any "user connections" or unconfigured connections will need {{ic|nmcli}} or an applet to configure and connect.<br />
<br />
You can enable NetworkManager at startup with the following command:<br />
<br />
{{bc|# systemctl enable NetworkManager}}<br />
<br />
You can start the NetworkManager daemon immediately with the following command:<br />
<br />
{{bc|# systemctl start NetworkManager}}<br />
<br />
==== Avoid warnings ====<br />
<br />
NetworkManager will print probably meaningless [https://bugs.archlinux.org/task/34971 warnings (Bug#34971)] to your system log, when [[Networkmanager#Network_services_with_NetworkManager_dispatcher|NetworkManager-dispatcher.service]] and [https://www.archlinux.org/packages/?name=modemmanager ModemManager.service] are not enabled. To keep the log clean and suppress this messages, enable both, even if they are not required by your environment:<br />
<br />
{{bc|# systemctl enable NetworkManager-dispatcher.service && systemctl enable ModemManager.service}}<br />
{{bc|# systemctl start NetworkManager-dispatcher.service && systemctl start ModemManager.service}}<br />
<br />
=== Enable NetworkManager Wait Online ===<br />
If you have services which fail if they are started before the network is up, you have to use {{ic|NetworkManager-wait-online.service}} in addition to the NetworkManager service. This is however hardly ever necessary since most network daemons start up fine, even if the network has not been configured yet.<br />
<br />
You can enable NetworkManager Wait Online at startup with the following command:<br />
<br />
{{bc|# systemctl enable NetworkManager-wait-online}}<br />
<br />
In some cases the service will still fail to start sucessfully on boot:<br />
<br />
NetworkManager-wait-online.service: main process exited, code=exited, status=1/FAILURE<br />
Failed to start Network Manager Wait Online<br />
Unit NetworkManger-wait-online.service entered failed state<br />
Starting Network.<br />
Reached target Network.<br />
<br />
This is due to the timeout setting in {{ic|/usr/lib/systemd/system/NetworkManager-wait-online.service}} being too short. Change the default timeout from 30 to a higher value.<br />
<br />
=== Set up PolicyKit permissions ===<br />
<br />
See [[General Troubleshooting#Session permissions]] for setting up a working session.<br />
<br />
With a working session, you have several options for granting the necessary privileges to NetworkManager:<br />
<br />
''Option 1.'' Run a [[PolicyKit]] authentication agent when you log in, such as {{ic|/usr/lib/polkit-gnome/polkit-gnome-authentication-agent-1}} (part of {{Pkg|polkit-gnome}}). You will be prompted for your password whenever you add or remove a network connection.<br />
<br />
''Option 2.'' Add yourself to the {{ic|wheel}} group. You will not have to enter your password, but your user account may be granted other permissions as well, such as the ability to use [[sudo]] without entering the root password.<br />
<br />
''Option 3.'' Add yourself to the {{ic|network}} group and create the following file:<br />
{{hc|/etc/polkit-1/rules.d/50-org.freedesktop.NetworkManager.rules|<nowiki><br />
polkit.addRule(function(action, subject) {<br />
if (action.id.indexOf("org.freedesktop.NetworkManager.") == 0 && subject.isInGroup("network")) {<br />
return polkit.Result.YES;<br />
}<br />
});</nowiki>}}<br />
All users in the {{ic|network}} group will be able to add and remove networks without a password. This will not work under systemd if you do not have an active session with [[Systemd#Using_systemd-logind|systemd-logind]].<br />
<br />
=== Network services with NetworkManager dispatcher===<br />
<br />
There are quite a few network services that you will not want running until NetworkManager brings up an interface. Good examples are [[NTPd]] and network filesystem mounts of various types (e.g. '''netfs'''). NetworkManager has the ability to start these services when you connect to a network and stop them when you disconnect. To activate the feature you need to enable/start the NetworkManager-dispatcher [[daemon|service]].<br />
<br />
Once the feature is active, scripts can be added to the {{ic|/etc/NetworkManager/dispatcher.d}} directory. These scripts will need to have executable, user permissions. For security, it is good practice to make them owned by '''root:root''' and writable only by the owner.<br />
<br />
The scripts will be run in alphabetical order at connection time, and in reverse alphabetical order at disconnect time. They receive two arguments: the name of the interface (e.g. ''eth0'') and the status (''up'' or ''down'' for interfaces and ''vpn-up'' or ''vpn-down'' for vpn connections). To ensure what order they come up in, it is common to use numerical characters prior to the name of the script (e.g. {{ic|10_portmap}} or {{ic|30_netfs}} (which ensures that the portmapper is up before NFS mounts are attempted).<br />
<br />
{{Warning|<br />
* For security reason. You should disable write access for group and other. For example use 755 mask. In other case it can refuse to execute script, with error message "nm-dispatcher.action: Script could not be executed: writable by group or other, or set-UID." in {{ic|/var/log/messages.log}}.<br />
* If you connect to foreign or public networks, be aware of what services you are starting and what servers you expect to be available for them to connect to. You could make a security hole by starting the wrong services while connected to a public network.<br />
}}<br />
<br />
==== Avoiding the three seconds timeout ====<br />
<br />
If the above is working, then this section is not relevant. However, there is a general problem related to running dispatcher scripts which take longer than 3 seconds to be executed. NetworkManager uses an internal timeout of 3 seconds (see the [https://bugzilla.redhat.com/show_bug.cgi?id=982734 Bugtracker] for more information) and automatically kills scripts that take longer than 3 seconds to finish. In this case, the dispatcher service file, located in {{ic|/usr/lib/systemd/system/NetworkManager-dispatcher.service}}, has to be modified to remain active after exist. Create the service file {{ic|/etc/systemd/system/NetworkManager-dispatcher.service}} with the following content:<br />
<br />
.include /usr/lib/systemd/system/NetworkManager-dispatcher.service<br />
[Service]<br />
RemainAfterExit=yes<br />
<br />
Now enable the modified {{ic|NetworkManager-dispatcher}} script.<br />
<br />
==== Start OpenNTPD ====<br />
<br />
The following example starts the OpenNTPD daemon when an interface is brought up. Save the file as {{ic|/etc/NetworkManager/dispatcher.d/20_openntpd}} and make it executable.<br />
<br />
{{bc|1=<br />
#!/bin/sh<br />
interface=$1 status=$2<br />
case $status in<br />
up)<br />
systemctl start openntpd<br />
;;<br />
down)<br />
if ! nm-tool | awk '/State:/{print $2}' | grep -qs ^connected; then<br />
systemctl stop openntpd<br />
fi<br />
;;<br />
esac<br />
}}<br />
<br />
==== Mount remote folder with sshfs ====<br />
<br />
As the script is run in a very restrictive environment, you have to export {{ic|SSH_AUTH_SOCK}} in order to connect to your SSH agent. There are different ways to accomplish this, see [https://bbs.archlinux.org/viewtopic.php?pid=1042030#p1042030 this link] for more information. The example below works with [[gnome-keyring]], and will ask you for the password if not unlocked already. In case NetworkManager connects automatically on login, it is likely gnome-keyring has not yet started and the export will fail (hence the sleep). The {{ic|UUID}} to match can be found with the command {{ic|nmcli con status}} or {{ic|nmcli con list}}. <br />
<br />
{{bc|<nowiki><br />
#!/bin/sh<br />
USER='username'<br />
REMOTE='user@host:/remote/path'<br />
LOCAL='/local/path'<br />
<br />
interface=$1 status=$2<br />
if [ "$CONNECTION_UUID" = "</nowiki>''uuid''<nowiki>" ]; then<br />
case $status in<br />
up)<br />
export SSH_AUTH_SOCK=$(find /tmp -maxdepth 1 -type s -user "$USER" -name 'ssh')<br />
su "$USER" -c "sshfs $REMOTE $LOCAL"<br />
;;<br />
down)<br />
fusermount -u "$LOCAL"<br />
;;<br />
esac<br />
fi<br />
</nowiki>}}<br />
<br />
==== Use dispatcher to connect to a VPN after a network-connection is established ====<br />
<br />
In this example we want to connect automatically to a previously defined VPN connection after connecting to a specific Wi-Fi network. First thing to do is to create the dispatcher script that defines what to do after we are connected to the network. <br />
<br />
:1. Create the dispatcher script:<br />
{{hc|/etc/NetworkManager/dispatcher.d/vpn-up|<nowiki><br />
#!/bin/sh<br />
VPN_NAME="name of VPN connection defined in NetworkManager"<br />
ESSID="wifi network ESSID (not connection name)"<br />
<br />
interface=$1 status=$2<br />
case $status in<br />
up|vpn-down)<br />
if iwgetid | grep -qs ":\"$ESSID\""; then<br />
nmcli con up id "$VPN_NAME"<br />
fi<br />
;;<br />
down)<br />
if iwgetid | grep -qs ":\"$ESSID\""; then<br />
if nmcli con status id "$VPN_NAME" | grep -qs activated; then<br />
nmcli con down id "$VPN_NAME"<br />
fi<br />
fi<br />
;;<br />
esac<br />
</nowiki>}}<br />
Remember to make it executable with {{ic|chmod +x}} and to make the VPN connection available to all users. <br />
<br />
Trying to connect using this setup will fail and NetworkManager will complain about 'no valid VPN secrets', because of [http://projects.gnome.org/NetworkManager/developers/migrating-to-09/secrets-flags.html the way VPN secrets are stored] which brings us to step 2:<br />
<br />
:2. Edit your VPN connection configuration file to make NetworkManager store the secrets by itself rather than inside a keyring [https://bugzilla.redhat.com/show_bug.cgi?id=710552 that will be inaccessible for root]: open up {{ic|/etc/NetworkManager/system-connections/''name of your VPN connection''}} and change the {{ic|password-flags}} and {{ic|secret-flags}} form {{ic|1}} to {{ic|0}}.<br />
<br />
{{Note|It may now be necessary to re-open the NetworkManager connection editor and re-enter the VPN passwords/secrets.}}<br />
<br />
=== Proxy settings ===<br />
<br />
NetworkManager does not directly handle proxy settings, but if you are using GNOME, you could use [http://marin.jb.free.fr/proxydriver/ proxydriver] wich handles proxy settings using NetworkManager's informations. You can find the package for {{AUR|proxydriver}} in the [[AUR]].<br />
<br />
In order for proxydriver to be able to change the proxy settings, you would need to execute this command, as part of the GNOME startup process (System -> Preferences -> Startup Applications):<br />
<br />
xhost +si:localuser:your_username<br />
<br />
See: [[Proxy settings]]<br />
<br />
== Testing ==<br />
<br />
NetworkManager applets are designed to load upon login so no further configuration should be necessary for most users. If you have already disabled your previous network settings and disconnected from your network, you can now test if NetworkManager will work. The first step is to [[Daemon|start]] the ''networkmanager'' daemon.<br />
<br />
Some applets will provide you with a {{ic|.desktop}} file so that the NetworkManager applet can be loaded through the application menu. If it does not, you are going to either have to discover the command to use or logout and login again to start the applet. Once the applet is started, it will likely begin polling network connections with for auto-configuration with a DHCP server.<br />
<br />
To start the GNOME applet in non-xdg-compliant window managers like [[Awesome]]:<br />
<br />
nm-applet --sm-disable &<br />
<br />
For static IPs you will have to configure NetworkManager to understand them. The process usually involves right-clicking the applet and selecting something like 'Edit Connections'.<br />
<br />
== Troubleshooting ==<br />
<br />
Some fixes to common problems.<br />
<br />
=== Using nm-applet, wifi networks don't prompt for password and just disconnect ===<br />
<br />
This happens when no keyring package is installed. An easy solution is to install {{Pkg|gnome-keyring}}.<br />
<br />
=== No traffic via PPTP tunnel ===<br />
<br />
PPTP connection logins successfully, you see ppp0 interface with correct VPN IP, but you cannot even ping remote IP. It is due to lack of MPPE (Microsoft Point-to-Point Encryption) support in stock Arch pppd. It is recommended to first try with the stock Arch {{Pkg|ppp}} as it may work as intended.<br />
<br />
To solve the problem it should be sufficient to install {{AUR|ppp-mppe}} from the [[AUR]].<br />
<br />
WPA2-Enterprise wireless networks demanding MSCHAPv2 type-2 authentication with PEAP sometimes require ppp-mppe rather than the stock ppp package. [[netctl]] seems to work out of the box without ppp-mppe, however. In either case, usage of MSCHAPv2 is discouraged as it is highly vulnerable, although using another method is usually not an option. See this [https://www.cloudcracker.com/blog/2012/07/29/cracking-ms-chap-v2/ blog post].<br />
<br />
=== Network management disabled ===<br />
<br />
Sometimes when NetworkManager shuts down but the pid (state) file does not get removed and you will get a 'Network management disabled' message. If this happens, you'll have to remove it manually:<br />
<br />
# rm /var/lib/NetworkManager/NetworkManager.state<br />
<br />
If this happens upon reboot, you can add an action to your {{ic|/etc/rc.local}} to have it removed upon bootup:<br />
<br />
{{bc|<nowiki>nmpid=/var/lib/NetworkManager/NetworkManager.state<br />
[ -f $nmpid ] && rm $nmpid</nowiki>}}<br />
<br />
=== Customizing resolv.conf ===<br />
<br />
See the main page: [[resolv.conf]]. Also make sure that NetworkManager uses {{Pkg|dhcpcd}} and not {{Pkg|dhclient}}. If you want to use {{Pkg|dhclient}}, you may try the {{AUR|networkmanager-dispatch-resolv}} package from the [[AUR]].<br />
<br />
=== DHCP problems ===<br />
<br />
{{Poor writing|solutions for {{Pkg|dhcpcd}} and {{Pkg|dhclient}} mixed together}}<br />
<br />
If you have problems with getting an IP via DHCP, try to add the following to your {{ic|/etc/dhclient.conf}}:<br />
interface "eth0" {<br />
send dhcp-client-identifier 01:aa:bb:cc:dd:ee:ff;<br />
}<br />
Where {{ic|aa:bb:cc:dd:ee:ff}} is the MAC address of this NIC. The MAC address can be found using the {{ic|ip link show eth0}} command from the {{Pkg|iproute2}} package.<br />
<br />
For some (incompliant) routers, you will not be able to connect properly unless you comment the line<br />
require dhcp_server_identifier<br />
in {{ic|/etc/dhcpcd.conf}} (note that this file is distinct from {{ic|dhcpd.conf}}). This should not cause issues unless you have multiple DHCP servers on your network (not typical); see [http://technet.microsoft.com/en-us/library/cc977442.aspx this page] for more information.<br />
<br />
=== Hostname problems ===<br />
<br />
{{Accuracy|no description of the problem|Talk:NetworkManager#Hostname_problems_Entry}}<br />
<br />
Add the following line to /etc/NetworkManager/NetworkManager.conf:<br />
dhcp=dhcpcd<br />
then restart.<br />
systemctl restart NetworkManager<br />
source https://bbs.archlinux.org/viewtopic.php?id=152376<br />
<br />
=== Missing default route ===<br />
<br />
On at least one KDE4 system, no default route was created when establishing wireless connections with NetworkManager. Changing the route settings of the wireless connection to remove the default selection "Use only for resources on this connection" solved the issue.<br />
<br />
=== 3G modem not detected ===<br />
<br />
See [[USB_3G_Modem#Network_Manager]].<br />
<br />
=== Switching off WLAN on laptops ===<br />
<br />
Sometimes NetworkManager will not work when you disable your Wi-Fi adapter with a switch on your laptop and try to enable it again afterwards. This is often a problem with {{ic|rfkill}}. Install {{Pkg|rfkill}} from the [[official repositories]] and use <br />
<br />
$ watch -n1 rfkill list all<br />
<br />
to check if the driver notifies {{ic|rfkill}} about the wireless adapter's status.<br />
If one identifier stays blocked after you switch on the adapter you could try to manually unblock it with (where X is the number of the identifier provided by the above output):<br />
<br />
# rfkill event unblock X<br />
<br />
=== Static IP settings revert to DHCP ===<br />
<br />
Due to an unresolved bug, when changing default connections to static IP, {{ic|nm-applet}} may not properly store the configuration change, and will revert to automatic DHCP.<br />
<br />
To work around this issue you have to edit the default connection (e.g. "Auto eth0") in {{ic|nm-applet}}, change the connection name (e.g. "my eth0"), uncheck the "Available to all users" checkbox, change your static IP settings as desired, and click '''Apply'''. This will save a new connection with the given name.<br />
<br />
Next, you will want to make the default connection not connect automatically. To do so, run {{ic|nm-connection-editor}} (''not'' as root). In the connection editor, edit the default connection (eg "Auto eth0") and uncheck "Connect automatically". Click '''Apply''' and close the connection editor.<br />
<br />
=== Cannot edit connections as normal user ===<br />
<br />
See [[#Set_up_PolicyKit_permissions]].<br />
<br />
=== Forget hidden wireless network ===<br />
<br />
Since hidden network are not displayed in the selection list of the Wireless view, they cannot be forgotten (removed) with the GUI. You can delete one with the following command:<br />
<br />
# rm /etc/NetworkManager/system-connections/[SSID]<br />
<br />
This works for any other connection.<br />
<br />
=== VPN not working in Gnome ===<br />
<br />
When setting up openconnect or vpnc connections in NetworkManager while using Gnome, you'll sometimes never see the dialog box pop up and the following error appears in /var/log/errors.log:<br />
<br />
localhost NetworkManager[399]: <error> [1361719690.10506] [nm-vpn-connection.c:1405] get_secrets_cb(): Failed to request VPN secrets #3: (6) No agents were available for this request.<br />
<br />
This is caused by the Gnome NM Applet expecting dialog scripts to be at /usr/lib/gnome-shell, when NetworkManager's packages put them in /usr/lib/networkmanager.<br />
As a "temporary" fix (this bug has been around for a while now), make the following symlink(s):<br />
<br />
# For OpenConnect<br />
ln -s /usr/lib/networkmanager/nm-openconnect-auth-dialog /usr/lib/gnome-shell/ <br />
<br />
# For VPNC (i.e. Cisco VPN)<br />
ln -s /usr/lib/networkmanager/nm-vpnc-auth-dialog /usr/lib/gnome-shell/<br />
<br />
This may need to be done for any other NM VPN plugins as well, but these are the two most common.<br />
<br />
== Tips and tricks ==<br />
<br />
=== Sharing internet connection over Wi-Fi ===<br />
<br />
You can share your internet connection (e.g.: 3G or wired) with a few clicks using nm. You will need a supported Wi-Fi card (Cards based on Atheros AR9xx or at least AR5xx are probably best choice)<br />
<br />
==== Ad-hoc ====<br />
<br />
* [[pacman|Install]] the {{Pkg|dnsmasq}} package to be able to actually share the connection.<br />
* Custom dnsmasq.conf may interfere with nm (not sure about this, but i think so).<br />
* Click on nm-applet -> Create new wireless network.<br />
* Follow wizard (if using WEP be sure to use 5 or 13 character long password, different lengths will fail).<br />
* Settings will remain stored for the next time you need it.<br />
<br />
==== Real AP ====<br />
<br />
Support of infrastructure mode (which is needed by Android phones as they intentionally do not support ad-hoc) is not currently supported by NetworkManager, but is in active development...<br />
<br />
See: http://fedoraproject.org/wiki/Features/RealHotspot<br />
<br />
=== Checking if networking is up inside a cron job or script ===<br />
<br />
Some cron jobs require networking to be up to succeed. You may wish to avoid running these jobs when the network is down. To accomplish this, add an '''if''' test for networking that queries NetworkManager's {{ic|nm-tool}} and checks the state of networking. The test shown here succeeds if any interface is up, and fails if they are all down. This is convenient for laptops that might be hardwired, might be on wireless, or might be off the network.<br />
{{bc|<nowiki><br />
if [ $(nm-tool|grep State|cut -f2 -d' ') == "connected" ]; then<br />
#Whatever you want to do if the network is online<br />
else<br />
#Whatever you want to do if the network is offline - note, this and the else above are optional<br />
fi<br />
</nowiki>}}<br />
<br />
This useful for a {{ic|cron.hourly}} script that runs {{ic|fpupdate}} for the F-Prot virus scanner signature update, as an example. Another way it might be useful, with a little modification, is to differentiate between networks using various parts of the output from {{ic|nm-tool}}; for example, since the active wireless network is denoted with an asterisk, you could grep for the network name and then grep for a literal asterisk.<br />
<br />
=== Automatically unlock keyring after login ===<br />
<br />
==== GNOME ====<br />
<br />
# Right click on the {{ic|nm-applet}} icon in your panel and select Edit Connections and open the Wireless tab<br />
# Select the connection you want to work with and click the Edit button<br />
# Check the boxes “Connect Automatically” and “Available to all users”<br />
Log out and log back in to complete.<br />
<br />
{{Note|The following method is dated and known not to work on at least one machine!}}<br />
* In {{ic|/etc/pam.d/gdm}} (or your corresponding daemon in {{ic|/etc/pam.d}}), add these lines at the end of the "auth" and "session" blocks if they do not exist already: <br />
auth optional pam_gnome_keyring.so<br />
session optional pam_gnome_keyring.so auto_start<br />
<br />
* In {{ic|/etc/pam.d/passwd}}, use this line for the 'password' block:<br />
password optional pam_gnome_keyring.so<br />
<br />
:Next time you log in, you should be asked if you want the password to be unlocked automatically on login.<br />
<br />
==== KDE ====<br />
{{Note|See http://live.gnome.org/GnomeKeyring/Pam for reference, and if you are using KDE with KDM, you can use {{AUR|pam-keyring-tool}} from the [[AUR]].}}<br />
<br />
Put a script like the following in {{ic|~/.kde4/Autostart}}:<br />
#!/bin/sh<br />
echo PASSWORD | /usr/bin/pam-keyring-tool --unlock --keyring=default -s<br />
Similar should work with Openbox, LXDE, etc.<br />
<br />
==== SLiM login manager ====<br />
See [[Slim#SLiM and Gnome Keyring]].<br />
<br />
=== Ignore specific devices ===<br />
<br />
Sometimes it may be desired that NetworkManager ignores specific devices and does not try to configure addresses and routes for them.You can quickly and easily ignore devices by MAC by using the following in {{ic|/etc/NetworkManager/NetworkManager.conf}} :<br />
[keyfile]<br />
unmanaged-devices=mac:00:22:68:1c:59:b1;mac:00:1E:65:30:D1:C4<br />
After you have put this in, [[Daemon|restart]] NetworkManager, and you should be able to configure interfaces without NetworkManager altering what you have set.<br />
<br />
=== Connect faster ===<br />
<br />
{{Merge|Network_Configuration#Additional_settings|This section contains mostly generally applicable tips.}}<br />
<br />
==== Disabling IPv6 ====<br />
<br />
Slow connection or reconnection to the network may be due to superfluous IPv6 queries in NetworkManager. If there is no IPv6 support on the local network, connecting to a network may take longer than normal while NetworkManager tries to establish an IPv6 connection that eventually times out. The solution is to disable IPv6 within NetworkManager which will make network connection faster. This has to be done once for every network you connect to.<br />
<br />
* Right-click on the network status icon.<br />
* Click on "Edit Connections".<br />
* Go to the "Wired" or "Wireless" tab, as appropriate.<br />
* Select the name of the network.<br />
* Click on "Edit".<br />
* Go to the "IPv6 Settings" tab.<br />
* In the "Method" dropdown, choose "Ignore/Disabled".<br />
* Click on "Save".<br />
<br />
==== Speed up DHCP by disabling ARP probing in DHCPCD ====<br />
<br />
{{ic|dhcpcd}} contains an implementation of a recommendation of the DHCP standard ([http://www.ietf.org/rfc/rfc2131.txt RFC2131] section 2.2) to check via ARP if the assigned IP address is really not taken. This seems mostly useless in home networks, so you can save about 5 seconds on every connect by adding the following line to {{ic|/etc/dhcpcd.conf}}:<br />
<br />
noarp<br />
<br />
This is equivalent to passing {{ic|--noarp}} to {{ic|dhcpcd}}, and disables the described ARP probing, speeding up connections to networks with DHCP.<br />
<br />
==== Use OpenDNS servers ====<br />
<br />
Create {{ic|/etc/resolv.conf.opendns}} with the nameservers:<br />
<br />
nameserver 208.67.222.222<br />
nameserver 208.67.220.220<br />
<br />
or use Google DNS servers, because people have been getting ads via the OpenDNS servers lately <br />
<br />
nameserver 8.8.8.8<br />
nameserver 8.8.4.4<br />
<br />
And have the dispatcher replace the discovered DHCP servers with the OpenDNS ones:<br />
<br />
{{hc|/etc/NetworkManager/dispatcher.d/dns-servers-opendns|<nowiki><br />
#!/bin/bash<br />
# Use OpenDNS servers over DHCP discovered servers<br />
<br />
cp -f /etc/resolv.conf.opendns /etc/resolv.conf</nowiki>}}<br />
<br />
Make the script executable:<br />
<br />
# chmod +x /etc/NetworkManager/dispatcher.d/dns-servers-opendns<br />
<br />
=== Enable DNS Caching ===<br />
<br />
DNS requests can be sped up by caching previous requests locally for subsequent lookup. NetworkManager has a plugin to enable DNS caching using dnsmasq, but it is not enabled in the default configuration. It is, however, easy to enable using the following instructions.<br />
<br />
Start by [[pacman|installing]] {{Pkg|dnsmasq}}. Then, edit {{ic|/etc/NetworkManager/NetworkManager.conf}} and add the following line under the {{ic|[main]}} section:<br />
<br />
dns=dnsmasq<br />
<br />
Now restart NetworkManager or reboot. NetworkManager will automatically start dnsmasq and add 127.0.0.1 to {{ic|/etc/resolv.conf}}. The actual DNS servers can be found in {{ic|/var/run/NetworkManager/dnsmasq.conf}}. You can verify dnsmasq is being used by doing the same DNS lookup twice with dig and verifying the server and query times.</div>Justin8https://wiki.archlinux.org/index.php?title=Music_Player_Daemon&diff=282529Music Player Daemon2013-11-13T02:17:15Z<p>Justin8: /* Autostart with systemd */</p>
<hr />
<div>[[Category:Player]]<br />
[[de:Music Player Daemon]]<br />
[[es:Music Player Daemon]]<br />
[[fr:MPD]]<br />
[[it:Music Player Daemon]]<br />
[[nl:Music Player Daemon]]<br />
[[pl:Music Player Daemon]]<br />
[[ru:Music Player Daemon]]<br />
[[sr:Music Player Daemon]]<br />
[[tr:Music_Player_Daemon]]<br />
[[zh-CN:Music Player Daemon]]<br />
{{Article summary start}}<br />
{{Article summary text|Installation, configuration and basic troubleshooting of MPD.}}<br />
{{Article summary heading|Related Articles}}<br />
{{Article summary wiki|MPD/Tips and Tricks}}<br />
{{Article summary wiki|MPD/Troubleshooting}}<br />
{{Article summary heading|Other sources}}<br />
{{Article summary text|[[Wikipedia:Music Player Daemon|Wikipedia article]]}}<br />
{{Article summary end}}<br />
<br />
'''[http://www.musicpd.org/ MPD]''' ('''m'''usic '''p'''layer '''d'''aemon) is an audio player that has a server-client architecture. It plays audio files, organizes playlists and maintains a music database all while using very few resources. In order to interface with it, a separate [[#Clients|client]] is needed.<br />
<br />
== Installation ==<br />
<br />
The latest stable version of {{Pkg|mpd}} is available in the [[official repositories]].<br />
<br />
Should users wish to run an experimental version, the [[AUR]] offers several from which to choose. For example, {{AUR|mpd-git}}.<br />
<br />
== Setup ==<br />
<br />
MPD is able to run locally (per user settings), globally (settings apply to all users), and in multiple instances. The way of setting up mpd depends on the way it is intended to be used: a local configuration may prove more useful on a desktop system, for example.<br />
<br />
For a proper MPD operation these are the necessary files and directories:<br />
<br />
* database - The music database<br />
* pid - The file where mpd stores its process ID<br />
* log - MPD logs here<br />
* state - MPD's current state is noted here<br />
* playlists - The folder where playlists are saved into<br />
* sticker.sql - The sticker database<br />
<br />
In order for MPD to be able to play back audio, [[ALSA]], [[PulseAudio]] or [[OSS]] needs to be setup and working.<br />
<br />
=== Local configuration (per user) ===<br />
<br />
MPD can be configured per user (rather than the typical method of configuring MPD globally). Running MPD as a normal user has the benefits of:<br />
<br />
* A single directory {{ic|~/.mpd}} (or any other directory under {{ic|/home/$USER/}}) that will contain all the MPD configuration files.<br />
* Easier to avoid unforeseen read/write permission errors.<br />
<br />
To setup: create a directory for the required files and the playlists; copy the example configuration locally; create all of the requisite files:<br />
<br />
$ mkdir -p ~/.mpd/playlists<br />
$ cp /usr/share/doc/mpd/mpdconf.example ~/.mpdconf<br />
$ touch ~/.mpd/{database,log,state,pid,sticker.sql}<br />
<br />
Edit {{ic|~/.mpdconf}} and specify the requisite files:<br />
<br />
{{hc|~/.mpdconf|<br />
music_directory "~/music" # Can keep commented if XDG music dir.<br />
playlist_directory "~/.mpd/playlists"<br />
db_file "~/.mpd/database"<br />
log_file "~/.mpd/log"<br />
pid_file "~/.mpd/pid"<br />
state_file "~/.mpd/state"<br />
sticker_file "~/.mpd/sticker.sql"<br />
}}<br />
<br />
MPD can now be started by typing {{ic|mpd}} on the command line (mpd first searches for {{ic|~/.mpdconf}}, then {{ic|~/etc/mpd.conf}} [there is no support for XDG-config directory {{ic|~/.config/mpd/mpd.conf}}]). To specify the location of the configuration file :<br />
<br />
mpd /path/to/mpd.conf<br />
<br />
To start MPD on login add to {{ic|~/.profile}} (or another [[Autostarting#Shells|Autostart file]]):<br />
<br />
# MPD daemon start (if no other user instance exists)<br />
[ ! -s ~/.mpd/pid ] && mpd<br />
<br />
To start with the X.org server add to either [[xprofile]] or [[xinitrc]]. Some DEs ignore these files (GNOME does) and a desktop file must be placed in {{ic|~/.config/autostart/mpd.desktop}}:<br />
<br />
{{bc|<nowiki><br />
[Desktop Entry]<br />
Encoding=UTF-8<br />
Type=Application<br />
Name=Music Player Daemon<br />
Comment=Server for playing audio files<br />
Exec=mpd<br />
StartupNotify=false<br />
Terminal=false<br />
Hidden=false<br />
X-GNOME-Autostart-enabled=false<br />
</nowiki>}}<br />
<br />
==== Autostart with systemd ====<br />
<br />
{{Note|It is assumed that you already have systemd user-session manager running. See the [[systemd/User]] page for details.}}<br />
<br />
The package {{Pkg|mpd}} provides user service file in {{ic|/usr/lib/systemd/user/mpd.service}}. Though it is a symlink to {{ic|/usr/lib/systemd/system/mpd.service}}, the difference is that the process is not started as root. The configuration file {{ic|~/.mpdconf}} is expected to exist (otherwise the global {{ic|/etc/mpd.conf}} is used). You should not use the {{ic|user}} and {{ic|group}} variables in the MPD configuration file, the process already has user permissions and therefore it is not necessary to change them further.<br />
<br />
{{Tip|If you don't like the configuration file being in {{ic|~/.mpdconf}}, copy {{ic|/usr/lib/systemd/system/mpd.service}} to {{ic|~/.config/systemd/user/mpd.service}} and edit it to contain the path of your configuration file:<br />
{{hc|~/.config/systemd/user/mpd.service|2=<br />
...<br />
ExecStart=/usr/bin/mpd --no-daemon /path/to/your/mpd.conf<br />
...<br />
}}}}<br />
<br />
{{Note|Since version 207 of systemd, it uses a different PAM service for user@.service, and includes an incorrect default PAM config.<br />
Fix it with: {{ic|# sed -i s/system-auth/system-login/g /etc/pam.d/systemd-user}} (or replace all occurrences of {{ic|system-auth}} in that file with {{ic|system-login}}). Skipping this will result in starting the mpd service returning 'Failed to issue method call: No such file or directory' See [[Systemd/User]] for more information}}<br />
<br />
All you have to do is enable and start the user service:<br />
<br />
$ systemctl --user enable mpd<br />
$ systemctl --user start mpd<br />
<br />
{{Note|Also make sure to disable every other method of starting mpd you used before.}}<br />
<br />
==== Scripted configuration ====<br />
<br />
Rasi has written a script that will create the proper directory structure, configuration files and prompt for the location of the user's Music directory; it can be downloaded [http://53280.de/dl/mpdsetup.sh here].<br />
<br />
=== Global configuration ===<br />
<br />
{{Warning|Users of PulseAudio with a local mpd have to implement a [[Music Player Daemon/Tips and Tricks#Local (with separate mpd user)|workaround]] in order to run mpd as its own user!}}<br />
<br />
The default Arch install keeps the setup in {{ic|/var/lib/mpd}} and uses ''mpd'' as default user.<br />
<br />
Edit {{ic|/etc/mpd.conf}} to reflect as such:<br />
{{hc|/etc/mpd.conf|<br />
music_directory "/path/to/music/dir"<br />
playlist_directory "/var/lib/mpd/playlists"<br />
db_file "/var/lib/mpd/mpd.db"<br />
log_file "syslog"<br />
pid_file "/run/mpd/mpd.pid"<br />
state_file "/var/lib/mpd/mpdstate"<br />
user "mpd"<br />
}}<br />
<br />
We just configured MPD to run as the ''mpd'' user, but {{ic|/var/lib/mpd}} is owned by ''root'' by default, we need to change this so ''mpd'' can write here:<br />
# chown -R mpd /var/lib/mpd<br />
<br />
==== Music directory ====<br />
<br />
MPD needs to have {{ic|+x}} permissions on ''all'' parent directories to the music collection (ie. if it's located outside of {{ic|/var/lib/mpd}}). Thus users will most likely need to remount the music directory under a directory that mpd has access to -- this only applies if running as the 'mpd' user.<br />
<br />
# mkdir /var/lib/mpd/music<br />
# echo "/path/to/music/dir /var/lib/mpd/music none bind" >> /etc/fstab<br />
# mount -a<br />
Also see [https://bbs.archlinux.org/viewtopic.php?id=86449 this forum thread].<br />
<br />
An additional solution would be to just create a symbolic link into {{ic|/var/lib/mpd/music}}.<br />
# mkdir /var/lib/mpd/music<br />
# ln -s /path/to/music/dir /var/lib/mpd/music/<br />
<br />
If the music collection is contained under multiple directories, create symbolic links under the main music directory in {{ic|/var/lib/mpd}}. Remember to set permissions accordingly on the directories being linked.<br />
<br />
==== Start MPD ====<br />
<br />
MPD can be controlled with the ''mpd'' [[daemon]]. The first startup can take some time as MPD will scan your music directory.<br />
<br />
Test everything by starting a client application ({{Pkg|ncmpc}} is a light and easy to use client), and play some music!<br />
<br />
==== Configure audio ====<br />
<br />
To change the volume for mpd independent from other programs, uncomment or add this switch in mpd.conf:<br />
{{hc|/etc/mpd.conf|<br />
mixer_type "software"<br />
}}<br />
<br />
Users of [[ALSA]] will want to have the following device definition, which allows software volume control in the MPD client to control the volume separately from other applications.<br />
{{hc|/etc/mpd.conf|2=<br />
audio_output {<br />
type "alsa"<br />
name "My Sound Card"<br />
mixer_type "software" # optional<br />
}<br />
}}<br />
<br />
Users of [[PulseAudio]] will need to make the following modification:<br />
{{hc|/etc/mpd.conf|2=<br />
audio_output {<br />
type "pulse"<br />
name "pulse audio"<br />
}<br />
}}<br />
<br />
PulseAudio supports multiple advanced operations, e.g. transferring the audio to a different machine. For advanced configuration with MPD see [http://mpd.wikia.com/wiki/PulseAudio Music Player Daemon Community Wiki].<br />
<br />
==== Changing user ====<br />
<br />
{{Note|This is only required if you change the user!}}<br />
Changing the group that MPD runs as may result in errors like "output: Failed to open "My ALSA Device"" "[alsa]: Failed to open ALSA device "default": No such file or directory" "player_thread: problems opening audio device while playing "Song Name.mp3""<br />
<br />
This is because by default MPD runs as member of '''audio''' group and the sound devices under {{Ic|/dev/snd/}} are owned by this group, so add user {{Ic|mpd}} to group {{Ic|audio}}:<br />
# gpasswd -a mpd audio<br />
<br />
==== Timeline of MPD startup ====<br />
<br />
To depict when MPD drops its superuser privileges and assumes those of the user set in the configuration, the timeline of a normal MPD startup is listed here:<br />
<br />
# Since MPD is started as root by systemd, it first reads the {{ic|/etc/mpd.conf}} file.<br />
# MPD reads the user variable in the {{ic|/etc/mpd.conf}} file, and changes from root to this user.<br />
# MPD then reads the contents of the {{ic|/etc/mpd.conf}} file and configures itself accordingly.<br />
<br />
Notice that MPD changes the running user from root to the one named in the {{ic|/etc/mpd.conf}} file. <br />
This way, uses of {{ic|~}} in the configuration file point correctly to the home user's directory, and not root's directory. <br />
It may be worthwhile to change all uses of {{ic|~}} to {{ic|/home/username}} to avoid any confusion over this aspect of MPD's behavior.<br />
<br />
=== Multi-mpd setup ===<br />
<br />
'''Useful if running an icecast server.'''<br />
<br />
For a second MPD (e.g., with icecast output to share music over the network) using the same music and playlist as the one above, simply copy the above configuration file and make a new file (e.g., {{ic|/home/username/.mpd/config-icecast}}), and only change the log_file, error_file, pid_file, and state_file parameters (e.g., {{ic|mpd-icecast.log}}, {{ic|mpd-icecast.error}}, and so on); using the same directory paths for the music and playlist directories would ensure that this second mpd would use the same music collection as the first one e.g., creating and editing a playlist under the first daemon would affect the second daemon as well. Users do not have to create the same playlists all over again for the second daemon. Call this second daemon the same way from {{ic|~/.xinitrc}} above. (Just be sure to have a different port number, so as to not conflict with the first mpd daemon).<br />
<br />
== Clients ==<br />
<br />
A separate client is needed to control mpd. See a long list of clients at the [http://mpd.wikia.com/wiki/Clients mpd wiki]. Popular options are:<br />
<br />
=== Console ===<br />
<br />
*{{App|mpc|Simple KISS client. All basic functionality available|http://mpd.wikia.com/wiki/Client:Mpc|{{Pkg|mpc}}}}<br />
*{{App|ncmpc|Ncurses client for mpd|http://mpd.wikia.com/wiki/Client:Ncmpc|{{Pkg|ncmpc}}}}<br />
*{{App|[[ncmpcpp]]|Almost exact clone of ncmpc with some new features written in C++ (tag editor, search engine)|http://unkart.ovh.org/ncmpcpp/|{{Pkg|ncmpcpp}}}}<br />
*{{App|pms|Highly configurable and accessible ncurses client|http://pms.sourceforge.net/|{{AUR|pmus}}}}<br />
*{{App|vimpc|Ncurses based MPD client with vi-like key bindings|http://sourceforge.net/projects/vimpc/|{{AUR|vimpc}}}}<br />
<br />
=== Graphical ===<br />
<br />
*{{App|Ario|Very feature-rich GTK2 GUI client for mpd, inspired by Rhythmbox|http://ario-player.sourceforge.net/|{{Pkg|ario}}}}<br />
*{{App|QmpdClient|GUI client written with Qt 4.x|http://bitcheese.net/wiki/QMPDClient|{{Pkg|qmpdclient}}}}<br />
*{{App|Sonata|Elegant Python GTK+ client|http://sonata.berlios.de/|{{Pkg|sonata}}}}<br />
*{{App|gmpc|GTK2 frontend for Music Player Daemon. It is designed to be lightweight and easy to use, while providing full access to all of MPD's features. Users are presented with several different methods to browse through their music. It can be extended by plugins, of which many are available.|http://gmpc.wikia.com/wiki/Gnome_Music_Player_Client|{{Pkg|gmpc}}}}<br />
*{{App|Dmpc|Dmenu-based MPC client with a playlist manager and state-saving on playlist changes|http://wintervenom.mine.nu/|{{AUR|dmpc}}}}<br />
*{{App|Cantata|High-feature, Qt4/KDE4 client for MPD with very configurable interface|https://code.google.com/p/cantata/|{{AUR|cantata-qt}}}}<br />
<br />
=== Web ===<br />
<br />
*{{App|Patchfork|Web client for MPD written in PHP and Ajax|http://mpd.wikia.com/wiki/Client:Pitchfork|{{AUR|patchfork-git}}}}.<br />
<br />
== See also ==<br />
<br />
* [http://forum.musicpd.org/ MPD Forum]<br />
* [http://www.musicpd.org/doc/user/ MPD User Manual]</div>Justin8https://wiki.archlinux.org/index.php?title=VirtualBox&diff=282422VirtualBox2013-11-12T02:10:38Z<p>Justin8: /* Install the Guest Additions */ Removed instructions to mount ISO before installing package; they are not needed and are confusing to readers.</p>
<hr />
<div>[[Category:Emulators]]<br />
[[Category:Virtualization]]<br />
[[cs:VirtualBox]]<br />
[[de:VirtualBox]]<br />
[[el:VirtualBox]]<br />
[[es:VirtualBox]]<br />
[[fr:VirtualBox]]<br />
[[hu:VirtualBox]]<br />
[[it:VirtualBox]]<br />
[[ja:VirtualBox]]<br />
[[pt:VirtualBox]]<br />
[[ru:VirtualBox]]<br />
[[zh-CN:VirtualBox]]<br />
{{Article summary start}}<br />
{{Article summary text|This article is about basic usage of VirtualBox, including running the VirtualBox software within an Arch Linux ''host'', and running an Arch Linux ''guest'' inside a VirtualBox virtual machine.}}<br />
{{Article summary heading|Required software}}<br />
{{Article summary link|VirtualBox|https://www.virtualbox.org}}<br />
{{Article summary heading|Related}}<br />
{{Article summary wiki|VirtualBox Extras}}<br />
{{Article summary wiki|PhpVirtualBox}}<br />
{{Article summary wiki|VirtualBox Arch Linux Guest On Physical Drive}}<br />
{{Article summary wiki|Installing Arch Linux from VirtualBox}}<br />
{{Article summary wiki|Moving an existing install into (or out of) a virtual machine}}<br />
{{Article summary end}}<br />
<br />
'''VirtualBox''' is a [[Wikipedia:Hypervisor|hypervisor]] used to run operating systems in a special environment, called virtual machine, on top of the existing operating system. It is in constant development and new features are implemented continuously. It comes with a [[Qt]] GUI interface, as well as headless and [[Wikipedia:SDL|SDL]] command-line tools for managing and running virtual machines.<br />
<br />
In order to integrate functions of the host system to the guests, including shared folders and clipboard, video acceleration and a seamless window integration mode, ''guest additions'' are provided for some guest operating systems.<br />
<br />
{{Wikipedia|VirtualBox}}<br />
<br />
== Installation steps for the host ==<br />
<br />
In order to launch VirtualBox virtual machines on your Arch Linux box, follow these installation steps.<br />
<br />
=== Basic packages ===<br />
<br />
First, from the [[official repositories]], install the {{Pkg|virtualbox}} package which contains the GPL-licensed VirtualBox suite. This package comes with the SDL and headless command-line tools included.<br />
<br />
This package comes with the {{Pkg|qt4}} package as an optional dependency, install it as well in order to use the graphical interface which is based on [[Qt]]. This is not required if you intend to use VirtualBox in command-line only. [[#Use the right front-end|See below to learn the differences]].<br />
<br />
=== VirtualBox kernel modules ===<br />
<br />
Next, in order for VirtualBox to virtualize your guest installation, you will need to add [[kernel modules]] to your host kernel. The needed package you need to install depends if you are using a kernel from the [[official repositories]] or a custom one (like those available in [[AUR]]).<br />
<br />
==== Hosts running an official kernel ====<br />
<br />
* If you are using the {{Pkg|linux}} kernel, install the {{pkg|virtualbox-host-modules}} package.<br />
* If you are using the LTS version of the kernel ({{pkg|linux-lts}}), you should install the {{pkg|virtualbox-host-modules-lts}} instead. <br />
<br />
{{Note|Each time an official Arch Linux kernel package gets upgraded, the VirtualBox kernel modules packages get updated accordingly. In order to use the newly updated modules, you need to reload them manually.}}<br />
<br />
==== Hosts running a custom kernel ====<br />
<br />
The {{ic|virtualbox-host-modules}} package is reported to work fine with some custom kernels such as {{AUR|Linux-ck}}. However, if you attempt to install the {{ic|virtualbox-host-modules}} package, the official Arch Linux kernel ({{Pkg|linux}}) will be installed with it as well, as it comes as a dependency. If you want to prevent your package manager from installing these unneeded packages, please install the {{Pkg|virtualbox-host-dkms}} package instead. The latter comes bundled with the source of the VirtualBox kernel modules that will be compiled to generate these modules for your kernel.<br />
<br />
{{Note|As the {{Pkg|virtualbox-host-dkms}} package requires compilation, make sure you have the kernel headers corresponding to your custom kernel version to prevent this error from happening {{ic|Your kernel headers for kernel ''your custom kernel version'' cannot be found at /usr/lib/modules/''your custom kernel version''/build or /usr/lib/modules/''your custom kernel version''/source}}.}}<br />
<br />
Once {{Pkg|virtualbox-host-dkms}} is installed, simply generate the kernel modules for your custom kernel by running the following command structure:<br />
# dkms install vboxhost/''virtualbox-host-source version'' -k ''your custom kernel version''/''your architecture''<br />
<br />
{{Tip|Use this all-in-one command instead, if you do not want to adapt the above command:<br />
# dkms install vboxhost/$(pacman -Q virtualbox|awk {'print $2'}|sed 's/\-.\+//') -k $(uname -rm|sed 's/\ /\//')}}<br />
<br />
To automatically recompile/reload the VirtualBox kernel modules when your custom kernel gets upgraded and avoid to use the above {{ic|dkms install}} command manually, enable the {{ic|dkms}} service with:<br />
# systemctl enable dkms<br />
<br />
{{Note|If you do not have the {{ic|dkms}} service enabled while the kernel is being updated, the VirtualBox modules will not be updated and will not work with your new kernel version any more. You will have to type in the {{ic|dkms install}} command described above manually.}}<br />
<br />
If you want to keep that {{ic|dkms}} deamon disabled, there is another way to perform the VirtualBox module recompilation automatically, but it requires reboot. You can use for that an [[mkinitcpio|initramfs hook]] that will automatically call the {{ic|dkms install}} command described above at boot time.<br />
<br />
To enable that feature, install the {{AUR|vboxhost-hook}} package from the [[Arch User Repository|AUR]] and add {{ic|vboxhost}} to your HOOKS array in {{ic|/etc/mkinitcpio.conf}}. Again, make sure the right linux headers are available for the new kernel otherwize the compilation will fail.<br />
<br />
{{Tip|Like the {{ic|dkms}} command, the {{ic|vboxhost}} hook will tell you if anything goes wrong during the recompilation of the VirtualBox modules.}}<br />
<br />
=== Load the VirtualBox kernel modules ===<br />
<br />
Among the [[kernel modules]] VirtualBox uses, there is a mandatory module named {{ic|vboxdrv}}, which must be loaded before any virtual machines can run. It can be automatically loaded when Arch Linux starts up, or it can be loaded manually when necessary.<br />
<br />
To load the module manually:<br />
# modprobe vboxdrv<br />
<br />
To load the VirtualBox module at boot time, refer to [[Kernel_modules#Loading]] and use the file name {{ic|virtualbox.conf}}<br />
<br />
{{Note|In order to avoid {{ic|no such file or directory}} errors when loading {{ic|vboxdrv}}, you may need to update the kernel modules database with {{ic|depmod -a}}.}}<br />
<br />
To ensure full functionality of bridged networking, ensure that the {{ic|vboxnetadp}}, {{ic|vboxnetflt}} and {{ic|vboxpci}} [[Kernel modules|kernel modules]] are loaded too and that the {{pkg|net-tools}} package is installed.<br />
<br />
=== Add usernames to the vboxusers group ===<br />
<br />
To use the USB ports of your host machine in your virtual machines, add to the {{ic|vboxusers}} [[group]] the usernames that will be authorized to use this feature. The new group does not automatically apply to existing sessions; the user has to log out and log in again, or start a new environment with a command like {{ic|newgrp}} or with {{ic|sudo -u $USER -s}}. To add the current user to the {{ic|vboxusers}} group, type:<br />
# gpasswd -a $USER vboxusers<br />
<br />
=== Guest additions disc ===<br />
<br />
It is also recommended to install the {{Pkg|virtualbox-guest-iso}} package on the host running VirtualBox. This package will act as a disc image that can be used to install the guest additions onto guest systems. See [[#Install Guest Additions|the section below]] to learn how to use these additions.<br />
<br />
=== Use the right front-end ===<br />
<br />
Now you are ready to use VirtualBox. Congratulations!<br />
<br />
Multiple front-ends are available to you which two are available by default:<br />
* If you want to use VirtualBox in command-line only (only launch and configure virtual machines), you can use the {{ic|VBoxSDL}} command. VBoxSDL does only provide a simple window that contains only the ''pure'' virtual machine, without menus or other controls.<br />
* If you want to use VirtualBox in command-line without any GUI running (e.g. on a server) to create, launch and configure virtual machines, use the {{ic|VBoxHeadless}} which produces no visible output on the host at all, but instead only delivers VRDP data.<br />
<br />
If you installed the {{Pkg|qt4}} optional dependency, you also have a nice looking GUI interface with menus and usable with the mouse.<br />
<br />
Finally, you can use [[PhpVirtualBox]] to administrate you virtual machines via a web interface.<br />
<br />
Refer to the [https://www.virtualbox.org/manual VirtualBox manual] to learn how to create virtual machines.<br />
<br />
== Arch Linux as a guest in a Virtual Machine ==<br />
<br />
Installing Arch Linux under VirtualBox is straightforward, and additions should be installed through pacman (following the instructions below), and not through the "Install Guest Additions" menu item in VirtualBox on your host machine, nor from a mounted ISO image.<br />
<br />
=== Install the Guest Additions ===<br />
<br />
Install the {{Pkg|virtualbox-guest-utils}} package. Manually load the modules with:<br />
<br />
# modprobe -a vboxguest vboxsf vboxvideo<br />
<br />
Create a {{ic|*.conf}} file (e.g. {{ic|virtualbox.conf}}) in {{ic|/etc/modules-load.d/}} with these lines:<br />
<br />
{{hc|/etc/modules-load.d/virtualbox.conf|<br />
vboxguest<br />
vboxsf<br />
vboxvideo}}<br />
<br />
Add the following line to the top of {{ic|~/.xinitrc}} above any {{ic|exec}} options. (create a new file if it does not exist):<br />
<br />
{{hc|~/.xinitrc|<br />
/usr/bin/VBoxClient-all}}<br />
<br />
{{Note|If you are creating a new {{ic|~/.xinitrc}} file you ''must'' also include a [[window manager]] or [[desktop environment]].}}<br />
<br />
=== Automatic re-compilation of the VirtualBox guest modules with every update of any kernel ===<br />
<br />
This is possible thanks to {{AUR|vboxguest-hook}} from the [[Arch User Repository|AUR]]. In '''vboxguest-hook''', the 'automatic re-compilation' functionality is done by a '''vboxguest hook''' on [[mkinitcpio]] after forcing to update the {{pkg|linux-headers}} package. You will need to add {{ic|vboxguest}} to the HOOKS array in {{ic|/etc/mkinitcpio.conf}}. You may need to manually recreate the initramfs after an upgrade of the {{pkg|linux-headers}} package.<br />
<br />
The hook will call the {{ic|dkms}} command to update the VirtualBox guest modules for the version of your new kernel.<br />
<br />
{{Note|If you are using this functionality, it is '''important''' to look at the installation process of the {{pkg|linux}} (or any other kernel) package. vboxguest hook will tell you if anything goes wrong.}}<br />
<br />
=== Start the sharing services ===<br />
<br />
After installing {{Pkg|virtualbox-guest-utils}} above, you should start {{ic|VBoxClient-all}} to start services for sharing the clipboard, resizing the screen, etc.<br />
* If you are running something that launches {{Ic|/etc/xdg/autostart/vboxclient.desktop}}, such as GNOME or KDE, then nothing needs to be done. <br />
* If you use {{Ic|.xinitrc}} to launch things instead, you must add the following to your {{Ic|.xinitrc}} before launching your WM.<br />
<br />
# VBoxClient-all &<br />
<br />
=== Using USB webcam / microphone ===<br />
<br />
{{Note|You will need to have VirtualBox extension pack installed before following the steps below. See [[VirtualBox_Extras#Extension_pack]] for details.}}<br />
<br />
# Make sure the virtual machine is not running and your webcam / microphone is not being used.<br />
# Bring up the main VirtualBox window and go to settings for Arch machine. Go to USB section.<br />
# Make sure "Enable USB Controller" is selected. Also make sure that "Enable USB 2.0 (EHCI) Controller" is selected too.<br />
# Click the "Add filter from device" button (the cable with the '+' icon).<br />
# Select your USB webcam/microphone device from the list.<br />
# Now click OK and start your VM.<br />
<br />
=== Using Arch under Virtualbox EFI mode ===<br />
<br />
My experience with this configuration was pretty terrible, but it does work.<br />
<br />
''UPD. Using efibootmgr has the same effect as using VirtualBox boot menu (see the note below): settings [https://www.virtualbox.org/ticket/11177 disappear] after VM shutdown.'' First, {{ic|efibootmgr}} does *not* work. It will appear to work, but all changes it makes appear to be overwritten on reboot. After performing a standard UEFI/GPT installation, reboot and you should get dumped to the EFI shell. Type exit and you will get a menu. Select the Boot Management Manager, Boot Options, Add Boot Option. Use the file browser to find the grub efi file and select it. Add a label if you want. Afterwards, select Change Boot Order from the menu, use arrow keys to select your Arch option, and {{ic|+}} to move it up to the top. GRUB should boot by default now.<br />
<br />
Other options are: 1) move your loader to {{ic|\EFI\boot\bootx64.efi}}, 2) create {{ic|\startup.nsh}} script, which executes desirable loader, like this:<br />
<br />
{{hc|\startup.nsh|<br />
HD16a0a1:\EFI\refind\refindx64.efi}}<br />
<br />
Here I'm using consistent mapping name (HD16a0a1). It is probably a good idea, because they do survive configuration changes.<br />
<br />
{{Note|Another useful way to get back to the EFI menu after autobooting is working is to press the {{ic|c}} key inside GRUB and type {{ic|exit}}. Obviously, this will only work with {{ic|grub-efi}}, not {{ic|grub-bios.}}<br />
<br />
Regenerating the {{ic|grub.cfg}} file may also be required to fix broken UUIDs. Check with the {{ic|lsblk -f}} command that they match.<br><br />
Yet another useful way to get to VirtualBox boot menu is pressing {{ic|F12}} right after starting virtual machine. It comes in handy when using rEFInd + EFISTUB, for example.}}<br />
<br />
=== Synchronize guest date with host ===<br />
<br />
To keep the date and time synchronized, make sure you have {{Pkg|virtualbox-guest-utils}} installed in your host (see [[#Install the Guest Additions|above]]). To enable the service for subsequent boots, run<br />
# systemctl enable vboxservice<br />
<br />
To start immediately, run<br />
# systemctl start vboxservice<br />
<br />
You also need run this daemon in order to use the auto-mounting feature of shared folders that are mentioned above.<br />
<br />
=== Enable shared folders ===<br />
<br />
Shared folders are managed via the VirtualBox program on the host. They may be added, auto-mounted and made read-only from there.<br />
<br />
If automounting is enabled, and the {{ic|vboxservice}} is enabled, creating a shared folder from the VirtualBox program on the host will mount that folder in {{ic|/media/sf_''SHAREDFOLDERNAME''}} on the guest. To have that folder created on the Arch Guest, after the Guest Additions have been installed, you need to add your username to the {{ic|vboxsf}} group.<br />
<br />
# groupadd vboxsf<br />
# gpasswd -a $USER vboxsf<br />
<br />
{{Note|For '''automounting''' to work, you have to enable the '''vboxservice''' service.}}<br />
<br />
If you want a shared folder (e.g {{ic|/media/sf_Dropbox}}) to be symlinked to another folder in your home directory for easy access, you can type on the guest:<br />
<br />
$ ln -s /media/sf_Dropbox/* ~/dropbox<br />
<br />
The {{ic|VBoxLinuxAdditions.run}} script provided in the Guest Additions iso does this for you, however, Arch does not recommend using it.<br />
<br />
If shared folders are not auto-mounted, try [https://bbs.archlinux.org/viewtopic.php?id=70780 manually mount].<br />
<br />
To prevent startup problems when you're using [[systemd]], you should add {{ic|1=comment=systemd.automount}} to your {{ic|/etc/fstab}}. This way, they are mounted only when you access those mount points and not during startup. Otherwise your system might become unusable after a kernel upgrade (if you install your guest additions manually).<br />
<br />
desktop /media/desktop vboxsf uid=user,gid=group,rw,dmode=700,fmode=600,comment=systemd.automount 0 0<br />
<br />
Don't waste your time to test the {{ic|nofail}} option. {{ic|mount.vboxsf}} is not able to handle this (2012-08-20).<br />
<br />
desktop /media/desktop vboxsf uid=user,gid=group,rw,dmode=700,fmode=600,nofail 0 0<br />
<br />
== Export VirtualBox virtual machines to other hypervisors ==<br />
<br />
If you plan to use your virtual machine, created with VirtualBox, on another computer which has not necessarily VirtualBox installed, you might be interested in following the next steps.<br />
<br />
=== Remove additions ===<br />
<br />
If you have installed the VirtualBox additions to your VirtualBox virtual machine, please uninstall them first. Your guest, especially if it is using an OS from the Windows family, might behave weirdly, crash or even might not boot at all if you are still using the specific VirtualBox drivers in another hypervisor.<br />
<br />
{{Tip|If you intend to use a virtualization solution from Parallels Inc for your Mac, the product ''Parallels Transporter'' can be used to create a virtual machine from a Windows or GNU/Linux virtual machine (or even from a native installation). With such a product, you do not need to apply follow the next step and can stop reading here.}}<br />
<br />
=== Use the right virtual disk format ===<br />
<br />
==== Supported formats by VirtualBox ====<br />
<br />
VirtualBox comes with its own container for the virtual hard drives: the Virtual Disk Image (VDI) file format. Even if this format is used by default when you create a virtual machine with VirtualBox, you can specify another one. Indeed VirtualBox does flawlessly support other formats:<br />
<br />
* VMDK: this format has been initially developed by VMware for their products, but it is now an open format. If you intend to use any VMware product, you will need to use this format since it is the only one supported by VMware.<br />
<br />
* VHD: this is the format used by Microsoft in Windows Virtual PC and Hyper-V. If you intend to use any of these Microsoft products, you will have to choose this format.<br />
:{{Tip|Since Windows 7, this format can be mounted directly without any additional application.}} <br />
<br />
* Version 2 of the HDD format used by Parallels (Desktop for Mac).<br />
<br />
* QED and QCOW used by QEMU.<br />
<br />
The format you will need to choose depends on the hypervisor that will be used.<br />
<br />
==== Specific virtual disk format differences ====<br />
<br />
Before converting your virtual drive, please keep in mind these specific virtual disk format differences:<br />
<br />
* The VMDK does offer the ability to be split into several files of up to 2GB. This feature is specially useful if you want to store the virtual machine on machines which do not support very large files. Other formats do not provide such an equivalent feature.<br />
<br />
* Changing the logical capacity of an existing virtual drive with VirtualBox {{ic|VBoxManage}} command is only supported for VDI and VHD formats used in dynamic allocation mode to expand (not shrink) their capacity.<br />
<br />
==== Convert your virtual disk format ====<br />
<br />
VirtualBox only supports the virtual disk convertion between VDI, VMDK and VHD formats. Here is an example of convertion from a VDI to VMDK vitual drive.<br />
<br />
$ VBoxManage clonehd ''ArchLinux_VM.vdi'' ''ArchLinux_VM.vmdk'' --format ''VMDK''<br />
<br />
If you want to replace the virtual disk you defined during the virtual machine creation process by the one you have just converted, use {{ic|[http://www.virtualbox.org/manual/ch08.html#vboxmanage-storagectl VBoxManage storagectl]}}, or use the GUI, or [[#Replace_the_virtual_disk_manually_from_the_.vbox_file|modify the ''.vbox'' configuration file]].<br />
<br />
=== Create the VM configuration for your hypervisor ===<br />
<br />
If your hypervisor (like VMware) does not support import of VirtualBox configuration files (''.vbox''), you will have to create a new virtual machine and specify its hardware configuration as close as possible as your initial VirtualBox virtual machine.<br />
<br />
{{Note|Pay a close attention to the installation mode (BIOS or UEFI) used to install the guest operating system. While an option is available on VirtualBox to choose between these 2 modes, on VMware, you will have to add the following line to your ''.vmx'' file.<br />
<br />
{{hc|ArchLinux_vm.vmx|2=<br />
firmware = "efi"<br />
}}<br />
}}<br />
<br />
Finally, ask your hypervisor to use the existing virtual disk you have converted and launch the virtual machine.<br />
{{Tip|If you are using VMware products and do not want to run through the whole GUI to find the right location to add your new virtual drive device, you can replace the location of the current ''.vmdk'' file by editing your ''.vmx'' configuration file manually.}}<br />
<br />
== Advanced configuration ==<br />
<br />
=== Replace the virtual disk manually from the ''.vbox'' file ===<br />
<br />
If you think that editing a simple ''XML'' file is more convenient than playing with the GUI or with {{ic|VBoxManage}} and you want to replace (or add) a virtual disk to your virtual machine, simply replace in the ''.vbox'' configuration file corresponding to your virtual machine the GUID, the file location and the format to your needs:<br />
<br />
{{hc|ArchLinux_vm.vbox|2=<br />
<HardDisk uuid="''{670157e5-8bd4-4f7b-8b96-9ee412a712b5}''" location="''ArchLinux_vm.vdi''" format="''VDI''" type="Normal"/><br />
}}<br />
<br />
then in the {{ic|<AttachedDevice>}} sub-tag of {{ic|<StorageController>}}, replace the GUID by the new one.<br />
<br />
{{hc|ArchLinux_vm.vbox|2=<br />
<AttachedDevice type="HardDisk" port="0" device="0"><br />
<Image uuid="''{670157e5-8bd4-4f7b-8b96-9ee412a712b5}''"/><br />
</AttachedDevice><br />
}}<br />
<br />
{{Note|If you do not know the GUID of the drive you want to add, but you have just used {{ic|VBoxManage}} for the convertion, this command will output the GUID just after the convertion. Using a random GUID does not work, as each [http://www.virtualbox.org/manual/ch05.html#cloningvdis UUID is stored inside each disk images].}}<br />
<br />
=== Starting virtual machines with a service ===<br />
<br />
Find hereinafter the implementation details of a systemd service that will be used to consider a virtual machine as a service.<br />
<br />
{{hc|/etc/systemd/system/vboxvmservice@.service|<nowiki><br />
[Unit]<br />
Description=VBox Virtual Machine %i Service<br />
Requires=systemd-modules-load.service<br />
After=systemd-modules-load.service<br />
<br />
[Service]<br />
User=</nowiki>{{ic|'''<user>'''}}<nowiki><br />
Group=vboxusers<br />
ExecStart=/usr/bin/VBoxHeadless -s %i<br />
ExecStop=/usr/bin/VBoxManage controlvm %i savestate<br />
<br />
[Install]<br />
WantedBy=multi-user.target</nowiki>}}<br />
<br />
{{Note|Replace {{ic|'''<user>'''}} with a user that is a member of the {{ic|vboxusers}} group. Make sure the user chosen is the same user that will create/import virtual machines, otherwise the user will not see the VM appliances.}}<br />
<br />
To enable the service that will launch the virtual machine at next boot, use:<br />
# systemctl enable vboxvmservice@'''your virtual machine name'''<br />
<br />
To start the service that will launch directly the virtual machine, use:<br />
# systemctl start vboxvmservice@'''your virtual machine name'''<br />
<br />
VirtualBox 4.2 introduces [http://lifeofageekadmin.com/how-to-set-your-virtualbox-vm-to-automatically-startup/ a new way] for UNIX-like systems to have virtual machines started automatically, other than using a systemd service.<br />
<br />
== Troubleshooting ==<br />
<br />
=== VBOX_E_INVALID_OBJECT_STATE (0x80BB0007) ===<br />
<br />
This can occur if a VM is exited ungracefully. The solution to unlock the VM is trivial:<br />
$ VBoxManage controlvm <your virtual machine name> poweroff<br />
<br />
=== USB subsystem is not working on the host or guest ===<br />
<br />
Sometimes, on old Linux hosts, the USB subsystem is not auto-detected resulting in an error {{ic|Could not load the Host USB Proxy service: VERR_NOT_FOUND}} or in a not visible USB drive on the host, [https://bbs.archlinux.org/viewtopic.php?id=121377 even when the user is in the '''vboxusers''' group]. This problem is due to the fact that VirtualBox switched from ''usbfs'' to ''sysfs'' in version 3.0.8. If the host doesn't understand this change, you can revert to the old behaviour by defining the following environment variable in any file that is sourced by your shell (e.g. your {{ic|~/.bashrc}} if you're using ''bash''):<br />
<br />
{{hc|~/.bashrc|VBOX_USB<nowiki>=</nowiki>usbfs}}<br />
<br />
Then make sure, the environment has been made aware of this change (reconnect, source the file manually, launch a new shell instance or reboot).<br />
<br />
Also make sure that your user is a member of the {{ic|storage}} group.<br />
<br />
=== Failed to create the host-only network interface ===<br />
<br />
To be able to create a ''Host-Only Network Adapter'' or a ''Bridged Network Adapter'', the kernel modules {{ic|vboxnetadp}} and {{ic|vboxnetflt}} need to be loaded, you also need to make sure the {{pkg|net-tools}} package is installed. You can load these kernel modules manually with:<br />
# modprobe -a vboxdrv vboxnetadp vboxnetflt<br />
<br />
To load these modules automatically at boot, refer to [[Kernel_modules#Loading]] and use a program name of {{ic|virtualbox}}.<br />
<br />
=== WinXP: Bit-depth cannot be greater than 16 ===<br />
<br />
If you are running at 16-bit color depth, then the icons may appear fuzzy/choppy. However, upon attempting to change the color depth to a higher level, the system may restrict you to a lower resolution or simply not enable you to change the depth at all. To fix this, run {{ic|regedit}} in Windows and add the following key to the Windows XP VM's registry:<br />
[HKEY_LOCAL_MACHINE\SOFTWARE\Policies\Microsoft\Windows NT\Terminal Services]<br />
"ColorDepth"=dword:00000004<br />
<br />
Then update the color depth in the "desktop properties" window. If nothing happens, force the screen to redraw through some method (i.e. {{ic|Host+f}} to redraw/enter full screen).<br />
<br />
=== Mounting .vdi images ===<br />
<br />
Mounting vdi images only works with fixed size (''static'') images; ''dynamic size'' images aren't easily mountable.<br />
<br />
First we need one information from your .vdi image:<br />
<br />
$ VBoxManage internalcommands dumphdinfo <your .vdi file location> | grep offData<br />
Header: offBlocks=4096 offData=69632<br />
<br />
Then, add to your {{ic|offData}} 32256. (e.g. 32256 + 69632 = 101888)<br />
<br />
Now you can mount your vdi image with the following command:<br />
<br />
# mount -t ext4 -o rw,noatime,noexec,loop,offset=101888 <your .vdi file location> /mnt/<br />
<br />
=== Copy and paste not working on Arch Linux guest ===<br />
<br />
After {{ic|virtualbox-guest-additions}} has been updated to version {{ic|4.2.0-2}}, the copy and paste feature from the host to Arch Linux guest does not work any more. It seems to be due to {{ic|VBoxClient-all}} requiring root access. In previous versions adding {{ic|VBoxClient-all &}} to {{ic|~/.xinitrc}} was sufficient to make copy and paste work again. Now, in order to solve the problem, update {{ic|~/.xinitrc}} to match {{ic|sudo VBoxClient-all &}} and add the line {{ic|, NOPASSWD: /usr/bin/VBoxClient-all}} to your username in {{ic|/etc/sudoers}} and restart X. It should all work again. The line in the sudoers file should look similar to this:<br />
<br />
# Allow sudo for user 'you' and let him run VBoxClient-all without requiring a password<br />
you ALL = PASSWD: ALL, NOPASSWD: /usr/bin/VBoxClient-all<br />
<br />
{{Note|Use {{ic|visudo}} to edit the {{ic|/etc/sudoers}} file. This will check for syntax errors when saving.}}<br />
<br />
=== Use serial port in guest OS ===<br />
<br />
Check you permission for the serial port:<br />
$ /bin/ls -l /dev/ttyS*<br />
crw-rw---- 1 root uucp 4, 64 Feb 3 09:12 /dev/ttyS0<br />
crw-rw---- 1 root uucp 4, 65 Feb 3 09:12 /dev/ttyS1<br />
crw-rw---- 1 root uucp 4, 66 Feb 3 09:12 /dev/ttyS2<br />
crw-rw---- 1 root uucp 4, 67 Feb 3 09:12 /dev/ttyS3<br />
<br />
Add your user to the {{ic|uucp}} group.<br />
# gpasswd -a $USER uucp <br />
and log out and log in again.<br />
<br />
=== Abort on resume ===<br />
<br />
There is [https://www.virtualbox.org/ticket/11289 a known bug] that causes abort on resume. The workaround is simple: always use {{ic|Host+q}} keyboard keys combination or use the menu to close the VM.<br />
<br />
=== System images in Btrfs ===<br />
<br />
Back in 2010, there were reports that OS disk images would not start if they were attached via a virtual SATA device. It was reportedly fixed, and seemed to be. But as of around March 2013, that particular bug report has been [https://www.virtualbox.org/ticket/6905 reopened]. It can be fixed by enabling the use of the host I/O cache, which is disabled by default with virtual SATA interfaces.<br />
<br />
=== Windows 8.x Error Code 0x000000C4===<br />
<br />
If you get this error code while booting, even if you choose OS Type Win 8, try to enable the {{ic|CMPXCHG16B}} CPU instruction:<br />
<br />
$ vboxmanage setextradata <your virtual machine name> VBoxInternal/CPUM/CMPXCHG16B 1<br />
<br />
=== Windows 8 VM fails to boot with error "ERR_DISK_FULL" ===<br />
<br />
Situation: Your Windows 8 VM refuses to start. VirtualBox throws an error stating the virtual disk is full. However, you are certain that the disk is not full. <br />
Bring up your VM's settings at ''Settings > Storage > Controller:SATA'' and select "Use Host I/O Cache".<br />
<br />
== External links ==<br />
<br />
* [https://www.virtualbox.org/manual/UserManual.html VirtualBox User Manual]</div>Justin8https://wiki.archlinux.org/index.php?title=Bcache&diff=282307Bcache2013-11-11T08:51:54Z<p>Justin8: Added non-installation instructions.</p>
<hr />
<div>[[Category:File systems]]<br />
== Introduction ==<br />
<br />
Bcache allows one to use an SSD as a read/write cache (in writeback mode) or read cache (writethrough or writearound) for another blockdevice (generally a rotating HDD or array). This article will show how to install arch using Bcache as the root partition. For an intro to bcache itself, see [http://bcache.evilpiepirate.org/ the bcache homepage]. Be sure to read and reference [http://atlas.evilpiepirate.org/git/linux-bcache.git/tree/Documentation/bcache.txt?h=bcache the bcache manual]. Bcache is in the mainline kernel since 3.10. The kernel on the arch install disk includes the bcache module since 2013.08.01.<br />
<br />
An alternative to Bcache is Facebook's [[Flashcache]].<br />
<br />
Bcache needs the backing device to be formatted as a bcache block device. In most cases, [https://github.com/g2p/blocks blocks to-bcache] can do an in-place conversion.<br />
{{Warning|Be sure you back up any important data first.}}<br />
<br />
== Setting up a bcache device on an existing system ==<br />
<br />
1. Install the {{AUR|bcache-tools-git}} package from [[AUR]].<br />
<br />
2. Create a backing device (This will typically be your mechanical drive). The backing device can be a whole device, a partition or any other standard block device. This will create /dev/bcache0<br />
# make-bcache -B /dev/sdx1<br />
<br />
3. Create a cache device (This will typically be your SSD). The cache device can be a whole device, a partition or any other standard block device<br />
# make-bcache -C /dev/sdx1<br />
<br />
4. Register the cache device against your backing device. We need to find the UUID of your cache device and then add it to the bcache device initially. Udev rules will take care of this on reboot and will only need to be done once.<br />
# cd /sys/fs/bcache<br />
# ls<br />
# echo "UUID" > /sys/block/bcache0/bcache/attach<br />
<br />
5. Change your cache mode (if you want to cache writes as well as reads):<br />
# echo writeback > /sys/block/bcache0/bcache/cache_mode<br />
<br />
6. If you want to have this partition available during the initcpio (i.e. you require it at some point in the boot process) you need to add 'bcache' to your modules array in /etc/mkinitcpio.conf as well as adding the 'bcache_udev' hook in your list between block and filesystem.<br />
<br />
<br />
== Installation to a bcache device ==<br />
<br />
1. Boot on the install disk (2013.08.01 minimum).<br />
<br />
2. Install the {{AUR|bcache-tools-git}} package from [[AUR]].<br />
<br />
3. Partition your hdd<br />
<br />
grub can't handle bcache, so you'll need at least 2 partitions (boot and one for the bcache backing device). If you're doing UEFI, you'll need an EFI System Partition (ESP) as well. E.g.:<br />
1 2048 22527 10.0 MiB EF00 EFI System<br />
2 22528 432127 200.0 MiB 8300 arch_boot<br />
3 432128 625142414 297.9 GiB 8300 bcache_backing<br />
<br />
{{Note|This example has no swapfile/partition. For a swap partition on the cache, use LVM in step 7. For a swap partition outside the cache, be sure to make a swap partition now.}}<br />
<br />
4. Configure your HDD as a bcache backing device.<br />
<br />
# make-bcache -B /dev/sda3<br />
<br />
{{Note|<br />
* [[GRUB2]] either needs an EFI partition, "bios boot" partition, or an 'embedding area' of 2KB or so after the MBR that doesn't get overwritten. Also, since GRUB2 does not know about bcache, so you also need a {{ic|/boot}} partition that grub can read. Partitioning {{ic|/dev/sda}} is a must unless you're booting from a different device.<br />
* If all associated disks are partitioned at once as below bcache will automatically attach "-B backing stores" to the "-C ssd cache" and step 4 is unnecessary on reboots.<br />
}}<br />
<br />
# make-bcache -B /dev/sd? /dev/sd? -C /dev/sd?<br />
<br />
5. Register any bcache devices with the kernel (this needs to done every bootup)<br />
<br />
# for i in /dev/sd*; do echo $i; echo $i > /sys/fs/bcache/register_quiet; done<br />
<br />
You now have a {{ic|/dev/bcache0}} device.<br />
<br />
{{Note|the bcache user manual says to do {{ic|echo /dev/sd* > /sys/fs/bcache/register_quiet}}, but this did not work for me.}}<br />
<br />
6. Configure your SSD<br />
<br />
Format the SSD as a caching device and link it to the backing device<br />
<br />
# make-bcache -C /dev/sdb<br />
# echo /dev/sdb > /sys/fs/bcache/register <br />
# echo ''UUID__from_previous_command'' > /sys/block/bcache0/bcache/attach<br />
<br />
{{Note|If the UUID is forgotten, it can be found with {{ic|ls /sys/fs/bcache/}} after the cache device has been registered.}}<br />
<br />
7. Format the bcache device. Use LVM or btrfs subvolumes if you want to divide up the {{ic|/dev/bcache0}} device how you like (ex for seperate {{ic|/}}, {{ic|/home}}, {{ic|/var}}, etc):<br />
<br />
# mkfs.btrfs /dev/bcache0<br />
# mount /dev/bcache0 /mnt/<br />
# btrfs subvolume create /mnt/root<br />
# btrfs subvolume create /mnt/home<br />
# umount /mnt<br />
<br />
8. Prepare the installation mount point:<br />
<br />
# mkfs.ext4 /dev/sda2<br />
# mkfs.msdos /dev/sda1 (if your ESP is at least 500MB, use mkfs.vfat to make a FAT32 partition instead)<br />
# pacman -S arch-install-scripts<br />
# mount /dev/bcache0 -o subvol=root,compress=lzo /mnt/<br />
# mkdir /mnt/boot /mnt/home<br />
# mount /dev/bcache0 -o subvol=home,compress=lzo /mnt/home<br />
# mount /dev/sda2 /mnt/boot<br />
# mkdir /boot/efi<br />
# mount /dev/sda1 /mnt/boot/efi/<br />
<br />
9. Install the system<br />
<br />
Do the rest of the installation [[Installation_Guide#Connect_to_the_internet|as normal]] except this:<br />
<br />
Before you edit {{ic|/etc/mkinitcpio.conf}} and run {{ic|mkinitcpio -p linux}}:<br />
<br />
* install {{AUR|bcache-tools-git}} package from the [[AUR]].<br />
* Edit {{ic|/etc/mkinitcpio.conf}}:<br />
** add the "bcache" module<br />
** add the "bcache_udev" hook between block and filesystem hooks<br />
<br />
8. Reboot and make sure it works from the {{ic|/dev/bcache0}} device.<br />
<br />
== Configuring ==<br />
<br />
There are many options that can be configured (such as cache mode, cache flush interval, sequential write heuristic, etc.) This is currently done by writing to files in {{ic|/sys}}. See the [http://atlas.evilpiepirate.org/git/linux-bcache.git/tree/Documentation/bcache.txt bcache user documentation].<br />
<br />
Changing the cache mode is done by echoing one of 'writethrough', 'writeback', 'writearound' or 'none' to /sys/block/bcache[0-9]/bcache/cache_mode.<br />
<br />
== Advanced operations ==<br />
<br />
=== Resize backing device ===<br />
<br />
It's possible to resize the backing device so long as you don't move the partition start. This process is described on [http://comments.gmane.org/gmane.linux.kernel.bcache.devel/249 the mailing list]. Here is an example using btrfs volume directly on bcache0. For LVM containers or for other filesystems, procedure will differ.<br />
<br />
==== Example of growing ====<br />
<br />
In this example, I grow the filesystem by 4GB. <br />
<br />
1. Reboot to a live CD/USB Drive (need not be bcache enabled) and use fdisk, gdisk, parted, or your other favorite tool to delete the backing partition and recreate it with the same start and a total size 4G larger. <br />
<br />
{{Warning|Do not use a tool like GParted that might perform filesystem operations! It will not recognize the bcache partition and might overwrite part of it!!}}<br />
<br />
2. Reboot to your normal install. Your filesystem will be currently mounted. That is fine. Issue the command to resize the partition to its maximum. For btrfs, that is<br />
<br />
# btrfs filesystem resize max /<br />
<br />
For ext3/4, that is:<br />
<br />
# resize2fs /dev/bcache0<br />
<br />
==== Example of shrinking ====<br />
<br />
In this example, I shrink the filesystem by 4GB.<br />
<br />
1. Disable writeback cache (switch to writethrough cache) and wait for the disk to flush.<br />
<br />
# echo writethrough > /sys/block/bcache0/bcache/cache_mode<br />
$ watch cat /sys/block/bcache0/bcache/state<br />
<br />
wait until state reports "clean". This might take a while.<br />
<br />
2. Shrink the mounted filesystem by something more than the desired amount, to ensure we don't accidentally clip it later. For btrfs, that is:<br />
<br />
# btrfs filesystem resize -5G /<br />
<br />
For ext3/4 you can use ''resize2fs'', but only if the partition is unmounted<br />
<br />
$ df -h /home<br />
/dev/bcache0 290G 20G 270G 1% /home<br />
# umount /home<br />
# resize2fs /dev/bcache0 283G<br />
<br />
3. Reboot to a LiveCD/USB drive (doesn't need to support bcache) and use fdisk, gdisk, parted, or your other favorite tool to delete the backing partition and recreate it with the same start and a total size 4G smaller.<br />
<br />
{{Warning|Do not use a tool like GParted that might perform filesystem operations! It won't recognize the bcache partition and might overwrite part of it!!}}<br />
<br />
4. Reboot to your normal install. Your filesystem will be currently mounted. That is fine. Issue the command to resize the partition to its maximum (that is, the size we shrunk the actual partition to in step 3). For btrfs, that is:<br />
<br />
# btrfs filesystem resize max /<br />
<br />
For ext3/4, that is:<br />
<br />
# resize2fs /dev/bcache0<br />
<br />
5. Re-enable writeback cache if you want that enabled:<br />
<br />
# echo writeback > /sys/block/bcache0/bcache/cache_mode<br />
<br />
{{Note|If you're very careful you can shrink the filesystem to the exact size in step 2 and avoid step 4. Be careful, though, many partition tools don't do exactly what you want, but instead adjust the requested partition start/end points to end on sector boundaries. This may be difficult to calculate ahead of time}}<br />
<br />
== Troubleshooting ==<br />
<br />
=== /dev/bcache device doesn't exist on bootup ===<br />
<br />
If you are sent to a busy box shell with an error:<br />
<br />
{{bc|1=<br />
ERROR: Unable to find root device 'UUID=b6b2d82b-f87e-44d5-bbc5-c51dd7aace15'.<br />
You are being dropped to a recovery shell<br />
Type 'exit' to try and continue booting<br />
}}<br />
<br />
This might happen if the backing device is configured for "writeback" mode (default is writearound). When in "writeback" mode, the /dev/bcache0 device is not started until the cache device is both registered and attached. Registering is something that needs to happen every bootup, but attaching should only have to be done once. I've sometime had to re-attach.<br />
<br />
To continue booting, try one of the following:<br />
<br />
* Register both the backing device and the cacheing device<br />
<br />
# echo /dev/sda3 > /sys/fs/bcache/register<br />
# echo /dev/sdb > /sys/fs/bcache/register<br />
<br />
If the /dev/bcache0 device now exists, type exit and continue booting. You will need to fix your initcpio to ensure devices are registered before mounting the root device.<br />
<br />
{{Note|<br />
* An error of "sh: echo: write error: Invalid argument" means the device was already registered or is not recognized as either a bcache backing device or cache.<br />
* This can also happen if using udev's 69-bcache.rules in Installation's step 7b and blkid and bcache-probe "disagree" due to rogue superblocks. See [http://bcache.evilpiepirate.org/#index6h1 bcache's wiki] for a possible explanation/resolution.<br />
}}<br />
<br />
* Re-attach the cache to the backing device:<br />
<br />
If the cache device was registered, a folder with the UUID of the cache should exist in {{ic|/sys/fs/bcache}}. Use that UUID when following the example below:<br />
<br />
# ls /sys/fs/bcache/<br />
b6b2d82b-f87e-44d5-bbc5-c51dd7aace15 register register_quiet<br />
# echo b6b2d82b-f87e-44d5-bbc5-c51dd7aace15 > /sys/block/sda/sda3/bcache/attach<br />
<br />
If the {{ic|/dev/bcache0}} device now exists, type exit and continue booting. You should not have to do this again. If it persists, ask on the bcache mailing list.<br />
<br />
{{Note|An error of {{ic|sh: echo: write error: Invalid argument}} means the device was already attached. An error of {{ic|sh: echo: write error: No such file or directory}} means the UUID is not a valid cache (make sure you typed it correctly).}}<br />
<br />
* Invalidate the cache and force the backing device to run without it. You might want to check some stats, such as "dirty_data" so you have some idea of how much data will be lost.<br />
<br />
# cat /sys/block/sda/sda3/bcache/dirty_data<br />
-3.9M<br />
<br />
dirty data is data in the cache that has not been written to the backing device. If you force the backing device to run, this data will be lost, even if you later re-attach the cache.<br />
<br />
# cat /sys/block/sda/sda3/bcache/running<br />
0<br />
# echo 1 > /sys/block/sda/sda3/bcache/running<br />
<br />
The {{ic|/dev/bcache0}} device will now exist. Type exit and continue booting. You might want to unregister the cache device and run make-bcache again. An fsck on {{ic|/dev/bcache0}} would also be wise. See the [http://atlas.evilpiepirate.org/git/linux-bcache.git/tree/Documentation/bcache.txt?h=bcache bcache user documentation].<br />
<br />
{{Warning|Only invalidate the cache if one of the two options above did not work.}}<br />
<br />
=== /sys/fs/bcache/ doesn't exist ===<br />
<br />
The kernel you booted is not bcache enabled.</div>Justin8https://wiki.archlinux.org/index.php?title=Bcache&diff=282301Bcache2013-11-11T08:21:53Z<p>Justin8: Updated wiki to reflect latest bcache package</p>
<hr />
<div>[[Category:File systems]]<br />
== Introduction ==<br />
<br />
Bcache allows one to use an SSD as a read/write cache (in writeback mode) or read cache (writethrough or writearound) for another blockdevice (generally a rotating HDD or array). This article will show how to install arch using Bcache as the root partition. For an intro to bcache itself, see [http://bcache.evilpiepirate.org/ the bcache homepage]. Be sure to read and reference [http://atlas.evilpiepirate.org/git/linux-bcache.git/tree/Documentation/bcache.txt?h=bcache the bcache manual]. Bcache is in the mainline kernel since 3.10. The kernel on the arch install disk includes the bcache module since 2013.08.01.<br />
<br />
An alternative to Bcache is Facebook's [[Flashcache]].<br />
<br />
Bcache needs the backing device to be formatted as a bcache block device. In most cases, [https://github.com/g2p/blocks blocks to-bcache] can do an in-place conversion.<br />
{{Warning|Be sure you back up any important data first.}}<br />
<br />
== Preparation ==<br />
<br />
* Boot on the install disk (2013.08.01 minimum).<br />
* Install the {{AUR|bcache-tools-git}} package from [[AUR]].<br />
<br />
== Installation ==<br />
<br />
1. Partition your hdd<br />
<br />
grub can't handle bcache, so you'll need at least 2 partitions (boot and one for the bcache backing device). If you're doing UEFI, you'll need an EFI System Partition (ESP) as well. E.g.:<br />
1 2048 22527 10.0 MiB EF00 EFI System<br />
2 22528 432127 200.0 MiB 8300 arch_boot<br />
3 432128 625142414 297.9 GiB 8300 bcache_backing<br />
<br />
{{Note|This example has no swapfile/partition. For a swap partition on the cache, use LVM in step 7. For a swap partition outside the cache, be sure to make a swap partition now.}}<br />
<br />
2. Configure your HDD as a bcache backing device.<br />
<br />
# make-bcache -B /dev/sda3<br />
<br />
{{Note|<br />
* [[GRUB2]] either needs an EFI partition, "bios boot" partition, or an 'embedding area' of 2KB or so after the MBR that doesn't get overwritten. Also, since GRUB2 does not know about bcache, so you also need a {{ic|/boot}} partition that grub can read. Partitioning {{ic|/dev/sda}} is a must unless you're booting from a different device.<br />
* If all associated disks are partitioned at once as below bcache will automatically attach "-B backing stores" to the "-C ssd cache" and step 4 is unnecessary on reboots.<br />
}}<br />
<br />
# make-bcache -B /dev/sd? /dev/sd? -C /dev/sd?<br />
<br />
3. Register any bcache devices with the kernel (this needs to done every bootup)<br />
<br />
# for i in /dev/sd*; do echo $i; echo $i > /sys/fs/bcache/register_quiet; done<br />
<br />
You now have a {{ic|/dev/bcache0}} device.<br />
<br />
{{Note|the bcache user manual says to do {{ic|echo /dev/sd* > /sys/fs/bcache/register_quiet}}, but this did not work for me.}}<br />
<br />
4. Configure your SSD<br />
<br />
Format the SSD as a caching device and link it to the backing device<br />
<br />
# make-bcache -C /dev/sdb<br />
# echo /dev/sdb > /sys/fs/bcache/register <br />
# echo ''UUID__from_previous_command'' > /sys/block/bcache0/bcache/attach<br />
<br />
{{Note|If the UUID is forgotten, it can be found with {{ic|ls /sys/fs/bcache/}} after the cache device has been registered.}}<br />
<br />
5. Format the bcache device. Use LVM or btrfs subvolumes if you want to divide up the {{ic|/dev/bcache0}} device how you like (ex for seperate {{ic|/}}, {{ic|/home}}, {{ic|/var}}, etc):<br />
<br />
# mkfs.btrfs /dev/bcache0<br />
# mount /dev/bcache0 /mnt/<br />
# btrfs subvolume create /mnt/root<br />
# btrfs subvolume create /mnt/home<br />
# umount /mnt<br />
<br />
6. Prepare the installation mount point:<br />
<br />
# mkfs.ext4 /dev/sda2<br />
# mkfs.msdos /dev/sda1 (if your ESP is at least 500MB, use mkfs.vfat to make a FAT32 partition instead)<br />
# pacman -S arch-install-scripts<br />
# mount /dev/bcache0 -o subvol=root,compress=lzo /mnt/<br />
# mkdir /mnt/boot /mnt/home<br />
# mount /dev/bcache0 -o subvol=home,compress=lzo /mnt/home<br />
# mount /dev/sda2 /mnt/boot<br />
# mkdir /boot/efi<br />
# mount /dev/sda1 /mnt/boot/efi/<br />
<br />
7. Install the system<br />
<br />
Do the rest of the installation [[Installation_Guide#Connect_to_the_internet|as normal]] except this:<br />
<br />
Before you edit {{ic|/etc/mkinitcpio.conf}} and run {{ic|mkinitcpio -p linux}}:<br />
<br />
* install {{AUR|bcache-tools-git}} package from the [[AUR]].<br />
* Edit {{ic|/etc/mkinitcpio.conf}}:<br />
** add the "bcache" module<br />
** add the "bcache_udev" hook between block and filesystems <br />
<br />
8. Reboot and make sure it works from the {{ic|/dev/bcache0}} device.<br />
<br />
== Configuring ==<br />
<br />
There are many options that can be configured (such as cache mode, cache flush interval, sequential write heuristic, etc.) This is currently done by writing to files in {{ic|/sys}}. See the [http://atlas.evilpiepirate.org/git/linux-bcache.git/tree/Documentation/bcache.txt bcache user documentation].<br />
<br />
Changing the cache mode is done by echoing one of 'writethrough', 'writeback', 'writearound' or 'none' to /sys/block/bcache[0-9]/bcache/cache_mode.<br />
<br />
== Advanced operations ==<br />
<br />
=== Resize backing device ===<br />
<br />
It's possible to resize the backing device so long as you don't move the partition start. This process is described on [http://comments.gmane.org/gmane.linux.kernel.bcache.devel/249 the mailing list]. Here is an example using btrfs volume directly on bcache0. For LVM containers or for other filesystems, procedure will differ.<br />
<br />
==== Example of growing ====<br />
<br />
In this example, I grow the filesystem by 4GB. <br />
<br />
1. Reboot to a live CD/USB Drive (need not be bcache enabled) and use fdisk, gdisk, parted, or your other favorite tool to delete the backing partition and recreate it with the same start and a total size 4G larger. <br />
<br />
{{Warning|Do not use a tool like GParted that might perform filesystem operations! It will not recognize the bcache partition and might overwrite part of it!!}}<br />
<br />
2. Reboot to your normal install. Your filesystem will be currently mounted. That is fine. Issue the command to resize the partition to its maximum. For btrfs, that is<br />
<br />
# btrfs filesystem resize max /<br />
<br />
For ext3/4, that is:<br />
<br />
# resize2fs /dev/bcache0<br />
<br />
==== Example of shrinking ====<br />
<br />
In this example, I shrink the filesystem by 4GB.<br />
<br />
1. Disable writeback cache (switch to writethrough cache) and wait for the disk to flush.<br />
<br />
# echo writethrough > /sys/block/bcache0/bcache/cache_mode<br />
$ watch cat /sys/block/bcache0/bcache/state<br />
<br />
wait until state reports "clean". This might take a while.<br />
<br />
2. Shrink the mounted filesystem by something more than the desired amount, to ensure we don't accidentally clip it later. For btrfs, that is:<br />
<br />
# btrfs filesystem resize -5G /<br />
<br />
For ext3/4 you can use ''resize2fs'', but only if the partition is unmounted<br />
<br />
$ df -h /home<br />
/dev/bcache0 290G 20G 270G 1% /home<br />
# umount /home<br />
# resize2fs /dev/bcache0 283G<br />
<br />
3. Reboot to a LiveCD/USB drive (doesn't need to support bcache) and use fdisk, gdisk, parted, or your other favorite tool to delete the backing partition and recreate it with the same start and a total size 4G smaller.<br />
<br />
{{Warning|Do not use a tool like GParted that might perform filesystem operations! It won't recognize the bcache partition and might overwrite part of it!!}}<br />
<br />
4. Reboot to your normal install. Your filesystem will be currently mounted. That is fine. Issue the command to resize the partition to its maximum (that is, the size we shrunk the actual partition to in step 3). For btrfs, that is:<br />
<br />
# btrfs filesystem resize max /<br />
<br />
For ext3/4, that is:<br />
<br />
# resize2fs /dev/bcache0<br />
<br />
5. Re-enable writeback cache if you want that enabled:<br />
<br />
# echo writeback > /sys/block/bcache0/bcache/cache_mode<br />
<br />
{{Note|If you're very careful you can shrink the filesystem to the exact size in step 2 and avoid step 4. Be careful, though, many partition tools don't do exactly what you want, but instead adjust the requested partition start/end points to end on sector boundaries. This may be difficult to calculate ahead of time}}<br />
<br />
== Troubleshooting ==<br />
<br />
=== /dev/bcache device doesn't exist on bootup ===<br />
<br />
If you are sent to a busy box shell with an error:<br />
<br />
{{bc|1=<br />
ERROR: Unable to find root device 'UUID=b6b2d82b-f87e-44d5-bbc5-c51dd7aace15'.<br />
You are being dropped to a recovery shell<br />
Type 'exit' to try and continue booting<br />
}}<br />
<br />
This might happen if the backing device is configured for "writeback" mode (default is writearound). When in "writeback" mode, the /dev/bcache0 device is not started until the cache device is both registered and attached. Registering is something that needs to happen every bootup, but attaching should only have to be done once. I've sometime had to re-attach.<br />
<br />
To continue booting, try one of the following:<br />
<br />
* Register both the backing device and the cacheing device<br />
<br />
# echo /dev/sda3 > /sys/fs/bcache/register<br />
# echo /dev/sdb > /sys/fs/bcache/register<br />
<br />
If the /dev/bcache0 device now exists, type exit and continue booting. You will need to fix your initcpio to ensure devices are registered before mounting the root device.<br />
<br />
{{Note|<br />
* An error of "sh: echo: write error: Invalid argument" means the device was already registered or is not recognized as either a bcache backing device or cache.<br />
* This can also happen if using udev's 69-bcache.rules in Installation's step 7b and blkid and bcache-probe "disagree" due to rogue superblocks. See [http://bcache.evilpiepirate.org/#index6h1 bcache's wiki] for a possible explanation/resolution.<br />
}}<br />
<br />
* Re-attach the cache to the backing device:<br />
<br />
If the cache device was registered, a folder with the UUID of the cache should exist in {{ic|/sys/fs/bcache}}. Use that UUID when following the example below:<br />
<br />
# ls /sys/fs/bcache/<br />
b6b2d82b-f87e-44d5-bbc5-c51dd7aace15 register register_quiet<br />
# echo b6b2d82b-f87e-44d5-bbc5-c51dd7aace15 > /sys/block/sda/sda3/bcache/attach<br />
<br />
If the {{ic|/dev/bcache0}} device now exists, type exit and continue booting. You should not have to do this again. If it persists, ask on the bcache mailing list.<br />
<br />
{{Note|An error of {{ic|sh: echo: write error: Invalid argument}} means the device was already attached. An error of {{ic|sh: echo: write error: No such file or directory}} means the UUID is not a valid cache (make sure you typed it correctly).}}<br />
<br />
* Invalidate the cache and force the backing device to run without it. You might want to check some stats, such as "dirty_data" so you have some idea of how much data will be lost.<br />
<br />
# cat /sys/block/sda/sda3/bcache/dirty_data<br />
-3.9M<br />
<br />
dirty data is data in the cache that has not been written to the backing device. If you force the backing device to run, this data will be lost, even if you later re-attach the cache.<br />
<br />
# cat /sys/block/sda/sda3/bcache/running<br />
0<br />
# echo 1 > /sys/block/sda/sda3/bcache/running<br />
<br />
The {{ic|/dev/bcache0}} device will now exist. Type exit and continue booting. You might want to unregister the cache device and run make-bcache again. An fsck on {{ic|/dev/bcache0}} would also be wise. See the [http://atlas.evilpiepirate.org/git/linux-bcache.git/tree/Documentation/bcache.txt?h=bcache bcache user documentation].<br />
<br />
{{Warning|Only invalidate the cache if one of the two options above did not work.}}<br />
<br />
=== /sys/fs/bcache/ doesn't exist ===<br />
<br />
The kernel you booted is not bcache enabled.</div>Justin8https://wiki.archlinux.org/index.php?title=OpenSSH&diff=282279OpenSSH2013-11-11T00:21:24Z<p>Justin8: /* Autossh - automatically restarts SSH sessions and tunnels */</p>
<hr />
<div>[[Category:Secure Shell]]<br />
[[ar:Ssh]]<br />
[[es:Secure Shell]]<br />
[[de:SSH]]<br />
[[fr:ssh]]<br />
[[it:Secure Shell]]<br />
[[ko:Secure Shell]]<br />
[[pl:Secure Shell]]<br />
[[pt:Secure Shell]]<br />
[[ru:Secure Shell]]<br />
[[sr:Secure Shell]]<br />
[[zh-CN:Secure Shell]]<br />
'''Secure Shell''' ('''SSH''') is a network protocol that allows data to be exchanged over a secure channel between two computers. Encryption provides confidentiality and integrity of data. SSH uses public-key cryptography to authenticate the remote computer and allow the remote computer to authenticate the user, if necessary.<br />
<br />
SSH is typically used to log into a remote machine and execute commands, but it also supports tunneling, forwarding arbitrary TCP ports and X11 connections; file transfer can be accomplished using the associated SFTP or SCP protocols.<br />
<br />
An SSH server, by default, listens on the standard TCP port 22. An SSH client program is typically used for establishing connections to an ''sshd'' daemon accepting remote connections. Both are commonly present on most modern operating systems, including Mac OS X, GNU/Linux, Solaris and OpenVMS. Proprietary, freeware and open source versions of various levels of complexity and completeness exist.<br />
<br />
(Source: [[Wikipedia:Secure Shell]])<br />
<br />
== OpenSSH ==<br />
OpenSSH (OpenBSD Secure Shell) is a set of computer programs providing encrypted communication sessions over a computer network using the ssh protocol. It was created as an open source alternative to the proprietary Secure Shell software suite offered by SSH Communications Security. OpenSSH is developed as part of the OpenBSD project, which is led by Theo de Raadt.<br />
<br />
OpenSSH is occasionally confused with the similarly-named OpenSSL; however, the projects have different purposes and are developed by different teams, the similar name is drawn only from similar goals.<br />
<br />
=== Installing OpenSSH ===<br />
[[pacman|Install]] {{Pkg|openssh}} from the [[official repositories]].<br />
<br />
=== Configuring SSH ===<br />
====Client====<br />
The SSH client configuration file is {{ic|/etc/ssh/ssh_config}} or {{ic|~/.ssh/config}}.<br />
<br />
An example configuration: <br />
<br />
{{hc|/etc/ssh/ssh_config|<br />
# $OpenBSD: ssh_config,v 1.26 2010/01/11 01:39:46 dtucker Exp $<br />
<br />
# This is the ssh client system-wide configuration file. See<br />
# ssh_config(5) for more information. This file provides defaults for<br />
# users, and the values can be changed in per-user configuration files<br />
# or on the command line.<br />
<br />
# Configuration data is parsed as follows:<br />
# 1. command line options<br />
# 2. user-specific file<br />
# 3. system-wide file<br />
# Any configuration value is only changed the first time it is set.<br />
# Thus, host-specific definitions should be at the beginning of the<br />
# configuration file, and defaults at the end.<br />
<br />
# Site-wide defaults for some commonly used options. For a comprehensive<br />
# list of available options, their meanings and defaults, please see the<br />
# ssh_config(5) man page.<br />
<br />
# Host *<br />
# ForwardAgent no<br />
# ForwardX11 no<br />
# RhostsRSAAuthentication no<br />
# RSAAuthentication yes<br />
# PasswordAuthentication yes<br />
# HostbasedAuthentication no<br />
# GSSAPIAuthentication no<br />
# GSSAPIDelegateCredentials no<br />
# BatchMode no<br />
# CheckHostIP yes<br />
# AddressFamily any<br />
# ConnectTimeout 0<br />
# StrictHostKeyChecking ask<br />
# IdentityFile ~/.ssh/identity<br />
# IdentityFile ~/.ssh/id_rsa<br />
# IdentityFile ~/.ssh/id_dsa<br />
# Port 22<br />
# Protocol 2,1<br />
# Cipher 3des<br />
# Ciphers aes128-ctr,aes192-ctr,aes256-ctr,arcfour256,arcfour128,aes128-cbc,3des-cbc<br />
# MACs hmac-md5,hmac-sha1,umac-64@openssh.com,hmac-ripemd160<br />
# EscapeChar ~<br />
# Tunnel no<br />
# TunnelDevice any:any<br />
# PermitLocalCommand no<br />
# VisualHostKey no<br />
# ProxyCommand ssh -q -W %h:%p gateway.example.com<br />
}}<br />
<br />
It is not longer needed to change the Protocol line into:<br />
Protocol 2<br />
<br />
That means Protocol 1 will not be used as long as it is not explicitly enabled. (source: http://www.openssh.org/txt/release-5.4)<br />
<br />
====Daemon====<br />
The SSH daemon configuration file can be found and edited in {{ic|/etc/ssh/ssh'''d'''_config}}.<br />
<br />
To allow access only for some users add this line:<br />
AllowUsers user1 user2<br />
<br />
To disable root login over SSH, change the PermitRootLogin line into this:<br />
PermitRootLogin no<br />
<br />
To add a nice welcome message edit the file {{ic|/etc/issue}} and change the Banner line into this:<br />
Banner /etc/issue<br />
<br />
{{Tip| You may want to change the default port from 22 to any higher port (see [http://en.wikipedia.org/wiki/Security_through_obscurity security through obscurity]).}} <br />
<br />
Even though the port ssh is running on could be detected by using a port-scanner like nmap, changing it will reduce the number of log entries caused by automated authentication attempts. To help select a port review the [[Wikipedia:List of TCP and UDP port numbers|list of TCP and UDP port numbers]].<br />
<br />
{{Tip|Disabling password logins entirely will greatly increase security, see [[SSH Keys]] for more information.}}<br />
<br />
=== Managing the sshd daemon ===<br />
You can start the sshd daemon with the following command:<br />
# systemctl start sshd<br />
<br />
You can enable the sshd daemon at startup with the following command:<br />
# systemctl enable sshd.service<br />
<br />
{{Warning|Systemd is an asynchronous starting process. If you bind the SSH daemon to a specific IP address {{ic|ListenAddress 192.168.1.100}} it may fail to load during boot since the default sshd.service unit file has no dependency on network interfaces being enabled. When binding to an IP address, you will need to add {{ic|After&#61;network.target}} to a custom sshd.service unit file. See [[Systemd#Editing provided unit files]].}}<br />
<br />
Or you can enable SSH Daemon socket so the daemon is started on the first incoming connection:<br />
# systemctl enable sshd.socket<br />
If you use a different port than the default 22, you have to set "ListenStream" in the unit file. Copy /lib/systemd/system/sshd.socket to /etc/systemd/system/sshd.socket to keep your unit file from being overwritten on upgrades. In /etc/systemd/system/sshd.socket change "ListenStream" the appropriate port.<br />
<br />
{{Warning|Using sshd.socket effectively negates the {{ic|ListenAddress}} setting, so using the default sshd.socket will allow connections over any address. To achieve the effect of setting {{ic|ListenAddress}}, you must create a custom unit file and modify ListenStream (ie. {{ic|ListenStream&#61;192.168.1.100:22}} is equivalent to {{ic|ListenAddress 192.168.1.100}}). However, doing so has the same drawback as setting {{ic|ListenAddress}}: the socket will fail to start if the network is not up in time.}}<br />
<br />
=== Connecting to the server ===<br />
To connect to a server, run:<br />
$ ssh -p port user@server-address<br />
<br />
=== Protecting SSH ===<br />
Allowing remote log-on through SSH is good for administrative purposes, but can pose a threat to your server's security. Often the target of brute force attacks, SSH access needs to be limited properly to prevent third parties gaining access to your server.<br />
* Use non-standard account names and passwords<br />
* Only allow incoming SSH connections from trusted locations<br />
* Use [[fail2ban]] or [[sshguard]] to monitor for brute force attacks, and ban brute forcing IPs accordingly<br />
<br />
===== Protecting against brute force attacks =====<br />
Brute forcing is a simple concept: One continuously tries to log in to a webpage or server log-in prompt like SSH with a high number of random username and password combinations. You can protect yourself from brute force attacks by using an automated script that blocks anybody trying to brute force their way in, for example [[fail2ban]] or [[sshguard]].<br />
<br />
===== Deny root login =====<br />
It is generally considered bad practice to allow the user '''root''' to log in over SSH: The '''root''' account will exist on nearly any Linux system and grants full access to the system, once login has been achieved. Sudo provides root rights for actions requiring these and is the more secure solution, third parties would have to find a username present on the system, the matching password and the matching password for sudo to get root rights on your system. More barriers to be breached before full access to the system is reached.<br />
<br />
Configure SSH to deny remote logins with the root user by editing {{ic|/etc/ssh/sshd_config}} and look for this section:<br />
# Authentication:<br />
<br />
#LoginGraceTime 2m<br />
''#PermitRootLogin yes''<br />
#StrictModes yes<br />
#MaxAuthTries 6<br />
#MaxSessions 10<br />
<br />
Now simply change ''#PermitRootLogin yes'' to no, and uncomment the line:<br />
PermitRootLogin no<br />
<br />
Next, restart the SSH daemon:<br />
# systemctl restart sshd<br />
<br />
You will now be unable to log in through SSH under root, but will still be able to log in with your normal user and use ''su'' - or ''sudo'' to do system administration.<br />
<br />
== Other SSH clients and servers ==<br />
Apart from OpenSSH, there are many SSH [[Wikipedia:Comparison of SSH clients|clients]] and [[Wikipedia:Comparison of SSH servers|servers]] avaliable.<br />
<br />
=== Dropbear ===<br />
[[Wikipedia:Dropbear (software)|Dropbear]] is a SSH-2 client and server. {{AUR|dropbear}} is available in the [[AUR]].<br />
<br />
The commandline ssh client is named dbclient.<br />
<br />
=== SSH alternative: Mobile Shell - responsive, survives disconnects ===<br />
From the Mosh [http://mosh.mit.edu/ website]:<br />
<br />
Remote terminal application that allows roaming, supports intermittent connectivity, and provides intelligent local echo and line editing of user keystrokes. Mosh is a replacement for SSH. It's more robust and responsive, especially over Wi-Fi, cellular, and long-distance links.<br />
<br />
[[pacman|Install]] {{Pkg|mosh}} from the [[official repositories]] or the latest revision {{AUR|mosh-git}} in the [[AUR]].<br />
<br />
== Tips and tricks ==<br />
<br />
=== Encrypted SOCKS tunnel ===<br />
This is highly useful for laptop users connected to various unsafe wireless connections. The only thing you need is an SSH server running at a somewhat secure location, like your home or at work. It might be useful to use a dynamic DNS service like [http://www.dyndns.org/ DynDNS] so you do not have to remember your IP-address.<br />
<br />
==== Step 1: start the connection ====<br />
You only have to execute this single command to start the connection:<br />
$ ssh -TND 4711 user@host<br />
where {{Ic|"user"}} is your username at the SSH server running at the {{Ic|"host"}}. It will ask for your password, and then you're connected! The {{Ic|"N"}} flag disables the interactive prompt, and the {{Ic|"D"}} flag specifies the local port on which to listen on (you can choose any port number if you want). The {{Ic|"T"}} flag disables pseudo-tty allocation.<br />
<br />
It's nice to add the verbose {{Ic|"-v"}} flag, because then you can verify that it's actually connected from that output.<br />
<br />
==== Step 2: configure your browser (or other programs) ====<br />
The above step is completely useless if you do not configure your web browser (or other programs) to use this newly created socks tunnel. Since the current version of SSH supports both SOCKS4 and SOCKS5, you can use either of them.<br />
<br />
* For Firefox: ''Edit &rarr; Preferences &rarr; Advanced &rarr; Network &rarr; Connection &rarr; Setting'':<br />
: Check the ''"Manual proxy configuration"'' radio button, and enter "localhost" in the ''"SOCKS host"'' text field, and then enter your port number in the next text field (I used 4711 above).<br />
<br />
Firefox does not automatically make DNS requests through the socks tunnel. This potential privacy concern can be mitigated by the following steps:<br />
<br />
# Type about:config into the Firefox location bar.<br />
# Search for network.proxy.socks_remote_dns<br />
# Set the value to true.<br />
# Restart the browser.<br />
<br />
* For Chromium: You can set the SOCKS settings as environment variables or as command line options. I recommend to add one of the following functions to your {{ic|.bashrc}}:<br />
function secure_chromium {<br />
port=4711<br />
export SOCKS_SERVER=localhost:$port<br />
export SOCKS_VERSION=5<br />
chromium &<br />
exit<br />
}<br />
OR<br />
function secure_chromium {<br />
port=4711<br />
chromium --proxy-server="socks://localhost:$port" &<br />
exit<br />
}<br />
<br />
Now open a terminal and just do:<br />
$ secure_chromium<br />
<br />
Enjoy your secure tunnel!<br />
<br />
=== X11 forwarding ===<br />
To run graphical programs through a SSH connection you can enable X11 forwarding. An option needs to be set in the configuration files on the server and client (here "client" means your (desktop) machine your X11 Server runs on, and you will run X applications on the "server").<br />
<br />
[[pacman|Install]] {{Pkg|xorg-xauth}} from the [[official repositories]] onto the server.<br />
<br />
* Enable the '''AllowTcpForwarding''' option in {{ic|ssh'''d'''_config}} on the '''server'''.<br />
* Enable the '''X11Forwarding''' option in {{ic|ssh'''d'''_config}} on the '''server'''.<br />
* Set the '''X11DisplayOffset''' option in {{ic|ssh'''d'''_config}} on the '''server''' to 10.<br />
* Enable the '''X11UseLocalhost''' option in {{ic|ssh'''d'''_config}} on the '''server'''.<br />
Also:<br />
* Enable the '''ForwardX11''' option in {{ic|ssh_config}} on the '''client'''.<br />
* Enable the '''ForwardX11Trusted''' if gui is drawing badly.<br />
<br />
You need to restart the ssh daemon on the server for these changes to take effect, of course.<br />
<br />
To use the forwarding, log on to your server through ssh:<br />
$ ssh -X -p port user@server-address<br />
If you receive errors trying to run graphical applications try trusted forwarding instead:<br />
$ ssh -Y -p port user@server-address<br />
You can now start any X program on the remote server, the output will be forwarded to your local session:<br />
$ xclock<br />
<br />
<br />
If you get "Cannot open display" errors try the following command as the non root user:<br />
$ xhost +<br />
<br />
the above command will allow anybody to forward X11 applications. To restrict forwarding to a particular host type:<br />
$ xhost +hostname<br />
<br />
where hostname is the name of the particular host you want to forward to. Type "man xhost" for more details.<br />
<br />
Be careful with some applications as they check for a running instance on the local machine. Firefox is an example. Either close running Firefox or use the following start parameter to start a remote instance on the local machine<br />
$ firefox -no-remote<br />
<br />
If you get "X11 forwarding request failed on channel 0" when you connect (and the server /var/log/errors.log shows "Failed to allocate internet-domain X11 display socket"), try to either<br />
* Enable the '''AddressFamily any''' option in {{ic|ssh'''d'''_config}} on the '''server''', or<br />
* Set the '''AddressFamily''' option in {{ic|ssh'''d'''_config}} on the '''server''' to inet.<br />
Setting it to inet may fix problems with Ubuntu clients on IPv4.<br />
<br />
For running X applications as other user on the SSH server you need to {{Ic|xauth add}} the authentication line taken from {{Ic|xauth list}} of the SSH logged in user.<br />
<br />
=== Forwarding other ports ===<br />
In addition to SSH's built-in support for X11, it can also be used to securely tunnel any TCP connection, by use of local forwarding or remote forwarding.<br />
<br />
Local forwarding opens a port on the local machine, connections to which will be forwarded to the remote host and from there on to a given destination. Very often, the forwarding destination will be the same as the remote host, thus providing a secure shell and, e.g. a secure VNC connection, to the same machine. Local forwarding is accomplished by means of the {{Ic|-L}} switch and it's accompanying forwarding specification in the form of {{Ic|<tunnel port>:<destination address>:<destination port>}}.<br />
<br />
Thus:<br />
<br />
$ ssh -L 1000:mail.google.com:25 192.168.0.100<br />
<br />
will use SSH to login to and open a shell on 192.168.0.100, and will also create a tunnel from the local machine's TCP port 1000 to mail.google.com on port 25. Once established, connections to localhost:1000 will connect to the Gmail SMTP port. To Google, it will appear that any such connection (though not necessarily the data conveyed over the connection) originated from 192.168.0.100, and such data will be secure as between the local machine and 192.168.0.100, but not between 192.168.0.100, unless other measures are taken.<br />
<br />
Similarly:<br />
<br />
$ ssh -L 2000:192.168.0.100:6001 192.168.0.100<br />
<br />
will allow connections to localhost:2000 which will be transparently sent to the remote host on port 6001. The preceding example is useful for VNC connections using the vncserver utility--part of the tightvnc package--which, though very useful, is explicit about its lack of security.<br />
<br />
Remote forwarding allows the remote host to connect to an arbitrary host via the SSH tunnel and the local machine, providing a functional reversal of local forwarding, and is useful for situations where, e.g., the remote host has limited connectivity due to firewalling. It is enabled with the {{Ic|-R}} switch and a forwarding specification in the form of {{Ic|<tunnel port>:<destination address>:<destination port>}}.<br />
<br />
Thus:<br />
<br />
$ ssh -R 3000:irc.freenode.net:6667 192.168.0.200<br />
<br />
will bring up a shell on 192.168.0.200, and connections from 192.168.0.200 to itself on port 3000 (remotely speaking, localhost:3000) will be sent over the tunnel to the local machine and then on to irc.freenode.net on port 6667, thus, in this example, allowing the use of IRC programs on the remote host to be used, even if port 6667 would normally be blocked to it.<br />
<br />
Both local and remote forwarding can be used to provide a secure "gateway," allowing other computers to take advantage of an SSH tunnel, without actually running SSH or the SSH daemon by providing a bind-address for the start of the tunnel as part of the forwarding specification, e.g. {{Ic|<tunnel address>:<tunnel port>:<destination address>:<destination port>}}. The {{Ic|<tunnel address>}} can be any address on the machine at the start of the tunnel, {{Ic|localhost}}, {{Ic|*}} (or blank), which, respectively, allow connections via the given address, via the loopback interface, or via any interface. By default, forwarding is limited to connections from the machine at the "beginning" of the tunnel, i.e. the {{Ic|<tunnel address>}} is set to {{Ic|localhost}}. Local forwarding requires no additional configuration, however remote forwarding is limited by the remote server's SSH daemon configuration. See the {{Ic|GatewayPorts}} option in {{Ic|sshd_config(5)}} for more information.<br />
<br />
=== Speeding up SSH ===<br />
You can make all sessions to the same host use a single connection, which will greatly speed up subsequent logins, by adding these lines under the proper host in {{ic|/etc/ssh/ssh_config}}:<br />
Host examplehost.com<br />
ControlMaster auto<br />
ControlPath ~/.ssh/socket-%r@%h:%p<br />
<br />
Changing the ciphers used by SSH to less cpu-demanding ones can improve speed. In this aspect, the best choices are arcfour and blowfish-cbc. '''Please do not do this unless you know what you are doing; arcfour has a number of known weaknesses'''. To use them, run SSH with the {{Ic|"c"}} flag, like this:<br />
$ ssh -c arcfour,blowfish-cbc user@server-address<br />
To use them permanently, add this line under the proper host in {{ic|/etc/ssh/ssh_config}}:<br />
Ciphers arcfour,blowfish-cbc<br />
Another option to improve speed is to enable compression with the {{Ic|"C"}} flag. A permanent solution is to add this line under the proper host in {{ic|/etc/ssh/ssh_config}}:<br />
Compression yes<br />
Login time can be shorten by using the {{Ic|"4"}} flag, which bypasses IPv6 lookup. This can be made permanent by adding this line under the proper host in {{ic|/etc/ssh/ssh_config}}:<br />
AddressFamily inet<br />
Another way of making these changes permanent is to create an alias in {{ic|~/.bashrc}}:<br />
alias ssh='ssh -C4c arcfour,blowfish-cbc'<br />
<br />
=== Mounting a remote filesystem with SSHFS ===<br />
Please refer to the [[Sshfs]] article to use sshfs to mount a remote system - accessible via SSH - to a local folder, so you will be able to do any operation on the mounted files with any tool (copy, rename, edit with vim, etc.). Using sshfs instead of shfs is generally preferred as a new version of shfs hasn't been released since 2004.<br />
<br />
=== Keep alive ===<br />
Your ssh session will automatically log out if it is idle. To keep the connection active (alive) add this to {{ic|~/.ssh/config}} or to {{ic|/etc/ssh/ssh_config}} on the client.<br />
<br />
ServerAliveInterval 120<br />
<br />
This will send a "keep alive" signal to the server every 120 seconds.<br />
<br />
Conversely, to keep incoming connections alive, you can set<br />
<br />
ClientAliveInterval 120<br />
<br />
(or some other number greater than 0) in {{ic|/etc/ssh/sshd_config}} on the server.<br />
<br />
=== Saving connection data in ssh config ===<br />
Whenever you want to connect to a ssh server, you usually have to type at least its address and the username. To save that typing work for servers you regularly connect to, you can use the personal {{ic|$HOME/.ssh/config}} or the global {{ic|/etc/ssh/ssh_config}} files as shown in the following example:<br />
<br />
{{hc|$HOME/.ssh/config|<br />
Host myserver<br />
HostName 123.123.123.123<br />
Port 12345<br />
User bob<br />
Host other_server<br />
HostName test.something.org<br />
User alice<br />
CheckHostIP no<br />
Cipher blowfish<br />
}}<br />
<br />
Now you can simply connect to the server by using the name you specified:<br />
<br />
$ ssh myserver<br />
<br />
To see a complete list of the possible options, check out ssh_config's manpage on your system or the [http://www.openbsd.org/cgi-bin/man.cgi?query=ssh_config ssh_config documentation] on the official website.<br />
<br />
=== Autossh - automatically restarts SSH sessions and tunnels ===<br />
When a ssh session or tunnel cannot be kept alive, because for example bad network conditions cause the sshd client to disconnect, you can use [http://www.harding.motd.ca/autossh/ Autossh] to automatically restart them. Autossh can be installed from the [[official repositories]]. <br />
<br />
Usage examples:<br />
$ autossh -M 0 -o "ServerAliveInterval 45" -o "ServerAliveCountMax 2" username@example.com<br />
Combined with [[ sshfs ]]:<br />
$ sshfs -o reconnect,compression=yes,transform_symlinks,ServerAliveInterval=45,ServerAliveCountMax=2,ssh_command='autossh -M 0' username@example.com: /mnt/example <br />
Connecting through a SOCKS-proxy set by [[ Proxy_settings ]]:<br />
$ autossh -M 0 -o "ServerAliveInterval 45" -o "ServerAliveCountMax 2" -NCD 8080 username@example.com <br />
With the {{ic|-f}} option autossh can be made to run as a background process. Running it this way however means the passprase cannot be entered interactively.<br />
<br />
The session will end once you type {{ic|exit}} in the session, or the autossh process receives a SIGTERM, SIGINT of SIGKILL signal.<br />
<br />
If you want to automatically start autossh, it is now easy to get systemd to manage this for you. For example, you could create a systemd unit file like this:<br />
<br />
[Unit]<br />
Description=AutoSSH service for port 2222<br />
After=network.target<br />
<br />
[Service]<br />
ExecStart=/usr/bin/autossh -M 0 -NL 2222:localhost:2222 -o TCPKeepAlive=yes foo@bar.com<br />
<br />
[Install]<br />
WantedBy=multi-user.target<br />
<br />
Then place this in, for example, /etc/systemd/system/autossh.service. Of course, you can make this unit more complex if necessary (see the systemd documentation for details), and obviously you can use your own options for autossh.<br />
<br />
You can then enable your autossh tunnels with, e.g.:<br />
<br />
$ systemctl start autossh<br />
(or whatever you called the service file)<br />
<br />
If this works OK for you, you can make this permanent by running<br />
<br />
$ systemctl enable autossh<br />
<br />
That way autossh will start automatically at boot.<br />
<br />
It is also easy to maintain several autossh processes, to keep several tunnels alive. Just create multiple .service files with different names.<br />
<br />
== Troubleshooting ==<br />
=== SSH connection left hanging after poweroff/reboot ===<br />
SSH connection hangs after poweroff or reboot if systemd stop network before sshd. To fix that problem, comment and change the {{ic|After}} statement:<br />
{{hc|/usr/lib/systemd/system/systemd-user-sessions.service|2=<br />
#After=remote-fs.target<br />
After=network.target}}<br />
<br />
=== Connection refused or timeout problem ===<br />
<br />
==== Is your router doing port forwarding? ====<br />
<br />
SKIP THIS STEP IF YOU ARE NOT BEHIND A NAT MODEM/ROUTER (eg, a VPS or otherwise publicly addressed host). Most home and small businesses will have a NAT modem/router.<br />
<br />
The first thing is to make sure that your router knows to forward any incoming ssh connection to your machine. Your external IP is given to you by your ISP, and it is associated with any requests coming out of your router. So your router needs to know that any incoming ssh connection to your external IP needs to be forwarded to your machine running sshd.<br />
<br />
Find your internal network address.<br />
<br />
ip a<br />
<br />
Find your interface device and look for the inet field. Then access your router's configuration web interface, using your router's IP (find this on the web). Tell your router to forward it to your inet IP. Go to [http://portforward.com/] for more instructions on how to do so for your particular router.<br />
<br />
==== Is SSH running and listening? ====<br />
$ ss -tnlp<br />
<br />
If the above command do not show SSH port is open, SSH is NOT running. Check {{ic|/var/log/messages}} for errors etc.<br />
<br />
==== Are there firewall rules blocking the connection? ====<br />
<br />
[[Iptables]] may be blocking connections on port {{ic|22}}. Check this with:<br />
{{bc|# iptables -nvL}}<br />
and look for rules that might be dropping packets on the {{ic|INPUT}} chain. Then, if necessary, unblock the port with a command like: <br />
{{bc|<br />
# iptables -I INPUT 1 -p tcp --dport 22 -j ACCEPT<br />
}}<br />
For more help configuring firewalls, see [[firewalls]].<br />
<br />
==== Is the traffic even getting to your computer? ====<br />
Start a traffic dump on the computer you're having problems with:<br />
<br />
# tcpdump -lnn -i any port ssh and tcp-syn<br />
<br />
This should show some basic information, then wait for any matching traffic to happen before displaying it. Try your connection now. If you do not see any output when you attempt to connect, then something outside of your computer is blocking the traffic (e. g., hardware firewall, NAT router etc.).<br />
<br />
==== Your ISP or a third party blocking default port? ====<br />
{{Note|Try this step if you '''KNOW''' you aren't running any firewalls and you know you have configured the router for DMZ or have forwarded the port to your computer and it still doesn't work. Here you will find diagnostic steps and a possible solution.}}<br />
<br />
In some cases, your ISP might block the default port (SSH port 22) so whatever you try (opening ports, hardening the stack, defending against flood attacks, et al) ends up useless. To confirm this, create a server on all interfaces (0.0.0.0) and connect remotely. <br />
<br />
If you get an error message comparable to this:<br />
ssh: connect to host www.inet.hr port 22: Connection refused<br />
<br />
That means the port '''ISN'T ''' being blocked by the ISP, but the server doesn't run SSH on that port (See [http://en.wikipedia.org/wiki/Security_through_obscurity security through obscurity]).<br />
<br />
However, if you get an error message comparable to this:<br />
ssh: connect to host 111.222.333.444 port 22: Operation timed out <br />
<br />
That means that something is rejecting your TCP traffic on port 22. Basically that port is stealth, either by your firewall or 3rd party intervetion (like an ISP blocking and/or rejecting incoming traffic on port 22). If you know you aren't running any firewall on your computer, and you know that Gremlins aren't growing in your routers and switches, then your ISP is blocking the traffic.<br />
<br />
To double check, you can run Wireshark on your server and listen to traffic on port 22. Since Wireshark is a Layer 2 Packet Sniffing utility, and TCP/UDP are Layer 3 and above (See [http://en.wikipedia.org/wiki/TCP/IP_model IP Network stack]), if you don't receive anything while connecting remotely, a third party is most likely to be blocking the traffic on that port to your server.<br />
<br />
===== Diagnosis via Wireshark =====<br />
[[pacman|Install]] Wireshark with the {{Pkg|wireshark-cli}} package, available in the [[official repositories]].<br />
<br />
And then run it using,<br />
tshark -f "tcp port 22" -i NET_IF<br />
<br />
where NET_IF is the network interface for a WAN connection (see {{ic|ip a}} to check). If you aren't receiving any packets while trying to connect remotely, you can be very sure that your ISP is blocking the incoming traffic on port 22.<br />
<br />
===== Possible solution =====<br />
The solution is just to use some other port that the ISP isn't blocking. Open the {{ic|/etc/ssh/sshd_config}} and configure the file to use different ports. For example, add:<br />
<br />
Port 22<br />
Port 1234<br />
<br />
Also make sure that other "Port" configuration lines in the file are commented out. Just commenting "Port 22" and putting "Port 1234" won't solve the issue because then sshd will only listen on port 1234. Use both lines to run the SSH server on both ports. <br />
<br />
Restart the server {{ic|systemctl restart sshd.service}} and you're almost done. You still have to configure your client(s) to use the other port instead of the default port. There are numerous solutions to that problem, but let's cover two of them here.<br />
<br />
==== Read from socket failed: connection reset by peer ====<br />
Recent versions of openssh sometimes fail with the above error message, due to a bug involving elliptic curve cryptography. In that case add the following line to {{ic|~/.ssh/config}}:<br />
<br />
HostKeyAlgorithms ssh-rsa-cert-v01@openssh.com,ssh-dss-cert-v01@openssh.com,ssh-rsa-cert-v00@openssh.com,ssh-dss-cert-v00@openssh.com,ecdsa-sha2-nistp256,ecdsa-sha2-nistp384,ecdsa-sha2-nistp521,ssh-rsa,ssh-dss<br />
<br />
With openssh 5.9, the above fix doesn't work. Instead, put the following lines in {{ic|~/.ssh/config}}:<br />
<br />
Ciphers aes128-ctr,aes192-ctr,aes256-ctr,aes128-cbc,3des-cbc <br />
MACs hmac-md5,hmac-sha1,hmac-ripemd160<br />
<br />
See also the [http://www.gossamer-threads.com/lists/openssh/dev/51339 discussion] on the openssh bug forum.<br />
<br />
=== "[your shell]: No such file or directory" / ssh_exchange_identification problem ===<br />
One possible cause for this is the need of certain SSH clients to find an absolute path (one returned by {{Ic|whereis -b [your shell]}}, for instance) in {{Ic|$SHELL}}, even if the shell's binary is located in one of the {{Ic|$PATH}} entries. Another reason can be that the user is no member of the ''network'' group.<br />
<br />
==="Terminal unknown" or "Error opening terminal" error message===<br />
With ssh it is possible to receive errors like "Terminal unknown" upon logging in. Starting ncurses applications like nano fails with the message "Error opening terminal". There are two methods to this problem, a quick one using the $TERM variable and a profound one using the terminfo file.<br />
<br />
====Workaround by setting the $TERM variable====<br />
After connecting to the remote server set the $TERM variable to "xterm" with the following command.<br />
<br />
{{ic|TERM&#61;xterm}}<br />
<br />
This method is a workaround and should be used on ssh servers you do seldomly connect to, because it can have unwanted side effects. Also you have to repeat the command after every connection, or alternatively set it in ~.bashrc .<br />
====Solution using terminfo file====<br />
A profound solution is transferring the terminfo file of the terminal on your client computer to the ssh server. In this example we cover how to setup the terminfo file for the "rxvt-unicode-256color" terminal.<br />
Create the directory containing the terminfo files on the ssh server, while you are logged in to the server issue this command:<br />
<br />
{{ic| mkdir -p ~/.terminfo/r/}}<br />
<br />
Now copy the terminfo file of your terminal to the new directory. Replace ''"rxvt-unicode-256color"'' with your client's terminal in the following command and ''ssh-server'' with the relevant user and server adress.<br />
<br />
{{ic|$ scp /usr/share/terminfo/r/''rxvt-unicode-256color'' ssh-server:~/.terminfo/r/}}<br />
<br />
After logging in and out from the ssh server the problem should be fixed.<br />
<br />
== See also ==<br />
*[[SSH Keys]]<br />
*[[Pam abl]]<br />
*[[fail2ban]]<br />
*[[sshguard]]<br />
*[[Sshfs]]<br />
*[[Syslog-ng]] : To send ssh log data to another file<br />
<br />
== Links & references ==<br />
*[http://www.soloport.com/iptables.html A Cure for the Common SSH Login Attack]<br />
*[http://www.la-samhna.de/library/brutessh.html Defending against brute force ssh attacks]<br />
*[http://www.ibm.com/developerworks/library/l-keyc/index.html OpenSSH key management, Part 1] and [http://www.ibm.com/developerworks/library/l-keyc2 Part 2] on IBM developerWorks</div>Justin8https://wiki.archlinux.org/index.php?title=Bcache&diff=281720Bcache2013-11-06T17:50:49Z<p>Justin8: Updated refrences to the bcache udev rules to match upstream.</p>
<hr />
<div>[[Category:File systems]]<br />
== Introduction ==<br />
<br />
Bcache allows one to use an SSD as a read/write cache (in writeback mode) or read cache (writethrough or writearound) for another blockdevice (generally a rotating HDD or array). This article will show how to install arch using Bcache as the root partition. For an intro to bcache itself, see [http://bcache.evilpiepirate.org/ the bcache homepage]. Be sure to read and reference [http://atlas.evilpiepirate.org/git/linux-bcache.git/tree/Documentation/bcache.txt?h=bcache the bcache manual]. Bcache is in the mainline kernel since 3.10. The kernel on the arch install disk includes the bcache module since 2013.08.01.<br />
<br />
An alternative to Bcache is Facebook's [[Flashcache]].<br />
<br />
Bcache needs the backing device to be formatted as a bcache block device. In most cases, [https://github.com/g2p/blocks blocks to-bcache] can do an in-place conversion.<br />
{{Warning|Be sure you back up any important data first.}}<br />
<br />
== Preparation ==<br />
<br />
* Boot on the install disk (2013.08.01 minimum).<br />
* Install the {{AUR|bcache-tools-git}} package from [[AUR]].<br />
<br />
== Installation ==<br />
<br />
1. Partition your hdd<br />
<br />
grub can't handle bcache, so you'll need at least 2 partitions (boot and one for the bcache backing device). If you're doing UEFI, you'll need an EFI System Partition (ESP) as well. E.g.:<br />
1 2048 22527 10.0 MiB EF00 EFI System<br />
2 22528 432127 200.0 MiB 8300 arch_boot<br />
3 432128 625142414 297.9 GiB 8300 bcache_backing<br />
<br />
{{Note|This example has no swapfile/partition. Keep in mind swapfile on btrfs does not work. For a swap partition on the cache, use LVM in step 7. For a swap partition outside the cache, be sure to make a swap partition now.}}<br />
<br />
2. Configure your HDD as a bcache backing device.<br />
<br />
# make-bcache -B /dev/sda3<br />
<br />
{{Note|<br />
* [[GRUB2]] either needs an EFI partition, "bios boot" partition, or an 'embedding area' of 2KB or so after the MBR that doesn't get overwritten. Also, since GRUB2 does not know about bcache, so you also need a {{ic|/boot}} partition that grub can read. Partitioning {{ic|/dev/sda}} is a must unless you're booting from a different device.<br />
* If all associated disks are partitioned at once as below bcache will automatically attach "-B backing stores" to the "-C ssd cache" and step 4 is unnecessary.<br />
}}<br />
<br />
# make-bcache -B /dev/sd? /dev/sd? -C /dev/sd?<br />
<br />
3. Register any bcache devices with the kernel (this needs to done every bootup)<br />
<br />
# for i in /dev/sd*; do echo $i; echo $i > /sys/fs/bcache/register_quiet; done<br />
<br />
You now have a {{ic|/dev/bcache0}} device.<br />
<br />
{{Note|the bcache user manual says to do {{ic|echo /dev/sd* > /sys/fs/bcache/register_quiet}}, but this did not work for me.}}<br />
<br />
4. Configure your SSD<br />
<br />
Format the SSD as a caching device and link it to the backing device<br />
<br />
# make-bcache -C /dev/sdb<br />
# echo /dev/sdb > /sys/fs/bcache/register <br />
# echo ''UUID__from_previous_command'' > /sys/block/bcache0/bcache/attach<br />
<br />
{{Note|If the UUID is forgotten, it can be found with {{ic|ls /sys/fs/bcache/}} after the cache device has been registered.}}<br />
<br />
5. Format the bcache device. Use LVM or btrfs subvolumes if you want to divide up the {{ic|/dev/bcache0}} device how you like (ex for seperate {{ic|/}}, {{ic|/home}}, {{ic|/var}}, etc):<br />
<br />
# mkfs.btrfs /dev/bcache0<br />
# mount /dev/bcache0 /mnt/<br />
# btrfs subvolume create /mnt/root<br />
# btrfs subvolume create /mnt/home<br />
# umount /mnt<br />
<br />
6. Prepare the installation mount point:<br />
<br />
# mkfs.ext4 /dev/sda2<br />
# mkfs.msdos /dev/sda1 (if your ESP is at least 500MB, use mkfs.vfat to make a FAT32 partition instead)<br />
# pacman -S arch-install-scripts<br />
# mount /dev/bcache0 -o subvol=root,compress=lzo /mnt/<br />
# mkdir /mnt/boot /mnt/home<br />
# mount /dev/bcache0 -o subvol=home,compress=lzo /mnt/home<br />
# mount /dev/sda2 /mnt/boot<br />
# mkdir /boot/efi<br />
# mount /dev/sda1 /mnt/boot/efi/<br />
<br />
7a. Install the system<br />
<br />
Do the rest of the installation [[Installation_Guide#Connect_to_the_internet|as normal]] except this:<br />
<br />
Before you edit {{ic|/etc/mkinitcpio.conf}} and run {{ic|mkinitcpio -p linux}}:<br />
<br />
* install {{AUR|bcache-tools-git}} package from the [[AUR]].<br />
* Edit {{ic|/etc/mkinitcpio.conf}}:<br />
** add the "bcache" module<br />
** add the "bcache" hook between block and filesystems <br />
<br />
{{Note|The AUR package hook currently uses a for loop similar to the one in step 3 for scanning and assembling bcache arrays at boot. Try step 7b for a udev approach.}}<br />
<br />
7b. Alternatively, udev can be leveraged to assemble bcache arrays as it discovers them. Install the AUR package in step 7a then copy [http://evilpiepirate.org/git/bcache-tools.git/tree/69-bcache.rules 69-bcache.rules] to {{ic|/usr/lib/udev/rules.d/69-bcache.rules}}.<br />
<br />
You must then copy the following script adapted from mdadm_udev to {{ic|/usr/lib/initcpio/install/bcache_udev}} and edit {{ic|mkinitcpio.conf}} to include the {{ic|bcache_udev}} hook instead of {{ic|bcache}} before running {{ic|mkinitcpio -p linux}}:<br />
<br />
{{bc|1=<br />
#!/bin/bash<br />
build() {<br />
add_checked_modules -f 'bcache'<br />
add_binary "/usr/lib/udev/bcache-register"<br />
add_binary "/usr/bin/probe-bcache"<br />
add_file "/usr/lib/udev/rules.d/69-bcache.rules"<br />
}<br />
help() {<br />
cat <<HELPEOF<br />
This hook loads the necessary modules for a bcache ssd cached array and uses<br />
udev at runtime to create the devices. This hook will NOT work without the <br />
udev hook included on the image.<br />
HELPEOF<br />
}<br />
# vim: set ft=sh ts=4 sw=4 et:<br />
}}<br />
<br />
{{Note|It is important to ensure that all files referenced in the {{ic|bcache_udev}} hook can be found at the paths stated.}}<br />
<br />
8. Reboot and make sure it works from the {{ic|/dev/bcache0}} device.<br />
<br />
== Configuring ==<br />
<br />
There are many options that can be configured (such as cache mode, cache flush interval, sequential write heuristic, etc.) This is currently done by writing to files in {{ic|/sys}}. See the [http://atlas.evilpiepirate.org/git/linux-bcache.git/tree/Documentation/bcache.txt bcache user documentation].<br />
<br />
Changing the cache mode is done by echoing one of 'writethrough', 'writeback', 'writearound' or 'none' to /sys/block/bcache[0-9]/bcache/cache_mode.<br />
<br />
== Advanced operations ==<br />
<br />
=== Resize backing device ===<br />
<br />
It's possible to resize the backing device so long as you don't move the partition start. This process is described on [http://comments.gmane.org/gmane.linux.kernel.bcache.devel/249 the mailing list]. Here is an example using btrfs volume directly on bcache0. For LVM containers or for other filesystems, procedure will differ.<br />
<br />
==== Example of growing ====<br />
<br />
In this example, I grow the filesystem by 4GB. <br />
<br />
1. Reboot to a live CD/USB Drive (need not be bcache enabled) and use fdisk, gdisk, parted, or your other favorite tool to delete the backing partition and recreate it with the same start and a total size 4G larger. <br />
<br />
{{Warning|Do not use a tool like GParted that might perform filesystem operations! It will not recognize the bcache partition and might overwrite part of it!!}}<br />
<br />
2. Reboot to your normal install. Your filesystem will be currently mounted. That is fine. Issue the command to resize the partition to its maximum. For btrfs, that is<br />
<br />
# btrfs filesystem resize max /<br />
<br />
For ext3/4, that is:<br />
<br />
# resize2fs /dev/bcache0<br />
<br />
==== Example of shrinking ====<br />
<br />
In this example, I shrink the filesystem by 4GB.<br />
<br />
1. Disable writeback cache (switch to writethrough cache) and wait for the disk to flush.<br />
<br />
# echo writethrough > /sys/block/bcache0/bcache/cache_mode<br />
$ watch cat /sys/block/bcache0/bcache/state<br />
<br />
wait until state reports "clean". This might take a while.<br />
<br />
2. Shrink the mounted filesystem by something more than the desired amount, to ensure we don't accidentally clip it later. For btrfs, that is:<br />
<br />
# btrfs filesystem resize -5G /<br />
<br />
For ext3/4 you can use ''resize2fs'', but only if the partition is unmounted<br />
<br />
$ df -h /home<br />
/dev/bcache0 290G 20G 270G 1% /home<br />
# umount /home<br />
# resize2fs /dev/bcache0 283G<br />
<br />
3. Reboot to a LiveCD/USB drive (doesn't need to support bcache) and use fdisk, gdisk, parted, or your other favorite tool to delete the backing partition and recreate it with the same start and a total size 4G smaller.<br />
<br />
{{Warning|Do not use a tool like GParted that might perform filesystem operations! It won't recognize the bcache partition and might overwrite part of it!!}}<br />
<br />
4. Reboot to your normal install. Your filesystem will be currently mounted. That is fine. Issue the command to resize the partition to its maximum (that is, the size we shrunk the actual partition to in step 3). For btrfs, that is:<br />
<br />
# btrfs filesystem resize max /<br />
<br />
For ext3/4, that is:<br />
<br />
# resize2fs /dev/bcache0<br />
<br />
5. Re-enable writeback cache if you want that enabled:<br />
<br />
# echo writeback > /sys/block/bcache0/bcache/cache_mode<br />
<br />
{{Note|If you're very careful you can shrink the filesystem to the exact size in step 2 and avoid step 4. Be careful, though, many partition tools don't do exactly what you want, but instead adjust the requested partition start/end points to end on sector boundaries. This may be difficult to calculate ahead of time}}<br />
<br />
== Troubleshooting ==<br />
<br />
=== /dev/bcache device doesn't exist on bootup ===<br />
<br />
If you are sent to a busy box shell with an error:<br />
<br />
{{bc|1=<br />
ERROR: Unable to find root device 'UUID=b6b2d82b-f87e-44d5-bbc5-c51dd7aace15'.<br />
You are being dropped to a recovery shell<br />
Type 'exit' to try and continue booting<br />
}}<br />
<br />
This might happen if the backing device is configured for "writeback" mode (default is writearound). When in "writeback" mode, the /dev/bcache0 device is not started until the cache device is both registered and attached. Registering is something that needs to happen every bootup, but attaching should only have to be done once. I've sometime had to re-attach.<br />
<br />
To continue booting, try one of the following:<br />
<br />
* Register both the backing device and the cacheing device<br />
<br />
# echo /dev/sda3 > /sys/fs/bcache/register<br />
# echo /dev/sdb > /sys/fs/bcache/register<br />
<br />
If the /dev/bcache0 device now exists, type exit and continue booting. You will need to fix your initcpio to ensure devices are registered before mounting the root device.<br />
<br />
{{Note|<br />
* An error of "sh: echo: write error: Invalid argument" means the device was already registered or is not recognized as either a bcache backing device or cache.<br />
* This can also happen if using udev's 69-bcache.rules in Installation's step 7b and blkid and bcache-probe "disagree" due to rogue superblocks. See [http://bcache.evilpiepirate.org/#index6h1 bcache's wiki] for a possible explanation/resolution.<br />
}}<br />
<br />
* Re-attach the cache to the backing device:<br />
<br />
If the cache device was registered, a folder with the UUID of the cache should exist in {{ic|/sys/fs/bcache}}. Use that UUID when following the example below:<br />
<br />
# ls /sys/fs/bcache/<br />
b6b2d82b-f87e-44d5-bbc5-c51dd7aace15 register register_quiet<br />
# echo b6b2d82b-f87e-44d5-bbc5-c51dd7aace15 > /sys/block/sda/sda3/bcache/attach<br />
<br />
If the {{ic|/dev/bcache0}} device now exists, type exit and continue booting. You should not have to do this again. If it persists, ask on the bcache mailing list.<br />
<br />
{{Note|An error of {{ic|sh: echo: write error: Invalid argument}} means the device was already attached. An error of {{ic|sh: echo: write error: No such file or directory}} means the UUID is not a valid cache (make sure you typed it correctly).}}<br />
<br />
* Invalidate the cache and force the backing device to run without it. You might want to check some stats, such as "dirty_data" so you have some idea of how much data will be lost.<br />
<br />
# cat /sys/block/sda/sda3/bcache/dirty_data<br />
-3.9M<br />
<br />
dirty data is data in the cache that has not been written to the backing device. If you force the backing device to run, this data will be lost, even if you later re-attach the cache.<br />
<br />
# cat /sys/block/sda/sda3/bcache/running<br />
0<br />
# echo 1 > /sys/block/sda/sda3/bcache/running<br />
<br />
The {{ic|/dev/bcache0}} device will now exist. Type exit and continue booting. You might want to unregister the cache device and run make-bcache again. An fsck on {{ic|/dev/bcache0}} would also be wise. See the [http://atlas.evilpiepirate.org/git/linux-bcache.git/tree/Documentation/bcache.txt?h=bcache bcache user documentation].<br />
<br />
{{Warning|Only invalidate the cache if one of the two options above did not work.}}<br />
<br />
=== /sys/fs/bcache/ doesn't exist ===<br />
<br />
The kernel you booted is not bcache enabled.</div>Justin8https://wiki.archlinux.org/index.php?title=OpenSSH&diff=281706OpenSSH2013-11-06T15:33:33Z<p>Justin8: /* Autossh - automatically restarts SSH sessions and tunnels */ - Fixed incorrect syntax in examples that doesn't work.</p>
<hr />
<div>[[Category:Secure Shell]]<br />
[[ar:Ssh]]<br />
[[es:Secure Shell]]<br />
[[de:SSH]]<br />
[[fr:ssh]]<br />
[[it:Secure Shell]]<br />
[[ko:Secure Shell]]<br />
[[pl:Secure Shell]]<br />
[[pt:Secure Shell]]<br />
[[ru:Secure Shell]]<br />
[[sr:Secure Shell]]<br />
[[zh-CN:Secure Shell]]<br />
'''Secure Shell''' ('''SSH''') is a network protocol that allows data to be exchanged over a secure channel between two computers. Encryption provides confidentiality and integrity of data. SSH uses public-key cryptography to authenticate the remote computer and allow the remote computer to authenticate the user, if necessary.<br />
<br />
SSH is typically used to log into a remote machine and execute commands, but it also supports tunneling, forwarding arbitrary TCP ports and X11 connections; file transfer can be accomplished using the associated SFTP or SCP protocols.<br />
<br />
An SSH server, by default, listens on the standard TCP port 22. An SSH client program is typically used for establishing connections to an ''sshd'' daemon accepting remote connections. Both are commonly present on most modern operating systems, including Mac OS X, GNU/Linux, Solaris and OpenVMS. Proprietary, freeware and open source versions of various levels of complexity and completeness exist.<br />
<br />
(Source: [[Wikipedia:Secure Shell]])<br />
<br />
== OpenSSH ==<br />
OpenSSH (OpenBSD Secure Shell) is a set of computer programs providing encrypted communication sessions over a computer network using the ssh protocol. It was created as an open source alternative to the proprietary Secure Shell software suite offered by SSH Communications Security. OpenSSH is developed as part of the OpenBSD project, which is led by Theo de Raadt.<br />
<br />
OpenSSH is occasionally confused with the similarly-named OpenSSL; however, the projects have different purposes and are developed by different teams, the similar name is drawn only from similar goals.<br />
<br />
=== Installing OpenSSH ===<br />
[[pacman|Install]] {{Pkg|openssh}} from the [[official repositories]].<br />
<br />
=== Configuring SSH ===<br />
====Client====<br />
The SSH client configuration file is {{ic|/etc/ssh/ssh_config}} or {{ic|~/.ssh/config}}.<br />
<br />
An example configuration: <br />
<br />
{{hc|/etc/ssh/ssh_config|<br />
# $OpenBSD: ssh_config,v 1.26 2010/01/11 01:39:46 dtucker Exp $<br />
<br />
# This is the ssh client system-wide configuration file. See<br />
# ssh_config(5) for more information. This file provides defaults for<br />
# users, and the values can be changed in per-user configuration files<br />
# or on the command line.<br />
<br />
# Configuration data is parsed as follows:<br />
# 1. command line options<br />
# 2. user-specific file<br />
# 3. system-wide file<br />
# Any configuration value is only changed the first time it is set.<br />
# Thus, host-specific definitions should be at the beginning of the<br />
# configuration file, and defaults at the end.<br />
<br />
# Site-wide defaults for some commonly used options. For a comprehensive<br />
# list of available options, their meanings and defaults, please see the<br />
# ssh_config(5) man page.<br />
<br />
# Host *<br />
# ForwardAgent no<br />
# ForwardX11 no<br />
# RhostsRSAAuthentication no<br />
# RSAAuthentication yes<br />
# PasswordAuthentication yes<br />
# HostbasedAuthentication no<br />
# GSSAPIAuthentication no<br />
# GSSAPIDelegateCredentials no<br />
# BatchMode no<br />
# CheckHostIP yes<br />
# AddressFamily any<br />
# ConnectTimeout 0<br />
# StrictHostKeyChecking ask<br />
# IdentityFile ~/.ssh/identity<br />
# IdentityFile ~/.ssh/id_rsa<br />
# IdentityFile ~/.ssh/id_dsa<br />
# Port 22<br />
# Protocol 2,1<br />
# Cipher 3des<br />
# Ciphers aes128-ctr,aes192-ctr,aes256-ctr,arcfour256,arcfour128,aes128-cbc,3des-cbc<br />
# MACs hmac-md5,hmac-sha1,umac-64@openssh.com,hmac-ripemd160<br />
# EscapeChar ~<br />
# Tunnel no<br />
# TunnelDevice any:any<br />
# PermitLocalCommand no<br />
# VisualHostKey no<br />
# ProxyCommand ssh -q -W %h:%p gateway.example.com<br />
}}<br />
<br />
It is not longer needed to change the Protocol line into:<br />
Protocol 2<br />
<br />
That means Protocol 1 will not be used as long as it is not explicitly enabled. (source: http://www.openssh.org/txt/release-5.4)<br />
<br />
====Daemon====<br />
The SSH daemon configuration file can be found and edited in {{ic|/etc/ssh/ssh'''d'''_config}}.<br />
<br />
To allow access only for some users add this line:<br />
AllowUsers user1 user2<br />
<br />
To disable root login over SSH, change the PermitRootLogin line into this:<br />
PermitRootLogin no<br />
<br />
To add a nice welcome message edit the file {{ic|/etc/issue}} and change the Banner line into this:<br />
Banner /etc/issue<br />
<br />
{{Tip| You may want to change the default port from 22 to any higher port (see [http://en.wikipedia.org/wiki/Security_through_obscurity security through obscurity]).}} <br />
<br />
Even though the port ssh is running on could be detected by using a port-scanner like nmap, changing it will reduce the number of log entries caused by automated authentication attempts. To help select a port review the [[Wikipedia:List of TCP and UDP port numbers|list of TCP and UDP port numbers]].<br />
<br />
{{Tip|Disabling password logins entirely will greatly increase security, see [[SSH Keys]] for more information.}}<br />
<br />
=== Managing the sshd daemon ===<br />
You can start the sshd daemon with the following command:<br />
# systemctl start sshd<br />
<br />
You can enable the sshd daemon at startup with the following command:<br />
# systemctl enable sshd.service<br />
<br />
{{Warning|Systemd is an asynchronous starting process. If you bind the SSH daemon to a specific IP address {{ic|ListenAddress 192.168.1.100}} it may fail to load during boot since the default sshd.service unit file has no dependency on network interfaces being enabled. When binding to an IP address, you will need to add {{ic|After&#61;network.target}} to a custom sshd.service unit file. See [[Systemd#Editing provided unit files]].}}<br />
<br />
Or you can enable SSH Daemon socket so the daemon is started on the first incoming connection:<br />
# systemctl enable sshd.socket<br />
If you use a different port than the default 22, you have to set "ListenStream" in the unit file. Copy /lib/systemd/system/sshd.socket to /etc/systemd/system/sshd.socket to keep your unit file from being overwritten on upgrades. In /etc/systemd/system/sshd.socket change "ListenStream" the appropriate port.<br />
<br />
{{Warning|Using sshd.socket effectively negates the {{ic|ListenAddress}} setting, so using the default sshd.socket will allow connections over any address. To achieve the effect of setting {{ic|ListenAddress}}, you must create a custom unit file and modify ListenStream (ie. {{ic|ListenStream&#61;192.168.1.100:22}} is equivalent to {{ic|ListenAddress 192.168.1.100}}). However, doing so has the same drawback as setting {{ic|ListenAddress}}: the socket will fail to start if the network is not up in time.}}<br />
<br />
=== Connecting to the server ===<br />
To connect to a server, run:<br />
$ ssh -p port user@server-address<br />
<br />
=== Protecting SSH ===<br />
Allowing remote log-on through SSH is good for administrative purposes, but can pose a threat to your server's security. Often the target of brute force attacks, SSH access needs to be limited properly to prevent third parties gaining access to your server.<br />
* Use non-standard account names and passwords<br />
* Only allow incoming SSH connections from trusted locations<br />
* Use [[fail2ban]] or [[sshguard]] to monitor for brute force attacks, and ban brute forcing IPs accordingly<br />
<br />
===== Protecting against brute force attacks =====<br />
Brute forcing is a simple concept: One continuously tries to log in to a webpage or server log-in prompt like SSH with a high number of random username and password combinations. You can protect yourself from brute force attacks by using an automated script that blocks anybody trying to brute force their way in, for example [[fail2ban]] or [[sshguard]].<br />
<br />
===== Deny root login =====<br />
It is generally considered bad practice to allow the user '''root''' to log in over SSH: The '''root''' account will exist on nearly any Linux system and grants full access to the system, once login has been achieved. Sudo provides root rights for actions requiring these and is the more secure solution, third parties would have to find a username present on the system, the matching password and the matching password for sudo to get root rights on your system. More barriers to be breached before full access to the system is reached.<br />
<br />
Configure SSH to deny remote logins with the root user by editing {{ic|/etc/ssh/sshd_config}} and look for this section:<br />
# Authentication:<br />
<br />
#LoginGraceTime 2m<br />
''#PermitRootLogin yes''<br />
#StrictModes yes<br />
#MaxAuthTries 6<br />
#MaxSessions 10<br />
<br />
Now simply change ''#PermitRootLogin yes'' to no, and uncomment the line:<br />
PermitRootLogin no<br />
<br />
Next, restart the SSH daemon:<br />
# systemctl restart sshd<br />
<br />
You will now be unable to log in through SSH under root, but will still be able to log in with your normal user and use ''su'' - or ''sudo'' to do system administration.<br />
<br />
== Other SSH clients and servers ==<br />
Apart from OpenSSH, there are many SSH [[Wikipedia:Comparison of SSH clients|clients]] and [[Wikipedia:Comparison of SSH servers|servers]] avaliable.<br />
<br />
=== Dropbear ===<br />
[[Wikipedia:Dropbear (software)|Dropbear]] is a SSH-2 client and server. {{AUR|dropbear}} is available in the [[AUR]].<br />
<br />
The commandline ssh client is named dbclient.<br />
<br />
=== SSH alternative: Mobile Shell - responsive, survives disconnects ===<br />
From the Mosh [http://mosh.mit.edu/ website]:<br />
<br />
Remote terminal application that allows roaming, supports intermittent connectivity, and provides intelligent local echo and line editing of user keystrokes. Mosh is a replacement for SSH. It's more robust and responsive, especially over Wi-Fi, cellular, and long-distance links.<br />
<br />
[[pacman|Install]] {{Pkg|mosh}} from the [[official repositories]] or the latest revision {{AUR|mosh-git}} in the [[AUR]].<br />
<br />
== Tips and tricks ==<br />
<br />
=== Encrypted SOCKS tunnel ===<br />
This is highly useful for laptop users connected to various unsafe wireless connections. The only thing you need is an SSH server running at a somewhat secure location, like your home or at work. It might be useful to use a dynamic DNS service like [http://www.dyndns.org/ DynDNS] so you do not have to remember your IP-address.<br />
<br />
==== Step 1: start the connection ====<br />
You only have to execute this single command to start the connection:<br />
$ ssh -TND 4711 user@host<br />
where {{Ic|"user"}} is your username at the SSH server running at the {{Ic|"host"}}. It will ask for your password, and then you're connected! The {{Ic|"N"}} flag disables the interactive prompt, and the {{Ic|"D"}} flag specifies the local port on which to listen on (you can choose any port number if you want). The {{Ic|"T"}} flag disables pseudo-tty allocation.<br />
<br />
It's nice to add the verbose {{Ic|"-v"}} flag, because then you can verify that it's actually connected from that output.<br />
<br />
==== Step 2: configure your browser (or other programs) ====<br />
The above step is completely useless if you do not configure your web browser (or other programs) to use this newly created socks tunnel. Since the current version of SSH supports both SOCKS4 and SOCKS5, you can use either of them.<br />
<br />
* For Firefox: ''Edit &rarr; Preferences &rarr; Advanced &rarr; Network &rarr; Connection &rarr; Setting'':<br />
: Check the ''"Manual proxy configuration"'' radio button, and enter "localhost" in the ''"SOCKS host"'' text field, and then enter your port number in the next text field (I used 4711 above).<br />
<br />
Firefox does not automatically make DNS requests through the socks tunnel. This potential privacy concern can be mitigated by the following steps:<br />
<br />
# Type about:config into the Firefox location bar.<br />
# Search for network.proxy.socks_remote_dns<br />
# Set the value to true.<br />
# Restart the browser.<br />
<br />
* For Chromium: You can set the SOCKS settings as environment variables or as command line options. I recommend to add one of the following functions to your {{ic|.bashrc}}:<br />
function secure_chromium {<br />
port=4711<br />
export SOCKS_SERVER=localhost:$port<br />
export SOCKS_VERSION=5<br />
chromium &<br />
exit<br />
}<br />
OR<br />
function secure_chromium {<br />
port=4711<br />
chromium --proxy-server="socks://localhost:$port" &<br />
exit<br />
}<br />
<br />
Now open a terminal and just do:<br />
$ secure_chromium<br />
<br />
Enjoy your secure tunnel!<br />
<br />
=== X11 forwarding ===<br />
To run graphical programs through a SSH connection you can enable X11 forwarding. An option needs to be set in the configuration files on the server and client (here "client" means your (desktop) machine your X11 Server runs on, and you will run X applications on the "server").<br />
<br />
[[pacman|Install]] {{Pkg|xorg-xauth}} from the [[official repositories]] onto the server.<br />
<br />
* Enable the '''AllowTcpForwarding''' option in {{ic|ssh'''d'''_config}} on the '''server'''.<br />
* Enable the '''X11Forwarding''' option in {{ic|ssh'''d'''_config}} on the '''server'''.<br />
* Set the '''X11DisplayOffset''' option in {{ic|ssh'''d'''_config}} on the '''server''' to 10.<br />
* Enable the '''X11UseLocalhost''' option in {{ic|ssh'''d'''_config}} on the '''server'''.<br />
Also:<br />
* Enable the '''ForwardX11''' option in {{ic|ssh_config}} on the '''client'''.<br />
* Enable the '''ForwardX11Trusted''' if gui is drawing badly.<br />
<br />
You need to restart the ssh daemon on the server for these changes to take effect, of course.<br />
<br />
To use the forwarding, log on to your server through ssh:<br />
$ ssh -X -p port user@server-address<br />
If you receive errors trying to run graphical applications try trusted forwarding instead:<br />
$ ssh -Y -p port user@server-address<br />
You can now start any X program on the remote server, the output will be forwarded to your local session:<br />
$ xclock<br />
<br />
<br />
If you get "Cannot open display" errors try the following command as the non root user:<br />
$ xhost +<br />
<br />
the above command will allow anybody to forward X11 applications. To restrict forwarding to a particular host type:<br />
$ xhost +hostname<br />
<br />
where hostname is the name of the particular host you want to forward to. Type "man xhost" for more details.<br />
<br />
Be careful with some applications as they check for a running instance on the local machine. Firefox is an example. Either close running Firefox or use the following start parameter to start a remote instance on the local machine<br />
$ firefox -no-remote<br />
<br />
If you get "X11 forwarding request failed on channel 0" when you connect (and the server /var/log/errors.log shows "Failed to allocate internet-domain X11 display socket"), try to either<br />
* Enable the '''AddressFamily any''' option in {{ic|ssh'''d'''_config}} on the '''server''', or<br />
* Set the '''AddressFamily''' option in {{ic|ssh'''d'''_config}} on the '''server''' to inet.<br />
Setting it to inet may fix problems with Ubuntu clients on IPv4.<br />
<br />
For running X applications as other user on the SSH server you need to {{Ic|xauth add}} the authentication line taken from {{Ic|xauth list}} of the SSH logged in user.<br />
<br />
=== Forwarding other ports ===<br />
In addition to SSH's built-in support for X11, it can also be used to securely tunnel any TCP connection, by use of local forwarding or remote forwarding.<br />
<br />
Local forwarding opens a port on the local machine, connections to which will be forwarded to the remote host and from there on to a given destination. Very often, the forwarding destination will be the same as the remote host, thus providing a secure shell and, e.g. a secure VNC connection, to the same machine. Local forwarding is accomplished by means of the {{Ic|-L}} switch and it's accompanying forwarding specification in the form of {{Ic|<tunnel port>:<destination address>:<destination port>}}.<br />
<br />
Thus:<br />
<br />
$ ssh -L 1000:mail.google.com:25 192.168.0.100<br />
<br />
will use SSH to login to and open a shell on 192.168.0.100, and will also create a tunnel from the local machine's TCP port 1000 to mail.google.com on port 25. Once established, connections to localhost:1000 will connect to the Gmail SMTP port. To Google, it will appear that any such connection (though not necessarily the data conveyed over the connection) originated from 192.168.0.100, and such data will be secure as between the local machine and 192.168.0.100, but not between 192.168.0.100, unless other measures are taken.<br />
<br />
Similarly:<br />
<br />
$ ssh -L 2000:192.168.0.100:6001 192.168.0.100<br />
<br />
will allow connections to localhost:2000 which will be transparently sent to the remote host on port 6001. The preceding example is useful for VNC connections using the vncserver utility--part of the tightvnc package--which, though very useful, is explicit about its lack of security.<br />
<br />
Remote forwarding allows the remote host to connect to an arbitrary host via the SSH tunnel and the local machine, providing a functional reversal of local forwarding, and is useful for situations where, e.g., the remote host has limited connectivity due to firewalling. It is enabled with the {{Ic|-R}} switch and a forwarding specification in the form of {{Ic|<tunnel port>:<destination address>:<destination port>}}.<br />
<br />
Thus:<br />
<br />
$ ssh -R 3000:irc.freenode.net:6667 192.168.0.200<br />
<br />
will bring up a shell on 192.168.0.200, and connections from 192.168.0.200 to itself on port 3000 (remotely speaking, localhost:3000) will be sent over the tunnel to the local machine and then on to irc.freenode.net on port 6667, thus, in this example, allowing the use of IRC programs on the remote host to be used, even if port 6667 would normally be blocked to it.<br />
<br />
Both local and remote forwarding can be used to provide a secure "gateway," allowing other computers to take advantage of an SSH tunnel, without actually running SSH or the SSH daemon by providing a bind-address for the start of the tunnel as part of the forwarding specification, e.g. {{Ic|<tunnel address>:<tunnel port>:<destination address>:<destination port>}}. The {{Ic|<tunnel address>}} can be any address on the machine at the start of the tunnel, {{Ic|localhost}}, {{Ic|*}} (or blank), which, respectively, allow connections via the given address, via the loopback interface, or via any interface. By default, forwarding is limited to connections from the machine at the "beginning" of the tunnel, i.e. the {{Ic|<tunnel address>}} is set to {{Ic|localhost}}. Local forwarding requires no additional configuration, however remote forwarding is limited by the remote server's SSH daemon configuration. See the {{Ic|GatewayPorts}} option in {{Ic|sshd_config(5)}} for more information.<br />
<br />
=== Speeding up SSH ===<br />
You can make all sessions to the same host use a single connection, which will greatly speed up subsequent logins, by adding these lines under the proper host in {{ic|/etc/ssh/ssh_config}}:<br />
Host examplehost.com<br />
ControlMaster auto<br />
ControlPath ~/.ssh/socket-%r@%h:%p<br />
<br />
Changing the ciphers used by SSH to less cpu-demanding ones can improve speed. In this aspect, the best choices are arcfour and blowfish-cbc. '''Please do not do this unless you know what you are doing; arcfour has a number of known weaknesses'''. To use them, run SSH with the {{Ic|"c"}} flag, like this:<br />
$ ssh -c arcfour,blowfish-cbc user@server-address<br />
To use them permanently, add this line under the proper host in {{ic|/etc/ssh/ssh_config}}:<br />
Ciphers arcfour,blowfish-cbc<br />
Another option to improve speed is to enable compression with the {{Ic|"C"}} flag. A permanent solution is to add this line under the proper host in {{ic|/etc/ssh/ssh_config}}:<br />
Compression yes<br />
Login time can be shorten by using the {{Ic|"4"}} flag, which bypasses IPv6 lookup. This can be made permanent by adding this line under the proper host in {{ic|/etc/ssh/ssh_config}}:<br />
AddressFamily inet<br />
Another way of making these changes permanent is to create an alias in {{ic|~/.bashrc}}:<br />
alias ssh='ssh -C4c arcfour,blowfish-cbc'<br />
<br />
=== Mounting a remote filesystem with SSHFS ===<br />
Please refer to the [[Sshfs]] article to use sshfs to mount a remote system - accessible via SSH - to a local folder, so you will be able to do any operation on the mounted files with any tool (copy, rename, edit with vim, etc.). Using sshfs instead of shfs is generally preferred as a new version of shfs hasn't been released since 2004.<br />
<br />
=== Keep alive ===<br />
Your ssh session will automatically log out if it is idle. To keep the connection active (alive) add this to {{ic|~/.ssh/config}} or to {{ic|/etc/ssh/ssh_config}} on the client.<br />
<br />
ServerAliveInterval 120<br />
<br />
This will send a "keep alive" signal to the server every 120 seconds.<br />
<br />
Conversely, to keep incoming connections alive, you can set<br />
<br />
ClientAliveInterval 120<br />
<br />
(or some other number greater than 0) in {{ic|/etc/ssh/sshd_config}} on the server.<br />
<br />
=== Saving connection data in ssh config ===<br />
Whenever you want to connect to a ssh server, you usually have to type at least its address and the username. To save that typing work for servers you regularly connect to, you can use the personal {{ic|$HOME/.ssh/config}} or the global {{ic|/etc/ssh/ssh_config}} files as shown in the following example:<br />
<br />
{{hc|$HOME/.ssh/config|<br />
Host myserver<br />
HostName 123.123.123.123<br />
Port 12345<br />
User bob<br />
Host other_server<br />
HostName test.something.org<br />
User alice<br />
CheckHostIP no<br />
Cipher blowfish<br />
}}<br />
<br />
Now you can simply connect to the server by using the name you specified:<br />
<br />
$ ssh myserver<br />
<br />
To see a complete list of the possible options, check out ssh_config's manpage on your system or the [http://www.openbsd.org/cgi-bin/man.cgi?query=ssh_config ssh_config documentation] on the official website.<br />
<br />
=== Autossh - automatically restarts SSH sessions and tunnels ===<br />
When a ssh session or tunnel cannot be kept alive, because for example bad network conditions cause the sshd client to disconnect, you can use [http://www.harding.motd.ca/autossh/ Autossh] to automatically restart them. Autossh can be installed from the [[official repositories]]. <br />
<br />
Usage examples:<br />
$ autossh -M 0 -o "ServerAliveInterval 45" -o "ServerAliveCountMax 2" username@example.com<br />
Combined with [[ sshfs ]]:<br />
$ sshfs -o reconnect,compression=yes,transform_symlinks,ServerAliveInterval=45,ServerAliveCountMax=2,ssh_command='autossh -M 0' username@example.com: /mnt/example <br />
Connecting through a SOCKS-proxy set by [[ Proxy_settings ]]:<br />
$ autossh -M 0 -o "ServerAliveInterval 45" -o "ServerAliveCountMax 2" -NCD 8080 username@example.com <br />
With the {{ic|-f}} option autossh can be made to run as a background process. Running it this way however means the passprase cannot be entered interactively.<br />
<br />
The session will end once you type {{ic|exit}} in the session, or the autossh process receives a SIGTERM, SIGINT of SIGKILL signal.<br />
<br />
If you want to automatically start autossh, it is now easy to get systemd to manage this for you. For example, you could create a systemd unit file like this:<br />
<br />
[Unit]<br />
Description=AutoSSH service for port 2222<br />
After=network.target<br />
<br />
[Service]<br />
ExecStart=/usr/bin/autossh -M 0 -NL 2222:localhost:2222 foo@bar.com<br />
<br />
[Install]<br />
WantedBy=multi-user.target<br />
<br />
Then place this in, for example, /etc/systemd/system/autossh.service. Of course, you can make this unit more complex if necessary (see the systemd documentation for details), and obviously you can use your own options for autossh.<br />
<br />
You can then enable your autossh tunnels with, e.g.:<br />
<br />
$ systemctl start autossh<br />
(or whatever you called the service file)<br />
<br />
If this works OK for you, you can make this permanent by running<br />
<br />
$ systemctl enable autossh<br />
<br />
That way autossh will start automatically at boot.<br />
<br />
It is also easy to maintain several autossh processes, to keep several tunnels alive. Just create multiple .service files with different names.<br />
<br />
== Troubleshooting ==<br />
=== SSH connection left hanging after poweroff/reboot ===<br />
SSH connection hangs after poweroff or reboot if systemd stop network before sshd. To fix that problem, comment and change the {{ic|After}} statement:<br />
{{hc|/usr/lib/systemd/system/systemd-user-sessions.service|2=<br />
#After=remote-fs.target<br />
After=network.target}}<br />
<br />
=== Connection refused or timeout problem ===<br />
<br />
==== Is your router doing port forwarding? ====<br />
<br />
SKIP THIS STEP IF YOU ARE NOT BEHIND A NAT MODEM/ROUTER (eg, a VPS or otherwise publicly addressed host). Most home and small businesses will have a NAT modem/router.<br />
<br />
The first thing is to make sure that your router knows to forward any incoming ssh connection to your machine. Your external IP is given to you by your ISP, and it is associated with any requests coming out of your router. So your router needs to know that any incoming ssh connection to your external IP needs to be forwarded to your machine running sshd.<br />
<br />
Find your internal network address.<br />
<br />
ip a<br />
<br />
Find your interface device and look for the inet field. Then access your router's configuration web interface, using your router's IP (find this on the web). Tell your router to forward it to your inet IP. Go to [http://portforward.com/] for more instructions on how to do so for your particular router.<br />
<br />
==== Is SSH running and listening? ====<br />
$ ss -tnlp<br />
<br />
If the above command do not show SSH port is open, SSH is NOT running. Check {{ic|/var/log/messages}} for errors etc.<br />
<br />
==== Are there firewall rules blocking the connection? ====<br />
<br />
[[Iptables]] may be blocking connections on port {{ic|22}}. Check this with:<br />
{{bc|# iptables -nvL}}<br />
and look for rules that might be dropping packets on the {{ic|INPUT}} chain. Then, if necessary, unblock the port with a command like: <br />
{{bc|<br />
# iptables -I INPUT 1 -p tcp --dport 22 -j ACCEPT<br />
}}<br />
For more help configuring firewalls, see [[firewalls]].<br />
<br />
==== Is the traffic even getting to your computer? ====<br />
Start a traffic dump on the computer you're having problems with:<br />
<br />
# tcpdump -lnn -i any port ssh and tcp-syn<br />
<br />
This should show some basic information, then wait for any matching traffic to happen before displaying it. Try your connection now. If you do not see any output when you attempt to connect, then something outside of your computer is blocking the traffic (e. g., hardware firewall, NAT router etc.).<br />
<br />
==== Your ISP or a third party blocking default port? ====<br />
{{Note|Try this step if you '''KNOW''' you aren't running any firewalls and you know you have configured the router for DMZ or have forwarded the port to your computer and it still doesn't work. Here you will find diagnostic steps and a possible solution.}}<br />
<br />
In some cases, your ISP might block the default port (SSH port 22) so whatever you try (opening ports, hardening the stack, defending against flood attacks, et al) ends up useless. To confirm this, create a server on all interfaces (0.0.0.0) and connect remotely. <br />
<br />
If you get an error message comparable to this:<br />
ssh: connect to host www.inet.hr port 22: Connection refused<br />
<br />
That means the port '''ISN'T ''' being blocked by the ISP, but the server doesn't run SSH on that port (See [http://en.wikipedia.org/wiki/Security_through_obscurity security through obscurity]).<br />
<br />
However, if you get an error message comparable to this:<br />
ssh: connect to host 111.222.333.444 port 22: Operation timed out <br />
<br />
That means that something is rejecting your TCP traffic on port 22. Basically that port is stealth, either by your firewall or 3rd party intervetion (like an ISP blocking and/or rejecting incoming traffic on port 22). If you know you aren't running any firewall on your computer, and you know that Gremlins aren't growing in your routers and switches, then your ISP is blocking the traffic.<br />
<br />
To double check, you can run Wireshark on your server and listen to traffic on port 22. Since Wireshark is a Layer 2 Packet Sniffing utility, and TCP/UDP are Layer 3 and above (See [http://en.wikipedia.org/wiki/TCP/IP_model IP Network stack]), if you don't receive anything while connecting remotely, a third party is most likely to be blocking the traffic on that port to your server.<br />
<br />
===== Diagnosis via Wireshark =====<br />
[[pacman|Install]] Wireshark with the {{Pkg|wireshark-cli}} package, available in the [[official repositories]].<br />
<br />
And then run it using,<br />
tshark -f "tcp port 22" -i NET_IF<br />
<br />
where NET_IF is the network interface for a WAN connection (see {{ic|ip a}} to check). If you aren't receiving any packets while trying to connect remotely, you can be very sure that your ISP is blocking the incoming traffic on port 22.<br />
<br />
===== Possible solution =====<br />
The solution is just to use some other port that the ISP isn't blocking. Open the {{ic|/etc/ssh/sshd_config}} and configure the file to use different ports. For example, add:<br />
<br />
Port 22<br />
Port 1234<br />
<br />
Also make sure that other "Port" configuration lines in the file are commented out. Just commenting "Port 22" and putting "Port 1234" won't solve the issue because then sshd will only listen on port 1234. Use both lines to run the SSH server on both ports. <br />
<br />
Restart the server {{ic|systemctl restart sshd.service}} and you're almost done. You still have to configure your client(s) to use the other port instead of the default port. There are numerous solutions to that problem, but let's cover two of them here.<br />
<br />
==== Read from socket failed: connection reset by peer ====<br />
Recent versions of openssh sometimes fail with the above error message, due to a bug involving elliptic curve cryptography. In that case add the following line to {{ic|~/.ssh/config}}:<br />
<br />
HostKeyAlgorithms ssh-rsa-cert-v01@openssh.com,ssh-dss-cert-v01@openssh.com,ssh-rsa-cert-v00@openssh.com,ssh-dss-cert-v00@openssh.com,ecdsa-sha2-nistp256,ecdsa-sha2-nistp384,ecdsa-sha2-nistp521,ssh-rsa,ssh-dss<br />
<br />
With openssh 5.9, the above fix doesn't work. Instead, put the following lines in {{ic|~/.ssh/config}}:<br />
<br />
Ciphers aes128-ctr,aes192-ctr,aes256-ctr,aes128-cbc,3des-cbc <br />
MACs hmac-md5,hmac-sha1,hmac-ripemd160<br />
<br />
See also the [http://www.gossamer-threads.com/lists/openssh/dev/51339 discussion] on the openssh bug forum.<br />
<br />
=== "[your shell]: No such file or directory" / ssh_exchange_identification problem ===<br />
One possible cause for this is the need of certain SSH clients to find an absolute path (one returned by {{Ic|whereis -b [your shell]}}, for instance) in {{Ic|$SHELL}}, even if the shell's binary is located in one of the {{Ic|$PATH}} entries. Another reason can be that the user is no member of the ''network'' group.<br />
<br />
==="Terminal unknown" or "Error opening terminal" error message===<br />
With ssh it is possible to receive errors like "Terminal unknown" upon logging in. Starting ncurses applications like nano fails with the message "Error opening terminal". There are two methods to this problem, a quick one using the $TERM variable and a profound one using the terminfo file.<br />
<br />
====Workaround by setting the $TERM variable====<br />
After connecting to the remote server set the $TERM variable to "xterm" with the following command.<br />
<br />
{{ic|TERM&#61;xterm}}<br />
<br />
This method is a workaround and should be used on ssh servers you do seldomly connect to, because it can have unwanted side effects. Also you have to repeat the command after every connection, or alternatively set it in ~.bashrc .<br />
====Solution using terminfo file====<br />
A profound solution is transferring the terminfo file of the terminal on your client computer to the ssh server. In this example we cover how to setup the terminfo file for the "rxvt-unicode-256color" terminal.<br />
Create the directory containing the terminfo files on the ssh server, while you are logged in to the server issue this command:<br />
<br />
{{ic| mkdir -p ~/.terminfo/r/}}<br />
<br />
Now copy the terminfo file of your terminal to the new directory. Replace ''"rxvt-unicode-256color"'' with your client's terminal in the following command and ''ssh-server'' with the relevant user and server adress.<br />
<br />
{{ic|$ scp /usr/share/terminfo/r/''rxvt-unicode-256color'' ssh-server:~/.terminfo/r/}}<br />
<br />
After logging in and out from the ssh server the problem should be fixed.<br />
<br />
== See also ==<br />
*[[SSH Keys]]<br />
*[[Pam abl]]<br />
*[[fail2ban]]<br />
*[[sshguard]]<br />
*[[Sshfs]]<br />
*[[Syslog-ng]] : To send ssh log data to another file<br />
<br />
== Links & references ==<br />
*[http://www.soloport.com/iptables.html A Cure for the Common SSH Login Attack]<br />
*[http://www.la-samhna.de/library/brutessh.html Defending against brute force ssh attacks]<br />
*[http://www.ibm.com/developerworks/library/l-keyc/index.html OpenSSH key management, Part 1] and [http://www.ibm.com/developerworks/library/l-keyc2 Part 2] on IBM developerWorks</div>Justin8https://wiki.archlinux.org/index.php?title=Bcache&diff=281527Bcache2013-11-05T13:06:26Z<p>Justin8: /* Configuring */</p>
<hr />
<div>[[Category:File systems]]<br />
==Introduction==<br />
<br />
Bcache allows one to use an SSD as a read/write cache (in writeback mode) or read cache (writethrough or writearound) for another blockdevice (generally a rotating HDD or array). This article will show how to install arch using Bcache as the root partition. For an intro to bcache itself, see [http://bcache.evilpiepirate.org/ the bcache homepage]. Be sure to read and reference [http://atlas.evilpiepirate.org/git/linux-bcache.git/tree/Documentation/bcache.txt?h=bcache the bcache manual]. Bcache is in the mainline kernel since 3.10. The kernel on the arch install disk includes the bcache module since 2013.08.01.<br />
<br />
An alternative to Bcache is Facebook's [[Flashcache]].<br />
<br />
Bcache needs the backing device to be formatted as a bcache block device. In most cases, [https://github.com/g2p/blocks blocks to-bcache] can do an in-place conversion. {{Warning|Be sure you back up any important data first.}}<br />
<br />
== Preparation ==<br />
<br />
* Boot on the install disk (2013.08.01 minimum)<br />
* Install the {{AUR|bcache-tools-git}} package from [[AUR]].<br />
<br />
== Installation ==<br />
<br />
1. Partition your hdd<br />
<br />
grub can't handle bcache, so you'll need at least 2 partitions (boot and one for the bcache backing device). If you're doing UEFI, you'll need an EFI System Partition (ESP) as well.<br />
ex:<br />
1 2048 22527 10.0 MiB EF00 EFI System<br />
2 22528 432127 200.0 MiB 8300 arch_boot<br />
3 432128 625142414 297.9 GiB 8300 bcache_backing<br />
<br />
{{Note|This example has no swapfile/partition. Keep in mind swapfile on btrfs does not work. For a swap partition on the cache, use LVM in step 7. For a swap partition outside the cache, be sure to make a swap partition now.}}<br />
<br />
2. Configure your HDD as a bcache backing device.<br />
<br />
# make-bcache -B /dev/sda3<br />
<br />
{{Note|[[GRUB2]] either needs an EFI partition, "bios boot" partition, or an 'embedding area' of 2KB or so after the MBR that doesn't get overwritten. Also, since GRUB2 does not know about bcache, so you also need a /boot partition that grub can read. Partitioning /dev/sda is a must unless you're booting from a different device.}}<br />
<br />
{{Note| If all associated disks are partitioned at once as below bcache will automatically attach "-B backing stores" to the "-C ssd cache" and step 4 is unnecessary.}}<br />
<br />
# make-bcache -B /dev/sd? /dev/sd? -C /dev/sd?<br />
<br />
3. Register any bcache devices with the kernel (this needs to done every bootup)<br />
<br />
# for i in /dev/sd*; do echo $i; echo $i > /sys/fs/bcache/register_quiet; done<br />
<br />
You now have a /dev/bcache0 device<br />
<br />
{{Note|the bcache user manual says to do "echo /dev/sd* > /sys/fs/bcache/register_quiet", but this didn't work for me}}<br />
<br />
4. Configure your SSD<br />
<br />
Format the SSD as a caching device and link it to the backing device<br />
<br />
# make-bcache -C /dev/sdb<br />
# echo /dev/sdb > /sys/fs/bcache/register <br />
# echo <Set UUID from previous command> > /sys/block/bcache0/bcache/attach<br />
<br />
{{Note| If the UUID is forgotten, it can be found with ''ls /sys/fs/bcache/'' after the cache device has been registered.}}<br />
<br />
5. Format the bcache device. Use LVM or btrfs subvolumes if you want to divide up the /dev/bcache0 device how you like (ex for seperate /, /home, /var, etc).<br />
<br />
# mkfs.btrfs /dev/bcache0<br />
# mount /dev/bcache0 /mnt/<br />
# btrfs subvolume create /mnt/root<br />
# btrfs subvolume create /mnt/home<br />
# umount /mnt<br />
<br />
6. Prepare the installation mount point<br />
<br />
# mkfs.ext4 /dev/sda2<br />
# mkfs.msdos /dev/sda1 (if your ESP is at least 500MB, use mkfs.vfat to make a FAT32 partition instead)<br />
# pacman -S arch-install-scripts<br />
# mount /dev/bcache0 -o subvol=root,compress=lzo /mnt/<br />
# mkdir /mnt/boot<br />
# mkdir /mnt/home<br />
# mount /dev/bcache0 -o subvol=home,compress=lzo /mnt/home<br />
# mount /dev/sda2 /mnt/boot<br />
# mkdir /boot/efi<br />
# mount /dev/sda1 /mnt/boot/efi/<br />
<br />
7a. Install the system<br />
<br />
Do the rest of the installation [[Installation_Guide#Connect_to_the_internet|as normal]] except this:<br />
<br />
Before you edit /etc/mkinitcpio.conf and run run "mkinitcpio -p linux":<br />
<br />
* install "bcache-tools-git" package from [[AUR]].<br />
* in /etc/mkinitcpio.conf:<br />
** add the "bcache" module<br />
** add the "bcache" hook between block and filesystems <br />
<br />
{{Note| The AUR package hook currently uses a for loop similar to the one in step 3 for scanning and assembling bcache arrays at boot. Try step 7b for a udev approach.}}<br />
<br />
7b. Alternatively, udev can be leveraged to assemble bcache arrays as it discovers them. Install the AUR package in step 7a then copy [http://evilpiepirate.org/git/bcache-tools.git/tree/61-bcache.rules 61-bcache.rules] to /usr/lib/udev/rules.d/61-bcache.rules.<br />
<br />
You must then copy the following script adapted from mdadm_udev to /usr/lib/initcpio/install/bcache_udev and edit mkinitcpio.conf to include the "bcache_udev" hook instead of "bcache" before running "mkinitcpio -p linux":<br />
<br />
#!/bin/bash<br />
build() {<br />
add_checked_modules -f 'bcache'<br />
add_binary "/usr/lib/udev/bcache-register"<br />
add_binary "/usr/bin/probe-bcache"<br />
add_file "/usr/lib/udev/rules.d/61-bcache.rules"<br />
}<br />
help() {<br />
cat <<HELPEOF<br />
This hook loads the necessary modules for a bcache ssd cached array and uses<br />
udev at runtime to create the devices. This hook will NOT work without the <br />
udev hook included on the image.<br />
HELPEOF<br />
}<br />
# vim: set ft=sh ts=4 sw=4 et:<br />
<br />
{{Note| It's important to ensure that all files referenced in the bcache_udev hook can be found at the paths stated.}}<br />
<br />
8. Reboot and make sure it works from the /dev/bcache0 device<br />
<br />
== Configuring ==<br />
<br />
There are many options that can be configured (such as cache mode, cache flush interval, sequential write heuristic, etc.) This is currently done by writing to files in /sys. See the [http://atlas.evilpiepirate.org/git/linux-bcache.git/tree/Documentation/bcache.txt bcache user documentation].<br />
<br />
Changing the cache mode is done by echoing one of 'writethrough', 'writeback', 'writearound' or 'none' to /sys/block/bcache[0-9]/bcache/cache_mode.<br />
<br />
== Advanced Operations ==<br />
<br />
=== Resize backing device ===<br />
<br />
It's possible to resize the backing device so long as you don't move the partition start. This process is described on [http://comments.gmane.org/gmane.linux.kernel.bcache.devel/249 the mailing list]. Here's an example using btrfs volume directly on bcache0. For LVM containers or for other filesystems, procedure will differ.<br />
<br />
==== Example of growing ====<br />
<br />
In this example, I grow the filesystem by 4GB. <br />
<br />
1. Reboot to a live CD/USB Drive (need not be bcache enabled) and use fdisk, gdisk, parted, or your other favorite tool to delete the backing partition and recreate it with the same start and a total size 4G larger. <br />
<br />
{{Warning|Do not use a tool like GParted that might perform filesystem operations! It won't recognize the bcache partition and might overwrite part of it!!}}<br />
<br />
2. Reboot to your normal install. Your filesystem will be currently mounted. That's fine. Issue the command to resize the partition to its maximum. For btrfs, that's<br />
<br />
# btrfs filesystem resize max /<br />
<br />
For ext3/4, that's<br />
<br />
# resize2fs /dev/bcache0<br />
<br />
3. That's it!<br />
<br />
==== Example of shrinking ====<br />
<br />
In this example, I shrink the filesystem by 4GB.<br />
<br />
1. Disable writeback cache (switch to writethrough cache) and wait for the disk to flush.<br />
<br />
# echo writethrough > /sys/block/bcache0/bcache/cache_mode<br />
$ watch cat /sys/block/bcache0/bcache/state<br />
<br />
wait until state reports "clean". This might take a while.<br />
<br />
2. Shrink the mounted filesystem by something more than the desired amount, to ensure we don't accidentally clip it later. For btrfs, that's<br />
<br />
# btrfs filesystem resize -5G /<br />
<br />
For ext3/4 you can use resize2fs, but only if the partition is unmounted<br />
<br />
$ df -h /home<br />
/dev/bcache0 290G 20G 270G 1% /home<br />
# umount /home<br />
# resize2fs /dev/bcache0 283G<br />
<br />
3. Reboot to a LiveCD/USB drive (doesn't need to support bcache) and use fdisk, gdisk, parted, or your other favorite tool to delete the backing partition and recreate it with the same start and a total size 4G smaller.<br />
<br />
{{Warning|Do not use a tool like GParted that might perform filesystem operations! It won't recognize the bcache partition and might overwrite part of it!!}}<br />
<br />
4. Reboot to your normal install. Your filesystem will be currently mounted. That's fine. Issue the command to resize the partition to its maximum (that is, the size we shrunk the actual partition to in step 3). For btrfs, that's<br />
<br />
# btrfs filesystem resize max /<br />
<br />
For ext3/4, that's<br />
<br />
# resize2fs /dev/bcache0<br />
<br />
5. Re-enable writeback cache if you want that enabled:<br />
<br />
# echo writeback > /sys/block/bcache0/bcache/cache_mode<br />
<br />
6. That's it!<br />
<br />
{{Note|If you're very careful you can shrink the filesystem to the exact size in step 2 and avoid step 4. Be careful, though, many partition tools don't do exactly what you want, but instead adjust the requested partition start/end points to end on sector boundaries. This may be difficult to calculate ahead of time}}<br />
<br />
== Troubleshooting ==<br />
<br />
=== /dev/bcache device doesn't exist on bootup ===<br />
<br />
If you are sent to a busy box shell with an error:<br />
<br />
ERROR: Unable to find root device 'UUID=b6b2d82b-f87e-44d5-bbc5-c51dd7aace15'.<br />
You are being dropped to a recovery shell<br />
Type 'exit' to try and continue booting<br />
<br />
This might happen if the backing device is configured for "writeback" mode (default is writearound). When in "writeback" mode, the /dev/bcache0 device is not started until the cache device is both registered and attached. Registering is something that needs to happen every bootup, but attaching should only have to be done once. I've sometime had to re-attach.<br />
<br />
To continue booting, try one of the following:<br />
<br />
* Register both the backing device and the cacheing device<br />
<br />
# echo /dev/sda3 > /sys/fs/bcache/register<br />
# echo /dev/sdb > /sys/fs/bcache/register<br />
<br />
If the /dev/bcache0 device now exists, type exit and continue booting. You will need to fix your initcpio to ensure devices are registered before mounting the root device.<br />
<br />
{{Note|An error of "sh: echo: write error: Invalid argument" means the device was already registered or is not recognized as either a bcache backing device or cache.}}<br />
<br />
{{Note|This can also happen if using udev's 61-bcache.rules in Installation's step 7b and blkid and bcache-probe "disagree" due to rogue superblocks. See [http://bcache.evilpiepirate.org/#index6h1 bcache's wiki] for a possible explanation/resolution.}}<br />
<br />
* Re-attach the cache to the backing device:<br />
<br />
If the cache device was registered, a folder with the UUID of the cache should exist in /sys/fs/bcache. Use that UUID when following the example below:<br />
<br />
# ls /sys/fs/bcache/<br />
b6b2d82b-f87e-44d5-bbc5-c51dd7aace15 register register_quiet<br />
# echo b6b2d82b-f87e-44d5-bbc5-c51dd7aace15 > /sys/block/sda/sda3/bcache/attach<br />
<br />
If the /dev/bcache0 device now exists, type exit and continue booting. You should not have to do this again. If it persists, ask on the bcache mailing list.<br />
<br />
{{Note|An error of "sh: echo: write error: Invalid argument" means the device was already attached. An error of "sh: echo: write error: No such file or directory" means the UUID is not a valid cache (make sure you typed it correctly)}}<br />
<br />
* Invalidate the cache and force the backing device to run without it. You might want to check some stats, such as "dirty_data" so you have some idea of how much data will be lost.<br />
<br />
# cat /sys/block/sda/sda3/bcache/dirty_data<br />
-3.9M<br />
<br />
dirty data is data in the cache that has not been written to the backing device. If you force the backing device to run, this data will be lost, even if you later re-attach the cache.<br />
<br />
# cat /sys/block/sda/sda3/bcache/running<br />
0<br />
# echo 1 > /sys/block/sda/sda3/bcache/running<br />
<br />
The /dev/bcache0 device will now exist. Type exit and continue booting. You might want to unregister the cache device and run make-bcache again. An fsck on /dev/bcache0 would also be wise. See the [http://atlas.evilpiepirate.org/git/linux-bcache.git/tree/Documentation/bcache.txt?h=bcache bcache user documentation].<br />
<br />
{{Warning| Only invalidate the cache if one of the two options above did not work.}}<br />
<br />
=== /sys/fs/bcache/ doesn't exist ===<br />
<br />
The kernel you booted is not bcache enabled.</div>Justin8https://wiki.archlinux.org/index.php?title=I3&diff=267863I32013-07-25T01:44:45Z<p>Justin8: </p>
<hr />
<div>{{DISPLAYTITLE:i3}}<br />
[[Category:Tiling WMs]]<br />
[[Category:Dynamic WMs]]<br />
[[ko:i3]]<br />
[[ru:i3]]<br />
[[zh-CN:i3]]<br />
[http://i3wm.org/ i3] is a dynamic [[Wikipedia:Tiling window manager|tiling window manager]] inspired by [[wmii]] that is primarily targeted at developers and advanced users.<br />
<br />
Clients (windows) are organized in a tree data structure within containers. The tree branches via horizontal or vertical splits, and containers can also be set to tabbed or stacked layouts. Floating windows are available for corner cases that don't mix well with tiling, and remain on a separate layer above the tiled windows.<br />
<br />
== Installation ==<br />
Install the {{Pkg|i3}} [[Pacman#Installing package groups|package group]] from the [[official repositories]], which includes: {{Pkg|i3-wm}}, the window manager; {{Pkg|i3status}}, a package to write a status line to i3bar through [[Wikipedia:Standard streams#Standard output (stdout)|stdout]]; and {{Pkg|i3lock}}, an improved screenlocker.<br />
<br />
Additional packages are available in the [[Arch User Repository]]. Install {{AUR|i3-git}} for the development version. A GNOME session is included in the current packages.<br />
<br />
== Configuration ==<br />
<br />
Edit your {{ic|~/.xinitrc}} and add:<br />
exec i3<br />
If you want i3 to log its output (useful for debugging), add this line to {{ic|~/.xinitrc}}:<br />
exec i3 -V >> ~/.i3/i3log 2>&1<br />
If you use the Nvidia binary driver '''<302.17''' you need to add the --force-xinerama flag to {{ic|~/.xinitrc}}. A detailed explanation can be found at [http://i3wm.org/docs/multi-monitor.html i3wm.org].<br />
exec i3 --force-xinerama<br />
<br />
=== Status bar ===<br />
The internal status bar, i3-''ws''bar, was deprecated and replaced by i3bar in i3 v4.0.<br />
<br />
==== New method: i3bar ====<br />
Unlike i3-wsbar, which requires dzen2, i3bar does not have any dependencies other than {{Pkg|i3-wm}}. It can be used to view information generated by [[conky]] or i3status. For example (as of version 4.1):<br />
{{hc|~/.i3/config|<nowiki><br />
bar {<br />
output LVDS1<br />
status_command i3status<br />
position top<br />
mode hide<br />
workspace_buttons yes<br />
tray_output none<br />
<br />
font -misc-fixed-medium-r-normal--13-120-75-75-C-70-iso10646-1<br />
<br />
colors {<br />
background #000000<br />
statusline #ffffff<br />
<br />
focused_workspace #ffffff #285577<br />
active_workspace #ffffff #333333<br />
inactive_workspace #888888 #222222<br />
urgent_workspace #ffffff #900000<br />
}<br />
}<br />
</nowiki>}}<br />
<br />
For further information see the [http://i3wm.org/docs/userguide.html#_configuring_i3bar Configuring i3bar] section of the official User Guide.<br />
<br />
==== Comparison of i3bar and dzen2 ====<br />
This comparison of i3bar and dzen2 only takes into account how well the two programs can handle the input from conky or i3status.<br />
{| border="1" cellpadding="5" align="center"<br />
! Program !! Color Codes !! Formatting !! Special Fonts !! Dock !! Trayer<br />
|-<br />
| i3bar || Yes || No, right aligned || No (UTF8 only)|| Yes || Yes<br />
|-<br />
| dzen2 || Yes || No, left aligned || Yes || Yes (the svn version) || No<br />
|-<br />
|}<br />
<br />
Though development of i3bar is very active and support for custom formatting and fonts has been announced, dzen2-svn has an edge over i3bar (as of August 7th).<br />
<br />
==== Alternatives ====<br />
* [https://github.com/enkore/i3pystatus i3pystatus] - extensible i3status replacement with many modules and very flexible configuration. Multi-threaded and lock-up free.<br />
* [https://github.com/ultrabug/py3status py3status] – an extensible i3status wrapper written in python<br />
<br />
=== Quickly jumping to open window ===<br />
* [https://github.com/proxypoke/quickswitch-for-i3 quickswitch-for-i3] – A Python utility to quickly change to and locate windows in i3<br />
* [https://github.com/yiuin/i3-wm-scripts i3-wm-scripts] – search for and jump to windows with particular names matching regexp<br />
* [https://github.com/ziberna/i3-py/tree/master/examples#winmenupy winmenupy] launches dmenu with a list of clients, sorted after workspaces. Selecting a client jumps to that window.<br />
<br />
== Suspending with i3lock ==<br />
You need to add unit file below and enable it with {{ic|# systemctl enable suspend@<user>.service}}.<br />
<br />
{{hc|/etc/systemd/system/suspend@.service|2=<nowiki><br />
[Unit]<br />
Description=Starts i3lock at suspend time<br />
Before=sleep.target<br />
<br />
[Service]<br />
User=%I<br />
Type=forking<br />
Environment=DISPLAY=:0<br />
ExecStartPre= <br />
ExecStart=/usr/bin/i3lock<br />
<br />
[Install]<br />
WantedBy=sleep.target</nowiki>}}<br />
<br />
== Usage ==<br />
{{Box BLUE||See the [http://i3wm.org/docs official documentation] on this subject for more information: [http://i3wm.org/docs/userguide.html i3 User’s Guide]}}<br />
<br />
i3 currently uses [[dmenu]] as a application launcher, which is bound by default to {{Keypress|Mod1|background=#FF0}}+{{Keypress|d}}.<br />
<br />
=== Clipboard (copy & paste issues) ===<br />
<br />
By default, when you close a window, the buffer with the clipboard info will disappear. You have to use a clipboard manager like {{Pkg|clipit}} to avoid that.<br />
<br />
== See also ==<br />
* [[Comparison of Tiling Window Managers]]<br />
* [http://i3wm.org Official website]<br />
* [http://code.stapelberg.de/git/i3 Source code]<br />
* [https://wiki.archlinux.org/index.php/Systemd#Suspend.2Fresume_service_files Suspend/resume service files]<br />
* [https://github.com/ashinkarov/i3-extras Collection of scripts and patches]<br />
'''Arch Linux Forums'''<br />
* [https://bbs.archlinux.org/viewtopic.php?id=99064 ''The i3 thread''] - A general discussion about i3<br />
* [https://bbs.archlinux.org/viewtopic.php?pid=1229978 ''i3 desktop screenshots and config sharing'']</div>Justin8