Difference between revisions of "Btrfs"

From ArchWiki
Jump to navigation Jump to search
(→‎Defragmentation: Switched examples for clarity (more often used command first))
(→‎Preparation: remove outdated mention of the base group)
 
(212 intermediate revisions by 45 users not shown)
Line 1: Line 1:
 
[[Category:File systems]]
 
[[Category:File systems]]
 +
[[fr:Btrfs]]
 
[[ja:Btrfs]]
 
[[ja:Btrfs]]
[[zh-CN:Btrfs]]
+
[[ru:Btrfs]]
 +
[[zh-hans:Btrfs]]
 
{{Related articles start}}
 
{{Related articles start}}
 
{{Related|File systems}}
 
{{Related|File systems}}
{{Related|Mkinitcpio-btrfs}}
 
 
{{Related|Snapper}}
 
{{Related|Snapper}}
 
{{Related|dm-crypt/Encrypting an entire system#Btrfs subvolumes with swap}}
 
{{Related|dm-crypt/Encrypting an entire system#Btrfs subvolumes with swap}}
 
{{Related articles end}}
 
{{Related articles end}}
 
From [[Wikipedia:Btrfs]]:
 
 
:Btrfs (B-tree file system, pronounced as "butter F S", "better F S", "b-tree F S", or simply by spelling it out) is a file system based on the copy-on-write (COW) principle, initially designed at Oracle Corporation for use in Linux. The development of Btrfs began in 2007, and by August 2014, the file system's on-disk format has been marked as stable.
 
  
 
From [https://btrfs.wiki.kernel.org/index.php/Main_Page Btrfs Wiki]:
 
From [https://btrfs.wiki.kernel.org/index.php/Main_Page Btrfs Wiki]:
  
:Btrfs is a new copy on write (CoW) filesystem for Linux aimed at implementing advanced features while focusing on fault tolerance, repair and easy administration. Jointly developed at Oracle, Red Hat, Fujitsu, Intel, SUSE, STRATO and many others, Btrfs is licensed under the GPL and open for contribution from anyone.
+
:Btrfs is a modern copy on write (CoW) filesystem for Linux aimed at implementing advanced features while also focusing on fault tolerance, repair and easy administration. Jointly developed at multiple companies, Btrfs is licensed under the GPL and open for contribution from anyone.
  
{{Warning| Btrfs has some features that are considered experimental. See the Btrfs Wiki's [https://btrfs.wiki.kernel.org/index.php/Main_Page#Stability_status Stability status], [https://btrfs.wiki.kernel.org/index.php/FAQ#Is_btrfs_stable.3F Is Btrfs stable?] and [https://btrfs.wiki.kernel.org/index.php/Getting_started Getting started] for more detailed information. See the [[#Known issues]] section.
+
{{Warning|Btrfs has some features that are unstable. See the Btrfs Wiki's [https://btrfs.wiki.kernel.org/index.php/Status Status], [https://btrfs.wiki.kernel.org/index.php/FAQ#Is_btrfs_stable.3F Is Btrfs stable?] and [https://btrfs.wiki.kernel.org/index.php/Getting_started Getting started] for more detailed information. See the [[#Known issues]] section.}}  
}}
 
  
 
== Preparation ==
 
== Preparation ==
  
The official kernels {{Pkg|linux}} and {{Pkg|linux-lts}} include support for Btrfs. If you want to boot from a Btrfs file system, check if your [[boot loader]] supports Btrfs.
+
For user space utilities [[install]] the {{Pkg|btrfs-progs}} package, which is required for basic operations.
  
User space utilities are available by [[installing]] the {{Pkg|btrfs-progs}} package.
+
If you need to boot from a Btrfs file system (i.e., your kernel and initramfs reside on a Btrfs partition), check if your [[boot loader]] supports Btrfs.
 
 
== Partitionless Btrfs disk ==
 
 
 
Btrfs can occupy an entire data storage device, replacing the [[MBR]] or [[GPT]] partitioning schemes, using [[#Subvolumes|subvolumes]] to simulate partitions. However, using a partitionless setup is not required to simply [[#Creating a new file system|create a Btrfs filesystem]] on an existing [[partition]] that was created using another method. There are some limitations to partitionless single disk setups:
 
 
 
* Cannot use different [[file systems]] for different [[fstab|mount points]].
 
* Cannot use [[Swap|swap area]] as Btrfs does not support [[Swap#Swap_file|swap files]] and there is no place to create [[Swap#Swap_partition|swap partition]]. This also limits the use of hibernation/resume, which needs a swap area to store the hibernation image.
 
* Cannot use [[UEFI]] to boot.
 
 
 
To overwrite the existing partition table with Btrfs, run the following command:
 
 
 
# mkfs.btrfs /dev/sd''X''
 
 
 
For example, use {{ic|/dev/sda}} rather than {{ic|/dev/sda1}}. The latter would format an existing partition instead of replacing the entire partitioning scheme.
 
 
 
Install the [[boot loader]] like you would for a data storage device with a [[Master Boot Record]]. See [[Syslinux#Manual install]] or  [[GRUB#Install to partition or partitionless disk]].
 
 
 
{{Warning|GRUB strongly discourages installation to a partitionless disk.}}
 
  
 
== File system creation ==
 
== File system creation ==
  
A Btrfs file system can either be newly created or have one converted.
+
The following shows how to create a new Btrfs [[file system]]. To convert an ext3/4 partition to Btrfs, see [[#Ext3/4 to Btrfs conversion]]. To use a partitionless setup, see [[#Partitionless Btrfs disk]].
  
=== Creating a new file system ===
+
See {{man|8|mkfs.btrfs}} for more information.
  
==== File system on a single device ====
+
=== File system on a single device ===
  
To format a partition do:
+
To create a Btrfs filesystem on partition {{ic|/dev/''partition''}}:
  
 
  # mkfs.btrfs -L ''mylabel'' /dev/''partition''
 
  # mkfs.btrfs -L ''mylabel'' /dev/''partition''
Line 60: Line 38:
 
  # mkfs.btrfs -L ''mylabel'' -n 16k /dev/''partition''
 
  # mkfs.btrfs -L ''mylabel'' -n 16k /dev/''partition''
  
==== Multi-device file system ====
+
=== Multi-device file system ===
  
{{Warning|
+
{{Warning|The RAID 5 and RAID 6 modes of Btrfs are fatally flawed, and should not be used for "anything but testing with throw-away data." See [https://btrfs.wiki.kernel.org/index.php/RAID56 the Btrfs page on RAID5 and RAID6] for status updates.}}
* As of August 2016, the RAID 5, RAID 6 mode of Btrfs is considered ''fatally flawed'', and shouldn't be used for "anything but testing with throw-away data." [https://www.mail-archive.com/linux-btrfs@vger.kernel.org/msg55161.html]
 
* Some [[boot loader]]s such as [[Syslinux]] do not support multi-device file systems.}}
 
  
Multiple devices can be entered to create a RAID. Supported RAID levels include RAID 0, RAID 1, RAID 10, RAID 5 and RAID 6. The RAID levels can be configured separately for data and metadata using the {{ic|-d}} and {{ic|-m}} options respectively. By default the data is striped ({{ic|raid0}}) and the metadata is mirrored ({{ic|raid1}}). See [https://btrfs.wiki.kernel.org/index.php/Using_Btrfs_with_Multiple_Devices Using Btrfs with Multiple Devices] for more information about how to create a Btrfs RAID volume as well as the manpage for {{ic|mkfs.btrfs}}.
+
Multiple devices can be used to create a RAID. Supported RAID levels include RAID 0, RAID 1, RAID 10, RAID 5 and RAID 6. The RAID levels can be configured separately for data and metadata using the {{ic|-d}} and {{ic|-m}} options respectively. By default the data is striped ({{ic|raid0}}) and the metadata is mirrored ({{ic|raid1}}). See [https://btrfs.wiki.kernel.org/index.php/Using_Btrfs_with_Multiple_Devices Using Btrfs with Multiple Devices] for more information about how to create a Btrfs RAID volume.
  
 
  # mkfs.btrfs -d raid0 -m raid1 /dev/''part1'' /dev/''part2'' ...
 
  # mkfs.btrfs -d raid0 -m raid1 /dev/''part1'' /dev/''part2'' ...
  
You '''must''' include either the {{ic|udev}} hook or the {{ic|btrfs}} hook in {{ic|/etc/mkinitcpio.conf}} in order to use multiple btrfs devices in a pool. See the [[Mkinitcpio#Common hooks]] article for more information.
+
You can create a [[W:JBOD|JBOD configuration]], where disks are seen as one filesystem, but files are not duplicated, using {{ic|-d single -m raid1}}.
 +
 
 +
You must include either the {{ic|udev}} hook or the {{ic|btrfs}} hook in {{ic|/etc/mkinitcpio.conf}} in order to use multiple Btrfs devices in a pool. See the [[Mkinitcpio#Common hooks]] article for more information.
  
{{Note|Mounting such a filesystem may result in all but one of the according ''.device''-jobs getting stuck and systemd never finishing startup due to a [https://github.com/systemd/systemd/issues/1921 bug] in handling this type of filesystem.}}
+
{{Note|
 +
* It is possible to add devices to a multiple-device filesystem later on. See the [https://btrfs.wiki.kernel.org/index.php/Using_Btrfs_with_Multiple_Devices Btrfs wiki article] for more information.
 +
* Devices can be of different sizes. However, if one drive in a RAID configuration is bigger than the others, this extra space will not be used.
 +
* Some [[boot loader]]s such as [[Syslinux]] do not support multi-device file systems.
 +
}}
  
 
See [[#RAID]] for advice on maintenance specific to multi-device Btrfs file systems.
 
See [[#RAID]] for advice on maintenance specific to multi-device Btrfs file systems.
 
=== Ext3/4 to Btrfs conversion ===
 
 
{{Warning| As of mid-to-late 2015, there are many reports on the btrfs mailing list about incomplete/corrupt/broken conversions. The situation is improving as patches are being submitted, but proceed very carefully. Make sure you have ''working'' backups of any data you cannot afford to lose. See [https://btrfs.wiki.kernel.org/index.php/Conversion_from_Ext3 Conversion from Ext3] on the btrfs wiki.}}
 
 
Boot from an install CD, then convert by doing:
 
 
# btrfs-convert /dev/''partition''
 
 
Mount the partion and test the conversion by checking the files.  Be sure to change the {{ic|/etc/fstab}} to reflect the change ('''type''' to {{ic|btrfs}} and '''fs_passno''' [the last field] to {{ic|0}} as Btrfs does not do a file system check on boot). Also note that the UUID of the partition will have changed, so update fstab accordingly when using UUIDs. {{ic|chroot}} into the system and rebuild the GRUB menu list (see [[Install from existing Linux]] and [[GRUB]] articles). If converting a root filesystem, while still chrooted run {{ic|mkinitcpio -p linux}} to regenerate the initramfs or the system will not successfully boot. If you get stuck in grub with 'unknown filesystem' try reinstalling grub with {{ic|grub-install /dev/''partition''}} and regenerate the config as well {{ic|grub-mkconfig -o /boot/grub/grub.cfg}}.
 
 
After confirming that there are no problems, complete the conversion by deleting the backup {{ic|ext2_saved}} sub-volume. Note that you cannot revert back to ext3/4 without it.
 
 
# btrfs subvolume delete /ext2_saved
 
 
Finally [[#Balance|balance]] the file system to reclaim the space.
 
  
 
== Configuring the file system ==
 
== Configuring the file system ==
  
=== Copy-On-Write (CoW) ===
+
=== Copy-on-Write (CoW) ===
  
By default, Btrfs uses [[Wikipedia:copy-on-write]] for all files all the time. See [https://btrfs.wiki.kernel.org/index.php/SysadminGuide#Copy_on_Write_.28CoW.29 the Btrfs Sysadmin Guide section] for implementation details, as well as advantages and disadvantages.
+
By default, Btrfs uses [[Wikipedia:copy-on-write|copy-on-write]] for all files all the time. See the [https://btrfs.wiki.kernel.org/index.php/SysadminGuide#Copy_on_Write_.28CoW.29 Btrfs Sysadmin Guide section] for implementation details, as well as advantages and disadvantages.
  
 
==== Disabling CoW ====
 
==== Disabling CoW ====
  
To disable copy-on-write for newly created files in a mounted subvolume, use the {{ic|nodatacow}} mount option. This will only affect newly created files. Copy-on-write will still happen for existing files.
+
To disable copy-on-write for newly created files in a mounted subvolume, use the {{ic|nodatacow}} mount option. This will only affect newly created files. Copy-on-write will still happen for existing files. The {{ic|nodatacow}} option also disables compression. See {{man|5|btrfs}} for details.
 +
 
 +
{{Note|From {{man|5|btrfs|MOUNT OPTIONS}}: "within a single file system, it is not possible to mount some subvolumes with {{ic|nodatacow}} and others with {{ic|datacow}}. The mount option of the first mounted subvolume applies to any other subvolumes."}}
  
 
To disable copy-on-write for single files/directories do:
 
To disable copy-on-write for single files/directories do:
Line 111: Line 79:
  
 
{{Tip|In accordance with the note above, you can use the following trick to disable copy-on-write on existing files in a directory:
 
{{Tip|In accordance with the note above, you can use the following trick to disable copy-on-write on existing files in a directory:
 +
 
  $ mv ''/path/to/dir'' ''/path/to/dir''_old
 
  $ mv ''/path/to/dir'' ''/path/to/dir''_old
 
  $ mkdir ''/path/to/dir''
 
  $ mkdir ''/path/to/dir''
Line 116: Line 85:
 
  $ cp -a ''/path/to/dir''_old/* ''/path/to/dir''
 
  $ cp -a ''/path/to/dir''_old/* ''/path/to/dir''
 
  $ rm -rf ''/path/to/dir''_old
 
  $ rm -rf ''/path/to/dir''_old
 +
 
Make sure that the data are not used during this process. Also note that {{ic|mv}} or {{ic|cp --reflink}} as described below will not work.
 
Make sure that the data are not used during this process. Also note that {{ic|mv}} or {{ic|cp --reflink}} as described below will not work.
 
}}
 
}}
  
==== Forcing CoW ====
+
==== Creating lightweight copies ====
  
To force copy-on-write when copying files use:
+
By default, when copying files on a Btrfs filesystem with {{ic|cp}}, actual copies are created. To create a lightweight copy referencing to the original data, use the ''reflink'' option:
  
 
  $ cp --reflink ''source'' ''dest''  
 
  $ cp --reflink ''source'' ''dest''  
  
This would only be required if CoW was disabled for the file to be copied (as implemented above). See the man page on {{ic|cp}} for more details on the {{ic|--reflink}} flag.
+
See the man page on {{man|1|cp}} for more details on the {{ic|--reflink}} flag.
  
 
=== Compression ===
 
=== Compression ===
  
Btrfs supports transparent compression, meaning every file on the partition is automatically compressed. This not only reduces the size of files, but also [http://www.phoronix.com/scan.php?page=article&item=btrfs_compress_2635&num=1 improves performance], in particular if using the [http://www.phoronix.com/scan.php?page=article&item=btrfs_lzo_2638&num=1 lzo algorithm], in some specific use cases (e.g. single thread with heavy file IO), while obviously harming performance on other cases (e.g. multithreaded and/or cpu intensive tasks with large file IO).
+
Btrfs supports transparent and automatic compression. This not only reduces the size of files, but can also [http://www.phoronix.com/scan.php?page=article&item=btrfs_compress_2635&num=1 improve performance], in some specific use cases (e.g. single thread with heavy file I/O), while obviously harming performance in other cases (e.g. multithreaded and/or cpu intensive tasks with large file I/O). Better performance is generally achieved with the fastest compress algorithms, ''zstd'' and ''lzo'', and some [https://www.phoronix.com/scan.php?page=article&item=btrfs-zstd-compress benchmarks] provide detailed comparisons.
  
Compression is enabled using the {{ic|1=compress=zlib}} or {{ic|1=compress=lzo}} mount options. Only files created or modified after the mount option is added will be compressed. However, it can be applied quite easily to existing files (e.g. after a conversion from ext3/4) using the {{ic|btrfs filesystem defragment -c''alg''}} command, where {{ic|''alg''}} is either {{ic|zlib}} or {{ic|lzo}}. In order to re-compress the whole file system with {{pkg|lzo}}, run the following command:
+
The {{ic|1=compress=''alg''}} mount option enables automatically considering every file for compression, where {{ic|''alg''}} is either {{ic|zlib}}, {{ic|lzo}}, {{ic|zstd}}, or {{ic|no}} (for no compression). Using this option, btrfs will check if compressing the first portion of the data shrinks it.  If it does, the entire write to that file will be compressed.  If it does not, none of it is compressed.  With this option, if the first portion of the write does not shrink, no compression will be applied to the write even if the rest of the data would shrink tremendously. [https://btrfs.wiki.kernel.org/index.php/Compression#What_happens_to_incompressible_files.3F]  This is done to prevent making the disk wait to start writing until all of the data to be written is fully given to btrfs and compressed.
  
  # btrfs filesystem defragment -r -v -clzo /
+
The {{ic|1=compress-force=''alg''}} mount option can be used instead, which makes btrfs skip checking if compression shrinks the first portion, and enables automatic compression for every file. In a worst case scenario, this can cause (slightly) more space to be used, and CPU usage for no purpose.
  
{{Tip|Compression can also be enabled per-file without using the {{ic|compress}} mount option; simply apply {{ic|chattr +c}} to the file. When applied to directories, it will cause new files to be automatically compressed as they come.}}
+
Only files created or modified after the mount option is added will be compressed.
  
When installing Arch to an empty Btrfs partition, use the {{ic|compress}} option when [[Beginners' guide#Format the file systems and enable swap|mounting the filesystem]]{{Broken section link}}: {{ic|1=mount -o compress=lzo /dev/sd''xY'' /mnt/}}. During [[Beginners' guide#Configuration|configuration]], add {{ic|1=compress=lzo}} to the mount options of the root file system in [[fstab]].
+
To apply compression to existing files, use the {{ic|btrfs filesystem defragment -c''alg''}} command, where {{ic|''alg''}} is either {{ic|zlib}}, {{ic|lzo}} or {{ic|zstd}}. For example, in order to re-compress the whole file system with {{pkg|zstd}}, run the following command:
 +
 
 +
# btrfs filesystem defragment -r -v -czstd /
 +
 
 +
To enable compression when installing Arch to an empty Btrfs partition, use the {{ic|compress}} option when [[mounting]] the file system: {{ic|1=mount -o compress=zstd /dev/sd''xY'' /mnt/}}. During configuration, add {{ic|1=compress=zstd}} to the mount options of the root file system in [[fstab]].
 +
 
 +
{{Tip|Compression can also be enabled per-file without using the {{ic|compress}} mount option; to do so apply {{ic|chattr +c}} to the file. When applied to directories, it will cause new files to be automatically compressed as they come.}}
 +
 
 +
{{Warning|
 +
* Systems using older kernels or {{pkg|btrfs-progs}} without {{ic|zstd}} support may be unable to read or repair your filesystem if you use this option.
 +
* [[GRUB]] introduced ''zstd'' support in 2.04.  Make sure you have actually upgraded the bootloader installed in your MBR/ESP since then, by running {{ic|grub-install}} with the appropriate options for your BIOS/UEFI setup, since that is not done automatically. See {{Bug|63235}}.
 +
* [[rEFInd]] before version 0.11.4 lacks support for ''zstd'', either switch to {{AUR|refind-efi-git}}, use a separate boot partition without ''zstd'', or reset compression of boot files to something supported using for example the command: {{bc|$ btrfs filesystem defragment -v -clzo /boot/*}}
 +
}}
 +
 
 +
==== View compression types and ratios ====
 +
 
 +
{{pkg|compsize}} takes a list of files (or an entire btrfs filesystem) and measures compression types used and effective compression ratios.  Uncompressed size may not match the number given by other programs such as {{ic|du}}, because every extent is counted once, even if it is reflinked several times, and even if part of it is no longer used anywhere but has not been garbage collected.  The {{ic|-x}} option keeps it on a single filesystem, which is useful in situations like {{ic|compsize -x /}} to avoid it from attempting to look in non-btrfs subdirectories and fail the entire run.
  
 
=== Subvolumes ===
 
=== Subvolumes ===
Line 143: Line 129:
 
"A btrfs subvolume is not a block device (and cannot be treated as one) instead, a btrfs subvolume can be thought of as a POSIX file namespace. This namespace can be accessed via the top-level subvolume of the filesystem, or it can be mounted in its own right." [https://btrfs.wiki.kernel.org/index.php/SysadminGuide#Subvolumes]
 
"A btrfs subvolume is not a block device (and cannot be treated as one) instead, a btrfs subvolume can be thought of as a POSIX file namespace. This namespace can be accessed via the top-level subvolume of the filesystem, or it can be mounted in its own right." [https://btrfs.wiki.kernel.org/index.php/SysadminGuide#Subvolumes]
  
Each Btrfs file system has a top-level subvolume with ID 5. It can be mounted as {{ic|/}} (by default), or another subvolume can be [[#Mounting subvolumes|mounted]] instead.
+
Each Btrfs file system has a top-level subvolume with ID 5. It can be mounted as {{ic|/}} (by default), or another subvolume can be [[#Mounting subvolumes|mounted]] instead. Subvolumes can be moved around in the filesystem and are rather identified by their id than their path.
  
 
See the following links for more details:
 
See the following links for more details:
Line 172: Line 158:
 
==== Mounting subvolumes ====
 
==== Mounting subvolumes ====
  
Subvolumes can be mounted like file system partitions using the {{ic|1=subvol=''/path/to/subvolume''}} or {{ic|1=subvolid=''objectid''}} mount flags. For example, you could have a subvolume named {{ic|subvol_root}} and mount it as {{ic|/}}.  One can mimic traditional file system partitions by creating various subvolumes under the top level of the file system and then mounting them at the appropriate mount points. Thus one can easily restore a file system (or part of it) to a previous state easily using [[#Snapshots]].
+
Subvolumes can be mounted like file system partitions using the {{ic|1=subvol=''/path/to/subvolume''}} or {{ic|1=subvolid=''objectid''}} mount flags. For example, you could have a subvolume named {{ic|subvol_root}} and mount it as {{ic|/}}.  One can mimic traditional file system partitions by creating various subvolumes under the top level of the file system and then mounting them at the appropriate mount points. Thus one can easily restore a file system (or part of it) to a previous state using [[#Snapshots]].
  
 
{{Tip|1= Changing subvolume layouts is made simpler by not using the toplevel subvolume (ID=5) as {{ic|/}} (which is done by default). Instead, consider creating a subvolume to house your actual data and mounting it as {{ic|/}}.}}
 
{{Tip|1= Changing subvolume layouts is made simpler by not using the toplevel subvolume (ID=5) as {{ic|/}} (which is done by default). Instead, consider creating a subvolume to house your actual data and mounting it as {{ic|/}}.}}
 +
 +
{{Note|From {{man|5|btrfs|MOUNT OPTIONS}}: "Most mount options apply to the '''whole filesystem''', and only the options for the first subvolume to be mounted will take effect. This is due to lack of implementation and may change in the future.". See the [https://btrfs.wiki.kernel.org/index.php/FAQ#Can_I_mount_subvolumes_with_different_mount_options.3F Btrfs Wiki FAQ] for which mount options can be used per subvolume.}}
  
 
See [[Snapper#Suggested filesystem layout]], [https://btrfs.wiki.kernel.org/index.php/SysadminGuide#Managing_Snapshots Btrfs SysadminGuide#Managing Snapshots], and [https://btrfs.wiki.kernel.org/index.php/SysadminGuide#Layout Btrfs SysadminGuide#Layout] for example file system layouts using subvolumes.
 
See [[Snapper#Suggested filesystem layout]], [https://btrfs.wiki.kernel.org/index.php/SysadminGuide#Managing_Snapshots Btrfs SysadminGuide#Managing Snapshots], and [https://btrfs.wiki.kernel.org/index.php/SysadminGuide#Layout Btrfs SysadminGuide#Layout] for example file system layouts using subvolumes.
  
===== Mount options =====
+
See {{man|5|btrfs}} for a full list of btrfs-specific mount options.
 
 
When mounting subvolumes with {{ic|1=subvol=}} several mount options are available. For example, mount options that affect [[#Compression]] or [[#Copy-On-Write (CoW)]] can be used.
 
 
 
See [https://btrfs.wiki.kernel.org/index.php/Mount_options Btrfs Wiki Mount options] and [https://btrfs.wiki.kernel.org/index.php/Gotchas Btrfs Wiki Gotchas] for more information. In addition to configurations that can be made during or after file system creation, the various mount options for Btrfs can drastically change its performance characteristics. As this is a file system that is still in active development, changes and regressions should be expected. See links in the [[#See also]] section for some benchmarks.
 
 
 
{{Warning|Specific mount options can disable safety features and increase the risk of complete file system corruption.}}
 
  
 
==== Changing the default sub-volume ====
 
==== Changing the default sub-volume ====
Line 196: Line 178:
 
{{Note|1=After changing the default subvolume on a system with [[GRUB]], you should run {{ic|grub-install}} again to notify the bootloader of the changes. See [https://bbs.archlinux.org/viewtopic.php?pid=1615373 this forum thread].}}
 
{{Note|1=After changing the default subvolume on a system with [[GRUB]], you should run {{ic|grub-install}} again to notify the bootloader of the changes. See [https://bbs.archlinux.org/viewtopic.php?pid=1615373 this forum thread].}}
  
{{Warning|Changing the default subvolume with {{ic|btrfs subvolume set-default}} will make the top level of the filesystem inaccessible when the default subvolume is mounted . Reference: [https://btrfs.wiki.kernel.org/index.php/SysadminGuide Btrfs Wiki Sysadmin Guide].}}
+
Changing the default subvolume with {{ic|btrfs subvolume set-default}} will make the top level of the filesystem inaccessible, except by use of the {{ic|1=subvol=/}} or {{ic|1=subvolid=5}} mount options [https://btrfs.wiki.kernel.org/index.php/SysadminGuide].
 +
 
 +
=== Quota ===
 +
 
 +
{{warning|Qgroup is not stable yet and combining quota with (too many) snapshots of subvolumes can cause performance problems, for example when deleting snapshots. Plus there are several more [https://btrfs.wiki.kernel.org/index.php/Quota_support#Known_issues known issues].}}
 +
 
 +
Quota support in Btrfs is implemented at a subvolume level by the use of quota groups or qgroup: Each subvolume is assigned a quota groups in the form of ''0/subvolume_id'' by default. However it is possible to create a quota group using any number if desired.
 +
 
 +
To use qgroups you need to enable quota first using
  
=== Commit interval settings ===
+
# btrfs quota enable ''path''
  
The resolution at which data are written to the filesystem is dictated by Btrfs itself and by system-wide settings. Btrfs defaults to a 30 seconds checkpoint interval in which new data are committed to the filesystem. This is tuneable using mount options (see below)
+
From this point onwards newly created subvolumes will be controlled by those groups.
 +
In order to retrospectively enable them for already existing subvolumes, enable quota normally, then create a qgroup (quota group) for each of those subvolume using their ''subvolume_id'' and rescan them:
  
System-wide settings also affect commit intervals. They include the files under {{ic|/proc/sys/vm/*}} and are out-of-scope of this wiki article. The kernel documentation for them resides in {{ic|Documentation/sysctl/vm.txt}}.
+
# btrfs subvolume list ''path'' | cut -d' ' -f2 | xargs -I{} -n1 btrfs qgroup create 0/{} ''path''
 +
# btrfs quota rescan ''path''
 +
 
 +
Quota groups in Btrfs form a tree hierarchy, whereby qgroups are attached to subvolumes. The size limits are set per qgroup and apply when any limit is reached in tree that contains a given subvolume.
 +
 
 +
Limits on quota groups can be applied either to the total data usage, un-shared data usage, compressed data usage or both. File copy and file deletion may both affect limits since the unshared limit of another qgroup can change if the original volume's files are deleted and only one copy is remaining. For example a fresh snapshot shares almost all the blocks with the original subvolume, new writes to either subvolume will raise towards the exclusive limit, deletions of common data in one volume raises towards the exclusive limit in the other one.
 +
 
 +
To apply a limit to a qgroup, use the command {{ic|btrfs qgroup limit}}. Depending on your usage either use a total limit, unshared limit ({{ic|-e}}) or compressed limit ({{ic|-c}}).
 +
To show usage and limits for a given path within a filesystem use
 +
 
 +
# btrfs qgroup show -reF ''path''
 +
 
 +
=== Commit interval ===
  
=== Checkpoint interval ===
+
The resolution at which data are written to the filesystem is dictated by Btrfs itself and by system-wide settings. Btrfs defaults to a 30 seconds checkpoint interval in which new data are committed to the filesystem. This can be changed by appending the {{ic|commit}} mount option in {{ic|/etc/fstab}} for the btrfs partition.
  
Users are able to change the checkpoint interval from the default 30 s to any value by appending the {{ic|commit}} mount option in {{ic|/etc/fstab}} for the btrfs partition.
+
LABEL=arch64 / btrfs defaults,noatime,compress=lzo,commit=120 0 0
  
LABEL=arch64 / btrfs defaults,noatime,ssd,compress=lzo,commit=120 0 0
+
System-wide settings also affect commit intervals. They include the files under {{ic|/proc/sys/vm/*}} and are out-of-scope of this wiki article. The kernel documentation for them resides in {{ic|Documentation/sysctl/vm.txt}}.
  
 
=== SSD TRIM ===
 
=== SSD TRIM ===
A Btrfs filesystem will automatically free unused blocks from an SSD drive supporting the TRIM command.
+
 
 +
A Btrfs filesystem is able to free unused blocks from an SSD drive supporting the TRIM command.
  
 
More information about enabling and using TRIM can be found in [[Solid State Drives#TRIM]].
 
More information about enabling and using TRIM can be found in [[Solid State Drives#TRIM]].
Line 217: Line 221:
 
== Usage ==
 
== Usage ==
  
=== Displaying used/free space ===
+
=== Swap file ===
  
General linux userspace tools such as {{ic|/usr/bin/df}} will inaccurately report free space on a Btrfs partition since it does not take into account space allocated for and used by the metadata. It is recommended to use {{ic|/usr/bin/btrfs}} to query a Btrfs partition. Below is an illustration of this effect, first querying using {{ic|df -h}}, and then using {{ic|btrfs filesystem df}}:
+
[[Swap file]]s in Btrfs are supported since Linux kernel 5.0.[https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=ed46ff3d423780fa5173b38a844bf0fdb210a2a7] The proper way to initialize a swap file is described in [[Swap file#Swap file creation]].
  
{{hc|$ df -h /|
+
{{Note|For kernels version 5.0+, Btfrs has native swap file support with some limitations:
Filesystem      Size  Used Avail Use% Mounted on
+
* The swap file cannot be on a snapshotted subvolume. The proper procedure is to create a new subvolume to place the swap file in.
/dev/sda3      119G  3.0G  116G  3% /
+
* It does not support swap files on file systems that span multiple devices. See [https://btrfs.wiki.kernel.org/index.php/FAQ#Does_btrfs_support_swap_files.3F Btrfs wiki: Does btrfs support swap files?] and [https://bbs.archlinux.org/viewtopic.php?pid=1849371#p1849371 Arch forums discussion].
 +
* Hibernation onto a swap file is currently not supported by {{pkg|systemd}} [https://github.com/systemd/systemd/issues/11939].
 
}}
 
}}
  
{{hc|$ btrfs filesystem df /|2=
+
{{Warning|Linux kernels before version 5.0, including {{pkg|linux-lts}}, do not support swap files. Using a swap file with Btrfs and a kernel prior to 5.0 can lead to file system corruption.}}
Data: total=3.01GB, used=2.73GB
 
System: total=4.00MB, used=16.00KB
 
Metadata: total=1.01GB, used=181.83MB
 
}}
 
  
Notice that {{ic|df -h}} reports 3.0GB used but {{ic|btrfs filesystem df}} reports 2.73GB for the data. This is due to the way Btrfs allocates space into the pool. The true disk usage is the sum of all three 'used' values which is inferior to 3.0GB as reported by {{ic|df -h}}.
+
=== Displaying used/free space ===
 
 
{{Note|1=If you see an entry of type {{ic|unknown}} in the output of {{ic|btrfs filesystem df}} at kernel >= 3.15, this is a display bug. As of [http://thread.gmane.org/gmane.comp.file-systems.btrfs/34419 this patch], the entry means GlobalReserve, which is kind of a buffer for changes not yet flushed. This entry is displayed as {{ic|unknown, single}} in RAID setups and is not possible to re-balance.}}
 
 
 
Another useful command to show a less verbose readout of used space is {{ic|btrfs filesystem show}}:
 
  
# btrfs filesystem show /dev/sda3
+
General linux userspace tools such as {{ic|df}} will inaccurately report free space on a Btrfs partition. It is recommended to use {{ic|btrfs filesystem usage}} to query Btrfs partitions. For example:
  
The newest command to get information on free/used space of a is {{ic|btrfs filesystem usage}}:
+
# btrfs filesystem usage /
  
# btrfs filesystem usage
+
{{Note|The {{ic|btrfs filesystem usage}} command does not currently work correctly with {{ic|RAID5/RAID6}} RAID levels.}}
  
{{Note|1=The {{ic|btrfs filesystem usage}} command does not currently work correctly with {{ic|RAID5/RAID6}} RAID levels.}}
+
See [https://btrfs.wiki.kernel.org/index.php/FAQ#How_much_free_space_do_I_have.3F] for more information.
  
 
=== Defragmentation ===
 
=== Defragmentation ===
  
Btrfs supports online defragmentation. To defragment the entire file system verbosely:
+
Btrfs supports online defragmentation through the mount option {{ic|autodefrag}}, see {{man|5|btrfs|MOUNT OPTIONS}}. To manually defragment your root, use:
 
 
# btrfs filesystem defragment -r -v /
 
  
To defragment the metadata of the root folder:
+
# btrfs filesystem defragment -r /
  
# btrfs filesystem defragment /
+
Using the above command without the {{ic|-r}} switch will result in only the metadata held by the subvolume containing the directory being defragmented. This allows for single file defragmentation by simply specifying the path.
  
This ''will not'' defragment the entire file system. For more information read [https://btrfs.wiki.kernel.org/index.php/Problem_FAQ#Defragmenting_a_directory_doesn.27t_work this page] on the Btrfs wiki.
+
Defragmenting a file which has a COW copy (either a snapshot copy or one made with {{ic|cp --reflink}} or bcp) plus using the {{ic|-c}} switch with a compression algorithm may result in two unrelated files effectively increasing the disk usage.
  
 
=== RAID ===
 
=== RAID ===
Line 264: Line 259:
 
{{Warning| Parity RAID (RAID 5/6) code has multiple serious data-loss bugs in it. See the Btrfs Wiki's [https://btrfs.wiki.kernel.org/index.php/RAID56 RAID5/6 page] and a bug report on [https://www.mail-archive.com/linux-btrfs@vger.kernel.org/msg55161.html linux-btrfs mailing list] for more detailed information.}}
 
{{Warning| Parity RAID (RAID 5/6) code has multiple serious data-loss bugs in it. See the Btrfs Wiki's [https://btrfs.wiki.kernel.org/index.php/RAID56 RAID5/6 page] and a bug report on [https://www.mail-archive.com/linux-btrfs@vger.kernel.org/msg55161.html linux-btrfs mailing list] for more detailed information.}}
  
==== Scrub ====
+
=== Scrub ===
  
 
The [https://btrfs.wiki.kernel.org/index.php/Glossary Btrfs Wiki Glossary] says that Btrfs scrub is "[a]n online filesystem checking tool. Reads all the data and metadata on the filesystem, and uses checksums and the duplicate copies from RAID storage to identify and repair any corrupt data."
 
The [https://btrfs.wiki.kernel.org/index.php/Glossary Btrfs Wiki Glossary] says that Btrfs scrub is "[a]n online filesystem checking tool. Reads all the data and metadata on the filesystem, and uses checksums and the duplicate copies from RAID storage to identify and repair any corrupt data."
Line 270: Line 265:
 
{{Warning|A running scrub process will prevent the system from suspending, see [http://comments.gmane.org/gmane.comp.file-systems.btrfs/33106 this thread] for details.}}
 
{{Warning|A running scrub process will prevent the system from suspending, see [http://comments.gmane.org/gmane.comp.file-systems.btrfs/33106 this thread] for details.}}
  
===== Start manually =====
+
==== Start manually ====
  
To start a scrub for the subvolume mounted at root do:
+
To start a (background) scrub on the filesystem which contains {{ic|/}}:
  
 
  # btrfs scrub start /
 
  # btrfs scrub start /
  
To check the status of the scrub do:
+
To check the status of a running scrub:
  
 
  # btrfs scrub status /
 
  # btrfs scrub status /
  
===== Start with a service or timer =====
+
==== Start with a service or timer ====
  
The {{Pkg|btrfs-progs}} package brings the {{ic|btrfs-scrub@.timer}} unit for monthly scrubbing the specified mountpoint. [[Enable]] the timer with an escaped path, e.g. {{ic|btrfs-scrub@-.timer}} for {{ic|/}} and {{ic|btrfs-scrub@home.timer}} for {{ic|/home}}. You can use the ''systemd-escape'' tool to escape a given string, see {{ic|systemd-escape(1)}} for examples.
+
The {{Pkg|btrfs-progs}} package brings the {{ic|btrfs-scrub@.timer}} unit for monthly scrubbing the specified mountpoint. [[Enable]] the timer with an escaped path, e.g. {{ic|btrfs-scrub@-.timer}} for {{ic|/}} and {{ic|btrfs-scrub@home.timer}} for {{ic|/home}}. You can use {{ic|systemd-escape -p ''/path/to/mountpoint''}} to escape the path, see {{man|1|systemd-escape}} for details.
  
 
You can also run the scrub by [[starting]] {{ic|btrfs-scrub@.service}} (with the same encoded path). The advantage of this over {{ic|# btrfs scrub}} is that the results of the scrub will be logged in the [[systemd journal]].
 
You can also run the scrub by [[starting]] {{ic|btrfs-scrub@.service}} (with the same encoded path). The advantage of this over {{ic|# btrfs scrub}} is that the results of the scrub will be logged in the [[systemd journal]].
  
==== Balance ====
+
=== Balance ===
  
 
"A balance passes all data in the filesystem through the allocator again. It is primarily intended to rebalance the data in the filesystem across the devices when a device is added or removed. A balance will regenerate missing copies for the redundant RAID levels, if a device has failed." [https://btrfs.wiki.kernel.org/index.php/Glossary] See [https://btrfs.wiki.kernel.org/index.php/FAQ#What_does_.22balance.22_do.3F Upstream FAQ page].
 
"A balance passes all data in the filesystem through the allocator again. It is primarily intended to rebalance the data in the filesystem across the devices when a device is added or removed. A balance will regenerate missing copies for the redundant RAID levels, if a device has failed." [https://btrfs.wiki.kernel.org/index.php/Glossary] See [https://btrfs.wiki.kernel.org/index.php/FAQ#What_does_.22balance.22_do.3F Upstream FAQ page].
 +
 +
On a single-device filesystem a balance may be also useful for (temporarily) reducing the amount of allocated but unused (meta)data chunks. Sometimes this is needed for fixing [https://btrfs.wiki.kernel.org/index.php/FAQ#Help.21_Btrfs_claims_I.27m_out_of_space.2C_but_it_looks_like_I_should_have_lots_left.21 "filesystem full" issues].
  
 
  # btrfs balance start /
 
  # btrfs balance start /
Line 303: Line 300:
 
To create a readonly snapshot add the {{ic|-r}} flag. To create writable version of a readonly snapshot, simply create a snapshot of it.
 
To create a readonly snapshot add the {{ic|-r}} flag. To create writable version of a readonly snapshot, simply create a snapshot of it.
  
{{Note|Snapshots are not recursive. Every sub-volume inside sub-volume will be an empty directory inside the snapshot.}}
+
{{Note|Snapshots are not recursive. Every nested subvolume will be an empty directory inside the snapshot.}}
  
 
=== Send/receive ===
 
=== Send/receive ===
Line 311: Line 308:
 
   # btrfs send /root_backup | btrfs receive /backup
 
   # btrfs send /root_backup | btrfs receive /backup
  
The snapshot that is sent ''must'' be readonly. The above command is useful for copying a subvolume to an external device (''e.g.'', a USB disk mounted at {{ic|/backup}} above).
+
The snapshot that is sent ''must'' be readonly. The above command is useful for copying a subvolume to an external device (e.g. a USB disk mounted at {{ic|/backup}} above).
  
 
You can also send only the difference between two snapshots. For example, if you have already sent a copy of {{ic|root_backup}} above and have made a new readonly snapshot on your system named {{ic|root_backup_new}}, then to send only the incremental difference to {{ic|/backup}} do:
 
You can also send only the difference between two snapshots. For example, if you have already sent a copy of {{ic|root_backup}} above and have made a new readonly snapshot on your system named {{ic|root_backup_new}}, then to send only the incremental difference to {{ic|/backup}} do:
  
   # btrfs send -p /root_backup /root_backup_new | btrfs send /backup
+
   # btrfs send -p /root_backup /root_backup_new | btrfs receive /backup
  
 
Now a new subvolume named {{ic|root_backup_new}} will be present in {{ic|/backup}}.
 
Now a new subvolume named {{ic|root_backup_new}} will be present in {{ic|/backup}}.
  
See [https://btrfs.wiki.kernel.org/index.php/Incremental_Backup Btrfs Wiki's Incremental Backup page] on how to use this for an incremental backup and for tools that automate the process.
+
See [https://btrfs.wiki.kernel.org/index.php/Incremental_Backup Btrfs Wiki's Incremental Backup page] on how to use this for incremental backups and for tools that automate the process.
 +
 
 +
=== Deduplication ===
 +
 
 +
Using copy-on-write, Btrfs is able to copy files or whole subvolumes without actually copying the data. However whenever a file is altered a new ''proper'' copy is created. Deduplication takes this a step further, by actively identifying blocks of data which share common sequences and combining them into an extent with the same copy-on-write semantics.
 +
 
 +
Tools dedicated to deduplicate a Btrfs formatted partition include {{aur|duperemove}}, {{aur|bedup}} and ''btrfs-dedup''. One may also want to merely deduplicate data on a file based level instead using e.g. {{pkg|rmlint}} or {{aur|jdupes}}. For an overview of available features of those programs and additional information have a look at the [https://btrfs.wiki.kernel.org/index.php/Deduplication#Batch upstream Wiki entry].
 +
 
 +
Furthermore Btrfs developers are working on inband (also known as synchronous or inline) deduplication, meaning deduplication done when writing new data to the filesystem. Currently it is still an experiment which is developed out-of-tree. Users willing to test the new feature should read the [https://btrfs.wiki.kernel.org/index.php/User_notes_on_dedupe appropriate kernel wiki page].
  
 
== Known issues ==
 
== Known issues ==
Line 327: Line 332:
 
=== Encryption ===
 
=== Encryption ===
  
Btrfs has no built-in encryption support, but this may come in future. Users can encrypt the partition before running {{ic|mkfs.btrfs}}. See [[dm-crypt/Encrypting an entire system#Btrfs subvolumes with swap]].  
+
Btrfs has no built-in encryption support, but this [https://lwn.net/Articles/700487/ may] come in the future. Users can encrypt the partition before running {{ic|mkfs.btrfs}}. See [[dm-crypt/Encrypting an entire system#Btrfs subvolumes with swap]].  
  
 
Existing Btrfs file systems can use something like [[EncFS]] or [[TrueCrypt]], though perhaps without some of Btrfs' features.
 
Existing Btrfs file systems can use something like [[EncFS]] or [[TrueCrypt]], though perhaps without some of Btrfs' features.
  
=== Swap file ===
+
=== TLP ===
  
Btrfs does not yet support [[Swap#Swap_file|swap files]]. This is due to swap files requiring a function that Btrfs does not have for possibility of file system corruption [https://btrfs.wiki.kernel.org/index.php/FAQ#Does_btrfs_support_swap_files.3F]. Patches for swapfile support are already available [https://lkml.org/lkml/2014/12/9/718] and may be included in an upcoming kernel release. As an alternative a swap file can be mounted on a loop device with poorer performance but will not be able to hibernate. Install the package {{Pkg|systemd-swap}} to automate this.
+
Using TLP requires special precautions in order to avoid filesystem corruption. Refer to the [[TLP#Btrfs|according TLP section]] for more information.
 
 
=== Linux-rt kernel ===
 
 
 
{{Out of date|We're on 4.6.4. Is this still an issue?}}
 
 
 
As of version 3.14.12_rt9, the [[Kernel#-rt|linux-rt]] kernel does not boot with the Btrfs file system. This is due to the slow development of the ''rt'' patchset.
 
  
 
== Tips and tricks ==
 
== Tips and tricks ==
  
=== Corruption recovery ===
+
=== Partitionless Btrfs disk ===
  
''btrfs-check'' cannot be used on a mounted file system. To be able to use ''btrfs-check'' without booting from a live USB, add it to the initial ramdisk:
+
{{Warning|Most users do not want this type of setup and instead should install Btrfs on a regular partition. Furthermore, GRUB strongly discourages installation to a partitionless disk. Consider using {{ic|--alloc-start}} for mkfs.btrfs to give larger space to GRUB. }}
  
{{hc|/etc/mkinitcpio.conf|output=
+
Btrfs can occupy an entire data storage device, replacing the [[MBR]] or [[GPT]] partitioning schemes, using [[#Subvolumes|subvolumes]] to simulate partitions. However, using a partitionless setup is not required to simply [[#File system creation|create a Btrfs filesystem]] on an existing [[partition]] that was created using another method. There are some limitations to partitionless single disk setups:
BINARIES="/usr/bin/btrfs"
 
}}
 
  
Regenerate the initial ramdisk using [[mkinitcpio]].
+
* Cannot use different [[file systems]] for different [[fstab|mount points]].
 +
* If using a Linux kernel version before 5.0, you cannot use [[Swap|swap area]] as Btrfs did not support [[Swap#Swap_file|swap files]] pre-5.0 and there is no place to create [[Swap#Swap_partition|swap partition]]. This also limits the use of hibernation/resume, which needs a swap area to store the hibernation image. This includes the current LTS version of Linux which is on 4.xx.
 +
* Cannot use [[UEFI]] to boot.
  
Then if there is a problem booting, the utility is available for repair.
+
To overwrite the existing partition table with Btrfs, run the following command:
  
{{Note|If the fsck process has to invalidate the space cache (and/or other caches?) then it is normal for a subsequent boot to hang up for a while (it may give console messages about btrfs-transaction being hung). The system should recover from this after a while.}}
+
# mkfs.btrfs /dev/sd''X''
  
See the [https://btrfs.wiki.kernel.org/index.php/Btrfsck Btrfs Wiki page] for more information.
+
For example, use {{ic|/dev/sda}} rather than {{ic|/dev/sda1}}. The latter would format an existing partition instead of replacing the entire partitioning scheme. Because the root partition is Btrfs, make sure {{ic|btrfs}} is compiled into the kernel, or put {{ic|btrfs}} into [[mkinitcpio.conf#MODULES]] and activate.
  
=== Booting into snapshots with GRUB ===
+
Install the [[boot loader]] like you would for a data storage device with a [[Master Boot Record]]. See [[Syslinux#Manual install]] or  [[GRUB/Tips and tricks#Install to partition or partitionless disk]]. If your kernel does not boot due to {{ic|Failed to mount /sysroot.}}, please add {{ic|1=GRUB_PRELOAD_MODULES="btrfs"}} in {{ic|/etc/default/grub}} and generate the grub configuration ([[GRUB#Generate the main configuration file]]).
  
You can manually create a [[GRUB#GNU/Linux menu entry]] with the {{ic|1=rootflags=subvol=}} argument. The {{ic|1=subvol=}} mount options in {{ic|/etc/fstab}} of the snapshot to boot into also have to be specified correctly.
+
=== Ext3/4 to Btrfs conversion ===
  
Alternatively, you can automatically populate your GRUB menu with btrfs snapshots when regenerating the GRUB configuration file by using {{AUR|grub-btrfs}} or {{AUR|grub-btrfs-git}}.
+
{{Warning|There are many reports on the btrfs mailing list about incomplete/corrupt/broken conversions. Make sure you have ''working'' backups of any data you cannot afford to lose. See [https://btrfs.wiki.kernel.org/index.php/Conversion_from_Ext3 Conversion from Ext3] on the btrfs wiki for more information.}}
  
=== Use Btrfs subvolumes with systemd-nspawn ===
+
Boot from an install CD, then convert by doing:
  
See the [[Systemd-nspawn#Use Btrfs subvolume as container root]] and [[Systemd-nspawn#Use temporary Btrfs snapshot of container]] articles.
+
# btrfs-convert /dev/''partition''
  
=== Automatic snapshots on each boot ===
+
Mount the partion and test the conversion by checking the files.  Be sure to change the {{ic|/etc/fstab}} to reflect the change ('''type''' to {{ic|btrfs}} and '''fs_passno''' [the last field] to {{ic|0}} as Btrfs does not do a file system check on boot). Also note that the UUID of the partition will have changed, so update fstab accordingly when using UUIDs. {{ic|chroot}} into the system and rebuild the GRUB menu list (see [[Install from existing Linux]] and [[GRUB]] articles). If converting a root filesystem, while still chrooted run {{ic|mkinitcpio -p linux}} to regenerate the initramfs or the system will not successfully boot. If you get stuck in grub with 'unknown filesystem' try reinstalling grub with {{ic|grub-install /dev/''partition''}} and regenerate the config as well {{ic|grub-mkconfig -o /boot/grub/grub.cfg}}.
  
{{Deletion|This seems like it could be summed up with a systemd unit to call ''snapper'' at boot. The script provided here is for a very specific use case. Although interesting, I'm not sure if this is really a "tip".|Talk:Btrfs - Tips and tricks#Automatic snapshots on each boot}}
+
After confirming that there are no problems, complete the conversion by deleting the backup {{ic|ext2_saved}} sub-volume. Note that you cannot revert back to ext3/4 without it.
  
It is possible to automatically save the then current state of a btrfs subvolume during system boot. Two files are needed to accomplish this, a script which snapshots btrfs subvolumes and a [[systemd]] unit file to run the script during the boot process.
+
# btrfs subvolume delete /ext2_saved
  
==== Requirements ====
+
Finally [[#Balance|balance]] the file system to reclaim the space.
  
The root filesystem must have been installed into its own dedicated btrfs subvolume and the btrfs-root where all subvolumes reside has to be accessible.
+
Remember that some applications which were installed prior have to be adapted to Btrfs. Notably [[TLP#Btrfs]] needs special care to avoid filesystem corruption but other applications may profit from certain features as well.
  
{{hc|/etc/fstab|
+
=== Checksum hardware acceleration ===
<nowiki># ...
 
UUID=XXXXXXXX-XXXX-XXXX-XXXX-XXXXXXXXXXXX  /                btrfs  defaults,subvol=__current/ROOT  0 0
 
UUID=XXXXXXXX-XXXX-XXXX-XXXX-XXXXXXXXXXXX  /run/btrfs-root  btrfs  defaults,compress=lzo            0 0
 
  
# just for the below example:
+
To verify if Btrfs checksum is hardware accelerated:
UUID=XXXXXXXX-XXXX-XXXX-XXXX-XXXXXXXXXXXX  /var              btrfs  defaults,subvol=__current/var    0 0
+
{{hc|$ dmesg {{!}} grep crc32c|2=
UUID=XXXXXXXX-XXXX-XXXX-XXXX-XXXXXXXXXXXX  /opt              btrfs  defaults,subvol=__current/opt    0 0
+
Btrfs loaded, crc32c=crc32c-intel
UUID=XXXXXXXX-XXXX-XXXX-XXXX-XXXXXXXXXXXX  /home            btrfs  defaults,subvol=__current/home  0 0
+
}}
# ...
 
</nowiki>}}
 
  
==== Snapshot script ====
+
If you see {{ic|1=crc32c=crc32c-generic}}, it is probably because your root partition is Btrfs, and you will have to compile {{ic|crc32c-intel}} into the kernel to make it work. Putting {{ic|crc32c-intel}} into [[mkinitcpio.conf]] does ''not'' work.
  
An example script to snapshot the btrfs subvolumes mounted under ''/'' is listed further below. The script will automatically detect the names of all subvolumes mounted under ''/'' and create snapshots of the same names under a directory specified by the caller.
+
=== Corruption recovery ===
  
What the script does, is to delete all snapshots from the state the system was in at last boot, make new snapshots, and alter the /etc/[[fstab]] file of the snapshot of the root subvolume to allow it to be booted without manual configuration.
+
''btrfs-check'' cannot be used on a mounted file system. To be able to use ''btrfs-check'' without booting from a live USB, add it to the initial ramdisk:
  
When called like this ...
+
{{hc|/etc/mkinitcpio.conf|output=
{{bc|# bash /usr/local/bin/snapshot_current_system_state.sh '/run/btrfs-root' '__current/ROOT' '__snapshot/__state_at_last_successful_boot'}}
+
BINARIES=("/usr/bin/btrfs")
the following structure will result:
+
}}
  
{{hc|<nowiki>$ btrfs subvolume list '/'</nowiki>|
+
[[Regenerate the initramfs]].
<nowiki>ID 257 gen 1134 top level 5 path __current/ROOT
 
ID 258 gen 1137 top level 5 path __current/var
 
ID 259 gen 1129 top level 5 path __current/opt
 
ID 260 gen 1137 top level 5 path __current/home
 
ID 277 gen 1128 top level 5 path __snapshot/__state_at_last_successful_boot/__current/ROOT
 
ID 278 gen 1128 top level 5 path __snapshot/__state_at_last_successful_boot/__current/var
 
ID 279 gen 1129 top level 5 path __snapshot/__state_at_last_successful_boot/__current/opt
 
ID 280 gen 1130 top level 5 path __snapshot/__state_at_last_successful_boot/__current/home
 
</nowiki>}}
 
Note that ''var, opt and home'' are subvolumes created by the caller in this example and mounted under '/'. The script detected and snapshotted them automatically. The resulting directory structure is:
 
{{bc|<nowiki>
 
/run/btrfs-root/__current/ROOT
 
/run/btrfs-root/__current/var
 
/run/btrfs-root/__current/opt
 
/run/btrfs-root/__current/home
 
/run/btrfs-root/__current/net
 
/run/btrfs-root/__current/bak
 
/run/btrfs-root/__snapshot/__state_at_last_successful_boot/__current/ROOT
 
/run/btrfs-root/__snapshot/__state_at_last_successful_boot/__current/var
 
/run/btrfs-root/__snapshot/__state_at_last_successful_boot/__current/opt
 
/run/btrfs-root/__snapshot/__state_at_last_successful_boot/__current/home
 
</nowiki>}}
 
  
The script below takes 3 parameters:
+
Then if there is a problem booting, the utility is available for repair.
# The path of the btrfs filesystem root. E. g. {{ic|/run/btrfs-root}},
 
# The name of the btrfs root subvolume as specified in /etc/[[fstab]]. E. g. {{ic|__current/ROOT}},
 
# The path where the newly created snapshots will reside without its 1st parameter portion. E. g. {{ic|__snapshot/__state_at_last_successful_boot}} (if the actual path is ''/run/btrfs-root/__snapshot/__state_at_last_successful_boot'')
 
  
{{Warning|This script will delete all snapshots of the same names as the regular subvolume names in the path its third parameter is pointing to. Be careful that the ''3rd parameter is not pointing at the place where your subvolumes reside in'' --- In this example, ''/run/btrfs-root/__current'' as the 3rd parameter would be incorrect and lead to data loss.}}
+
{{Note|If the fsck process has to invalidate the space cache (and/or other caches?) then it is normal for a subsequent boot to hang up for a while (it may give console messages about btrfs-transaction being hung). The system should recover from this after a while.}}
  
{{hc|/usr/local/bin/snapshot_current_system_state.sh|
+
See the [https://btrfs.wiki.kernel.org/index.php/Btrfsck Btrfs Wiki page] for more information.
<nowiki>
 
#!/bin/sh
 
  
# example call: # bash /usr/local/bin/snapshot_current_system_state.sh '/run/btrfs-root' '__current/ROOT' '__snapshot/__state_at_last_successful_boot'
+
=== Booting into snapshots ===
  
if [ $# -ne 3 ]
+
In order to boot into a snapshot you must specify the subvolume via a [[Kernel parameters#Configuration|kernel parameter]] using {{ic|1=rootflags=subvol=''/path/to/subvolume''}} and alter your {{ic|/etc/fstab}} to point to the same subvolume using {{ic|1=subvol=}}. Alternatively the subvolume can be specified with its id - retrievable with e.g. {{ic|btrfs subvolume list ''/root/path''}} - and {{ic|1=rootflags=subvolid=''objectid''}} as kernel parameter respectively {{ic|1=subvolid=''objectid''}} as mount option in {{ic|/etc/fstab}}.
then
 
  /usr/bin/echo -e "This script requires three parameters:\n1st parameter: The path of the btrfs filesystem root. e. g. /run/btrfs-root\n2nd parameter: The name of the btrfs root volume as specified in /etc/fstab. E. g. __current/ROOT\n3rd parameter: The path where the newly created snapshots will reside without its 1st parameter portion. E. g. __snapshot/__state_at_last_successful_boot\nCAUTION: This script will delete all snapshots of the same name as the regular volume names in the path parameter 3 is pointing to."
 
  exit 0
 
fi
 
  
btrfs_root="${1}" # example: '/run/btrfs-root'
+
If using GRUB you can automatically populate your boot menu with btrfs snapshots when regenerating the configuration file with the help of {{Pkg|grub-btrfs}} or {{AUR|grub-btrfs-git}}.
path_to_root_volume="${2}" # example: '__current/ROOT'
 
path_to_snapshots="${3}" # example: '__snapshot/__state_at_last_successful_boot'
 
  
# take no snapshots when booted into a snapshot
+
=== Use Btrfs subvolumes with systemd-nspawn ===
if [ -e '/SNAPSHOT-TIMESTAMP' ]
 
then
 
  exit 0
 
fi
 
  
# anti recursive snapshots
+
See the [[Systemd-nspawn#Use Btrfs subvolume as container root]] and [[Systemd-nspawn#Use temporary Btrfs snapshot of container]] articles.
for subvolume in $(/usr/bin/btrfs subvolume list '/' | /usr/bin/awk '{print $NF}') # scan
 
do
 
  path_to_snapshot="${btrfs_root}/${path_to_snapshots}/${subvolume}"
 
 
 
  if [ -d "${path_to_snapshot}" ]
 
  then
 
    /usr/bin/btrfs subvolume delete "${path_to_snapshot}"
 
  fi 
 
done
 
 
 
subvolumes="$(/usr/bin/btrfs subvolume list '/' | /usr/bin/awk '{print $NF}')" # rescan
 
for subvolume in $subvolumes
 
do
 
  snapshot_directory="${btrfs_root}/${path_to_snapshots}/$(/usr/bin/dirname ${subvolume})"
 
 
 
  if [ ! -d "${snapshot_directory}" ]
 
  then
 
    /usr/bin/mkdir -p "${snapshot_directory}"
 
  fi 
 
 
 
  /usr/bin/btrfs subvolume snapshot "${btrfs_root}/${subvolume}" "${btrfs_root}/${path_to_snapshots}/${subvolume}"
 
 
 
  if [ "${subvolume}" = "${path_to_root_volume}" ]
 
  then
 
    timestamp="$(/usr/bin/date +%d.%m.%Y-%H:%M:%S)"
 
 
 
    /usr/bin/echo -e "Arch Linux --- state at last successful boot (nonpersistent) [${timestamp}]\n" > "${btrfs_root}/${path_to_snapshots}/${path_to_root_volume}/etc/issue"
 
 
 
    /usr/bin/echo "${timestamp}" > "${btrfs_root}/${path_to_snapshots}/${path_to_root_volume}/SNAPSHOT-TIMESTAMP"
 
 
 
    sed_path_to_snapshots="$(/usr/bin/echo ${path_to_snapshots} | /usr/bin/sed --posix --regexp-extended 's/\//\\\//g')"
 
 
 
    for subvolumeX in $(echo $subvolumes | /usr/bin/sed --posix --regexp-extended 's/\//\\\//g')
 
    do
 
      /usr/bin/sed --posix --regexp-extended "s/subvol=${subvolumeX}/subvol=${sed_path_to_snapshots}\/${subvolumeX}/g" --in-place "${btrfs_root}/${path_to_snapshots}/${path_to_root_volume}/etc/fstab"
 
    done
 
  fi
 
done
 
 
 
/usr/bin/sync
 
</nowiki>}}
 
 
 
====systemd unit file====
 
 
 
Following [[systemd]] unit file will run the script every time the system manages to successfully boot into ''multi-user.target'':
 
* Example unit file {{ic|/etc/systemd/system/snapshot_current_system_state_upon_boot.service}}:
 
{{bc|<nowiki>
 
[Unit]
 
Description=Takes a snapshot of each btrfs subvolume mounted under / after multi-user.target has been reached.
 
After=multi-user.target
 
 
 
[Service]
 
Type=oneshot
 
ExecStart=/bin/sh /usr/local/bin/snapshot_current_system_state.sh '/run/btrfs-root' '__current/ROOT' '__snapshot/__state_at_last_successful_boot'
 
 
 
[Install]
 
WantedBy=multi-user.target
 
</nowiki>}}
 
 
 
The unit file has to be symlinked by systemd:
 
 
 
# systemctl enable snapshot_current_system_state_upon_boot.service
 
  
 
== Troubleshooting ==
 
== Troubleshooting ==
Line 526: Line 421:
 
==== Partition offset ====
 
==== Partition offset ====
  
{{Note|1=The offset problem may happen when you try to embed {{ic|core.img}} into a partitioned disk. It means that [https://wiki.archlinux.org/index.php?title=Talk:Btrfs&diff=319474&oldid=292530 it is OK] to embed grub's {{ic|corg.img}} into a Btrfs pool on a partitionless disk (e.g. {{ic|/dev/sd''X''}}) directly.}}
+
The offset problem may happen when you try to embed {{ic|core.img}} into a partitioned disk. It means that [https://wiki.archlinux.org/index.php?title=Talk:Btrfs&diff=319474&oldid=292530 it is OK] to embed grub's {{ic|core.img}} into a Btrfs pool on a partitionless disk (e.g. {{ic|/dev/sd''X''}}) directly.
  
[[GRUB]] can boot Btrfs partitions however the module may be larger than other [[file systems]]. And the {{ic|core.img}} file made by {{ic|grub-install}} may not fit in the first 63 sectors (31.5KiB) of the drive between the MBR and the first partition. Up-to-date partitioning tools such as {{ic|fdisk}} and {{ic|gdisk}} avoid this issue by offsetting the first partition by roughly 1MiB or 2MiB.
+
[[GRUB]] can boot Btrfs partitions, however the module may be larger than other [[file systems]]. And the {{ic|core.img}} file made by {{ic|grub-install}} may not fit in the first 63 sectors (31.5KiB) of the drive between the MBR and the first partition. Up-to-date partitioning tools such as {{ic|fdisk}} and {{ic|gdisk}} avoid this issue by offsetting the first partition by roughly 1MiB or 2MiB.
  
 
==== Missing root ====
 
==== Missing root ====
 +
 +
{{Accuracy|Suggests editing a non-configuration file manually.|Talk:Btrfs#Should not suggest to edit files in /usr/share}}
  
 
Users experiencing the following: {{ic|1=error no such device: root}} when booting from a RAID style setup then edit /usr/share/grub/grub-mkconfig_lib and remove both quotes from the line {{ic|1=echo "  search --no-floppy --fs-uuid --set=root ${hints} ${fs_uuid}"}}. Regenerate the config for grub and the system should boot without an error.
 
Users experiencing the following: {{ic|1=error no such device: root}} when booting from a RAID style setup then edit /usr/share/grub/grub-mkconfig_lib and remove both quotes from the line {{ic|1=echo "  search --no-floppy --fs-uuid --set=root ${hints} ${fs_uuid}"}}. Regenerate the config for grub and the system should boot without an error.
Line 549: Line 446:
 
A workaround is to remove {{ic|btrfs}} from the {{ic|HOOKS}} array in {{ic|/etc/mkinitcpio.conf}} and instead add {{ic|btrfs}} to the {{ic|MODULES}} array. Then regenerate the initramfs with {{ic|mkinitcpio -p linux}} (adjust the preset if needed) and reboot.
 
A workaround is to remove {{ic|btrfs}} from the {{ic|HOOKS}} array in {{ic|/etc/mkinitcpio.conf}} and instead add {{ic|btrfs}} to the {{ic|MODULES}} array. Then regenerate the initramfs with {{ic|mkinitcpio -p linux}} (adjust the preset if needed) and reboot.
  
See the [https://bbs.archlinux.org/viewtopic.php?id=189845 original forums thread] and {{Bug|42884}} for further information and discussion.
+
You will get the same error if you try to mount a raid array without one of the devices. In that case you must add the {{ic|degraded}} mount option to {{ic|/etc/fstab}}. If your root resides on the array, you must also add {{ic|1=rootflags=degraded}} to your [[kernel parameters]].
  
You will get the same error if you try to mount a raid array without one of the devices. In that case you must add the {{ic|degraded}} mount option to {{ic|/etc/fstab}}. If your root resides on the array, you must also add {{ic|1=rootflags=degraded}} to your [[kernel parameters]].
+
As of August 2016, a potential workaround for this bug is to mount the array by a single drive only in {{ic|/etc/fstab}}, and allow btrfs to discover and append the other drives automatically. Group-based identifiers such as UUID and LABEL appear to contribute to the failure. For example, a two-device RAID1 array consisting of 'disk1' and disk2' will have a UUID allocated to it, but instead of using the UUID, use only {{ic|/dev/mapper/disk1}} in {{ic|/etc/fstab}}. For a more detailed explanation, see the following [https://blog.samcater.com/fix-for-btrfs-open_ctree-failed-when-running-root-fs-on-raid-1-or-raid10-arch-linux/ blog post].
  
{{Note|As of August 2016, a potential workaround for this bug is to mount the array by a single drive only in {{ic|/etc/fstab}}, and allow btrfs to discover and append the other drives automatically. Group-based identifiers such as UUID and LABEL appear to contribute to the failure. For example, a two-device RAID1 array consisting of 'disk1' and disk2' will have a UUID allocated to it, but instead of using the UUID, use only {{ic|/dev/mapper/disk1}} in {{ic|/etc/fstab}}.
+
Another possible workaround is to remove the {{ic|udev}} hook in [[mkinitcpio.conf]] and replace it with the {{ic|systemd}} hook. In this case, {{ic|btrfs}} should ''not'' be in the {{ic|HOOKS}} or {{ic|MODULES}} arrays.
  
For a more detailed explanation, see the following [https://blog.samcater.com/fix-for-btrfs-open_ctree-failed-when-running-root-fs-on-raid-1-or-raid10-arch-linux/ blog post.]
+
See the [https://bbs.archlinux.org/viewtopic.php?id=189845 original forums thread] and {{Bug|42884}} for further information and discussion.
}}
 
  
 
=== btrfs check ===
 
=== btrfs check ===
 +
 
{{Warning|Since Btrfs is under heavy development, especially the {{ic|btrfs check}} command, it is highly recommended to create a '''backup''' and consult the following Btfrs documentation before executing {{ic|btrfs check}} with the {{ic|--repair}} switch.}}
 
{{Warning|Since Btrfs is under heavy development, especially the {{ic|btrfs check}} command, it is highly recommended to create a '''backup''' and consult the following Btfrs documentation before executing {{ic|btrfs check}} with the {{ic|--repair}} switch.}}
  
Line 571: Line 468:
 
* '''Performance related'''
 
* '''Performance related'''
 
** [http://superuser.com/questions/432188/should-i-put-my-multi-device-btrfs-filesystem-on-disk-partitions-or-raw-devices Btrfs on raw disks?]
 
** [http://superuser.com/questions/432188/should-i-put-my-multi-device-btrfs-filesystem-on-disk-partitions-or-raw-devices Btrfs on raw disks?]
** [http://comments.gmane.org/gmane.comp.file-systems.btrfs/19440 Varying leafsize and nodesize in Btrfs]
+
** [https://www.spinics.net/lists/linux-btrfs/msg18652.html Varying leafsize and nodesize in Btrfs]
 
** [http://comments.gmane.org/gmane.comp.file-systems.btrfs/15646 Btrfs support for efficient SSD operation (data blocks alignment)]
 
** [http://comments.gmane.org/gmane.comp.file-systems.btrfs/15646 Btrfs support for efficient SSD operation (data blocks alignment)]
 
** [https://btrfs.wiki.kernel.org/index.php/FAQ#Is_Btrfs_optimized_for_SSD.3F Is Btrfs optimized for SSDs?]
 
** [https://btrfs.wiki.kernel.org/index.php/FAQ#Is_Btrfs_optimized_for_SSD.3F Is Btrfs optimized for SSDs?]
 
** '''Phoronix mount option benchmarking'''
 
** '''Phoronix mount option benchmarking'''
 +
*** [http://www.phoronix.com/scan.php?page=article&item=btrfs-mount-linux49 Linux 4.9]
 
*** [http://www.phoronix.com/scan.php?page=article&item=linux_314_btrfs Linux 3.14]
 
*** [http://www.phoronix.com/scan.php?page=article&item=linux_314_btrfs Linux 3.14]
 
*** [http://www.phoronix.com/scan.php?page=article&item=linux_btrfs_311&num=1 Linux 3.11]
 
*** [http://www.phoronix.com/scan.php?page=article&item=linux_btrfs_311&num=1 Linux 3.11]

Latest revision as of 13:56, 12 October 2019

From Btrfs Wiki:

Btrfs is a modern copy on write (CoW) filesystem for Linux aimed at implementing advanced features while also focusing on fault tolerance, repair and easy administration. Jointly developed at multiple companies, Btrfs is licensed under the GPL and open for contribution from anyone.
Warning: Btrfs has some features that are unstable. See the Btrfs Wiki's Status, Is Btrfs stable? and Getting started for more detailed information. See the #Known issues section.

Preparation

For user space utilities install the btrfs-progs package, which is required for basic operations.

If you need to boot from a Btrfs file system (i.e., your kernel and initramfs reside on a Btrfs partition), check if your boot loader supports Btrfs.

File system creation

The following shows how to create a new Btrfs file system. To convert an ext3/4 partition to Btrfs, see #Ext3/4 to Btrfs conversion. To use a partitionless setup, see #Partitionless Btrfs disk.

See mkfs.btrfs(8) for more information.

File system on a single device

To create a Btrfs filesystem on partition /dev/partition:

# mkfs.btrfs -L mylabel /dev/partition

The Btrfs default blocksize is 16KB. To use a larger blocksize for data/metadata, specify a value for the nodesize via the -n switch as shown in this example using 16KB blocks:

# mkfs.btrfs -L mylabel -n 16k /dev/partition

Multi-device file system

Warning: The RAID 5 and RAID 6 modes of Btrfs are fatally flawed, and should not be used for "anything but testing with throw-away data." See the Btrfs page on RAID5 and RAID6 for status updates.

Multiple devices can be used to create a RAID. Supported RAID levels include RAID 0, RAID 1, RAID 10, RAID 5 and RAID 6. The RAID levels can be configured separately for data and metadata using the -d and -m options respectively. By default the data is striped (raid0) and the metadata is mirrored (raid1). See Using Btrfs with Multiple Devices for more information about how to create a Btrfs RAID volume.

# mkfs.btrfs -d raid0 -m raid1 /dev/part1 /dev/part2 ...

You can create a JBOD configuration, where disks are seen as one filesystem, but files are not duplicated, using -d single -m raid1.

You must include either the udev hook or the btrfs hook in /etc/mkinitcpio.conf in order to use multiple Btrfs devices in a pool. See the Mkinitcpio#Common hooks article for more information.

Note:
  • It is possible to add devices to a multiple-device filesystem later on. See the Btrfs wiki article for more information.
  • Devices can be of different sizes. However, if one drive in a RAID configuration is bigger than the others, this extra space will not be used.
  • Some boot loaders such as Syslinux do not support multi-device file systems.

See #RAID for advice on maintenance specific to multi-device Btrfs file systems.

Configuring the file system

Copy-on-Write (CoW)

By default, Btrfs uses copy-on-write for all files all the time. See the Btrfs Sysadmin Guide section for implementation details, as well as advantages and disadvantages.

Disabling CoW

To disable copy-on-write for newly created files in a mounted subvolume, use the nodatacow mount option. This will only affect newly created files. Copy-on-write will still happen for existing files. The nodatacow option also disables compression. See btrfs(5) for details.

Note: From btrfs(5): "within a single file system, it is not possible to mount some subvolumes with nodatacow and others with datacow. The mount option of the first mounted subvolume applies to any other subvolumes."

To disable copy-on-write for single files/directories do:

$ chattr +C /dir/file

This will disable copy-on-write for those operation in which there is only one reference to the file. If there is more than one reference (e.g. through cp --reflink=always or because of a filesystem snapshot), copy-on-write still occurs.

Note: From chattr man page: "For btrfs, the 'C' flag should be set on new or empty files. If it is set on a file which already has data blocks, it is undefined when the blocks assigned to the file will be fully stable. If the 'C' flag is set on a directory, it will have no effect on the directory, but new files created in that directory will have the No_COW attribute."
Tip: In accordance with the note above, you can use the following trick to disable copy-on-write on existing files in a directory:
$ mv /path/to/dir /path/to/dir_old
$ mkdir /path/to/dir
$ chattr +C /path/to/dir
$ cp -a /path/to/dir_old/* /path/to/dir
$ rm -rf /path/to/dir_old

Make sure that the data are not used during this process. Also note that mv or cp --reflink as described below will not work.

Creating lightweight copies

By default, when copying files on a Btrfs filesystem with cp, actual copies are created. To create a lightweight copy referencing to the original data, use the reflink option:

$ cp --reflink source dest 

See the man page on cp(1) for more details on the --reflink flag.

Compression

Btrfs supports transparent and automatic compression. This not only reduces the size of files, but can also improve performance, in some specific use cases (e.g. single thread with heavy file I/O), while obviously harming performance in other cases (e.g. multithreaded and/or cpu intensive tasks with large file I/O). Better performance is generally achieved with the fastest compress algorithms, zstd and lzo, and some benchmarks provide detailed comparisons.

The compress=alg mount option enables automatically considering every file for compression, where alg is either zlib, lzo, zstd, or no (for no compression). Using this option, btrfs will check if compressing the first portion of the data shrinks it. If it does, the entire write to that file will be compressed. If it does not, none of it is compressed. With this option, if the first portion of the write does not shrink, no compression will be applied to the write even if the rest of the data would shrink tremendously. [1] This is done to prevent making the disk wait to start writing until all of the data to be written is fully given to btrfs and compressed.

The compress-force=alg mount option can be used instead, which makes btrfs skip checking if compression shrinks the first portion, and enables automatic compression for every file. In a worst case scenario, this can cause (slightly) more space to be used, and CPU usage for no purpose.

Only files created or modified after the mount option is added will be compressed.

To apply compression to existing files, use the btrfs filesystem defragment -calg command, where alg is either zlib, lzo or zstd. For example, in order to re-compress the whole file system with zstd, run the following command:

# btrfs filesystem defragment -r -v -czstd /

To enable compression when installing Arch to an empty Btrfs partition, use the compress option when mounting the file system: mount -o compress=zstd /dev/sdxY /mnt/. During configuration, add compress=zstd to the mount options of the root file system in fstab.

Tip: Compression can also be enabled per-file without using the compress mount option; to do so apply chattr +c to the file. When applied to directories, it will cause new files to be automatically compressed as they come.
Warning:
  • Systems using older kernels or btrfs-progs without zstd support may be unable to read or repair your filesystem if you use this option.
  • GRUB introduced zstd support in 2.04. Make sure you have actually upgraded the bootloader installed in your MBR/ESP since then, by running grub-install with the appropriate options for your BIOS/UEFI setup, since that is not done automatically. See FS#63235.
  • rEFInd before version 0.11.4 lacks support for zstd, either switch to refind-efi-gitAUR, use a separate boot partition without zstd, or reset compression of boot files to something supported using for example the command:
    $ btrfs filesystem defragment -v -clzo /boot/*

View compression types and ratios

compsize takes a list of files (or an entire btrfs filesystem) and measures compression types used and effective compression ratios. Uncompressed size may not match the number given by other programs such as du, because every extent is counted once, even if it is reflinked several times, and even if part of it is no longer used anywhere but has not been garbage collected. The -x option keeps it on a single filesystem, which is useful in situations like compsize -x / to avoid it from attempting to look in non-btrfs subdirectories and fail the entire run.

Subvolumes

"A btrfs subvolume is not a block device (and cannot be treated as one) instead, a btrfs subvolume can be thought of as a POSIX file namespace. This namespace can be accessed via the top-level subvolume of the filesystem, or it can be mounted in its own right." [2]

Each Btrfs file system has a top-level subvolume with ID 5. It can be mounted as / (by default), or another subvolume can be mounted instead. Subvolumes can be moved around in the filesystem and are rather identified by their id than their path.

See the following links for more details:

Creating a subvolume

To create a subvolume:

# btrfs subvolume create /path/to/subvolume

Listing subvolumes

To see a list of current subvolumes under path:

# btrfs subvolume list -p path

Deleting a subvolume

To delete a subvolume:

# btrfs subvolume delete /path/to/subvolume

Attempting to remove the directory /path/to/subvolume without using the above command will not delete the subvolume.

Mounting subvolumes

Subvolumes can be mounted like file system partitions using the subvol=/path/to/subvolume or subvolid=objectid mount flags. For example, you could have a subvolume named subvol_root and mount it as /. One can mimic traditional file system partitions by creating various subvolumes under the top level of the file system and then mounting them at the appropriate mount points. Thus one can easily restore a file system (or part of it) to a previous state using #Snapshots.

Tip: Changing subvolume layouts is made simpler by not using the toplevel subvolume (ID=5) as / (which is done by default). Instead, consider creating a subvolume to house your actual data and mounting it as /.
Note: From btrfs(5): "Most mount options apply to the whole filesystem, and only the options for the first subvolume to be mounted will take effect. This is due to lack of implementation and may change in the future.". See the Btrfs Wiki FAQ for which mount options can be used per subvolume.

See Snapper#Suggested filesystem layout, Btrfs SysadminGuide#Managing Snapshots, and Btrfs SysadminGuide#Layout for example file system layouts using subvolumes.

See btrfs(5) for a full list of btrfs-specific mount options.

Changing the default sub-volume

The default sub-volume is mounted if no subvol= mount option is provided. To change the default subvolume, do:

# btrfs subvolume set-default subvolume-id /

where subvolume-id can be found by listing.

Note: After changing the default subvolume on a system with GRUB, you should run grub-install again to notify the bootloader of the changes. See this forum thread.

Changing the default subvolume with btrfs subvolume set-default will make the top level of the filesystem inaccessible, except by use of the subvol=/ or subvolid=5 mount options [3].

Quota

Warning: Qgroup is not stable yet and combining quota with (too many) snapshots of subvolumes can cause performance problems, for example when deleting snapshots. Plus there are several more known issues.

Quota support in Btrfs is implemented at a subvolume level by the use of quota groups or qgroup: Each subvolume is assigned a quota groups in the form of 0/subvolume_id by default. However it is possible to create a quota group using any number if desired.

To use qgroups you need to enable quota first using

# btrfs quota enable path

From this point onwards newly created subvolumes will be controlled by those groups. In order to retrospectively enable them for already existing subvolumes, enable quota normally, then create a qgroup (quota group) for each of those subvolume using their subvolume_id and rescan them:

# btrfs subvolume list path | cut -d' ' -f2 | xargs -I{} -n1 btrfs qgroup create 0/{} path
# btrfs quota rescan path

Quota groups in Btrfs form a tree hierarchy, whereby qgroups are attached to subvolumes. The size limits are set per qgroup and apply when any limit is reached in tree that contains a given subvolume.

Limits on quota groups can be applied either to the total data usage, un-shared data usage, compressed data usage or both. File copy and file deletion may both affect limits since the unshared limit of another qgroup can change if the original volume's files are deleted and only one copy is remaining. For example a fresh snapshot shares almost all the blocks with the original subvolume, new writes to either subvolume will raise towards the exclusive limit, deletions of common data in one volume raises towards the exclusive limit in the other one.

To apply a limit to a qgroup, use the command btrfs qgroup limit. Depending on your usage either use a total limit, unshared limit (-e) or compressed limit (-c). To show usage and limits for a given path within a filesystem use

# btrfs qgroup show -reF path

Commit interval

The resolution at which data are written to the filesystem is dictated by Btrfs itself and by system-wide settings. Btrfs defaults to a 30 seconds checkpoint interval in which new data are committed to the filesystem. This can be changed by appending the commit mount option in /etc/fstab for the btrfs partition.

LABEL=arch64 / btrfs defaults,noatime,compress=lzo,commit=120 0 0

System-wide settings also affect commit intervals. They include the files under /proc/sys/vm/* and are out-of-scope of this wiki article. The kernel documentation for them resides in Documentation/sysctl/vm.txt.

SSD TRIM

A Btrfs filesystem is able to free unused blocks from an SSD drive supporting the TRIM command.

More information about enabling and using TRIM can be found in Solid State Drives#TRIM.

Usage

Swap file

Swap files in Btrfs are supported since Linux kernel 5.0.[4] The proper way to initialize a swap file is described in Swap file#Swap file creation.

Note: For kernels version 5.0+, Btfrs has native swap file support with some limitations:
Warning: Linux kernels before version 5.0, including linux-lts, do not support swap files. Using a swap file with Btrfs and a kernel prior to 5.0 can lead to file system corruption.

Displaying used/free space

General linux userspace tools such as df will inaccurately report free space on a Btrfs partition. It is recommended to use btrfs filesystem usage to query Btrfs partitions. For example:

# btrfs filesystem usage /
Note: The btrfs filesystem usage command does not currently work correctly with RAID5/RAID6 RAID levels.

See [6] for more information.

Defragmentation

Btrfs supports online defragmentation through the mount option autodefrag, see btrfs(5). To manually defragment your root, use:

# btrfs filesystem defragment -r /

Using the above command without the -r switch will result in only the metadata held by the subvolume containing the directory being defragmented. This allows for single file defragmentation by simply specifying the path.

Defragmenting a file which has a COW copy (either a snapshot copy or one made with cp --reflink or bcp) plus using the -c switch with a compression algorithm may result in two unrelated files effectively increasing the disk usage.

RAID

Btrfs offers native "RAID" for #Multi-device file systems. Notable features which set btrfs RAID apart from mdadm are self-healing redundant arrays and online balancing. See the Btrfs wiki page for more information. The Btrfs sysadmin page also has a section with some more technical background.

Warning: Parity RAID (RAID 5/6) code has multiple serious data-loss bugs in it. See the Btrfs Wiki's RAID5/6 page and a bug report on linux-btrfs mailing list for more detailed information.

Scrub

The Btrfs Wiki Glossary says that Btrfs scrub is "[a]n online filesystem checking tool. Reads all the data and metadata on the filesystem, and uses checksums and the duplicate copies from RAID storage to identify and repair any corrupt data."

Warning: A running scrub process will prevent the system from suspending, see this thread for details.

Start manually

To start a (background) scrub on the filesystem which contains /:

# btrfs scrub start /

To check the status of a running scrub:

# btrfs scrub status /

Start with a service or timer

The btrfs-progs package brings the btrfs-scrub@.timer unit for monthly scrubbing the specified mountpoint. Enable the timer with an escaped path, e.g. btrfs-scrub@-.timer for / and btrfs-scrub@home.timer for /home. You can use systemd-escape -p /path/to/mountpoint to escape the path, see systemd-escape(1) for details.

You can also run the scrub by starting btrfs-scrub@.service (with the same encoded path). The advantage of this over # btrfs scrub is that the results of the scrub will be logged in the systemd journal.

Balance

"A balance passes all data in the filesystem through the allocator again. It is primarily intended to rebalance the data in the filesystem across the devices when a device is added or removed. A balance will regenerate missing copies for the redundant RAID levels, if a device has failed." [7] See Upstream FAQ page.

On a single-device filesystem a balance may be also useful for (temporarily) reducing the amount of allocated but unused (meta)data chunks. Sometimes this is needed for fixing "filesystem full" issues.

# btrfs balance start /
# btrfs balance status /

Snapshots

"A snapshot is simply a subvolume that shares its data (and metadata) with some other subvolume, using btrfs's COW capabilities." See Btrfs Wiki SysadminGuide#Snapshots for details.

To create a snapshot:

# btrfs subvolume snapshot source [dest/]name

To create a readonly snapshot add the -r flag. To create writable version of a readonly snapshot, simply create a snapshot of it.

Note: Snapshots are not recursive. Every nested subvolume will be an empty directory inside the snapshot.

Send/receive

A subvolume can be sent to stdout or a file using the send command. This is usually most useful when piped to a Btrfs receive command. For example, to send a snapshot named /root_backup (perhaps of a snapshot you made of / earlier) to /backup you would do the following:

 # btrfs send /root_backup | btrfs receive /backup

The snapshot that is sent must be readonly. The above command is useful for copying a subvolume to an external device (e.g. a USB disk mounted at /backup above).

You can also send only the difference between two snapshots. For example, if you have already sent a copy of root_backup above and have made a new readonly snapshot on your system named root_backup_new, then to send only the incremental difference to /backup do:

 # btrfs send -p /root_backup /root_backup_new | btrfs receive /backup

Now a new subvolume named root_backup_new will be present in /backup.

See Btrfs Wiki's Incremental Backup page on how to use this for incremental backups and for tools that automate the process.

Deduplication

Using copy-on-write, Btrfs is able to copy files or whole subvolumes without actually copying the data. However whenever a file is altered a new proper copy is created. Deduplication takes this a step further, by actively identifying blocks of data which share common sequences and combining them into an extent with the same copy-on-write semantics.

Tools dedicated to deduplicate a Btrfs formatted partition include duperemoveAUR, bedupAUR and btrfs-dedup. One may also want to merely deduplicate data on a file based level instead using e.g. rmlint or jdupesAUR. For an overview of available features of those programs and additional information have a look at the upstream Wiki entry.

Furthermore Btrfs developers are working on inband (also known as synchronous or inline) deduplication, meaning deduplication done when writing new data to the filesystem. Currently it is still an experiment which is developed out-of-tree. Users willing to test the new feature should read the appropriate kernel wiki page.

Known issues

A few limitations should be known before trying.

Encryption

Btrfs has no built-in encryption support, but this may come in the future. Users can encrypt the partition before running mkfs.btrfs. See dm-crypt/Encrypting an entire system#Btrfs subvolumes with swap.

Existing Btrfs file systems can use something like EncFS or TrueCrypt, though perhaps without some of Btrfs' features.

TLP

Using TLP requires special precautions in order to avoid filesystem corruption. Refer to the according TLP section for more information.

Tips and tricks

Partitionless Btrfs disk

Warning: Most users do not want this type of setup and instead should install Btrfs on a regular partition. Furthermore, GRUB strongly discourages installation to a partitionless disk. Consider using --alloc-start for mkfs.btrfs to give larger space to GRUB.

Btrfs can occupy an entire data storage device, replacing the MBR or GPT partitioning schemes, using subvolumes to simulate partitions. However, using a partitionless setup is not required to simply create a Btrfs filesystem on an existing partition that was created using another method. There are some limitations to partitionless single disk setups:

  • Cannot use different file systems for different mount points.
  • If using a Linux kernel version before 5.0, you cannot use swap area as Btrfs did not support swap files pre-5.0 and there is no place to create swap partition. This also limits the use of hibernation/resume, which needs a swap area to store the hibernation image. This includes the current LTS version of Linux which is on 4.xx.
  • Cannot use UEFI to boot.

To overwrite the existing partition table with Btrfs, run the following command:

# mkfs.btrfs /dev/sdX

For example, use /dev/sda rather than /dev/sda1. The latter would format an existing partition instead of replacing the entire partitioning scheme. Because the root partition is Btrfs, make sure btrfs is compiled into the kernel, or put btrfs into mkinitcpio.conf#MODULES and activate.

Install the boot loader like you would for a data storage device with a Master Boot Record. See Syslinux#Manual install or GRUB/Tips and tricks#Install to partition or partitionless disk. If your kernel does not boot due to Failed to mount /sysroot., please add GRUB_PRELOAD_MODULES="btrfs" in /etc/default/grub and generate the grub configuration (GRUB#Generate the main configuration file).

Ext3/4 to Btrfs conversion

Warning: There are many reports on the btrfs mailing list about incomplete/corrupt/broken conversions. Make sure you have working backups of any data you cannot afford to lose. See Conversion from Ext3 on the btrfs wiki for more information.

Boot from an install CD, then convert by doing:

# btrfs-convert /dev/partition

Mount the partion and test the conversion by checking the files. Be sure to change the /etc/fstab to reflect the change (type to btrfs and fs_passno [the last field] to 0 as Btrfs does not do a file system check on boot). Also note that the UUID of the partition will have changed, so update fstab accordingly when using UUIDs. chroot into the system and rebuild the GRUB menu list (see Install from existing Linux and GRUB articles). If converting a root filesystem, while still chrooted run mkinitcpio -p linux to regenerate the initramfs or the system will not successfully boot. If you get stuck in grub with 'unknown filesystem' try reinstalling grub with grub-install /dev/partition and regenerate the config as well grub-mkconfig -o /boot/grub/grub.cfg.

After confirming that there are no problems, complete the conversion by deleting the backup ext2_saved sub-volume. Note that you cannot revert back to ext3/4 without it.

# btrfs subvolume delete /ext2_saved

Finally balance the file system to reclaim the space.

Remember that some applications which were installed prior have to be adapted to Btrfs. Notably TLP#Btrfs needs special care to avoid filesystem corruption but other applications may profit from certain features as well.

Checksum hardware acceleration

To verify if Btrfs checksum is hardware accelerated:

$ dmesg | grep crc32c
Btrfs loaded, crc32c=crc32c-intel

If you see crc32c=crc32c-generic, it is probably because your root partition is Btrfs, and you will have to compile crc32c-intel into the kernel to make it work. Putting crc32c-intel into mkinitcpio.conf does not work.

Corruption recovery

btrfs-check cannot be used on a mounted file system. To be able to use btrfs-check without booting from a live USB, add it to the initial ramdisk:

/etc/mkinitcpio.conf
BINARIES=("/usr/bin/btrfs")

Regenerate the initramfs.

Then if there is a problem booting, the utility is available for repair.

Note: If the fsck process has to invalidate the space cache (and/or other caches?) then it is normal for a subsequent boot to hang up for a while (it may give console messages about btrfs-transaction being hung). The system should recover from this after a while.

See the Btrfs Wiki page for more information.

Booting into snapshots

In order to boot into a snapshot you must specify the subvolume via a kernel parameter using rootflags=subvol=/path/to/subvolume and alter your /etc/fstab to point to the same subvolume using subvol=. Alternatively the subvolume can be specified with its id - retrievable with e.g. btrfs subvolume list /root/path - and rootflags=subvolid=objectid as kernel parameter respectively subvolid=objectid as mount option in /etc/fstab.

If using GRUB you can automatically populate your boot menu with btrfs snapshots when regenerating the configuration file with the help of grub-btrfs or grub-btrfs-gitAUR.

Use Btrfs subvolumes with systemd-nspawn

See the Systemd-nspawn#Use Btrfs subvolume as container root and Systemd-nspawn#Use temporary Btrfs snapshot of container articles.

Troubleshooting

See the Btrfs Problem FAQ for general troubleshooting.

GRUB

Partition offset

The offset problem may happen when you try to embed core.img into a partitioned disk. It means that it is OK to embed grub's core.img into a Btrfs pool on a partitionless disk (e.g. /dev/sdX) directly.

GRUB can boot Btrfs partitions, however the module may be larger than other file systems. And the core.img file made by grub-install may not fit in the first 63 sectors (31.5KiB) of the drive between the MBR and the first partition. Up-to-date partitioning tools such as fdisk and gdisk avoid this issue by offsetting the first partition by roughly 1MiB or 2MiB.

Missing root

Tango-inaccurate.pngThe factual accuracy of this article or section is disputed.Tango-inaccurate.png

Reason: Suggests editing a non-configuration file manually. (Discuss in Talk:Btrfs#Should not suggest to edit files in /usr/share)

Users experiencing the following: error no such device: root when booting from a RAID style setup then edit /usr/share/grub/grub-mkconfig_lib and remove both quotes from the line echo " search --no-floppy --fs-uuid --set=root ${hints} ${fs_uuid}". Regenerate the config for grub and the system should boot without an error.

BTRFS: open_ctree failed

As of November 2014 there seems to be a bug in systemd or mkinitcpio causing the following error on systems with multi-device Btrfs filesystem using the btrfs hook in mkinitcpio.conf:

BTRFS: open_ctree failed
mount: wrong fs type, bad option, bad superblock on /dev/sdb2, missing codepage or helper program, or other error

In some cases useful info is found in syslog - try dmesg|tail or so.

You are now being dropped into an emergency shell.

A workaround is to remove btrfs from the HOOKS array in /etc/mkinitcpio.conf and instead add btrfs to the MODULES array. Then regenerate the initramfs with mkinitcpio -p linux (adjust the preset if needed) and reboot.

You will get the same error if you try to mount a raid array without one of the devices. In that case you must add the degraded mount option to /etc/fstab. If your root resides on the array, you must also add rootflags=degraded to your kernel parameters.

As of August 2016, a potential workaround for this bug is to mount the array by a single drive only in /etc/fstab, and allow btrfs to discover and append the other drives automatically. Group-based identifiers such as UUID and LABEL appear to contribute to the failure. For example, a two-device RAID1 array consisting of 'disk1' and disk2' will have a UUID allocated to it, but instead of using the UUID, use only /dev/mapper/disk1 in /etc/fstab. For a more detailed explanation, see the following blog post.

Another possible workaround is to remove the udev hook in mkinitcpio.conf and replace it with the systemd hook. In this case, btrfs should not be in the HOOKS or MODULES arrays.

See the original forums thread and FS#42884 for further information and discussion.

btrfs check

Warning: Since Btrfs is under heavy development, especially the btrfs check command, it is highly recommended to create a backup and consult the following Btfrs documentation before executing btrfs check with the --repair switch.

The btrfs check command can be used to check or repair an unmounted Btrfs filesystem. However, this repair tool is still immature and not able to repair certain filesystem errors even those that do not render the filesystem unmountable.

See Btrfsck for more information.

See also