Difference between revisions of "Btrfs"

From ArchWiki
Jump to: navigation, search
(Resources)
(Creating a Btrfs Partition: add warning about grub core.img size)
(42 intermediate revisions by 20 users not shown)
Line 1: Line 1:
 
[[Category:File systems]]
 
[[Category:File systems]]
{{i18n|Btrfs}}
+
[[ja:Btrfs]]
 
+
[[zh-CN:Btrfs]]
 
{{Article summary start}}
 
{{Article summary start}}
 
{{Article summary text|Provides an overview and setup of Btrfs on Arch Linux.}}
 
{{Article summary text|Provides an overview and setup of Btrfs on Arch Linux.}}
Line 8: Line 8:
 
{{Article summary end}}
 
{{Article summary end}}
  
Btrfs is a new copy on write (COW) filesystem for Linux aimed at implementing advanced features while focusing on fault tolerance, repair and easy administration. Jointly developed at Oracle, Red Hat, Fujitsu, Intel, SUSE and many others, Btrfs s licensed under the GPL and open for contribution from anyone.
+
Btrfs is a new copy-on-write (COW) filesystem for Linux aimed at implementing advanced features while focusing on fault tolerance, repair and easy administration. Jointly developed by Oracle, Red Hat, Fujitsu, Intel, SUSE and many others, Btrfs is licensed under the GPL and open for contribution from anyone.
  
 
== Recent Developments and News Links ==
 
== Recent Developments and News Links ==
 +
*[http://www.phoronix.com/scan.php?page=news_item&px=MTA0ODU Avi Miller presenting BTRFS] at SCALE 10x.  Jan/2012.
 
*[http://www.phoronix.com/scan.php?page=news_item&px=MTA4Mzc Summary of Chris Mason's talk from LFCS 2012]
 
*[http://www.phoronix.com/scan.php?page=news_item&px=MTA4Mzc Summary of Chris Mason's talk from LFCS 2012]
*On 28-Mar-2012, {{Pkg|btrfs-progs}} includes btrfsck, '''a tool that can fix errors on btrfs filesystems.'''
+
*On 2012-03-28, {{Pkg|btrfs-progs}} includes btrfsck, a tool that can fix errors on btrfs filesystems.
*Oracle has packaged this version of btrfs-progs and released it to their customers Oracle Linux 6 and backported to 5.
+
*Oracle has packaged this version of btrfs-progs and released it to their customers of Oracle Linux 6 and backported to 5.
 
*Arch Linux supplies this version in core/btrfs-progs (since version 0.19.20120328-1).
 
*Arch Linux supplies this version in core/btrfs-progs (since version 0.19.20120328-1).
  
 
== Installation ==
 
== Installation ==
  
Btrfs support is included in the the core/linux package (hardcoded into the kernel). User space utils are available from [core]:
+
Btrfs support is included in the {{pkg|linux}} package (as a module). User space utilities are available in {{pkg|btrfs-progs}}.
  
# pacman -S btrfs-progs
+
For multi-devices support (RAID like feature of btrfs) aka '''btrfs volume''' in early boot, you have to enable ''btrfs'' mkinitcpio hook (provided by mkinitcpio package) to be able to use, for example, a root btrfs volume. However, if you only use a bare btrfs partition, this hook is not necessary.
  
If using btrfs as the root filesystem, users ''may'' want to install {{AUR|mkinitcpio-btrfs}} from AUR. This package will install a mkinitcpio hook intended for those who wish to have a single or multi-drive BTRFS file system as their / (root)The hook will ensure that the chosen root device from the kernel command line is intact and safe to bootIf root is not a BTRFS device, the hook is quietly skipped.
+
For more options/features, users ''may'' want to install {{AUR|mkinitcpio-btrfs}} from the [[Arch User Repository|AUR]]. This package will install a mkinitcpio hook called ''btrfs_advanced'' intended for those who wish to have automatic rollback feature.
 +
 
 +
== Creating a Btrfs Partition ==
 +
{{Warning|[[GRUB|Grub 2]] can boot btrfs partitions, but the module is larger than e.g. ext4This means that the core.img file made by grub-install might not fit between the MBR and the first partitionYou can solve this by using GPT or simply putting an extra 1 or 2 MB of free space before the first partion.}}
  
== Basic Use ==
 
 
=== Format a New Partition to Btrfs ===
 
=== Format a New Partition to Btrfs ===
  
 
  # mkfs.btrfs [options] dev [dev ...]
 
  # mkfs.btrfs [options] dev [dev ...]
  
One can select multiple devices to create a raid. Supported raids include raid1, raid0, and raid 10. By default, metadata is mirrored and data is striped.
+
One can select multiple devices to create a RAID. Supported RAID levels include RAID 0, RAID 1 and RAID 10. By default, metadata is mirrored and data is striped.
  
 
=== Convert Ext3/4 to Btrfs ===
 
=== Convert Ext3/4 to Btrfs ===
{{Warning|Grub cannot boot btrfs root. Users need to install either [[Grub2]] or [[Syslinux]]This guide assumes users are aware of this limitation to Grub.}}
+
{{Warning|[[GRUB Legacy]] cannot boot with btrfs as root. Users need to install either [[GRUB]] or [[Syslinux]]. This guide assumes users are aware of this limitation.}}
  
 
# Boot a live CD (Arch for example)
 
# Boot a live CD (Arch for example)
# Enable [remote-core] and [remote-testing]
+
# Run {{ic|btrfs-convert <root-partition>}}
# Setup the network
+
# Mount the converted partition and modify the {{ic|/etc/fstab}} file specifying either {{ic|auto}} or {{ic|btrfs}} for the partition type. You should also set the {{ic|fs_passno}} parameter (the last field) to {{ic|0}} since btrfs does not require a filesystem check on every boot.
# modprobe btrfs
+
# Chroot into the system and rebuild the GRUB entry (see [[Install from Existing Linux]] and [[GRUB]] articles, if unfamiliar with this procedure).
# Install btrfs-progs (make sure versions of dependencies match: glibc,e2fsprogs)
+
 
# Run btrfs-convert
+
After <tt>btrfs-convert</tt>, and confirming everything works, if you want to make the change permanent, you need to delete the saved image, delete the subvolume that image is on, then balance the drive to reclaim the space. For example, after converting <tt>/home</tt> to btrfs:
# Mount the converted partition and modify the /etc/fstab specifying either 'auto' or 'btrfs' for the partition type.
+
 
# Chroot into the system and rebuild the grub2 entry (see [[Install from Existing Linux]] and [[Grub2]] articles if unfamiliar with this procedure.
+
rm /home/ext2_saved/*
 +
btrfs subvolume delete /home/ext2_saved
 +
 
 +
=== Encryption ===
 +
Btrfs has no built-in encryption support (this may come in future), but you can encrypt the partition before running <code>mkfs.btrfs</code>. See [[Dm-crypt with LUKS]].
 +
 
 +
(If you've already created a btrfs file system, you can also use something like [[EncFS]] or [[TrueCrypt]], though perhaps without some of btrfs' features.)
 +
 
 +
== Btrfs Features ==
 +
 
 +
=== Copy-On-Write (CoW) ===
 +
CoW comes with some advantages, but can negatively affect performance with large files that have small random writes. It is recomended to disable CoW for database files and virtual machine images.
 +
You can disable CoW for the entire block device by mounting it with "nodatacow" option. However, this will disable CoW for the entire file system.
 +
To disable CoW for single files/directories, use the following command:
 +
# chattr +C [file/directory path]
 +
Note, from chattr man page: For btrfs, the 'C' flag should be set on new or empty files.  If it is set on a file which already has data blocks, it is undefined when the blocks assigned to the file will be fully stable. If the 'C' flag is set on a directory, it will have no effect on the directory, but new files created in that directory will have the No_COW attribute.
 +
 
 +
=== Multi-device filesystem and RAID feature ===
 +
====Multi-device filesystem====
 +
When creating a ''btrfs'' filesystem, you can pass as many partitions or disk devices as you want to ''mkfs.btrfs''. The filesystem will be created across these devices. You can '''"'''merge'''"''' this way, multiple partitions or devices to get a big ''btrfs'' filesystem.
 +
 
 +
You can also add or remove device from an existing btrfs filesystem (caution is mandatory).
 +
 
 +
A multi-device ''btrfs'' filesystem (also called a btrfs volume) is not recognized until
 +
  # btrfs device scan
 +
has been run. This is the purpose of the ''btrfs'' mkinitcpio hook or the ''USEBTRFS'' variable in /etc/rc.conf
 +
 
 +
====RAID features====
 +
When creating multi-device filesystem, you can also specify to use RAID0, RAID1 or RAID10 across the devices you have added to the filesystem.
  
 
=== Subvolumes ===
 
=== Subvolumes ===
  
One of the features of btrfs is the use of subvolumes. Subvolumes are basically a named btree that holds files and directories. They have inodes inside the tree of tree roots and can have non-root owners and groups. Subvolumes can optionally be given a quota of blocks.  All of the blocks and file extents inside of subvolumes are reference counted to allow snapshotting. Similar to the dynamically expanding storage of a virtual machine that will only use as much space on a device as needed. Eliminating several half-filled partitions.  One can also mount the subvolumes with different mount options giving more flexibility in security.  
+
One of the features of btrfs is the use of subvolumes. Subvolumes are basically a named btree that holds files and directories. They have inodes inside the tree of tree roots and can have non-root owners and groups. Subvolumes can optionally be given a quota of blocks.  All of the blocks and file extents inside of subvolumes are reference counted to allow snapshotting. This is similar to the dynamically expanding storage of a virtual machine that will only use as much space on a device as needed, eliminating several half-filled partitions.  One can also mount the subvolumes with different mount options, giving more flexibility in security.  
  
 
To create a subvolume:
 
To create a subvolume:
Line 51: Line 82:
 
  # btrfs subvolume create [<dest>/]
 
  # btrfs subvolume create [<dest>/]
  
For increased flexibility, install your system INTO A DEDICATED SUBVOLUME, and use:
+
For increased flexibility, install your system into a dedicated subvolume, and, in the kernel boot parameters, use:
  
 
{{bc|1=rootflags=subvol=<whatever you called the subvol>}}
 
{{bc|1=rootflags=subvol=<whatever you called the subvol>}}
  
In the kernel boot parameters. It makes system rollbacks possible.
+
This makes system rollbacks possible.
  
If using for the root partition, it is advisable to add '''crc32c''' to the modules array in {{ic|/etc/mkinitcpio.conf}} as well as adding {{ic|btrfs}} to the HOOKS array in {{ic|/etc/mkinitcpio.conf}}.
+
If using for the root partition, it is advisable to add '''crc32c''' (or '''crc32c-intel''' for Intel machines) to the modules array in {{ic|/etc/mkinitcpio.conf}} as well as adding {{ic|btrfs}} to the HOOKS.
  
 
=== Snapshots ===
 
=== Snapshots ===
Line 67: Line 98:
 
Snapshots are not recursive, this means that every subvolume inside subvolume will be an empty directory inside the snapshot.
 
Snapshots are not recursive, this means that every subvolume inside subvolume will be an empty directory inside the snapshot.
  
== Defragmentation ==
+
=== Defragmentation ===
 
Btrfs supports online defragmentation. To defragment the metadata of the root folder, simply do:
 
Btrfs supports online defragmentation. To defragment the metadata of the root folder, simply do:
 
  # btrfs filesystem defragment /
 
  # btrfs filesystem defragment /
This ''will not'' defragment the entire system. For more information, see [https://btrfs.wiki.kernel.org/articles/p/r/o/Problem_FAQ_68af.html#Defragmenting_a_directory_doesn.27t_work this page] on the btrfs wiki.
+
This ''will not'' defragment the entire system. For more information, see [https://btrfs.wiki.kernel.org/index.php/Problem_FAQ#Defragmenting_a_directory_doesn.27t_work this page] on the btrfs wiki.
 +
To defragment the entire system verbosely, try instead:
 +
# find / -xdev -type f -print -exec btrfs filesystem defrag '{}' \;
  
== Compression ==
+
=== Compression ===
Btrfs supports transparent compression, which means every file on the partition is automatically compressed. This does not only reduce the size of those files, but also [http://www.phoronix.com/scan.php?page=article&item=btrfs_compress_2635&num=1 improves performance], in particular if using the [http://www.phoronix.com/scan.php?page=article&item=btrfs_lzo_2638&num=1 lzo algorithm]. Compression is enabled using the compress=gzip or compress=lzo mount options. Only files created or modified after the mount option is added will be compressed, so to fully benefit from compression it should be enabled during installation. After [[Beginners%27_Guide#Prepare_hard_drive|preparing the hard drive]], simply switch to another terminal (Ctrl+Alt+number), and run the following command:
+
Btrfs supports transparent compression, which means every file on the partition is automatically compressed. This does not only reduce the size of those files, but also [http://www.phoronix.com/scan.php?page=article&item=btrfs_compress_2635&num=1 improves performance], in particular if using the [http://www.phoronix.com/scan.php?page=article&item=btrfs_lzo_2638&num=1 lzo algorithm]. Compression is enabled using the {{ic|1=compress=gzip}} or {{ic|1=compress=lzo}} mount options. Only files created or modified after the mount option is added will be compressed, so to fully benefit from compression it should be enabled during installation. After [[Beginners%27_Guide#Prepare_the_storage_drive|preparing the storage drive]], simply switch to another terminal ({{keypress|Ctrl+Alt+number}}), and run the following command:
  # mount -o remount,compress=lzo /dev/partition /mnt/target # note: replace /dev/partition by the partition on which Arch Linux is installed.
+
  # mount -o remount,compress=lzo /dev/sdXY /mnt/target
  
Verify if compression is enabled with the mount command. After the installation is finished, add compress=lzo to the mount options of the root filesystem in {{ic|/etc/fstab}}.
+
After the installation is finished, add {{ic|1=compress=lzo}} to the mount options of the root filesystem in {{ic|/etc/[[fstab]]}}.
  
 
== Resources ==
 
== Resources ==
 
* [https://btrfs.wiki.kernel.org/ Btrfs Wiki]
 
* [https://btrfs.wiki.kernel.org/ Btrfs Wiki]
 
* [https://btrfs.wiki.kernel.org/index.php/Problem_FAQ BTRFS Problem FAQ] - Official FAQ
 
* [https://btrfs.wiki.kernel.org/index.php/Problem_FAQ BTRFS Problem FAQ] - Official FAQ
* [http://www.funtoo.org/wiki/BTRFS_Fun Funtoo Btrfs wiki entry] - Very well-written albeit slightly out dated article.
+
* [http://www.funtoo.org/wiki/BTRFS_Fun Funtoo Btrfs wiki entry] - Very well-written article

Revision as of 21:51, 25 February 2013

Template:Article summary start Template:Article summary text Template:Article summary heading Template:Article summary wiki Template:Article summary end

Btrfs is a new copy-on-write (COW) filesystem for Linux aimed at implementing advanced features while focusing on fault tolerance, repair and easy administration. Jointly developed by Oracle, Red Hat, Fujitsu, Intel, SUSE and many others, Btrfs is licensed under the GPL and open for contribution from anyone.

Recent Developments and News Links

Installation

Btrfs support is included in the linux package (as a module). User space utilities are available in btrfs-progs.

For multi-devices support (RAID like feature of btrfs) aka btrfs volume in early boot, you have to enable btrfs mkinitcpio hook (provided by mkinitcpio package) to be able to use, for example, a root btrfs volume. However, if you only use a bare btrfs partition, this hook is not necessary.

For more options/features, users may want to install mkinitcpio-btrfsAUR from the AUR. This package will install a mkinitcpio hook called btrfs_advanced intended for those who wish to have automatic rollback feature.

Creating a Btrfs Partition

Warning: Grub 2 can boot btrfs partitions, but the module is larger than e.g. ext4. This means that the core.img file made by grub-install might not fit between the MBR and the first partition. You can solve this by using GPT or simply putting an extra 1 or 2 MB of free space before the first partion.

Format a New Partition to Btrfs

# mkfs.btrfs [options] dev [dev ...]

One can select multiple devices to create a RAID. Supported RAID levels include RAID 0, RAID 1 and RAID 10. By default, metadata is mirrored and data is striped.

Convert Ext3/4 to Btrfs

Warning: GRUB Legacy cannot boot with btrfs as root. Users need to install either GRUB or Syslinux. This guide assumes users are aware of this limitation.
  1. Boot a live CD (Arch for example)
  2. Run btrfs-convert <root-partition>
  3. Mount the converted partition and modify the /etc/fstab file specifying either auto or btrfs for the partition type. You should also set the fs_passno parameter (the last field) to 0 since btrfs does not require a filesystem check on every boot.
  4. Chroot into the system and rebuild the GRUB entry (see Install from Existing Linux and GRUB articles, if unfamiliar with this procedure).

After btrfs-convert, and confirming everything works, if you want to make the change permanent, you need to delete the saved image, delete the subvolume that image is on, then balance the drive to reclaim the space. For example, after converting /home to btrfs:

rm /home/ext2_saved/*
btrfs subvolume delete /home/ext2_saved

Encryption

Btrfs has no built-in encryption support (this may come in future), but you can encrypt the partition before running mkfs.btrfs. See Dm-crypt with LUKS.

(If you've already created a btrfs file system, you can also use something like EncFS or TrueCrypt, though perhaps without some of btrfs' features.)

Btrfs Features

Copy-On-Write (CoW)

CoW comes with some advantages, but can negatively affect performance with large files that have small random writes. It is recomended to disable CoW for database files and virtual machine images. You can disable CoW for the entire block device by mounting it with "nodatacow" option. However, this will disable CoW for the entire file system. To disable CoW for single files/directories, use the following command:

# chattr +C [file/directory path]

Note, from chattr man page: For btrfs, the 'C' flag should be set on new or empty files. If it is set on a file which already has data blocks, it is undefined when the blocks assigned to the file will be fully stable. If the 'C' flag is set on a directory, it will have no effect on the directory, but new files created in that directory will have the No_COW attribute.

Multi-device filesystem and RAID feature

Multi-device filesystem

When creating a btrfs filesystem, you can pass as many partitions or disk devices as you want to mkfs.btrfs. The filesystem will be created across these devices. You can "merge" this way, multiple partitions or devices to get a big btrfs filesystem.

You can also add or remove device from an existing btrfs filesystem (caution is mandatory).

A multi-device btrfs filesystem (also called a btrfs volume) is not recognized until

 # btrfs device scan

has been run. This is the purpose of the btrfs mkinitcpio hook or the USEBTRFS variable in /etc/rc.conf

RAID features

When creating multi-device filesystem, you can also specify to use RAID0, RAID1 or RAID10 across the devices you have added to the filesystem.

Subvolumes

One of the features of btrfs is the use of subvolumes. Subvolumes are basically a named btree that holds files and directories. They have inodes inside the tree of tree roots and can have non-root owners and groups. Subvolumes can optionally be given a quota of blocks. All of the blocks and file extents inside of subvolumes are reference counted to allow snapshotting. This is similar to the dynamically expanding storage of a virtual machine that will only use as much space on a device as needed, eliminating several half-filled partitions. One can also mount the subvolumes with different mount options, giving more flexibility in security.

To create a subvolume:

# btrfs subvolume create [<dest>/]

For increased flexibility, install your system into a dedicated subvolume, and, in the kernel boot parameters, use:

rootflags=subvol=<whatever you called the subvol>

This makes system rollbacks possible.

If using for the root partition, it is advisable to add crc32c (or crc32c-intel for Intel machines) to the modules array in /etc/mkinitcpio.conf as well as adding btrfs to the HOOKS.

Snapshots

To create a snapshot:

# btrfs subvolume snapshot <source> [<dest>/]<name>

Snapshots are not recursive, this means that every subvolume inside subvolume will be an empty directory inside the snapshot.

Defragmentation

Btrfs supports online defragmentation. To defragment the metadata of the root folder, simply do:

# btrfs filesystem defragment /

This will not defragment the entire system. For more information, see this page on the btrfs wiki. To defragment the entire system verbosely, try instead:

# find / -xdev -type f -print -exec btrfs filesystem defrag '{}' \;

Compression

Btrfs supports transparent compression, which means every file on the partition is automatically compressed. This does not only reduce the size of those files, but also improves performance, in particular if using the lzo algorithm. Compression is enabled using the compress=gzip or compress=lzo mount options. Only files created or modified after the mount option is added will be compressed, so to fully benefit from compression it should be enabled during installation. After preparing the storage drive, simply switch to another terminal (Template:Keypress), and run the following command:

# mount -o remount,compress=lzo /dev/sdXY /mnt/target

After the installation is finished, add compress=lzo to the mount options of the root filesystem in /etc/fstab.

Resources