Difference between revisions of "Bcache"

From ArchWiki
Jump to: navigation, search
m (Update ESP link.)
 
(72 intermediate revisions by 20 users not shown)
Line 1: Line 1:
 
[[Category:File systems]]
 
[[Category:File systems]]
==Introduction==
+
{{Style|Some first-person comments, see [[Help:Style]].}}
  
Bcache allows one to use an SSD as a read/write cache (in writeback mode) or read cache (writethrough or writearound) for another blockdevice (generally a rotating HDD or array. This article will show how to install arch using Bcache as the root partition. For an intro to bcache itself, see [http://bcache.evilpiepirate.org/ the bcache homepage]. Be sure to read and reference [http://atlas.evilpiepirate.org/git/linux-bcache.git/tree/Documentation/bcache.txt?h=bcache the bcache manual]. Bcache is currently [http://lkml.indiana.edu/hypermail/linux/kernel/1301.1/03583.html in the process] of getting merged into the kernel.
+
Bcache allows one to use an SSD as a read/write cache (in writeback mode) or read cache (writethrough or writearound) for another blockdevice (generally a rotating HDD or array). This article will show how to install arch using Bcache as the root partition. For an intro to bcache itself, see [http://bcache.evilpiepirate.org/ the bcache homepage]. Be sure to read and reference [http://atlas.evilpiepirate.org/git/linux-bcache.git/tree/Documentation/bcache.txt the bcache manual]. Bcache is in the mainline kernel since 3.10. The kernel on the arch install disk includes the bcache module since 2013.08.01.
  
An alternative to Bcache is Facebook's [[Flashcache]]
+
An alternative to Bcache is Facebook's [[Flashcache]] and its offspring [[EnhanceIO]]. Another alternative - {{AUR|btier-dkms}}{{Broken package link|{{aur-mirror|btier-dkms}}}}.
  
{{Warning|Until bcache patches are accepted into mainline, installation is non-trivial. Follow the instructions carefully and read the upstream bcache documentation. Commands are included as examples only, and may differ slightly on your setup.}}
+
{{Accuracy|Backwards incompatible changes to on-disk format for linux 3.18 appears to be false. bcache on-disk format changes have not yet landed upstream yet as of 3.19}}
  
{{Warning|Bcache requires the backing device is formatted as a bcache block device. This is a destructive processes, so be sure you back up any important data first.}}
+
Bcache needs the backing device to be formatted as a bcache block device. In most cases, [https://github.com/g2p/blocks blocks to-bcache] can do an in-place conversion.
 +
{{Warning|
 +
* Be sure you back up any important data first.
 +
* The bcache-dev branch is under heavy development. The on-disk format has undergone changes in 3.18 which are not backwards compatible with previous formats[http://www.spinics.net/lists/linux-bcache/msg02721.html]. Note: This only applies to users who compile-in bcache-dev. The version built-in to the upstream Linux kernel is unaffected[http://www.spinics.net/lists/linux-bcache/msg02724.html].
 +
* Bcache and [[btrfs]] could leave you with a corrupted filesystem. Please visit [https://www.hdevalence.ca/blog/2013-09-21-notes-on-my-archlinux-install this post] for more information. Btrfs wiki reports that it was fixed in kernels 3.19+ [https://btrfs.wiki.kernel.org/index.php/Gotchas#Historical_references].
 +
}}
  
==Temporary install to SSD==
+
== Setting up a bcache device on an existing system ==
  
The ArchLinux install disk does not include bcache in its kernel. You could either re-master to the install disk to include this kernel, or do the install twice. The example below is for installing twice.
+
1. Install the {{AUR|bcache-tools}} package from [[AUR]].
  
# Install archlinux [[Installation_Guide|as normal]] to the SSD. Don't go crazy with partitions, just a / and an EFI system partition if you're not booting bios mode. This is just the temporary system we'll use to install the final system.
+
2. Create a backing device (This will typically be your mechanical drive). The backing device can be a whole device, a partition or any other standard block device. This will create /dev/bcache0
# Reboot to verify the install in step 1 worked.
+
    # make-bcache -B /dev/sdx1
# Install the "linux-bcache-git" and "bcache-tools-git" packages from [[AUR]].
+
# Reboot to the bcache enabled kernel
+
  
{{Note| The PKGBUILD for the aur/linux-bcache-git package can be edited to build a v3.2 Linux kernel. By default it builds the latest (currently 3.8).}}
+
3. Create a cache device (This will typically be your SSD). The cache device can be a whole device, a partition or any other standard block device
 +
    # make-bcache -C /dev/sdy2
 +
In this example the default block and bucket sizes of 512B and 128kB are used. The block size should match the backing devices sector size which will usually be either 512 or 4k. The bucket size should match the erase block size of the caching device with the intent of reducing write amplification. For example, using a HDD with 4k sectors and an SSD with an erase block size of 2MB this command would look like
 +
    # make-bcache --block 4k --bucket 2M -C /dev/sdy2
  
== Final install to the HDD ==
+
4. Register the cache device against your backing device. To find its ''cache set UUID'', run {{ic|# bcache-super-show /dev/sdy2 | grep cset.uuid}} and then add it to the bcache device initially. Udev rules will take care of this on reboot and will only need to be done once.
 +
    # echo '''cset.uuid''' > /sys/block/bcache0/bcache/attach
  
1. Partition your hdd<br>
+
5. Change your cache mode (if you want to cache writes as well as reads):
grub can't handle bcache, so you'll need at least 2 partitions (boot and one for the bcache backing device). If you're doing UEFI, you'll need an EFI System Partition (ESP) as well.
+
    # echo writeback > /sys/block/bcache0/bcache/cache_mode
  ex:
+
 
 +
6. If you want to have this partition available during the initcpio (i.e. you require it at some point in the boot process) you need to add 'bcache' to your modules array in /etc/mkinitcpio.conf as well as adding the 'bcache' hook in your list between block and filesystems. You must then rebuild the initramfs image. This is typically done with
 +
 
 +
    # mkinitcpio -p linux
 +
 
 +
=== Bcache management ===
 +
1. Check that everything has been correctly setup
 +
 
 +
    # cat /sys/block/bcache0/bcache/state
 +
 
 +
The output can be:
 +
* '''no cache''': this means you have not attached a caching device to your backing bcache device
 +
* '''clean''': this means everything is ok. The cache is clean.
 +
* '''dirty''': this means everything is setup fine and that you have enabled ''writeback'' and that the cache is dirty.
 +
* '''inconsistent''': you are in trouble because the backing device is not in sync with the caching device
 +
 
 +
You can have a {{ic|/dev/bcache0}} device associated with a backing device with no caching device attached. This means that all I/O (read/write) are passed directly to the backing device (pass-through mode)
 +
 
 +
2. See what caching mode is in use
 +
{{bc|# cat /sys/block/bcache0/bcache/cache_mode
 +
[writethrough] writeback writearound none
 +
}}
 +
In the above example, the ''writethrough'' mode is enabled.
 +
 
 +
3. Show info about a bcached device:
 +
 
 +
    # bcache-super-show /dev/sdXY
 +
 
 +
4. Stop the backing device:
 +
 
 +
    # echo 1 > /sys/block/sdX/sdX[Y]/bcache/stop
 +
 
 +
5. Detach a caching device:
 +
 
 +
    # echo 1 > /sys/block/sdX/sdX[Y]/bcache/detach
 +
 
 +
6. Safely remove the cache device
 +
    # echo <cache-set-uuid> > /sys/block/bcache0/bcache/detach
 +
 
 +
7. Release attached devices
 +
    # echo 1 > /sys/fs/bcache/<cache-set-uuid>/stop
 +
 
 +
=== Bcache on top of another block layer ===
 +
{{Accuracy| it is strongly advises to make Bcache first, underneath any other block layer}}
 +
{{Warning|
 +
* it is widely recommended to use Bcache underneath any other block layer.
 +
* the '''blocks''' package is broken with Python3. Please visit [https://bugs.debian.org/cgi-bin/bugreport.cgi?bug&#61;769737 this thread] for a workaround, or use [[Python VirtualEnv]].
 +
* when on top of a '''LVM2''' layer, the ''discard'' mount option may not be handled well.
 +
}}
 +
With {{ic|blocks}}, you can add a Bcache layer on a device with an existing filesystem or layer (normal partition, LUKS partition, logical volume).
 +
 
 +
First, install the {{AUR|blocks-git}}{{Broken package link|{{aur-mirror|blocks-git}}}} package from [[AUR]].
 +
 
 +
''blocks'' needs to add the '''bcache''' metadata at the start of the targeted partition, here {{ic|/dev/sdz4}}; to be able to do so, it will resize the partition just before the targeted one ({{ic|/dev/sdz3}}) to make necessary room, typically 2048 bits (2k). It will ask for your passphrase if the partition({{ic|/dev/sdz3}}) is an LUKS partition. Then it will recreate another partition with the start just lower of 2048 bits and run ''make-bcache'', so you do not need to run it yourself.
 +
 
 +
    # blocks to-bcache /dev/sdz4
 +
 
 +
You can check that everything is ok by running:
 +
 
 +
    # bcache-super-show /dev/sdz4
 +
 
 +
which should output metadata about the new backing '''bcache''' device
 +
 
 +
You can proceed to create the caching device and attach it to the backing device.
 +
 
 +
==== Troubleshooting ====
 +
If {{ic|blocks}} complains complains about ''overlapping metadata'' when running {{ic|blocks to-bcache}}, you will need to shrink by 100Mb the partition you want to cache and create a new 100Mb free partition.
 +
 
 +
== Installation to a bcache device ==
 +
 
 +
1. Boot on the install disk (2013.08.01 minimum).
 +
 
 +
2. Install the {{AUR|bcache-tools}} package from [[AUR]].
 +
 
 +
3. Partition your hdd
 +
 
 +
{{Note|While it may be true that Grub2 does not offer support for bcache as noted below, it does, however, fully support UEFI. It follows then, that so long as the necessary modules for the linux kernel to properly handle your boot device are either compiled into the kernel or are included in an initramfs, and you can include these files on it, the separate boot partition described below may be omitted in favor of the FAT EFI system partition. See [[GRUB]] and/or [[UEFI]] for more.}}
 +
grub cannot handle bcache, so you will need at least 2 partitions (boot and one for the bcache backing device). If you are doing UEFI, you will need an [[EFI System Partition]] (ESP) as well. E.g.:
 
       1            2048          22527  10.0 MiB    EF00  EFI System
 
       1            2048          22527  10.0 MiB    EF00  EFI System
 
       2          22528          432127  200.0 MiB  8300  arch_boot
 
       2          22528          432127  200.0 MiB  8300  arch_boot
 
       3          432128      625142414  297.9 GiB  8300  bcache_backing
 
       3          432128      625142414  297.9 GiB  8300  bcache_backing
{{Note|This example has no swapfile/partition. Keep in mind swapfile on btrfs does not work. For a swap partition on the cache, use LLVM in step 7. For a swap partition outside the cache, be sure to make a swap partition now.}}
 
  
2. Configure your HDD as a bcache backing device.
+
{{Note|This example has no swapfile/partition. For a swap partition on the cache, use LVM in step 7. For a swap partition outside the cache, be sure to make a swap partition now.}}
  # make-bcache -B /dev/sda3
+
{{Note|[[GRUB2]] either needs an EFI partition, "bios boot" partition, or an 'embedding area' of 2KB or so after the MBR that doesn't get overwritten. Also, since GRUB2 does not know about bcache, so you also need a /boot partition that grub can read. Partitioning /dev/sda is a must unless you're booting from a different device.}}
+
  
3. Register any bcache devices with the kernel (this needs to done every bootup)
+
4. Configure your HDD as a bcache backing device.
  for i in /dev/sd*; do echo $i; echo $i > /sys/fs/bcache/register_quiet; done
+
You now have a /dev/bcache0 device
+
{{Note|the bcache user manual says to do "echo /dev/sd* > /sys/fs/bcache/register_quiet", but this didn't work for me}}
+
  
7. Format the bcache device. Use LLVM or btrfs subvolumes if you want to divide up the /dev/bcache0 device how you like (ex for seperate /, /home, /var, etc).
+
# make-bcache -B /dev/sda3
  # mkfs.btrfs /dev/bcache0
+
  # mount /dev/bcache0 /mnt/
+
  # btrfs subvolume create /mnt/root
+
  # btrfs subvolume create /mnt/home
+
  # umount /mnt
+
  
8. Reinstall to the bcache volume (alternatively you can rsync the install you made on the SSD)
+
{{Note|
  # mkfs.ext4 /dev/sda2
+
* When preparing any boot disk it is important to know the ramifications of any decision you may make. Please review and review again the documentation for your chosen boot-loader/-manager and consider seriously how it might relate to bcache.
  # mkfs.msdos /dev/sda1 (if your ESP is at least 500MB, use mkfs.vfat to make a FAT32 partition instead)
+
* If all associated disks are partitioned at once as below bcache will automatically attach "-B backing stores" to the "-C ssd cache" and step 6 is unnecessary.
  # pacman -S arch-install-scripts
+
}}
  # mount /dev/bcache0 -o subvol=root,compress=lzo /mnt/
+
  # mkdir /mnt/boot
+
  # mkdir /mnt/home
+
  # mount /dev/bcache0 -o subvol=home,compress=lzo /mnt/home
+
  # mount /dev/sda2 /mnt/boot
+
  # mkdir /boot/efi
+
  # mount /dev/sda1 /mnt/boot/efi/
+
do the rest of the installation [[Installation_Guide#Connect_to_the_internet|as normal]]
+
  
9, Edit /etc/mkinitcpio.conf and add the bcache hook between block and filesystems.
+
# make-bcache -B /dev/sd? /dev/sd? -C /dev/sd?
  
10. In the arch-chroot, first install "bcache-tools-git" then the "linux-bcache-git" packages from [[AUR]b ]. (Don't forget to update your bootloader to use the new kernel!)
+
You now have a {{ic|/dev/bcache0}} device.
  
11. Reboot and make sure it works from the /dev/bcache0 device
+
5. Configure your SSD
  
12. delete the partitions from /dev/sdb, format it as a caching device, and link it to the backing device
+
Format the SSD as a caching device and link it to the backing device
    # parted /dev/sdb mklabel msdos; dd if=/dev/zero of=/dev/sdb bs=512 count=1
+
 
    # make-bcache -C /dev/sdb
+
# make-bcache -C /dev/sdb
    # echo /dev/sdb > /sys/fs/bcache/register  
+
# echo /dev/sdb > /sys/fs/bcache/register  
    # echo <Set UUID from previous command> > /sys/block/bcache0/bcache/attach
+
# echo ''UUID__from_previous_command'' > /sys/block/bcache0/bcache/attach
+
 
{{Note| If the UUID is forgotten, it can be found with ''ls /sys/fs/bcache/'' after the cache device has been registered.}}
+
{{Note|If the UUID is forgotten, it can be found with {{ic|ls /sys/fs/bcache/}} after the cache device has been registered.}}
 +
 
 +
6. Format the bcache device. Use LVM or btrfs subvolumes if you want to divide up the {{ic|/dev/bcache0}} device how you like (ex for separate {{ic|/}}, {{ic|/home}}, {{ic|/var}}, etc):
 +
 
 +
# mkfs.btrfs /dev/bcache0
 +
# mount /dev/bcache0 /mnt/
 +
# btrfs subvolume create /mnt/root
 +
# btrfs subvolume create /mnt/home
 +
# umount /mnt
 +
 
 +
7. Prepare the installation mount point:
 +
 
 +
# mkfs.ext4 /dev/sda2
 +
# mkfs.msdos /dev/sda1 (if your ESP is at least 500MB, use mkfs.vfat to make a FAT32 partition instead)
 +
# pacman -S arch-install-scripts
 +
# mount /dev/bcache0 -o subvol=root,compress=lzo /mnt/
 +
# mkdir /mnt/boot /mnt/home
 +
# mount /dev/bcache0 -o subvol=home,compress=lzo /mnt/home
 +
# mount /dev/sda2 /mnt/boot
 +
# mkdir /boot/efi
 +
# mount /dev/sda1 /mnt/boot/efi/
 +
 
 +
8. Install the system as per the [[Installation guide#Connect to the Internet|Installation Guide]] as normal except this:
 +
 
 +
Before you edit {{ic|/etc/mkinitcpio.conf}} and run {{ic|mkinitcpio -p linux}}:
 +
 
 +
* install {{AUR|bcache-tools}} package from the [[AUR]].
 +
* Edit {{ic|/etc/mkinitcpio.conf}}:
 +
** add the "bcache" module
 +
** add the "bcache" hook between block and filesystem hooks
  
 
== Configuring ==
 
== Configuring ==
There are many options that can be configured (such as writeback cache mode, cache flush interval, sequential write heuristic, etc.) This is currently done by writing to files in /sys. See the [http://atlas.evilpiepirate.org/git/linux-bcache.git/tree/Documentation/bcache.txt?h=bcache bcache user documentation].
 
  
== Advanced Operations ==
+
There are many options that can be configured (such as cache mode, cache flush interval, sequential write heuristic, etc.) This is currently done by writing to files in {{ic|/sys}}. See the [http://atlas.evilpiepirate.org/git/linux-bcache.git/tree/Documentation/bcache.txt bcache user documentation].
  
===Resize backing device===
+
Changing the cache mode is done by echoing one of 'writethrough', 'writeback', 'writearound' or 'none' to /sys/block/bcache[0-9]/bcache/cache_mode.
  
It's possible to resize the backing device so long as you don't move the partition start. This process is described on [http://comments.gmane.org/gmane.linux.kernel.bcache.devel/249 the mailing list]. Here's an example using btrfs volume directly on bcache0. For LLVM containers or for other filesystems, procedure will differ.
+
Note that changes to /sys are temporary, and will revert back after a reboot. To set custom configurations at boot create a .conf file in {{ic|/etc/tmpfile.d}}. To set, in a persistent fashion, the sequential cutoff for bcache0 to 1 MB  and write back you could create a file {{ic|/etc/tmpfile.d/my-bcache.conf}} with the contents
 +
 
 +
  w /sys/block/bcache0/bcache/sequential_cutoff - - - - 1M
 +
  w /sys/block/bcache0/bcache/cache_mode        - - - - writeback
 +
 
 +
== Advanced operations ==
 +
 
 +
=== Resize backing device ===
 +
 
 +
It is possible to resize the backing device so long as you do not move the partition start. This process is described on [http://comments.gmane.org/gmane.linux.kernel.bcache.devel/249 the mailing list]. Here is an example using btrfs volume directly on bcache0. For LVM containers or for other filesystems, procedure will differ.
 +
 
 +
==== Example of growing ====
  
====Example of growing====
 
 
In this example, I grow the filesystem by 4GB.  
 
In this example, I grow the filesystem by 4GB.  
  
 
1. Reboot to a live CD/USB Drive (need not be bcache enabled) and use fdisk, gdisk, parted, or your other favorite tool to delete the backing partition and recreate it with the same start and a total size 4G larger.  
 
1. Reboot to a live CD/USB Drive (need not be bcache enabled) and use fdisk, gdisk, parted, or your other favorite tool to delete the backing partition and recreate it with the same start and a total size 4G larger.  
  
{{Warning|Do not use a tool like GParted that might perform filesystem operations! It won't recognize the bcache partition and might overwrite part of it!!}}
+
{{Warning|Do not use a tool like GParted that might perform filesystem operations! It will not recognize the bcache partition and might overwrite part of it!!}}
  
2. Reboot to your normal install. Your filesystem will be currently mounted. That's fine. Issue the command to resize the partition to its maximum. For btrfs, that's
+
2. Reboot to your normal install. Your filesystem will be currently mounted. That is fine. Issue the command to resize the partition to its maximum. For btrfs, that is
    sudo btrfs filesystem resize max /
+
For ext3/4, that's
+
    sudo resize2fs /dev/bcache0
+
  
3. That's it!
+
# btrfs filesystem resize max /
 +
 
 +
For ext3/4, that is:
 +
 
 +
# resize2fs /dev/bcache0
 +
 
 +
==== Example of shrinking ====
  
====Example of shrinking====
 
 
In this example, I shrink the filesystem by 4GB.
 
In this example, I shrink the filesystem by 4GB.
  
 
1. Disable writeback cache (switch to writethrough cache) and wait for the disk to flush.
 
1. Disable writeback cache (switch to writethrough cache) and wait for the disk to flush.
    echo writethrough | sudo tee /sys/block/bcache0/bcache/cache_mode
+
 
    watch cat /sys/block/bcache0/bcache/state
+
# echo writethrough > /sys/block/bcache0/bcache/cache_mode
 +
$ watch cat /sys/block/bcache0/bcache/state
 +
 
 
wait until state reports "clean". This might take a while.
 
wait until state reports "clean". This might take a while.
  
2. Shrink the mounted filesystem by something more than the desired amount, to ensure we don't accidentally clip it later.
+
====== Force flush of cache to backing device ======
for btrfs, that's
+
I suggest to use  
  sudo btrfs filesystem resize -5G /
+
for ext3/4 you can use resize2fs, but only if the partition is unmounted
+
  $ df -h /home
+
  /dev/bcache0    290G  20G  270G  1% /home
+
  $ sudo umount /home
+
  $ sudo resize2fs /dev/bcache0 283G
+
  
3. Reboot to a LiveCD/USB drive (doesn't need to support bcache) and use fdisk, gdisk, parted, or your other favorite tool to delete the backing partition and recreate it with the same start and a total size 4G smaller.
+
  # echo 0 > /sys/block/bcache0/bcache/writeback_percent
  
{{Warning|Do not use a tool like GParted that might perform filesystem operations! It won't recognize the bcache partition and might overwrite part of it!!}}
+
This will flush the dirty data of the cache to the backing device in less a minute.
  
4. Reboot to your normal install. Your filesystem will be currently mounted. That's fine. Issue the command to resize the partition to its maximum (that is, the size we shrunk the actual partition to in step 3).
+
Revert back the value after with
For btrfs, that's
+
    sudo btrfs filesystem resize max /
+
For ext3/4, that's
+
    sudo resize2fs /dev/bcache0
+
  
5. Re-enable writeback cache if you want that enabled:
+
  # echo 10 > /sys/block/bcache0/bcache/writeback_percent
    echo writeback | sudo tee /sys/block/bcache0/bcache/cache_mode
+
  
6. That's it!
+
2. Shrink the mounted filesystem by something more than the desired amount, to ensure we do not accidentally clip it later. For btrfs, that is:
 +
 
 +
# btrfs filesystem resize -5G /
 +
 
 +
For ext3/4 you can use ''resize2fs'', but only if the partition is unmounted
 +
 
 +
$ df -h /home
 +
/dev/bcache0    290G  20G  270G  1% /home
 +
# umount /home
 +
# resize2fs /dev/bcache0 283G
 +
 
 +
3. Reboot to a LiveCD/USB drive (does not need to support bcache) and use fdisk, gdisk, parted, or your other favorite tool to delete the backing partition and recreate it with the same start and a total size 4G smaller.
 +
 
 +
{{Warning|Do not use a tool like GParted that might perform filesystem operations! It will not recognize the bcache partition and might overwrite part of it!!}}
 +
 
 +
4. Reboot to your normal install. Your filesystem will be currently mounted. That is fine. Issue the command to resize the partition to its maximum (that is, the size we shrunk the actual partition to in step 3). For btrfs, that is:
 +
 
 +
# btrfs filesystem resize max /
 +
 
 +
For ext3/4, that is:
 +
 
 +
# resize2fs /dev/bcache0
 +
 
 +
5. Re-enable writeback cache if you want that enabled:
  
{{Note|If you're very careful you can shrink the filesystem to the exact size in step 2 and avoid step 4. Be careful, though, many partition tools don't do exactly what you want, but instead adjust the requested partition start/end points to end on sector boundaries. This may be difficult to calculate ahead of time}}
+
# echo writeback > /sys/block/bcache0/bcache/cache_mode
  
 +
{{Note|If you are very careful you can shrink the filesystem to the exact size in step 2 and avoid step 4. Be careful, though, many partition tools do not do exactly what you want, but instead adjust the requested partition start/end points to end on sector boundaries. This may be difficult to calculate ahead of time}}
  
 
== Troubleshooting ==
 
== Troubleshooting ==
  
=== /dev/bcache device doesn't exist on bootup ===
+
=== /dev/bcache device does not exist on bootup ===
 +
 
 
If you are sent to a busy box shell with an error:
 
If you are sent to a busy box shell with an error:
    ERROR: Unable to find root device 'UUID=b6b2d82b-f87e-44d5-bbc5-c51dd7aace15'.
 
    You are being dropped to a recovery shell
 
        Type 'exit' to try and continue booting
 
  
This might happen if the backing device is configured for "writeback" mode (default is writearound). When in "writeback" mode, the /dev/bcache0 device is not started until the cache device is both registered and attached. Registering is something that needs to happen every bootup, but attaching should only have to be done once. I've sometime had to re-attach.
+
{{bc|1=
 +
ERROR: Unable to find root device 'UUID=b6b2d82b-f87e-44d5-bbc5-c51dd7aace15'.
 +
You are being dropped to a recovery shell
 +
    Type 'exit' to try and continue booting
 +
}}
 +
 
 +
This might happen if the backing device is configured for "writeback" mode (default is writearound). When in "writeback" mode, the /dev/bcache0 device is not started until the cache device is both registered and attached. Registering is something that needs to happen every bootup, but attaching should only have to be done once.  
  
 
To continue booting, try one of the following:
 
To continue booting, try one of the following:
* Register both the backing device and the cacheing device
+
 
  # echo /dev/sda3 > /sys/fs/bcache/register
+
* Register both the backing device and the caching device
  # echo /dev/sdb > /sys/fs/bcache/register
+
 
 +
# echo /dev/sda3 > /sys/fs/bcache/register
 +
# echo /dev/sdb > /sys/fs/bcache/register
 +
 
 
If the /dev/bcache0 device now exists, type exit and continue booting. You will need to fix your initcpio to ensure devices are registered before mounting the root device.
 
If the /dev/bcache0 device now exists, type exit and continue booting. You will need to fix your initcpio to ensure devices are registered before mounting the root device.
{{Note|An error of "sh: echo: write error: Invalid argument" means the device was already registered or is not recognized as either a bcache backing device or cache.}}
+
 
 +
{{Note|
 +
* An error of "sh: echo: write error: Invalid argument" means the device was already registered or is not recognized as either a bcache backing device or cache. If using the udev rule on boot it should only attempt to register a device if it finds a bcache superblock
 +
* This can also happen if using udev's 69-bcache.rules in Installation's step 7 and blkid and bcache-probe "disagree" due to rogue superblocks. See [http://bcache.evilpiepirate.org/#index6h1 bcache's wiki] for a possible explanation/resolution.
 +
}}
  
 
* Re-attach the cache to the backing device:
 
* Re-attach the cache to the backing device:
If the cache device was registered, a folder with the UUID of the cache should exist in /sys/fs/bcache. Use that UUID when following the example below:
 
  # ls /sys/fs/bcache/
 
  b6b2d82b-f87e-44d5-bbc5-c51dd7aace15    register    register_quiet
 
  # echo b6b2d82b-f87e-44d5-bbc5-c51dd7aace15 > /sys/block/sda/sda3/bcache/attach
 
If the /dev/bcache0 device now exists, type exit and continue booting. You should not have to do this again. If it persists, ask on the bcache mailing list.
 
{{Note|An error of "sh: echo: write error: Invalid argument" means the device was already attached. An error of "sh: echo: write error: No such file or directory" means the UUID is not a valid cache (make sure you typed it correctly)}}
 
  
* Invalidate the cache and force the backing device to run without it.
+
If the cache device was registered, a folder with the UUID of the cache should exist in {{ic|/sys/fs/bcache}}. Use that UUID when following the example below:
You might want to check some stats, such as "dirty_data" so you have some idea of how much data will be lost.
+
 
  # cat /sys/block/sda/sda3/bcache/dirty_data
+
# ls /sys/fs/bcache/
  -3.9M
+
b6b2d82b-f87e-44d5-bbc5-c51dd7aace15    register    register_quiet
 +
# echo b6b2d82b-f87e-44d5-bbc5-c51dd7aace15 > /sys/block/sda/sda3/bcache/attach
 +
 
 +
If the {{ic|/dev/bcache0}} device now exists, type exit and continue booting. You should not have to do this again. If it persists, ask on the bcache mailing list.
 +
 
 +
{{Note|An error of {{ic|sh: echo: write error: Invalid argument}} means the device was already attached. An error of {{ic|sh: echo: write error: No such file or directory}} means the UUID is not a valid cache (make sure you typed it correctly).}}
 +
 
 +
* Invalidate the cache and force the backing device to run without it. You might want to check some stats, such as "dirty_data" so you have some idea of how much data will be lost.
 +
 
 +
# cat /sys/block/sda/sda3/bcache/dirty_data
 +
-3.9M
 +
 
 
dirty data is data in the cache that has not been written to the backing device. If you force the backing device to run, this data will be lost, even if you later re-attach the cache.
 
dirty data is data in the cache that has not been written to the backing device. If you force the backing device to run, this data will be lost, even if you later re-attach the cache.
  # cat /sys/block/sda/sda3/bcache/running
 
  0
 
  # echo 1 > /sys/block/sda/sda3/bcache/running
 
The /dev/bcache0 device will now exist. Type exit and continue booting. You might want to unregister the cache device and run make-bcache again. An fsck on /dev/bcache0 would also be wise. See the [http://atlas.evilpiepirate.org/git/linux-bcache.git/tree/Documentation/bcache.txt?h=bcache bcache user documentation].
 
{{Warning| Only invalidate the cache if one of the two options above did not work.}}
 
  
 +
# cat /sys/block/sda/sda3/bcache/running
 +
0
 +
# echo 1 > /sys/block/sda/sda3/bcache/running
 +
 +
The {{ic|/dev/bcache0}} device will now exist. Type exit and continue booting. You might want to unregister the cache device and run make-bcache again. An fsck on {{ic|/dev/bcache0}} would also be wise. See the [http://atlas.evilpiepirate.org/git/linux-bcache.git/tree/Documentation/bcache.txt?h=bcache bcache user documentation].
 +
 +
{{Warning|Only invalidate the cache if one of the two options above did not work.}}
 +
 +
=== /sys/fs/bcache/ does not exist ===
  
=== /sys/fs/bcache/ doesn't exist ===
 
 
The kernel you booted is not bcache enabled.
 
The kernel you booted is not bcache enabled.
 +
 +
== See also ==
 +
 +
* [http://bcache.evilpiepirate.org Bcache Homepage]
 +
* [https://www.kernel.org/doc/Documentation/bcache.txt Bcache Manual]

Latest revision as of 08:31, 12 April 2016

Tango-edit-clear.pngThis article or section needs language, wiki syntax or style improvements.Tango-edit-clear.png

Reason: Some first-person comments, see Help:Style. (Discuss in Talk:Bcache#)

Bcache allows one to use an SSD as a read/write cache (in writeback mode) or read cache (writethrough or writearound) for another blockdevice (generally a rotating HDD or array). This article will show how to install arch using Bcache as the root partition. For an intro to bcache itself, see the bcache homepage. Be sure to read and reference the bcache manual. Bcache is in the mainline kernel since 3.10. The kernel on the arch install disk includes the bcache module since 2013.08.01.

An alternative to Bcache is Facebook's Flashcache and its offspring EnhanceIO. Another alternative - btier-dkmsAUR[broken link: archived in aur-mirror].

Tango-inaccurate.pngThe factual accuracy of this article or section is disputed.Tango-inaccurate.png

Reason: Backwards incompatible changes to on-disk format for linux 3.18 appears to be false. bcache on-disk format changes have not yet landed upstream yet as of 3.19 (Discuss in Talk:Bcache#)

Bcache needs the backing device to be formatted as a bcache block device. In most cases, blocks to-bcache can do an in-place conversion.

Warning:
  • Be sure you back up any important data first.
  • The bcache-dev branch is under heavy development. The on-disk format has undergone changes in 3.18 which are not backwards compatible with previous formats[1]. Note: This only applies to users who compile-in bcache-dev. The version built-in to the upstream Linux kernel is unaffected[2].
  • Bcache and btrfs could leave you with a corrupted filesystem. Please visit this post for more information. Btrfs wiki reports that it was fixed in kernels 3.19+ [3].

Setting up a bcache device on an existing system

1. Install the bcache-toolsAUR package from AUR.

2. Create a backing device (This will typically be your mechanical drive). The backing device can be a whole device, a partition or any other standard block device. This will create /dev/bcache0

   # make-bcache -B /dev/sdx1

3. Create a cache device (This will typically be your SSD). The cache device can be a whole device, a partition or any other standard block device

   # make-bcache -C /dev/sdy2

In this example the default block and bucket sizes of 512B and 128kB are used. The block size should match the backing devices sector size which will usually be either 512 or 4k. The bucket size should match the erase block size of the caching device with the intent of reducing write amplification. For example, using a HDD with 4k sectors and an SSD with an erase block size of 2MB this command would look like

   # make-bcache --block 4k --bucket 2M -C /dev/sdy2

4. Register the cache device against your backing device. To find its cache set UUID, run # bcache-super-show /dev/sdy2 | grep cset.uuid and then add it to the bcache device initially. Udev rules will take care of this on reboot and will only need to be done once.

   # echo cset.uuid > /sys/block/bcache0/bcache/attach

5. Change your cache mode (if you want to cache writes as well as reads):

   # echo writeback > /sys/block/bcache0/bcache/cache_mode

6. If you want to have this partition available during the initcpio (i.e. you require it at some point in the boot process) you need to add 'bcache' to your modules array in /etc/mkinitcpio.conf as well as adding the 'bcache' hook in your list between block and filesystems. You must then rebuild the initramfs image. This is typically done with

   # mkinitcpio -p linux

Bcache management

1. Check that everything has been correctly setup

   # cat /sys/block/bcache0/bcache/state

The output can be:

  • no cache: this means you have not attached a caching device to your backing bcache device
  • clean: this means everything is ok. The cache is clean.
  • dirty: this means everything is setup fine and that you have enabled writeback and that the cache is dirty.
  • inconsistent: you are in trouble because the backing device is not in sync with the caching device

You can have a /dev/bcache0 device associated with a backing device with no caching device attached. This means that all I/O (read/write) are passed directly to the backing device (pass-through mode)

2. See what caching mode is in use

# cat /sys/block/bcache0/bcache/cache_mode 
[writethrough] writeback writearound none

In the above example, the writethrough mode is enabled.

3. Show info about a bcached device:

   # bcache-super-show /dev/sdXY

4. Stop the backing device:

   # echo 1 > /sys/block/sdX/sdX[Y]/bcache/stop

5. Detach a caching device:

   # echo 1 > /sys/block/sdX/sdX[Y]/bcache/detach

6. Safely remove the cache device

   # echo <cache-set-uuid> > /sys/block/bcache0/bcache/detach

7. Release attached devices

   # echo 1 > /sys/fs/bcache/<cache-set-uuid>/stop

Bcache on top of another block layer

Tango-inaccurate.pngThe factual accuracy of this article or section is disputed.Tango-inaccurate.png

Reason: it is strongly advises to make Bcache first, underneath any other block layer (Discuss in Talk:Bcache#)
Warning:
  • it is widely recommended to use Bcache underneath any other block layer.
  • the blocks package is broken with Python3. Please visit this thread for a workaround, or use Python VirtualEnv.
  • when on top of a LVM2 layer, the discard mount option may not be handled well.

With blocks, you can add a Bcache layer on a device with an existing filesystem or layer (normal partition, LUKS partition, logical volume).

First, install the blocks-gitAUR[broken link: archived in aur-mirror] package from AUR.

blocks needs to add the bcache metadata at the start of the targeted partition, here /dev/sdz4; to be able to do so, it will resize the partition just before the targeted one (/dev/sdz3) to make necessary room, typically 2048 bits (2k). It will ask for your passphrase if the partition(/dev/sdz3) is an LUKS partition. Then it will recreate another partition with the start just lower of 2048 bits and run make-bcache, so you do not need to run it yourself.

   # blocks to-bcache /dev/sdz4

You can check that everything is ok by running:

   # bcache-super-show /dev/sdz4

which should output metadata about the new backing bcache device

You can proceed to create the caching device and attach it to the backing device.

Troubleshooting

If blocks complains complains about overlapping metadata when running blocks to-bcache, you will need to shrink by 100Mb the partition you want to cache and create a new 100Mb free partition.

Installation to a bcache device

1. Boot on the install disk (2013.08.01 minimum).

2. Install the bcache-toolsAUR package from AUR.

3. Partition your hdd

Note: While it may be true that Grub2 does not offer support for bcache as noted below, it does, however, fully support UEFI. It follows then, that so long as the necessary modules for the linux kernel to properly handle your boot device are either compiled into the kernel or are included in an initramfs, and you can include these files on it, the separate boot partition described below may be omitted in favor of the FAT EFI system partition. See GRUB and/or UEFI for more.

grub cannot handle bcache, so you will need at least 2 partitions (boot and one for the bcache backing device). If you are doing UEFI, you will need an EFI System Partition (ESP) as well. E.g.:

     1            2048           22527   10.0 MiB    EF00  EFI System
     2           22528          432127   200.0 MiB   8300  arch_boot
     3          432128       625142414   297.9 GiB   8300  bcache_backing
Note: This example has no swapfile/partition. For a swap partition on the cache, use LVM in step 7. For a swap partition outside the cache, be sure to make a swap partition now.

4. Configure your HDD as a bcache backing device.

# make-bcache -B /dev/sda3
Note:
  • When preparing any boot disk it is important to know the ramifications of any decision you may make. Please review and review again the documentation for your chosen boot-loader/-manager and consider seriously how it might relate to bcache.
  • If all associated disks are partitioned at once as below bcache will automatically attach "-B backing stores" to the "-C ssd cache" and step 6 is unnecessary.
# make-bcache -B /dev/sd? /dev/sd? -C /dev/sd?

You now have a /dev/bcache0 device.

5. Configure your SSD

Format the SSD as a caching device and link it to the backing device

# make-bcache -C /dev/sdb
# echo /dev/sdb > /sys/fs/bcache/register 
# echo UUID__from_previous_command > /sys/block/bcache0/bcache/attach
Note: If the UUID is forgotten, it can be found with ls /sys/fs/bcache/ after the cache device has been registered.

6. Format the bcache device. Use LVM or btrfs subvolumes if you want to divide up the /dev/bcache0 device how you like (ex for separate /, /home, /var, etc):

# mkfs.btrfs /dev/bcache0
# mount /dev/bcache0 /mnt/
# btrfs subvolume create /mnt/root
# btrfs subvolume create /mnt/home
# umount /mnt

7. Prepare the installation mount point:

# mkfs.ext4 /dev/sda2
# mkfs.msdos /dev/sda1 (if your ESP is at least 500MB, use mkfs.vfat to make a FAT32 partition instead)
# pacman -S arch-install-scripts
# mount /dev/bcache0 -o subvol=root,compress=lzo /mnt/
# mkdir /mnt/boot /mnt/home
# mount /dev/bcache0 -o subvol=home,compress=lzo /mnt/home
# mount /dev/sda2 /mnt/boot
# mkdir /boot/efi
# mount /dev/sda1 /mnt/boot/efi/

8. Install the system as per the Installation Guide as normal except this:

Before you edit /etc/mkinitcpio.conf and run mkinitcpio -p linux:

  • install bcache-toolsAUR package from the AUR.
  • Edit /etc/mkinitcpio.conf:
    • add the "bcache" module
    • add the "bcache" hook between block and filesystem hooks

Configuring

There are many options that can be configured (such as cache mode, cache flush interval, sequential write heuristic, etc.) This is currently done by writing to files in /sys. See the bcache user documentation.

Changing the cache mode is done by echoing one of 'writethrough', 'writeback', 'writearound' or 'none' to /sys/block/bcache[0-9]/bcache/cache_mode.

Note that changes to /sys are temporary, and will revert back after a reboot. To set custom configurations at boot create a .conf file in /etc/tmpfile.d. To set, in a persistent fashion, the sequential cutoff for bcache0 to 1 MB and write back you could create a file /etc/tmpfile.d/my-bcache.conf with the contents

  w /sys/block/bcache0/bcache/sequential_cutoff - - - - 1M
  w /sys/block/bcache0/bcache/cache_mode        - - - - writeback

Advanced operations

Resize backing device

It is possible to resize the backing device so long as you do not move the partition start. This process is described on the mailing list. Here is an example using btrfs volume directly on bcache0. For LVM containers or for other filesystems, procedure will differ.

Example of growing

In this example, I grow the filesystem by 4GB.

1. Reboot to a live CD/USB Drive (need not be bcache enabled) and use fdisk, gdisk, parted, or your other favorite tool to delete the backing partition and recreate it with the same start and a total size 4G larger.

Warning: Do not use a tool like GParted that might perform filesystem operations! It will not recognize the bcache partition and might overwrite part of it!!

2. Reboot to your normal install. Your filesystem will be currently mounted. That is fine. Issue the command to resize the partition to its maximum. For btrfs, that is

# btrfs filesystem resize max /

For ext3/4, that is:

# resize2fs /dev/bcache0

Example of shrinking

In this example, I shrink the filesystem by 4GB.

1. Disable writeback cache (switch to writethrough cache) and wait for the disk to flush.

# echo writethrough > /sys/block/bcache0/bcache/cache_mode
$ watch cat /sys/block/bcache0/bcache/state

wait until state reports "clean". This might take a while.

Force flush of cache to backing device

I suggest to use

 # echo 0 > /sys/block/bcache0/bcache/writeback_percent

This will flush the dirty data of the cache to the backing device in less a minute.

Revert back the value after with

 # echo 10 > /sys/block/bcache0/bcache/writeback_percent

2. Shrink the mounted filesystem by something more than the desired amount, to ensure we do not accidentally clip it later. For btrfs, that is:

# btrfs filesystem resize -5G /

For ext3/4 you can use resize2fs, but only if the partition is unmounted

$ df -h /home
/dev/bcache0    290G   20G   270G   1% /home
# umount /home
# resize2fs /dev/bcache0 283G

3. Reboot to a LiveCD/USB drive (does not need to support bcache) and use fdisk, gdisk, parted, or your other favorite tool to delete the backing partition and recreate it with the same start and a total size 4G smaller.

Warning: Do not use a tool like GParted that might perform filesystem operations! It will not recognize the bcache partition and might overwrite part of it!!

4. Reboot to your normal install. Your filesystem will be currently mounted. That is fine. Issue the command to resize the partition to its maximum (that is, the size we shrunk the actual partition to in step 3). For btrfs, that is:

# btrfs filesystem resize max /

For ext3/4, that is:

# resize2fs /dev/bcache0

5. Re-enable writeback cache if you want that enabled:

# echo writeback > /sys/block/bcache0/bcache/cache_mode
Note: If you are very careful you can shrink the filesystem to the exact size in step 2 and avoid step 4. Be careful, though, many partition tools do not do exactly what you want, but instead adjust the requested partition start/end points to end on sector boundaries. This may be difficult to calculate ahead of time

Troubleshooting

/dev/bcache device does not exist on bootup

If you are sent to a busy box shell with an error:

ERROR: Unable to find root device 'UUID=b6b2d82b-f87e-44d5-bbc5-c51dd7aace15'.
You are being dropped to a recovery shell
    Type 'exit' to try and continue booting

This might happen if the backing device is configured for "writeback" mode (default is writearound). When in "writeback" mode, the /dev/bcache0 device is not started until the cache device is both registered and attached. Registering is something that needs to happen every bootup, but attaching should only have to be done once.

To continue booting, try one of the following:

  • Register both the backing device and the caching device
# echo /dev/sda3 > /sys/fs/bcache/register
# echo /dev/sdb > /sys/fs/bcache/register

If the /dev/bcache0 device now exists, type exit and continue booting. You will need to fix your initcpio to ensure devices are registered before mounting the root device.

Note:
  • An error of "sh: echo: write error: Invalid argument" means the device was already registered or is not recognized as either a bcache backing device or cache. If using the udev rule on boot it should only attempt to register a device if it finds a bcache superblock
  • This can also happen if using udev's 69-bcache.rules in Installation's step 7 and blkid and bcache-probe "disagree" due to rogue superblocks. See bcache's wiki for a possible explanation/resolution.
  • Re-attach the cache to the backing device:

If the cache device was registered, a folder with the UUID of the cache should exist in /sys/fs/bcache. Use that UUID when following the example below:

# ls /sys/fs/bcache/
b6b2d82b-f87e-44d5-bbc5-c51dd7aace15     register     register_quiet
# echo b6b2d82b-f87e-44d5-bbc5-c51dd7aace15 > /sys/block/sda/sda3/bcache/attach

If the /dev/bcache0 device now exists, type exit and continue booting. You should not have to do this again. If it persists, ask on the bcache mailing list.

Note: An error of sh: echo: write error: Invalid argument means the device was already attached. An error of sh: echo: write error: No such file or directory means the UUID is not a valid cache (make sure you typed it correctly).
  • Invalidate the cache and force the backing device to run without it. You might want to check some stats, such as "dirty_data" so you have some idea of how much data will be lost.
# cat /sys/block/sda/sda3/bcache/dirty_data
-3.9M

dirty data is data in the cache that has not been written to the backing device. If you force the backing device to run, this data will be lost, even if you later re-attach the cache.

# cat /sys/block/sda/sda3/bcache/running
0
# echo 1 > /sys/block/sda/sda3/bcache/running

The /dev/bcache0 device will now exist. Type exit and continue booting. You might want to unregister the cache device and run make-bcache again. An fsck on /dev/bcache0 would also be wise. See the bcache user documentation.

Warning: Only invalidate the cache if one of the two options above did not work.

/sys/fs/bcache/ does not exist

The kernel you booted is not bcache enabled.

See also