From ArchWiki
Revision as of 11:11, 14 November 2013 by Mikeserv (Talk | contribs) (Installing the Kernel)

Jump to: navigation, search

Example installation

Here is mine Bcache installation. Maybe you can use some idea's. :)

Prepare the storage drive

Bcache-tools installation

fakeroot and binutils are only needed. but it will fail in a error if you only install them. wget is from a failed archbang instal :(

pacman -Syy
pacman -S base-devel fakeroot binutils wget git
cd ~/
tar -xzvf bcache-tools-git.tar.gz
cd bcache-tools-git
makepkg -si --asroot

Making the partitions

For /boot i use the slower HDD because i had troubles that mine SSD loaded the kernel while mine HDD was still spinning up :P

Starting with the faster SSD drive

fdisk /dev/sda

Partition type: p
Partition-number: 1
First-sector: [enter]
Last-sector: +90G

Partition type: p
Partition-number: 3
First-sector: [enter]
Last-sector: [enter]
write and exit

Now the slower 2 TB Harddrive

fdisk /dev/sdb

Partition type: p
Partition-number: 1
First-sector: [enter]
Last-sector: +1G

Partition type: p
Partition-number: 2
First-sector: [enter]
Last-sector: +16G [enter]

Partition type: p
Partition-number: 3
First-sector: [enter]
Last-sector: [enter]

write and exit

Making Bcache Device

The --wipe-bcache is to remove a error :) and the dd commands is or cleaning out old partition data.

make-bcache --wipe-bcache -B /dev/sdb3 -C /dev/sda2
dd if=/dev/zero count=1 bs=1024 seek=1 of=/dev/sda2
dd if=/dev/zero count=1 bs=1024 seek=1 of=/dev/sdb3

If next commands give a error: invalid argument Then the device is already registered.

echo /dev/sda2 > /sys/fs/bcache/register
echo /dev/sdb3 > /sys/fs/bcache/register

Formating the drives

mkfs.ext4 -E discard /dev/sda1
mkfs.ext4 -E discard /dev/sdb1
mkfs.ext4 /dev/bcache0
mkswap /dev/sdb2
swapon /dev/sdb2

Mount the partitions

cd ~/
mkdir -p /mnt/install
mount /dev/sda1 /mnt/install
mkdir -p /mnt/install/{boot,home,srv,data,var,mnt/.hdd}
mount /dev/sdb1 /mnt/install/boot
mount /dev/bcache0 /mnt/install/mnt/.hdd
mkdir -p /mnt/install/mnt/.hdd/{home,srv,data,var}
mount -o bind /mnt/install/mnt/.hdd/home /mnt/install/home
mount -o bind /mnt/install/mnt/.hdd/srv /mnt/install/srv
mount -o bind /mnt/install/mnt/.hdd/data /mnt/install/data
mount -o bind /mnt/install/mnt/.hdd/var /mnt/install/var

Bcache Check After mounting

Reason for /boot on the HDD is because of boot up failure because of HDD spin-up.


sda                 8:0    0 167.7G  0 disk 
|-sda1              8:1    0    90G  0 part /mnt/install
`-sda2              8:2    0  77.7G  0 part 
  `-bcache0       253:0    0   1.8T  0 disk /mnt/install/mnt/.hdd
sdb                 8:16   0   1.8T  0 disk 
|-sdb1              8:17   0     1G  0 part /mnt/install/boot
|-sdb2              8:18   0    16G  0 part [SWAP]
`-sdb3              8:19   0   1.8T  0 part 
  `-bcache0       253:0    0   1.8T  0 disk /mnt/install/mnt/.hdd

Generate an fstab

Generate an fstab file with the following command. UUIDs will be used because they have certain advantages.

genfstab -U -p /mnt/install >> /mnt/install/etc/fstab
sed -i '/dev\/sda/ s/rw,relatime,data=ordered/defaults,noatime,discard/g' /mnt/install/etc/fstab
sed -i '/mnt\/install/ {s/\/mnt\/install//g; s/rw,relatime,data=ordered/defaults/g; s/none/bind/g}' /mnt/install/etc/fstab
more /mnt/install/etc/fstab

Install and configure kernel and bootloader

Adding Bcache module and hook

Edit /etc/mkinitcpio.conf as needed and re-generate the initramfs image with: adding the "bcache" module adding the "bcache_udev" hook between block and filesystems

sed -i '/^MODULES=/ s/"$/ bcache"/' /etc/mkinitcpio.conf
sed -i '/^HOOKS=/ s/block filesystems/block bcache_udev filesystems/' /etc/mkinitcpio.conf
more /etc/mkinitcpio.conf

Registering and attaching Bcache

This part is a pain in the A.. The problem is that Bcache wil give a error at startup if this is not done.

ERROR: Unable to find root device 'UUID=b6b2d82b-f87e-44d5-bbc5-c51dd7aace15'.
You are being dropped to a recovery shell
Type 'exit' to try and continue booting

First get the correct "SET UUID" with this ls command. And use the UUID like the example underneath. Also check for the use of the correct devices. echo: write error: Invalid argument <<< This error is good !!! means that your value was already stored.

ls /sys/fs/bcache/
2300f944-0eb5-4a9e-b052-8d5ea63cbb8f  register  register_quiet
sudo echo /dev/sda2 > /sys/fs/bcache/register
sudo echo /dev/sdb3 > /sys/fs/bcache/register
sudo echo 2300f944-0eb5-4a9e-b052-8d5ea63cbb8f > /sys/block/sdb/sdb3/bcache/attach
mv /usr/lib/initcpio/hooks/bcache ~/bcache.old
cat >> /usr/lib/initcpio/hooks/bcache << EOF
run_hook() {
    echo /dev/sda2 > /sys/fs/bcache/register
    echo /dev/sdb3 > /sys/fs/bcache/register

Installing the Kernel

mkinitcpio -p linux

I hope some stuff is helpfull :P 17:58, 11 November 2013 (UTC)

I'm curious about two things. First, though I forget what the '-E' switch signifies, it seems you're formatting the raw, underlayer partitions themselves. I don't understand why.
Also, the "pain in the a?" So udev is not assembling your array automatically as the devices are discovered? That is unexpected behavior, and I suspect it has to do with the formatting you performed and udev's superblock discovery. Check the very bottom of the bcache wiki on their site. At least try lsblk to verify "bcache" partition types are not registering as "ext4" before assembling the array.
Oh. And please sign talk pages.

Mikeserv (talk) 03:35, 12 November 2013 (UTC)

Well this is a experiment with Bcache with mine own Wiki page and 100x testing the install, i am not a linux expert just a newbie who was reading a lot of howto's. Just see it as a compilation of different web pages and some logical thinking. :)

The -E is just a copy and past ... not sure if it was needed.I just thought it was needed to get the discard flag while formating.

The pain in the A.... this was before Udev ... i was busy getting Bcache to work 3 months ago ... 75% of boot up failed when booting up cold. Later i found out that the SSD was too fast with booting up and the normal HDD was still spinning up.... After i splitted up the SSD for root directory and bcache and moved the /boot to the HDD since then i can bootup 100% of the time without any error... This was before the udev thingy (wich i dont understand and need to read up) :)

So please understand that i am just a linux noob trying to contribute :)

Emesix (talk)Emesix (talk) 00:02, 13 November 2013 (UTC)

No, I get it, and I didn't mean to insinuate that you were doing anything wrong, exactly, just that, as designed, it could operate better.
Basically, and you should check the man-pages and the wiki page here to be sure just in case I'm not entirely correct (which does happen), udev is the program that runs during boot (and pretty much all of the rest of the time, too) to detect your hardware and load drivers as necessary. Udev is a relatively recent addition to the Linux boot process and is unique in that it discovers and handles hardware as it shows up, can handle multiple devices simultaneously with non-blocking parallell operations, and, as a result, is significantly faster than the methods it has replaced.
I was the one who added the udev rules and step 7b to this wiki, and I was the one who wrote the "Rogue Superblock" section of "Troubleshooting" at the bottom of bcache's wiki. I only gained the knowledge to write either after spending several frustrating days with an issue that appears very similar to your "pain in the a."
As step 7a shows, the prevailing Arch Linux method for reassembling a bcache array at boot used to look something like:
1) Wait 5 secs.
2) Check to see if udev has discovered and added {disk UUIDS}. If yes goto 3), no, repeat 1) and 2).
3) echo {pain in the a stuff} /sys/bla/bla/bla
This does work, of course, but you're adding an unnecessary wait loop to your boot process and you're performing a task with brittle shell script that depends on specific UUIDs that could be performed flexibly and at discovery time.
That's what the udev rules do. Basically, the rules instruct udev to treat as part of your bcache array any partition it finds that reports itself as a "bcache" partition type. It builds the bcache array from disk partitions at every boot only as soon as those partitions are ready, which would of course resolve the race condition you mentioned and allow you to boot from SSD if you liked.
And here's where our experience might converge: I added a partition to my bcache array after previously formatting it as ext4. If you don't know about superblocks (or magic blocks or whatever they're called) then you're just like I was a few months ago. In short the filesystem has to have a way to report on itself, so it sets aside a small marker on disk that says, "Hey, OS, I'm a disk partition of type EXT4 and I start at block offset whatever and end at block offset whatever. Also, I enjoy long walks on the beach and prefer red wine to white. See you around."
The problem I experienced was that by default ext4 tended to begin at an earlier offset than bcache, and so even though I formatted over the previously ext4 partition with bcache as instructed, bcache never overwrote ext4's first superblock. When udev attempted to build my array, it would always miss my first partition because it would read the first superblock, identify the partition as ext4, then skip it. Echoing the add instruction to /sys/ after boot was my fix for a couple of days, but eventually I figured it all out and wrote that short bit on superblocks in bcache's wiki.
Maybe that helps?
EDIT: Just looked back at the wiki proper and somebody has performed some major changes recently. While it does appear cleaner, it no longer makes sense. There are references to non-existent steps throughout, and the actual why's that were included at least to some degree seem to have been occluded in favor of dubious, though possibly more efficient how's. Anyway, step 7b no longer exists, but it used to include the bcache_udev mkinitcpio hook I (very barely) adapted from mdadm_udev when trying to fix Arch's bcache boot problems. It's now included in the AUR package the wiki recommends I guess, despite the wiki also telling us we must "echo ... /sys/... at every bootup."

Mikeserv (talk) 07:08, 14 November 2013 (UTC)

Sorry about that, I missed a couple of references back to the sections I changed. Hopefully it makes more sense now. Mikeserv's udev rule is crucial to having a working bcache device on boot, as Emesix had discovered, it will often result in a failed boot. The echo in to /sys/bcache* step was missed when I updated it to mention the udev rule as the 'default' method.

--Justin8 (talk) 10:45, 14 November 2013 (UTC)

For clarity's sake, the udev rule was never mine. The used rule has been included in the AUR package's git pull from at least the time I first installed bcache some months ago. The package maintainer at that time opted not to use it, I guess, and instead installed the legacy-type shell-scripted for-loop mkinitcpio hook I outlined above. Frustrated with having to wait for such things which seemed to me to defeat the whole purpose of buying and configuring an SSD boot device, I eventually stumbled upon the mdadm_udev hook in /use/share/initcpio and (only slightly) modified it to apply the 69-bcache-rules instead.
Justin, thanks for your recent change. I'm curious why you specify "unless assembling after boot?" Why include anything to do with bcache in the initramfs at all if you're building the array after the boot sequence?

Mikeserv (talk) 11:11, 14 November 2013 (UTC)