Difference between revisions of "Talk:Btrfs"

From ArchWiki
Jump to: navigation, search
(RAID-1 or RAID-5?: see btrfs wiki)
(Things to do: re)
 
(65 intermediate revisions by 18 users not shown)
Line 1: Line 1:
== Comment about stability ==
+
{{Note|The name of the file system does not seem to have an official capitalization, so the rule for the ArchWiki has been arbitrarily set to use "Btrfs" (capital "B" and the rest lower-case); "btrfs" and "BTRFS" are therefore wrong spellings in this wiki. -- [[User:Kynikos|Kynikos]] ([[User talk:Kynikos|talk]]) 09:36, 6 January 2014 (UTC)}}
Btrfs is still under heavy development; it might be useful if someone familiar with btrfs were to add a section commenting on its overall stability. [[User:Vikingurinn|Vikingurinn]] ([[User talk:Vikingurinn|talk]]) 16:52, 17 December 2012 (UTC)
+
  
== RAID-1 or RAID-5? ==
+
== Things to do ==
  
"''3 1TB disks in an md based raid1 yields a /dev/md0 with 1TB free space and the ability to safely loose 2 disks without losing data. '''3 1TB disks in a btrfs volume with data=raid1 will allow the storage of approximately 1.5TB of data before reporting full. Only 1 disk can safely be lost without losing data'''.''"
+
Here are some items that seem to need addressing on this article. I am working on various parts of it at the moment.
 +
# Reflow headings:
 +
## Goals: [[Effective_Use_of_Headers#Use_of_headers_and_headings]]
 +
### Helpful ideas:
 +
#### [[Effective_Use_of_Headers#False_multi-level_structure]]
 +
##### Side-note
 +
##### Auxiliary information
 +
#### [[Effective_Use_of_Headers#Multi-level_structure]]
 +
##### Group arguments
 +
##### Alternative arguments
 +
##### Contradictory arguments
 +
#### [[Effective_Use_of_Headers#The_header_text]]
 +
#### [[Article_Naming_Guidelines]]
 +
##### Specificity, "Descriptivity," Brevity, "Expandibility,"
 +
### See [https://btrfs.wiki.kernel.org/index.php/Main_Page Btrfs Wiki] for ideas.
 +
### See [[Beginners' Guide]] for ideas.
 +
## Proposed changes
 +
### "Installation" -> "'''Preparation'''"
 +
#### New -> "'''System Requirements'''"
 +
#### Remove -> "Additional packages"
 +
##### Move -> links to "See also" or "Related".
 +
#### "Partitioning" -> "'''Prepare the storage drive'''"
 +
##### '''OR:''' "Partitioning" (as a child) -> "'''''Choose a partition scheme'''''"
 +
### New -> "'''Create file systems'''"
 +
#### "Creating a new file system" -> "'''New file systems'''"
 +
##### "'''Examples'''"
 +
###### New -> "'''Single-drive file systems'''"
 +
###### "Multi-device filesystem and RAID feature" -> "'''Multi-drive file systems'''"
 +
####### Reduce content, provide links to "Btrfs Wiki" instead.
 +
#### "Convert from Ext3/4" -> "'''Convert existing file system'''"
 +
### New -> "'''Configuring the file system'''"
 +
#### '''OR:''' Move -> "'''Tips and tricks'''"
 +
#### Move -> "'''Mount options'''"
 +
##### "Copy-On-Write (CoW)" -> "'''Copy-on-write'''"
 +
###### "Checkpoint Interval" -> '''Merge'''
 +
##### Move -> "'''Compression'''"
 +
#### New -> "'''Skinny extents'''"
 +
#### Move -> "'''Sub-volumes'''"
 +
#### New -> "'''Quotas'''"
 +
### New -> "'''File system management'''"
 +
#### Move -> "'''Balance'''"
 +
#### Move -> "'''Defragmentation'''"
 +
#### Move -> "'''Display used/free space'''"
 +
#### Move -> "'''Balance'''"
 +
#### Move -> "'''Snapshots'''"
 +
### Move -> "'''Tips and tricks'''"
 +
### New -> "'''Troubleshooting'''"
 +
#### Move -> "'''Encryption'''"
 +
#### Move -> "'''GRUB'''"
 +
#### Move -> "'''Swap file'''"
 +
#### New -> "'''Btrfsck'''"
 +
##### See -> https://bbs.archlinux.org/viewtopic.php?id=156100
 +
### Move -> "'''See also'''"
  
That sounds more like some kind of weird, inefficient RAID-5 arrangement than RAID-1. I don't know btrfs enough to say it's wrong, but I know RAID-1 enough to question it?
+
Edits:
 +
[[User:AdamT|AdamT]] ([[User_talk:AdamT|Talk]]) 08:27, 28 November 2013 (UTC);
 +
[[User:AdamT|AdamT]] ([[User_talk:AdamT|Talk]]) 04:03, 29 November 2013 (UTC);
 +
[[User:AdamT|AdamT]] ([[User_talk:AdamT|Talk]]) 04:10, 29 November 2013 (UTC);
 +
[[User:AdamT|AdamT]] ([[User_talk:AdamT|Talk]]) 02:35, 4 December 2013 (UTC)
  
[[User:Fukawi2|Fukawi2]] ([[User talk:Fukawi2|talk]]) 23:52, 29 April 2013 (UTC)
+
: I hope to get to this soon if there are no objections or comments. [[User:AdamT|AdamT]] ([[User_talk:AdamT|Talk]]) 20:09, 4 January 2014 (UTC)
:See [https://btrfs.wiki.kernel.org/index.php/FAQ#What_are_the_differences_among_MD-RAID_.2F_device_mapper_.2F_btrfs_raid.3F btrfs wiki]:
+
:''btrfs combines all the drives into a storage pool first, and then duplicates the chunks as file data is created. RAID-1 is defined currently as "2 copies of all the data on different disks". This differs from MD-RAID and dmraid, in that those make exactly n copies for n disks. In a btrfs RAID-1 on 3 1TB drives we get 1.5TB of usable data. Because each block is only copied to 2 drives, writing a given block only requires exactly 2 drives spin up, reading requires only 1 drive to spinup.''
+
: -- [[User:Karol|Karol]] ([[User talk:Karol|talk]]) 15:59, 17 June 2013 (UTC)
+
  
== btrfs with fsck hook and fstab pass ==
+
:: I believe all of his has been addressed except for quotas. -- [[User:Rdeckard|Rdeckard]] ([[User_talk:Rdeckard|talk]]) 17:27, 25 July 2016 (UTC)
  
Apparently one does not need to have a btrfs partition fscked on boot, so there should be a '0' for 'pass' in the fstab for any btrfs partion.
+
== Examples ==
I had relatively slow boot ups with a pass of '1' (btrfs was root)
+
  
https://bugs.launchpad.net/ubuntu/+source/linux/+bug/791020/comments/27
+
* '''Linux 3.14'''
 +
** Btrfs on a SSD for system installation and an emphasis on maximizing performance.
 +
*:{{bc|1=noatime,compress=lzo,ssd,discard,autodefrag}}
 +
** Btrfs on a HDD for archival purposes with an emphasis on maximizing space.
 +
*: {{bc|1=noatime,compress-force=zlib,autodefrag,nospace_cache}}
  
the genfstab script puts a '1' in for 'pass' by default
+
Why I removed the Examples section:
 +
* autodefrag (since 3.0): Will detect random writes into existing files and kick off background defragging. It is well suited to bdb or sqlite databases, but not virtualization images or big databases (yet). Once the developers make sure it doesn't defrag files over and over again, they'll move this toward the default.
 +
** Unstable, may not be what the user wants (e.g. what is stored on the HDD/SSD?)
 +
* compress-force=<method> - Enable compression even for files that don't compress well, like videos and dd images of disk
 +
** Is this safe? Could this lead to corruption of files? There are bugs reported about using lzo as compression (e.g. https://bugs.archlinux.org/task/41155).
 +
* nospace_cache (since 3.2): Disable free space caching. This may slow down new writes
 +
** May slow down new writes.. Why would you disable free space caching? (see space_cache)
 +
 
 +
I'm sorry, but at the moment I simple not recommend to follow the examples; not that they are wrong, but simple because they differ for each user/machine.
 +
 
 +
[[User:Beta990|Beta990]] ([[User talk:Beta990|talk]]) 23:31, 8 August 2014 (UTC)
 +
 
 +
:The edit in question is [https://wiki.archlinux.org/index.php?title=Btrfs&diff=next&oldid=329381].
 +
:Does the argument about {{ic|autodefrag}} option apply also to "normal" foreground defragmentation, i.e. {{ic|btrfs filesystem defragment}}?
 +
:-- [[User:Lahwaacz|Lahwaacz]] ([[User talk:Lahwaacz|talk]]) 07:13, 9 August 2014 (UTC)
 +
 
 +
::To be honest, I don't really know. Maybe we need to contact the btrfs devs?
 +
::The bit that scares me off: 'Once the developers make sure it doesn't defrag files over and over again, they'll move this toward the default.'
 +
::[[User:Beta990|Beta990]] ([[User talk:Beta990|talk]]) 11:04, 9 August 2014 (UTC)
 +
 
 +
:::It's possible I don't understand the question, but wouldn't foreground defragging be unaffected by {{ic|autodefrag}}? I think the idea is that some applications constantly make random writes throughout their lifetimes, and Btrfs has a poor (or non-existent) algorithm for throttling defrag operations, meaning that Btrfs will constantly be starting new defrag jobs for as long as the application is running. This results in a lot of seeking and writing, and poor performance.
 +
:::On-demand (foreground) defragging, on the other hand, only happens when the user requests it. This means that the file will begin to become fragmented again after the defrag has ended, until the next one. The way I see it is: use {{ic|autodefrag}} if low seek times for reads are a requirement (sequential reads, use case: database lookup tables, row iteration), otherwise use {{ic|btrfs fi defragment}} (as needed, or via a cron job) if one wants to save performance from the writes caused by block relocation, when sequential reads are rare (use case: mostly write-only archival data, disk images).
 +
:::It really depends on the application. Does it cache lookup tables and then do random reads from there? Does it make random writes to files that are normally sequentially read? Does it simply archive data that is rarely read back? Personally I would lean towards '''not''' using {{ic|autodefrag}}, because many applications take fragmentation into consideration from the time the file is created. For example, databases often preallocate data in large chunks to attempt to avoid fragmentation. Disk images are often preallocated to a fixed size that rarely changes. Perhaps do an on-demand defrag every now and then, just in case.
 +
:::This is analogous to the "fstrim vs {{ic|-o discard}}" debate within Btrfs and other filesystems. The discard mount option causes a discard to be dispatched every time a block is freed. This is a (sometimes?) blocking operation, meaning the filesystem has to wait for the hardware to handle the discard before moving on. In this case, it makes more sense to dispatch discards for all free blocks in batches during periods of low I/O load.
 +
:::I'm by no means an expert on this, I just thought I would share my reasoning. Getting in touch with the developers definitely couldn't hurt. As for what this means to the article, I don't recommend users go overriding defaults just because they saw an options line on the wiki that claims better performance, unless they know what those options are actually doing. Defaults are defaults for a reason. [[User:EscapedNull|EscapedNull]] ([[User talk:EscapedNull|talk]]) 02:00, 19 February 2015 (UTC)
 +
 
 +
== btrfs check ==
 +
 
 +
''[Moved from [[ArchWiki talk:Reports#Btrfs]]. — [[User:Kynikos|Kynikos]] ([[User talk:Kynikos|talk]]) 03:39, 20 April 2015 (UTC)]''
 +
 
 +
I attempted to fix the style issues reported with [https://wiki.archlinux.org/index.php?title=Btrfs&diff=prev&oldid=370167 this] edit. I'm not convinced by the idea of a 'scenario' though. I would have thought just mentioning the existence of the command, outlining its use and linking to the relevant documentation would be enough. -- [[User:Chazza|Chazza]] ([[User talk:Chazza|talk]]) 11:55, 19 April 2015 (UTC)
 +
 
 +
== Subvolumes ==
 +
 
 +
The manpage ''btrf-subvolumes'' is really good. Do we really even need a section here about subvolumes? - [[User:Rdeckard|Rdeckard]] ([[User talk:Rdeckard|talk]]) 02:04, 31 March 2016 (UTC)
 +
 
 +
== Automatic snapshots on each boot  ==
 +
 
 +
''Moved from [[Talk:Btrfs_-_Tips_and_tricks]]'' -- [[User:Rdeckard|Rdeckard]] ([[User_talk:Rdeckard|talk]]) 13:09, 19 July 2016 (UTC)
 +
 
 +
I agree the whole article does not really fit in a "tip" category, but I am not sure removing the [[Btrfs_-_Tips and tricks#Automatic snapshots on each boot]] section is the best approach. Essentially this would render the rest of the article useless. It would be nice to have a similar section/systemd unit in [[Snapper]], indeed. It is a different (additional) tool though and to me (I have not tried either) this article reads like a smart solution for a problem that the snapper tool has not covered in a similarly easy way years later.
 +
I think it would be better to remove the template here and add a section with an appropriate expansion template into [[Snapper]], linking to this section. Once that is in place, it would be easier to decide about this article (see also above talk item). --[[User:Indigo|Indigo]] ([[User talk:Indigo|talk]]) 08:03, 30 March 2016 (UTC)
 +
 
 +
:I've begun this section here [Snapper#Snapshots_on_boot] on the snapper page that should solve the same problem this script solves. I do think that either the script needs to be moved off the wiki, or the section here needs to be removed completely. Thoughts? -- [[User:Rdeckard|Rdeckard]] ([[User_talk:Rdeckard|talk]]) 13:11, 19 July 2016 (UTC)

Latest revision as of 17:28, 25 July 2016

Note: The name of the file system does not seem to have an official capitalization, so the rule for the ArchWiki has been arbitrarily set to use "Btrfs" (capital "B" and the rest lower-case); "btrfs" and "BTRFS" are therefore wrong spellings in this wiki. -- Kynikos (talk) 09:36, 6 January 2014 (UTC)

Things to do

Here are some items that seem to need addressing on this article. I am working on various parts of it at the moment.

  1. Reflow headings:
    1. Goals: Effective_Use_of_Headers#Use_of_headers_and_headings
      1. Helpful ideas:
        1. Effective_Use_of_Headers#False_multi-level_structure
          1. Side-note
          2. Auxiliary information
        2. Effective_Use_of_Headers#Multi-level_structure
          1. Group arguments
          2. Alternative arguments
          3. Contradictory arguments
        3. Effective_Use_of_Headers#The_header_text
        4. Article_Naming_Guidelines
          1. Specificity, "Descriptivity," Brevity, "Expandibility,"
      2. See Btrfs Wiki for ideas.
      3. See Beginners' Guide for ideas.
    2. Proposed changes
      1. "Installation" -> "Preparation"
        1. New -> "System Requirements"
        2. Remove -> "Additional packages"
          1. Move -> links to "See also" or "Related".
        3. "Partitioning" -> "Prepare the storage drive"
          1. OR: "Partitioning" (as a child) -> "Choose a partition scheme"
      2. New -> "Create file systems"
        1. "Creating a new file system" -> "New file systems"
          1. "Examples"
            1. New -> "Single-drive file systems"
            2. "Multi-device filesystem and RAID feature" -> "Multi-drive file systems"
              1. Reduce content, provide links to "Btrfs Wiki" instead.
        2. "Convert from Ext3/4" -> "Convert existing file system"
      3. New -> "Configuring the file system"
        1. OR: Move -> "Tips and tricks"
        2. Move -> "Mount options"
          1. "Copy-On-Write (CoW)" -> "Copy-on-write"
            1. "Checkpoint Interval" -> Merge
          2. Move -> "Compression"
        3. New -> "Skinny extents"
        4. Move -> "Sub-volumes"
        5. New -> "Quotas"
      4. New -> "File system management"
        1. Move -> "Balance"
        2. Move -> "Defragmentation"
        3. Move -> "Display used/free space"
        4. Move -> "Balance"
        5. Move -> "Snapshots"
      5. Move -> "Tips and tricks"
      6. New -> "Troubleshooting"
        1. Move -> "Encryption"
        2. Move -> "GRUB"
        3. Move -> "Swap file"
        4. New -> "Btrfsck"
          1. See -> https://bbs.archlinux.org/viewtopic.php?id=156100
      7. Move -> "See also"

Edits: AdamT (Talk) 08:27, 28 November 2013 (UTC); AdamT (Talk) 04:03, 29 November 2013 (UTC); AdamT (Talk) 04:10, 29 November 2013 (UTC); AdamT (Talk) 02:35, 4 December 2013 (UTC)

I hope to get to this soon if there are no objections or comments. AdamT (Talk) 20:09, 4 January 2014 (UTC)
I believe all of his has been addressed except for quotas. -- Rdeckard (talk) 17:27, 25 July 2016 (UTC)

Examples

  • Linux 3.14
    • Btrfs on a SSD for system installation and an emphasis on maximizing performance.
    noatime,compress=lzo,ssd,discard,autodefrag
    • Btrfs on a HDD for archival purposes with an emphasis on maximizing space.
    noatime,compress-force=zlib,autodefrag,nospace_cache

Why I removed the Examples section:

  • autodefrag (since 3.0): Will detect random writes into existing files and kick off background defragging. It is well suited to bdb or sqlite databases, but not virtualization images or big databases (yet). Once the developers make sure it doesn't defrag files over and over again, they'll move this toward the default.
    • Unstable, may not be what the user wants (e.g. what is stored on the HDD/SSD?)
  • compress-force=<method> - Enable compression even for files that don't compress well, like videos and dd images of disk
  • nospace_cache (since 3.2): Disable free space caching. This may slow down new writes
    • May slow down new writes.. Why would you disable free space caching? (see space_cache)

I'm sorry, but at the moment I simple not recommend to follow the examples; not that they are wrong, but simple because they differ for each user/machine.

Beta990 (talk) 23:31, 8 August 2014 (UTC)

The edit in question is [1].
Does the argument about autodefrag option apply also to "normal" foreground defragmentation, i.e. btrfs filesystem defragment?
-- Lahwaacz (talk) 07:13, 9 August 2014 (UTC)
To be honest, I don't really know. Maybe we need to contact the btrfs devs?
The bit that scares me off: 'Once the developers make sure it doesn't defrag files over and over again, they'll move this toward the default.'
Beta990 (talk) 11:04, 9 August 2014 (UTC)
It's possible I don't understand the question, but wouldn't foreground defragging be unaffected by autodefrag? I think the idea is that some applications constantly make random writes throughout their lifetimes, and Btrfs has a poor (or non-existent) algorithm for throttling defrag operations, meaning that Btrfs will constantly be starting new defrag jobs for as long as the application is running. This results in a lot of seeking and writing, and poor performance.
On-demand (foreground) defragging, on the other hand, only happens when the user requests it. This means that the file will begin to become fragmented again after the defrag has ended, until the next one. The way I see it is: use autodefrag if low seek times for reads are a requirement (sequential reads, use case: database lookup tables, row iteration), otherwise use btrfs fi defragment (as needed, or via a cron job) if one wants to save performance from the writes caused by block relocation, when sequential reads are rare (use case: mostly write-only archival data, disk images).
It really depends on the application. Does it cache lookup tables and then do random reads from there? Does it make random writes to files that are normally sequentially read? Does it simply archive data that is rarely read back? Personally I would lean towards not using autodefrag, because many applications take fragmentation into consideration from the time the file is created. For example, databases often preallocate data in large chunks to attempt to avoid fragmentation. Disk images are often preallocated to a fixed size that rarely changes. Perhaps do an on-demand defrag every now and then, just in case.
This is analogous to the "fstrim vs -o discard" debate within Btrfs and other filesystems. The discard mount option causes a discard to be dispatched every time a block is freed. This is a (sometimes?) blocking operation, meaning the filesystem has to wait for the hardware to handle the discard before moving on. In this case, it makes more sense to dispatch discards for all free blocks in batches during periods of low I/O load.
I'm by no means an expert on this, I just thought I would share my reasoning. Getting in touch with the developers definitely couldn't hurt. As for what this means to the article, I don't recommend users go overriding defaults just because they saw an options line on the wiki that claims better performance, unless they know what those options are actually doing. Defaults are defaults for a reason. EscapedNull (talk) 02:00, 19 February 2015 (UTC)

btrfs check

[Moved from ArchWiki talk:Reports#Btrfs. — Kynikos (talk) 03:39, 20 April 2015 (UTC)]

I attempted to fix the style issues reported with this edit. I'm not convinced by the idea of a 'scenario' though. I would have thought just mentioning the existence of the command, outlining its use and linking to the relevant documentation would be enough. -- Chazza (talk) 11:55, 19 April 2015 (UTC)

Subvolumes

The manpage btrf-subvolumes is really good. Do we really even need a section here about subvolumes? - Rdeckard (talk) 02:04, 31 March 2016 (UTC)

Automatic snapshots on each boot

Moved from Talk:Btrfs_-_Tips_and_tricks -- Rdeckard (talk) 13:09, 19 July 2016 (UTC)

I agree the whole article does not really fit in a "tip" category, but I am not sure removing the Btrfs_-_Tips and tricks#Automatic snapshots on each boot section is the best approach. Essentially this would render the rest of the article useless. It would be nice to have a similar section/systemd unit in Snapper, indeed. It is a different (additional) tool though and to me (I have not tried either) this article reads like a smart solution for a problem that the snapper tool has not covered in a similarly easy way years later. I think it would be better to remove the template here and add a section with an appropriate expansion template into Snapper, linking to this section. Once that is in place, it would be easier to decide about this article (see also above talk item). --Indigo (talk) 08:03, 30 March 2016 (UTC)

I've begun this section here [Snapper#Snapshots_on_boot] on the snapper page that should solve the same problem this script solves. I do think that either the script needs to be moved off the wiki, or the section here needs to be removed completely. Thoughts? -- Rdeckard (talk) 13:11, 19 July 2016 (UTC)