Difference between revisions of "Talk:Btrfs"

From ArchWiki
Jump to: navigation, search
(Troubleshooting is now outdated)
(Subvolumes: new section)
 
(66 intermediate revisions by 18 users not shown)
Line 1: Line 1:
== Comment about stability ==
+
{{Note|The name of the file system does not seem to have an official capitalization, so the rule for the ArchWiki has been arbitrarily set to use "Btrfs" (capital "B" and the rest lower-case); "btrfs" and "BTRFS" are therefore wrong spellings in this wiki. -- [[User:Kynikos|Kynikos]] ([[User talk:Kynikos|talk]]) 09:36, 6 January 2014 (UTC)}}
Btrfs is still under heavy development; it might be useful if someone familiar with btrfs were to add a section commenting on its overall stability. [[User:Vikingurinn|Vikingurinn]] ([[User talk:Vikingurinn|talk]]) 16:52, 17 December 2012 (UTC)
+
  
== <s> Troubleshooting is now outdated </s> ==
+
== Things to do ==
Would be great of some early adopters would modernize this section now that the stable release of btrfs-progs has hit [testing].  As the warning says, the package now includes the btrfsfsck which can fix problems on the filesystem!
+
  
[[User:Graysky|Graysky]] 19:27, 28 March 2012 (EDT)
+
Here are some items that seem to need addressing on this article. I am working on various parts of it at the moment.
: Fixed. Close. -- [[User:Fengchao|Fengchao]] ([[User talk:Fengchao|talk]]) 03:31, 11 May 2013 (UTC)
+
# Reflow headings:
 +
## Goals: [[Effective_Use_of_Headers#Use_of_headers_and_headings]]
 +
### Helpful ideas:
 +
#### [[Effective_Use_of_Headers#False_multi-level_structure]]
 +
##### Side-note
 +
##### Auxiliary information
 +
#### [[Effective_Use_of_Headers#Multi-level_structure]]
 +
##### Group arguments
 +
##### Alternative arguments
 +
##### Contradictory arguments
 +
#### [[Effective_Use_of_Headers#The_header_text]]
 +
#### [[Article_Naming_Guidelines]]
 +
##### Specificity, "Descriptivity," Brevity, "Expandibility,"
 +
### See [https://btrfs.wiki.kernel.org/index.php/Main_Page Btrfs Wiki] for ideas.
 +
### See [[Beginners' Guide]] for ideas.
 +
## Proposed changes
 +
### "Installation" -> "'''Preparation'''"
 +
#### New -> "'''System Requirements'''"
 +
#### Remove -> "Additional packages"
 +
##### Move -> links to "See also" or "Related".
 +
#### "Partitioning" -> "'''Prepare the storage drive'''"
 +
##### '''OR:''' "Partitioning" (as a child) -> "'''''Choose a partition scheme'''''"
 +
### New -> "'''Create file systems'''"
 +
#### "Creating a new file system" -> "'''New file systems'''"
 +
##### "'''Examples'''"
 +
###### New -> "'''Single-drive file systems'''"
 +
###### "Multi-device filesystem and RAID feature" -> "'''Multi-drive file systems'''"
 +
####### Reduce content, provide links to "Btrfs Wiki" instead.
 +
#### "Convert from Ext3/4" -> "'''Convert existing file system'''"
 +
### New -> "'''Configuring the file system'''"
 +
#### '''OR:''' Move -> "'''Tips and tricks'''"
 +
#### Move -> "'''Mount options'''"
 +
##### "Copy-On-Write (CoW)" -> "'''Copy-on-write'''"
 +
###### "Checkpoint Interval" -> '''Merge'''
 +
##### Move -> "'''Compression'''"
 +
#### New -> "'''Skinny extents'''"
 +
#### Move -> "'''Sub-volumes'''"
 +
#### New -> "'''Quotas'''"
 +
### New -> "'''File system management'''"
 +
#### Move -> "'''Balance'''"
 +
#### Move -> "'''Defragmentation'''"
 +
#### Move -> "'''Display used/free space'''"
 +
#### Move -> "'''Balance'''"
 +
#### Move -> "'''Snapshots'''"
 +
### Move -> "'''Tips and tricks'''"
 +
### New -> "'''Troubleshooting'''"
 +
#### Move -> "'''Encryption'''"
 +
#### Move -> "'''GRUB'''"
 +
#### Move -> "'''Swap file'''"
 +
#### New -> "'''Btrfsck'''"
 +
##### See -> https://bbs.archlinux.org/viewtopic.php?id=156100
 +
### Move -> "'''See also'''"
  
== <s> Convert Ext3/4 to Btrfs </s> ==
+
Edits:
 +
[[User:AdamT|AdamT]] ([[User_talk:AdamT|Talk]]) 08:27, 28 November 2013 (UTC);
 +
[[User:AdamT|AdamT]] ([[User_talk:AdamT|Talk]]) 04:03, 29 November 2013 (UTC);
 +
[[User:AdamT|AdamT]] ([[User_talk:AdamT|Talk]]) 04:10, 29 November 2013 (UTC);
 +
[[User:AdamT|AdamT]] ([[User_talk:AdamT|Talk]]) 02:35, 4 December 2013 (UTC)
  
Just wondering about step #3: "Setup the network"
+
: I hope to get to this soon if there are no objections or comments. [[User:AdamT|AdamT]] ([[User_talk:AdamT|Talk]]) 20:09, 4 January 2014 (UTC)
I can't see any reason that this is needed. [[User:Capturts|Capturts]] ([[User talk:Capturts|talk]]) 18:35, 23 November 2012 (UTC)
+
  
: It's because of Step #5: Installing btrfs-progs. This is based off of the old install mediums, which  had local versions of programs, unless the remote repositories were enabled like in step #2. Since the new install mediums are remote-only (however they might have some filesystem tools on them), the entire section needs updating. [[User:Klink-a-dink-dink|Klink-a-dink-dink]] ([[User talk:Klink-a-dink-dink|talk]]) 01:06, 24 November 2012 (UTC)
+
== Examples ==
:: Fixed. Close. -- [[User:Fengchao|Fengchao]] ([[User talk:Fengchao|talk]]) 03:24, 11 May 2013 (UTC)
+
  
== RAID-1 or RAID-5? ==
+
* '''Linux 3.14'''
 +
** Btrfs on a SSD for system installation and an emphasis on maximizing performance.
 +
*:{{bc|1=noatime,compress=lzo,ssd,discard,autodefrag}}
 +
** Btrfs on a HDD for archival purposes with an emphasis on maximizing space.
 +
*: {{bc|1=noatime,compress-force=zlib,autodefrag,nospace_cache}}
  
"''3 1TB disks in an md based raid1 yields a /dev/md0 with 1TB free space and the ability to safely loose 2 disks without losing data. '''3 1TB disks in a btrfs volume with data=raid1 will allow the storage of approximately 1.5TB of data before reporting full. Only 1 disk can safely be lost without losing data'''.''"
+
Why I removed the Examples section:
 +
* autodefrag (since 3.0): Will detect random writes into existing files and kick off background defragging. It is well suited to bdb or sqlite databases, but not virtualization images or big databases (yet). Once the developers make sure it doesn't defrag files over and over again, they'll move this toward the default.
 +
** Unstable, may not be what the user wants (e.g. what is stored on the HDD/SSD?)
 +
* compress-force=<method> - Enable compression even for files that don't compress well, like videos and dd images of disk
 +
** Is this safe? Could this lead to corruption of files? There are bugs reported about using lzo as compression (e.g. https://bugs.archlinux.org/task/41155).
 +
* nospace_cache (since 3.2): Disable free space caching. This may slow down new writes
 +
** May slow down new writes.. Why would you disable free space caching? (see space_cache)
  
That sounds more like some kind of weird, inefficient RAID-5 arrangement than RAID-1. I don't know btrfs enough to say it's wrong, but I know RAID-1 enough to question it?
+
I'm sorry, but at the moment I simple not recommend to follow the examples; not that they are wrong, but simple because they differ for each user/machine.
  
[[User:Fukawi2|Fukawi2]] ([[User talk:Fukawi2|talk]]) 23:52, 29 April 2013 (UTC)
+
[[User:Beta990|Beta990]] ([[User talk:Beta990|talk]]) 23:31, 8 August 2014 (UTC)
 +
 
 +
:The edit in question is [https://wiki.archlinux.org/index.php?title=Btrfs&diff=next&oldid=329381].
 +
:Does the argument about {{ic|autodefrag}} option apply also to "normal" foreground defragmentation, i.e. {{ic|btrfs filesystem defragment}}?
 +
:-- [[User:Lahwaacz|Lahwaacz]] ([[User talk:Lahwaacz|talk]]) 07:13, 9 August 2014 (UTC)
 +
 
 +
::To be honest, I don't really know. Maybe we need to contact the btrfs devs?
 +
::The bit that scares me off: 'Once the developers make sure it doesn't defrag files over and over again, they'll move this toward the default.'
 +
::[[User:Beta990|Beta990]] ([[User talk:Beta990|talk]]) 11:04, 9 August 2014 (UTC)
 +
 
 +
:::It's possible I don't understand the question, but wouldn't foreground defragging be unaffected by {{ic|autodefrag}}? I think the idea is that some applications constantly make random writes throughout their lifetimes, and Btrfs has a poor (or non-existent) algorithm for throttling defrag operations, meaning that Btrfs will constantly be starting new defrag jobs for as long as the application is running. This results in a lot of seeking and writing, and poor performance.
 +
:::On-demand (foreground) defragging, on the other hand, only happens when the user requests it. This means that the file will begin to become fragmented again after the defrag has ended, until the next one. The way I see it is: use {{ic|autodefrag}} if low seek times for reads are a requirement (sequential reads, use case: database lookup tables, row iteration), otherwise use {{ic|btrfs fi defragment}} (as needed, or via a cron job) if one wants to save performance from the writes caused by block relocation, when sequential reads are rare (use case: mostly write-only archival data, disk images).
 +
:::It really depends on the application. Does it cache lookup tables and then do random reads from there? Does it make random writes to files that are normally sequentially read? Does it simply archive data that is rarely read back? Personally I would lean towards '''not''' using {{ic|autodefrag}}, because many applications take fragmentation into consideration from the time the file is created. For example, databases often preallocate data in large chunks to attempt to avoid fragmentation. Disk images are often preallocated to a fixed size that rarely changes. Perhaps do an on-demand defrag every now and then, just in case.
 +
:::This is analogous to the "fstrim vs {{ic|-o discard}}" debate within Btrfs and other filesystems. The discard mount option causes a discard to be dispatched every time a block is freed. This is a (sometimes?) blocking operation, meaning the filesystem has to wait for the hardware to handle the discard before moving on. In this case, it makes more sense to dispatch discards for all free blocks in batches during periods of low I/O load.
 +
:::I'm by no means an expert on this, I just thought I would share my reasoning. Getting in touch with the developers definitely couldn't hurt. As for what this means to the article, I don't recommend users go overriding defaults just because they saw an options line on the wiki that claims better performance, unless they know what those options are actually doing. Defaults are defaults for a reason. [[User:EscapedNull|EscapedNull]] ([[User talk:EscapedNull|talk]]) 02:00, 19 February 2015 (UTC)
 +
 
 +
== <s>Btrfs-progs 3.18 introduced 'btrfs filesystem usage' command</s> ==
 +
 
 +
From the btrfs changelog [https://btrfs.wiki.kernel.org/index.php/Changelog] :
 +
:filesystem usage - give an overview of fs usage in a way that's more comprehensible than existing 'fi df'
 +
I haven't upgraded quite yet so I don't have any sample output to post on the wiki. I'll post some info about it in [[Btrfs#Displaying_used.2Ffree_space]] when I get around to upgrading, but maybe someone will beat me to it! [[User:EscapedNull|EscapedNull]] ([[User talk:EscapedNull|talk]]) 02:06, 19 February 2015 (UTC)
 +
 
 +
:I added information about this command. [[User:Alad|Alad]] recommends that if you would like to see an example of the output of the command, you should enter it on your own system, despite other commands in this section including examples. [[User:ImNtReal|ImNtReal]] ([[User talk:ImNtReal|talk]]) 23:38, 12 February 2016 (UTC)
 +
 
 +
::See [https://wiki.archlinux.org/index.php?title=Btrfs&diff=420489&oldid=420285] -- [[User:Alad|Alad]] ([[User talk:Alad|talk]]) 14:34, 13 February 2016 (UTC)
 +
 
 +
:::Any further debate on this topic? Otherwise we're good. [[User:Silverhammermba|Silverhammermba]] ([[User talk:Silverhammermba|talk]]) 15:10, 15 February 2016 (UTC)
 +
 
 +
== btrfs check ==
 +
 
 +
''[Moved from [[ArchWiki talk:Reports#Btrfs]]. — [[User:Kynikos|Kynikos]] ([[User talk:Kynikos|talk]]) 03:39, 20 April 2015 (UTC)]''
 +
 
 +
I attempted to fix the style issues reported with [https://wiki.archlinux.org/index.php?title=Btrfs&diff=prev&oldid=370167 this] edit. I'm not convinced by the idea of a 'scenario' though. I would have thought just mentioning the existence of the command, outlining its use and linking to the relevant documentation would be enough. -- [[User:Chazza|Chazza]] ([[User talk:Chazza|talk]]) 11:55, 19 April 2015 (UTC)
 +
 
 +
== Subvolumes ==
 +
 
 +
The manpage ''btrf-subvolumes'' is really good. Do we really even need a section here about subvolumes? - [[User:Rdeckard|Rdeckard]] ([[User talk:Rdeckard|talk]]) 02:04, 31 March 2016 (UTC)

Latest revision as of 02:04, 31 March 2016

Note: The name of the file system does not seem to have an official capitalization, so the rule for the ArchWiki has been arbitrarily set to use "Btrfs" (capital "B" and the rest lower-case); "btrfs" and "BTRFS" are therefore wrong spellings in this wiki. -- Kynikos (talk) 09:36, 6 January 2014 (UTC)

Things to do

Here are some items that seem to need addressing on this article. I am working on various parts of it at the moment.

  1. Reflow headings:
    1. Goals: Effective_Use_of_Headers#Use_of_headers_and_headings
      1. Helpful ideas:
        1. Effective_Use_of_Headers#False_multi-level_structure
          1. Side-note
          2. Auxiliary information
        2. Effective_Use_of_Headers#Multi-level_structure
          1. Group arguments
          2. Alternative arguments
          3. Contradictory arguments
        3. Effective_Use_of_Headers#The_header_text
        4. Article_Naming_Guidelines
          1. Specificity, "Descriptivity," Brevity, "Expandibility,"
      2. See Btrfs Wiki for ideas.
      3. See Beginners' Guide for ideas.
    2. Proposed changes
      1. "Installation" -> "Preparation"
        1. New -> "System Requirements"
        2. Remove -> "Additional packages"
          1. Move -> links to "See also" or "Related".
        3. "Partitioning" -> "Prepare the storage drive"
          1. OR: "Partitioning" (as a child) -> "Choose a partition scheme"
      2. New -> "Create file systems"
        1. "Creating a new file system" -> "New file systems"
          1. "Examples"
            1. New -> "Single-drive file systems"
            2. "Multi-device filesystem and RAID feature" -> "Multi-drive file systems"
              1. Reduce content, provide links to "Btrfs Wiki" instead.
        2. "Convert from Ext3/4" -> "Convert existing file system"
      3. New -> "Configuring the file system"
        1. OR: Move -> "Tips and tricks"
        2. Move -> "Mount options"
          1. "Copy-On-Write (CoW)" -> "Copy-on-write"
            1. "Checkpoint Interval" -> Merge
          2. Move -> "Compression"
        3. New -> "Skinny extents"
        4. Move -> "Sub-volumes"
        5. New -> "Quotas"
      4. New -> "File system management"
        1. Move -> "Balance"
        2. Move -> "Defragmentation"
        3. Move -> "Display used/free space"
        4. Move -> "Balance"
        5. Move -> "Snapshots"
      5. Move -> "Tips and tricks"
      6. New -> "Troubleshooting"
        1. Move -> "Encryption"
        2. Move -> "GRUB"
        3. Move -> "Swap file"
        4. New -> "Btrfsck"
          1. See -> https://bbs.archlinux.org/viewtopic.php?id=156100
      7. Move -> "See also"

Edits: AdamT (Talk) 08:27, 28 November 2013 (UTC); AdamT (Talk) 04:03, 29 November 2013 (UTC); AdamT (Talk) 04:10, 29 November 2013 (UTC); AdamT (Talk) 02:35, 4 December 2013 (UTC)

I hope to get to this soon if there are no objections or comments. AdamT (Talk) 20:09, 4 January 2014 (UTC)

Examples

  • Linux 3.14
    • Btrfs on a SSD for system installation and an emphasis on maximizing performance.
    noatime,compress=lzo,ssd,discard,autodefrag
    • Btrfs on a HDD for archival purposes with an emphasis on maximizing space.
    noatime,compress-force=zlib,autodefrag,nospace_cache

Why I removed the Examples section:

  • autodefrag (since 3.0): Will detect random writes into existing files and kick off background defragging. It is well suited to bdb or sqlite databases, but not virtualization images or big databases (yet). Once the developers make sure it doesn't defrag files over and over again, they'll move this toward the default.
    • Unstable, may not be what the user wants (e.g. what is stored on the HDD/SSD?)
  • compress-force=<method> - Enable compression even for files that don't compress well, like videos and dd images of disk
  • nospace_cache (since 3.2): Disable free space caching. This may slow down new writes
    • May slow down new writes.. Why would you disable free space caching? (see space_cache)

I'm sorry, but at the moment I simple not recommend to follow the examples; not that they are wrong, but simple because they differ for each user/machine.

Beta990 (talk) 23:31, 8 August 2014 (UTC)

The edit in question is [1].
Does the argument about autodefrag option apply also to "normal" foreground defragmentation, i.e. btrfs filesystem defragment?
-- Lahwaacz (talk) 07:13, 9 August 2014 (UTC)
To be honest, I don't really know. Maybe we need to contact the btrfs devs?
The bit that scares me off: 'Once the developers make sure it doesn't defrag files over and over again, they'll move this toward the default.'
Beta990 (talk) 11:04, 9 August 2014 (UTC)
It's possible I don't understand the question, but wouldn't foreground defragging be unaffected by autodefrag? I think the idea is that some applications constantly make random writes throughout their lifetimes, and Btrfs has a poor (or non-existent) algorithm for throttling defrag operations, meaning that Btrfs will constantly be starting new defrag jobs for as long as the application is running. This results in a lot of seeking and writing, and poor performance.
On-demand (foreground) defragging, on the other hand, only happens when the user requests it. This means that the file will begin to become fragmented again after the defrag has ended, until the next one. The way I see it is: use autodefrag if low seek times for reads are a requirement (sequential reads, use case: database lookup tables, row iteration), otherwise use btrfs fi defragment (as needed, or via a cron job) if one wants to save performance from the writes caused by block relocation, when sequential reads are rare (use case: mostly write-only archival data, disk images).
It really depends on the application. Does it cache lookup tables and then do random reads from there? Does it make random writes to files that are normally sequentially read? Does it simply archive data that is rarely read back? Personally I would lean towards not using autodefrag, because many applications take fragmentation into consideration from the time the file is created. For example, databases often preallocate data in large chunks to attempt to avoid fragmentation. Disk images are often preallocated to a fixed size that rarely changes. Perhaps do an on-demand defrag every now and then, just in case.
This is analogous to the "fstrim vs -o discard" debate within Btrfs and other filesystems. The discard mount option causes a discard to be dispatched every time a block is freed. This is a (sometimes?) blocking operation, meaning the filesystem has to wait for the hardware to handle the discard before moving on. In this case, it makes more sense to dispatch discards for all free blocks in batches during periods of low I/O load.
I'm by no means an expert on this, I just thought I would share my reasoning. Getting in touch with the developers definitely couldn't hurt. As for what this means to the article, I don't recommend users go overriding defaults just because they saw an options line on the wiki that claims better performance, unless they know what those options are actually doing. Defaults are defaults for a reason. EscapedNull (talk) 02:00, 19 February 2015 (UTC)

Btrfs-progs 3.18 introduced 'btrfs filesystem usage' command

From the btrfs changelog [2] :

filesystem usage - give an overview of fs usage in a way that's more comprehensible than existing 'fi df'

I haven't upgraded quite yet so I don't have any sample output to post on the wiki. I'll post some info about it in Btrfs#Displaying_used.2Ffree_space when I get around to upgrading, but maybe someone will beat me to it! EscapedNull (talk) 02:06, 19 February 2015 (UTC)

I added information about this command. Alad recommends that if you would like to see an example of the output of the command, you should enter it on your own system, despite other commands in this section including examples. ImNtReal (talk) 23:38, 12 February 2016 (UTC)
See [3] -- Alad (talk) 14:34, 13 February 2016 (UTC)
Any further debate on this topic? Otherwise we're good. Silverhammermba (talk) 15:10, 15 February 2016 (UTC)

btrfs check

[Moved from ArchWiki talk:Reports#Btrfs. — Kynikos (talk) 03:39, 20 April 2015 (UTC)]

I attempted to fix the style issues reported with this edit. I'm not convinced by the idea of a 'scenario' though. I would have thought just mentioning the existence of the command, outlining its use and linking to the relevant documentation would be enough. -- Chazza (talk) 11:55, 19 April 2015 (UTC)

Subvolumes

The manpage btrf-subvolumes is really good. Do we really even need a section here about subvolumes? - Rdeckard (talk) 02:04, 31 March 2016 (UTC)