Difference between revisions of "User:Tomato"

From ArchWiki
Jump to: navigation, search
(lessfs)
m (Log parser: fix formatting)
 
(8 intermediate revisions by the same user not shown)
Line 2: Line 2:
  
 
== lessfs ==
 
== lessfs ==
 +
=== Sizes of different databases in lessfs ===
 +
With a 64k blocksize administration overhead would be approx 5~10% of the size of blockdata.tch
 +
=== Good all-round mount options ===
 +
<pre>-o kernel_cache,negative_timeout=0,entry_timeout=0,attr_timeout=0,use_ino,readdir_ino\
 +
      ,default_permissions,allow_other,big_writes,max_read=65536,max_write=65536</pre>
 +
 +
=== Filesystem, mounting but not working ===
 +
Each and every partition with datafiles need 10% of free space, lessfs will freeze all writes to the partition for as long as the free space isn't aviable.
 +
 +
Version 1.0.4 has new config parameters:
 +
<pre>
 +
MIN_SPACE_FREE=2
 +
MIN_SPACE_CLEAN=5
 +
CLEAN_PROGRAM=/tmp/clean.sh
 +
</pre>
 +
<code>MIN_SPACE_FREE</code> defines minimum space left (in percent?) on the device, <code>MIN_SPACE_CLEAN</code> defines the amount of space left that triggers the <code>CLEAN_PROGRAM</code>
 +
 
=== Bucket sizes ===
 
=== Bucket sizes ===
 
Usually when performance drops after reaching a certain database size this indicates that the bucketsize selected in <code>/etc/lessfs.cfg</code> is to low. The default 'lessfs.cfg' example included with the code only allows the database to grow up to: 2GB after which performance degrades.
 
Usually when performance drops after reaching a certain database size this indicates that the bucketsize selected in <code>/etc/lessfs.cfg</code> is to low. The default 'lessfs.cfg' example included with the code only allows the database to grow up to: 2GB after which performance degrades.
Line 15: Line 32:
 
</pre>
 
</pre>
 
(from http://sourceforge.net/projects/lessfs/forums/forum/932600/topic/3304815)
 
(from http://sourceforge.net/projects/lessfs/forums/forum/932600/topic/3304815)
 +
 +
If you ever need to change the _BS settings, just unmount lessfs, change the settings, mount lessfs and run defrag.
  
 
=== Files to put on different spindles ===
 
=== Files to put on different spindles ===
Line 20: Line 39:
 
(from http://www.lessfs.com/wordpress/?p=295)
 
(from http://www.lessfs.com/wordpress/?p=295)
  
Configuring Samba Domain/File server.
+
=== Log parser ===
 +
<pre>
 +
grep 'lessfs\[8118\]' /var/log/messages.log | grep -Eo \
 +
  'size .* bytes \: lzo compressed bytes .* and .* duplicate blocks' |\
 +
  awk ' { SUM += $2 ; SUMCP += $8 ; DUP += $10 } \
 +
  END { print "total size " SUM/1024/1024/1024 "GiB, compressed size " \
 +
  SUMCP/1024/1024/1024 "GiB (ratio 1 : " SUM/SUMCP ") with " DUP\
 +
  " duplicate blocks (" DUP*64/1024/1024 "GiB deduplicated)" }' </pre>
 +
 
 +
change <code>lessfs\[8118\]</code> to the PID of lessfs you want to monitor and <code>DUP*64</code> to the block size in kiB you use in it.
 +
 
 +
== Configuring Samba Domain/File server. ==
  
== What we want ==
+
=== What we want ===
 
basically, everything that Samba has to offer:
 
basically, everything that Samba has to offer:
 
* save and share files (duh)
 
* save and share files (duh)

Latest revision as of 11:40, 6 March 2010

just my personal scrapbook

lessfs

Sizes of different databases in lessfs

With a 64k blocksize administration overhead would be approx 5~10% of the size of blockdata.tch

Good all-round mount options

-o kernel_cache,negative_timeout=0,entry_timeout=0,attr_timeout=0,use_ino,readdir_ino\
      ,default_permissions,allow_other,big_writes,max_read=65536,max_write=65536

Filesystem, mounting but not working

Each and every partition with datafiles need 10% of free space, lessfs will freeze all writes to the partition for as long as the free space isn't aviable.

Version 1.0.4 has new config parameters:

MIN_SPACE_FREE=2
MIN_SPACE_CLEAN=5
CLEAN_PROGRAM=/tmp/clean.sh

MIN_SPACE_FREE defines minimum space left (in percent?) on the device, MIN_SPACE_CLEAN defines the amount of space left that triggers the CLEAN_PROGRAM

Bucket sizes

Usually when performance drops after reaching a certain database size this indicates that the bucketsize selected in /etc/lessfs.cfg is to low. The default 'lessfs.cfg' example included with the code only allows the database to grow up to: 2GB after which performance degrades. To store 50GB the bucketsize should be at least : 27262976

2MB bucketsize equals 1000000 database entries.
To store 50GB :
50GB = 50*1024*1024*1024 = 53687091200 bytes
53687091200/4096 (4k blocksize) = 13107200 entries in the databases.
13107200/1000000 (divide by 10^6 entries)
13 *2 (10^6 entries requires 2MB) = 26
26 * 1024*1024 = 27262976 -> This is the bucketsize you need for 50GB with a 4k blocksize.

(from http://sourceforge.net/projects/lessfs/forums/forum/932600/topic/3304815)

If you ever need to change the _BS settings, just unmount lessfs, change the settings, mount lessfs and run defrag.

Files to put on different spindles

For max performance compile with –with-sha3 and put the FILEBLOCK and BLOCKUSAGE databases each on there own raid one disk set. The faster these disks are the better. BLOCKDATA can be on raid5/6. (from http://www.lessfs.com/wordpress/?p=295)

Log parser

grep 'lessfs\[8118\]' /var/log/messages.log | grep -Eo \
  'size .* bytes \: lzo compressed bytes .* and .* duplicate blocks' |\
  awk ' { SUM += $2 ; SUMCP += $8 ; DUP += $10 } \
  END { print "total size " SUM/1024/1024/1024 "GiB, compressed size " \
  SUMCP/1024/1024/1024 "GiB (ratio 1 : " SUM/SUMCP ") with " DUP\
  " duplicate blocks (" DUP*64/1024/1024 "GiB deduplicated)" }' 

change lessfs\[8118\] to the PID of lessfs you want to monitor and DUP*64 to the block size in kiB you use in it.

Configuring Samba Domain/File server.

What we want

basically, everything that Samba has to offer:

guide: https://help.ubuntu.com/community/SettingUpSambaPDC