Difference between revisions of "User:Tomato"

From ArchWiki
Jump to: navigation, search
m (Log parser: fix formatting)
 
(34 intermediate revisions by the same user not shown)
Line 1: Line 1:
just my personal scrapbook
+
''just my personal scrapbook''
  
=== Configure file sharing from KDE4 using samba ===
+
== lessfs ==
Sadly, this functionality is implemented using SUID perl script, and as Archlinux does not support SUID scripts nor it has perl-suid package we will have to get our hands dirty.
+
=== Sizes of different databases in lessfs ===
 +
With a 64k blocksize administration overhead would be approx 5~10% of the size of blockdata.tch
 +
=== Good all-round mount options ===
 +
<pre>-o kernel_cache,negative_timeout=0,entry_timeout=0,attr_timeout=0,use_ino,readdir_ino\
 +
      ,default_permissions,allow_other,big_writes,max_read=65536,max_write=65536</pre>
  
{{Note|Works with simple file sharing only at the moment}}
+
=== Filesystem, mounting but not working ===
 +
Each and every partition with datafiles need 10% of free space, lessfs will freeze all writes to the partition for as long as the free space isn't aviable.
  
Work plan:
+
Version 1.0.4 has new config parameters:
# install packages
+
<pre>
# configure samba
+
MIN_SPACE_FREE=2
# modify {{Codeline|fileshareset}} scripts
+
MIN_SPACE_CLEAN=5
# create C wrapper for {{Codeline|fileshareset}} to work as a SUID binary
+
CLEAN_PROGRAM=/tmp/clean.sh
# configure KDE side of things
+
</pre>
# share a folder
+
<code>MIN_SPACE_FREE</code> defines minimum space left (in percent?) on the device, <code>MIN_SPACE_CLEAN</code> defines the amount of space left that triggers the <code>CLEAN_PROGRAM</code>
  
Packages we will need:
+
=== Bucket sizes ===
* {{Package Official|kdebase-dolphin}} this is the interface we will use to share folders
+
Usually when performance drops after reaching a certain database size this indicates that the bucketsize selected in <code>/etc/lessfs.cfg</code> is to low. The default 'lessfs.cfg' example included with the code only allows the database to grow up to: 2GB after which performance degrades.
* {{Package Official|samba}} for the server that will share files
+
To store 50GB the bucketsize should be at least : 27262976
* {{Package Official|kdenetwork-filesharing}} for System Settings File Sharing part
+
<pre>
* {{Package Official|kdebase-runtime}} basic KDE workspace (but if you're reading this, chances are you already have it installed)
+
2MB bucketsize equals 1000000 database entries.
* {{Package Official|kdelibs3}} for {{Codeline|fileshareset}} (not sure about it, but as of KDE4.4.0 it's still a dependency)
+
To store 50GB :
 +
50GB = 50*1024*1024*1024 = 53687091200 bytes
 +
53687091200/4096 (4k blocksize) = 13107200 entries in the databases.
 +
13107200/1000000 (divide by 10^6 entries)
 +
13 *2 (10^6 entries requires 2MB) = 26
 +
26 * 1024*1024 = 27262976 -> This is the bucketsize you need for 50GB with a 4k blocksize.
 +
</pre>
 +
(from http://sourceforge.net/projects/lessfs/forums/forum/932600/topic/3304815)
  
==== Installing requisites ====
+
If you ever need to change the _BS settings, just unmount lessfs, change the settings, mount lessfs and run defrag.
Install all packages and their dependencies:
+
  
{{Codeline|pacman -S kdebase-dolphin samba kdenetwork-filesharing kdebase-runtime kdelibs3}}
+
=== Files to put on different spindles ===
 +
For max performance compile with –with-sha3 and put the FILEBLOCK and BLOCKUSAGE databases each on there own raid one disk set. The faster these disks are the better. BLOCKDATA can be on raid5/6.
 +
(from http://www.lessfs.com/wordpress/?p=295)
  
Log out and log in again (kdelibs3 are in non standard place and so the <code>PATH</code> must be updated)
+
=== Log parser ===
 +
<pre>
 +
grep 'lessfs\[8118\]' /var/log/messages.log | grep -Eo \
 +
  'size .* bytes \: lzo compressed bytes .* and .* duplicate blocks' |\
 +
  awk ' { SUM += $2 ; SUMCP += $8 ; DUP += $10 } \
 +
  END { print "total size " SUM/1024/1024/1024 "GiB, compressed size " \
 +
  SUMCP/1024/1024/1024 "GiB (ratio 1 : " SUM/SUMCP ") with " DUP\
 +
  " duplicate blocks (" DUP*64/1024/1024 "GiB deduplicated)" }' </pre>
  
==== Configuring samba ====
+
change <code>lessfs\[8118\]</code> to the PID of lessfs you want to monitor and <code>DUP*64</code> to the block size in kiB you use in it.
A basic config file works OK, though you'll need to create Samba users (with <code>smbpasswd -a &lt;user name&gt;</code> as root):
+
{{File|name=smb.conf|content=[global]
+
workgroup=HOME
+
server string = Samba Server
+
log file = /var/log/samba/&m.log
+
max log size = 50
+
load printers = No
+
dns proxy = No
+
  
[homes]
+
== Configuring Samba Domain/File server. ==
comment = Home Directories
+
read only = No
+
browsable = No
+
browseable = No
+
}}
+
  
Remember to add samba to <code>DAEMONS</code> in <code>/etc/rc.conf</code> if you want it to start at boot time.
+
=== What we want ===
 +
basically, everything that Samba has to offer:
 +
* save and share files (duh)
 +
* edit files/folders ACLs to the maximum extent possible for Samba<br />http://www.samba.org/samba/docs/man/Samba-HOWTO-Collection/AccessControls.html
 +
** mapping of permission inheritance
 +
* save file attributes (archive, system, hidden, read only) on samba server without screwing UNIX permissions
 +
* have the ability to manage users/computers in domain without the need to use UID 0 (root) account<br />http://www.samba.org/samba/docs/man/Samba-HOWTO-Collection/rights.html
 +
* automatic creation of user folders
 +
* Previous Version share feature
 +
* enforcement of log-on hours, password complexity, password age and requests for password change, and group policies <br />http://wiki.samba.org/index.php/Implementing_System_Policies_with_Samba<br />http://www.samba.org/samba/docs/man/Samba-HOWTO-Collection/PolicyMgmt.html
 +
* view and set file system quota
 +
* time server/syncronisation
 +
* printing with generic drivers to CUPS (optional)
  
==== Modifying fileshareset scripts ====
+
guide:
 
+
https://help.ubuntu.com/community/SettingUpSambaPDC
 
+
magic:
+
kcmshell4 fileshare
+

Latest revision as of 11:40, 6 March 2010

just my personal scrapbook

lessfs

Sizes of different databases in lessfs

With a 64k blocksize administration overhead would be approx 5~10% of the size of blockdata.tch

Good all-round mount options

-o kernel_cache,negative_timeout=0,entry_timeout=0,attr_timeout=0,use_ino,readdir_ino\
      ,default_permissions,allow_other,big_writes,max_read=65536,max_write=65536

Filesystem, mounting but not working

Each and every partition with datafiles need 10% of free space, lessfs will freeze all writes to the partition for as long as the free space isn't aviable.

Version 1.0.4 has new config parameters:

MIN_SPACE_FREE=2
MIN_SPACE_CLEAN=5
CLEAN_PROGRAM=/tmp/clean.sh

MIN_SPACE_FREE defines minimum space left (in percent?) on the device, MIN_SPACE_CLEAN defines the amount of space left that triggers the CLEAN_PROGRAM

Bucket sizes

Usually when performance drops after reaching a certain database size this indicates that the bucketsize selected in /etc/lessfs.cfg is to low. The default 'lessfs.cfg' example included with the code only allows the database to grow up to: 2GB after which performance degrades. To store 50GB the bucketsize should be at least : 27262976

2MB bucketsize equals 1000000 database entries.
To store 50GB :
50GB = 50*1024*1024*1024 = 53687091200 bytes
53687091200/4096 (4k blocksize) = 13107200 entries in the databases.
13107200/1000000 (divide by 10^6 entries)
13 *2 (10^6 entries requires 2MB) = 26
26 * 1024*1024 = 27262976 -> This is the bucketsize you need for 50GB with a 4k blocksize.

(from http://sourceforge.net/projects/lessfs/forums/forum/932600/topic/3304815)

If you ever need to change the _BS settings, just unmount lessfs, change the settings, mount lessfs and run defrag.

Files to put on different spindles

For max performance compile with –with-sha3 and put the FILEBLOCK and BLOCKUSAGE databases each on there own raid one disk set. The faster these disks are the better. BLOCKDATA can be on raid5/6. (from http://www.lessfs.com/wordpress/?p=295)

Log parser

grep 'lessfs\[8118\]' /var/log/messages.log | grep -Eo \
  'size .* bytes \: lzo compressed bytes .* and .* duplicate blocks' |\
  awk ' { SUM += $2 ; SUMCP += $8 ; DUP += $10 } \
  END { print "total size " SUM/1024/1024/1024 "GiB, compressed size " \
  SUMCP/1024/1024/1024 "GiB (ratio 1 : " SUM/SUMCP ") with " DUP\
  " duplicate blocks (" DUP*64/1024/1024 "GiB deduplicated)" }' 

change lessfs\[8118\] to the PID of lessfs you want to monitor and DUP*64 to the block size in kiB you use in it.

Configuring Samba Domain/File server.

What we want

basically, everything that Samba has to offer:

guide: https://help.ubuntu.com/community/SettingUpSambaPDC