Difference between revisions of "User:Tomato"

From ArchWiki
Jump to: navigation, search
(Configure sudo)
m (Log parser: fix formatting)
 
(21 intermediate revisions by the same user not shown)
Line 1: Line 1:
just my personal scrapbook
+
''just my personal scrapbook''
  
=== Configure simple file sharing from KDE4 using samba ===
+
== lessfs ==
If you want to easily share files in read only mode go ahead and read this section, if you want to add write functionality too, you must read the section with advanced configuration.
+
=== Sizes of different databases in lessfs ===
 +
With a 64k blocksize administration overhead would be approx 5~10% of the size of blockdata.tch
 +
=== Good all-round mount options ===
 +
<pre>-o kernel_cache,negative_timeout=0,entry_timeout=0,attr_timeout=0,use_ino,readdir_ino\
 +
      ,default_permissions,allow_other,big_writes,max_read=65536,max_write=65536</pre>
  
{{Note|Sadly, simple file sharing functionality is implemented using SUID perl script, and as Archlinux does not support SUID scripts because of security reasons (just like any other recent UNIX like system) nor it has perl-suid package, we will have to get our hands dirty.}}
+
=== Filesystem, mounting but not working ===
 +
Each and every partition with datafiles need 10% of free space, lessfs will freeze all writes to the partition for as long as the free space isn't aviable.
  
Work plan:
+
Version 1.0.4 has new config parameters:
# install packages
+
<pre>
# configure samba
+
MIN_SPACE_FREE=2
# modify {{Codeline|fileshareset}} scripts
+
MIN_SPACE_CLEAN=5
# create C wrapper for {{Codeline|fileshareset}} to work as a SUID binary
+
CLEAN_PROGRAM=/tmp/clean.sh
# configure KDE side of things
+
</pre>
# share a folder
+
<code>MIN_SPACE_FREE</code> defines minimum space left (in percent?) on the device, <code>MIN_SPACE_CLEAN</code> defines the amount of space left that triggers the <code>CLEAN_PROGRAM</code>
  
Packages we will need:
+
=== Bucket sizes ===
* {{Package Official|kdebase-dolphin}} this is the interface we will use to share folders
+
Usually when performance drops after reaching a certain database size this indicates that the bucketsize selected in <code>/etc/lessfs.cfg</code> is to low. The default 'lessfs.cfg' example included with the code only allows the database to grow up to: 2GB after which performance degrades.
* {{Package Official|samba}} for the server that will share files
+
To store 50GB the bucketsize should be at least : 27262976
* {{Package Official|kdenetwork-filesharing}} for System Settings File Sharing part
+
<pre>
* {{Package Official|kdebase-runtime}} basic KDE workspace (but if you're reading this, chances are you already have it installed)
+
2MB bucketsize equals 1000000 database entries.
* {{Package Official|kdelibs3}} for {{Codeline|fileshareset}} (not sure about it, but as of KDE4.4.0 it's still a dependency)
+
To store 50GB :
* {{Package Official|gcc}} to compile SUID wrapper
+
50GB = 50*1024*1024*1024 = 53687091200 bytes
 +
53687091200/4096 (4k blocksize) = 13107200 entries in the databases.
 +
13107200/1000000 (divide by 10^6 entries)
 +
13 *2 (10^6 entries requires 2MB) = 26
 +
26 * 1024*1024 = 27262976 -> This is the bucketsize you need for 50GB with a 4k blocksize.
 +
</pre>
 +
(from http://sourceforge.net/projects/lessfs/forums/forum/932600/topic/3304815)
  
==== Installing requisites ====
+
If you ever need to change the _BS settings, just unmount lessfs, change the settings, mount lessfs and run defrag.
Install all packages and their dependencies:
+
  
{{Codeline|pacman -S kdebase-dolphin samba kdenetwork-filesharing kdebase-runtime kdelibs3 gcc}}
+
=== Files to put on different spindles ===
 +
For max performance compile with –with-sha3 and put the FILEBLOCK and BLOCKUSAGE databases each on there own raid one disk set. The faster these disks are the better. BLOCKDATA can be on raid5/6.
 +
(from http://www.lessfs.com/wordpress/?p=295)
  
Log out and log in again (kdelibs3 are in non standard place and so the <code>PATH</code> must be updated)
+
=== Log parser ===
 +
<pre>
 +
grep 'lessfs\[8118\]' /var/log/messages.log | grep -Eo \
 +
  'size .* bytes \: lzo compressed bytes .* and .* duplicate blocks' |\
 +
  awk ' { SUM += $2 ; SUMCP += $8 ; DUP += $10 } \
 +
  END { print "total size " SUM/1024/1024/1024 "GiB, compressed size " \
 +
  SUMCP/1024/1024/1024 "GiB (ratio 1 : " SUM/SUMCP ") with " DUP\
 +
  " duplicate blocks (" DUP*64/1024/1024 "GiB deduplicated)" }' </pre>
  
==== Configuring samba ====
+
change <code>lessfs\[8118\]</code> to the PID of lessfs you want to monitor and <code>DUP*64</code> to the block size in kiB you use in it.
A basic config file works OK, though you'll need to create Samba users (with <code>smbpasswd -a &lt;user name&gt;</code> as root):
+
{{File|name=smb.conf|content=[global]
+
workgroup=HOME
+
server string = Samba Server
+
log file = /var/log/samba/&m.log
+
max log size = 50
+
load printers = No
+
dns proxy = No
+
  
[homes]
+
== Configuring Samba Domain/File server. ==
comment = Home Directories
+
read only = No
+
browsable = No
+
browseable = No
+
}}
+
  
Alternatively you can configure samba with <code>security = share</code> and add <code>guest account = &lt;your user name&gt;</code> to get an anonymous Samba server
+
=== What we want ===
 +
basically, everything that Samba has to offer:
 +
* save and share files (duh)
 +
* edit files/folders ACLs to the maximum extent possible for Samba<br />http://www.samba.org/samba/docs/man/Samba-HOWTO-Collection/AccessControls.html
 +
** mapping of permission inheritance
 +
* save file attributes (archive, system, hidden, read only) on samba server without screwing UNIX permissions
 +
* have the ability to manage users/computers in domain without the need to use UID 0 (root) account<br />http://www.samba.org/samba/docs/man/Samba-HOWTO-Collection/rights.html
 +
* automatic creation of user folders
 +
* Previous Version share feature
 +
* enforcement of log-on hours, password complexity, password age and requests for password change, and group policies <br />http://wiki.samba.org/index.php/Implementing_System_Policies_with_Samba<br />http://www.samba.org/samba/docs/man/Samba-HOWTO-Collection/PolicyMgmt.html
 +
* view and set file system quota
 +
* time server/syncronisation
 +
* printing with generic drivers to CUPS (optional)
  
Remember to add samba to <code>DAEMONS</code> in <code>/etc/rc.conf</code> if you want it to start at boot time.
+
guide:
 
+
https://help.ubuntu.com/community/SettingUpSambaPDC
==== Modifying fileshareset scripts ====
+
{{Command|name=|output=for directory in /opt/kde/bin/ /usr/lib/kde4/libexec; do
+
  cd $directory
+
  sed 's/\/init.d\//\/rc.d\//g' fileshareset > fileshareset.pl
+
  rm filesharelist
+
  ln -s fileshareset.pl filesharelist
+
done}}
+
 
+
==== Create C wrapper ====
+
Create such files in your home directory:
+
 
+
{{File|name=kde4.c|content=#include <stdio.h>
+
#include <stdlib.h>
+
#include <string.h>
+
#include <sys/types.h>
+
#include <unistd.h>
+
 
+
#define EXEC "/usr/lib/kde4/libexec/fileshareset.pl"
+
 
+
int
+
main(int argc, char** argv)
+
{
+
  char **args;
+
 
+
  setegid(0);
+
  seteuid(0);
+
  setgid(0);
+
  setuid(0);
+
 
+
  args = calloc(argc+1, sizeof(char**));
+
  if (args == 0)
+
    exit(1);
+
 
+
  for (int i=0; i < argc; i++)
+
  {
+
    args[i] = calloc(1,strlen(argv[i]));
+
    if(args[i] == 0)
+
      exit(1);
+
    strcpy(args[i], argv[i]);
+
  }
+
  return execv(EXEC, args);
+
}
+
}}
+
 
+
{{File|name=kde3.c|content=#include <stdio.h>
+
#include <stdlib.h>
+
#include <string.h>
+
#include <sys/types.h>
+
#include <unistd.h>
+
 
+
#define EXEC "/opt/kde/bin/fileshareset.pl"
+
 
+
int
+
main(int argc, char** argv)
+
{
+
  char **args;
+
 
+
  setegid(0);
+
  seteuid(0);
+
  setgid(0);
+
  setuid(0);
+
 
+
  args = calloc(argc+1, sizeof(char**));
+
  if (args == 0)
+
    exit(1);
+
 
+
  for (int i=0; i < argc; i++)
+
  {
+
    args[i] = calloc(1,strlen(argv[i]));
+
    if(args[i] == 0)
+
      exit(1);
+
    strcpy(args[i], argv[i]);
+
  }
+
  return execv(EXEC, args);
+
}
+
}}
+
 
+
compile them using <code>gcc kde4.c -std=c99 -o kde4</code> and <code>gcc kde3.c -std=c99 -o kde3</code>.
+
Copy them to system folders:
+
{{Command|name=copying wrappers|output=cp kde3 /opt/kde/bin/fileshareset
+
chown root:root /opt/kde/bin/fileshareset
+
chmod u+s /opt/kde/bin/fileshareset
+
cp kde4 /usr/lib/kde4/libexec/fileshareset
+
chown root:root /usr/lib/kde4/libexec/fileshareset
+
chmod u+s /usr/lib/kde4/libexec/fileshareset
+
}}
+
 
+
==== Configure KDE ====
+
Right click any folder in your home directory and select Properties. Go to Share tab and click 'Configure File Sharing...', you'll be asked for root password.
+
(alternatively run <code>kcmshell4 fileshare</code> as root)
+
 
+
In the dialog you need to select checkbox near "Enable Local Network File Sharing".
+
Check radiobutton besides 'Advanced sharing', unselect NFS sharing, check 'Simple sharing' again.
+
 
+
Click 'Allowed Users' and select the second option -- 'Only users of a certain group are allowed to share folders'. Click 'Choose group...'. Create new group, call it <code>samba-share</code>, don't check any checkboxes in the dialog, click OK. In the new window add yourself to the group and all the other users you may want. Click OK.
+
 
+
Click OK in the main dialog.
+
 
+
Log out, log back in.
+
 
+
==== Share a folder ====
+
When the file sharing is configured you will see a new radio button on the Share tab -- Shared and Not shared. Change position to Shared, click OK. On the shared folder should appear now a small globe to indicate that it is shared.
+
 
+
=== Configure advanced file sharing in KDE4 using Samba ===
+
 
+
Work plan:
+
# install packages
+
# configure samba
+
# configure <code>sudo</code> and <code>kdesu</code>
+
# configure KDE side of things
+
# share a folder
+
 
+
Packages we will need:
+
* {{Package Official|kdebase-dolphin}} this is the interface we will use to share folders
+
* {{Package Official|samba}} for the server that will share files
+
* {{Package Official|kdenetwork-filesharing}} for System Settings File Sharing part
+
* {{Package Official|kdebase-runtime}} basic KDE workspace (but if you're reading this, chances are you already have it installed)
+
* {{Package Official|sudo}}
+
 
+
==== Installing requisites ====
+
Install all packages and their dependencies:
+
 
+
{{Codeline|pacman -S kdebase-dolphin samba kdenetwork-filesharing kdebase-runtime sudo}}
+
 
+
Log out and log in again (kdelibs3 are in non standard place and so the <code>PATH</code> must be updated)
+
 
+
==== Configuring samba ====
+
A basic config file works OK, though you'll need to create Samba users (with <code>smbpasswd -a &lt;user name&gt;</code> as root):
+
{{File|name=smb.conf|content=[global]
+
workgroup=HOME
+
server string = Samba Server
+
log file = /var/log/samba/&m.log
+
max log size = 50
+
load printers = No
+
dns proxy = No
+
 
+
[homes]
+
comment = Home Directories
+
read only = No
+
browsable = No
+
browseable = No
+
}}
+
 
+
Alternatively you can configure samba with <code>security = share</code> and add <code>guest account = &lt;your user name&gt;</code> to get an anonymous Samba server
+
 
+
Remember to add samba to <code>DAEMONS</code> in <code>/etc/rc.conf</code> if you want it to start at boot time.
+
 
+
==== Configure sudo ====
+
as root run <code>visudo</code> (it uses editor defined in <code>$EDITOR</code> or <code>$VISUAL</code> variables) and add line:
+
{{File|name=/etc/sudoers|content=...
+
&lt;your user name&gt; ALL=(ALL) ALL
+
}}
+
this will allow you to run all programs with <code>sudo</code> using your own password.
+
 
+
Run this command to change default super user command:
+
{{Codeline|kwriteconfig --file kdesurc --group super-user-command --key super-user-command sudo}}
+
 
+
==== Configure KDE ====
+
Right click any folder in your home directory and select Properties. Go to Share tab and click 'Configure File Sharing...', you'll be asked for your sudo password.
+
(alternatively run <code>kcmshell4 fileshare</code> as root)
+
 
+
In the dialog you need to select checkbox near "Enable Local Network File Sharing".
+
Check radiobutton besides 'Advanced sharing', unselect NFS sharing.
+
 
+
Click 'Allowed Users' and select the second option -- 'Only users of a certain group are allowed to share folders'. Click 'Choose group...'. Create new group, call it <code>samba-share</code>, don't check any checkboxes in the dialog, click OK. In the new window add yourself, root to the group and all the other users you may want. Click OK.
+
 
+
Click OK in the main dialog.
+
 
+
Log out, log back in.
+
 
+
==== Share a folder ====
+
When the file sharing is configured you will see a new checkbox in the Share tab: 'Share this folder in the local network'. Select checkbox beside 'Share with Samba', give the share some meaningful name and decide whatever it should be writable or not.
+
 
+
More Samba Options list all possible Samba options that can be applied to a share, so it's quite comprehensive.
+

Latest revision as of 11:40, 6 March 2010

just my personal scrapbook

lessfs

Sizes of different databases in lessfs

With a 64k blocksize administration overhead would be approx 5~10% of the size of blockdata.tch

Good all-round mount options

-o kernel_cache,negative_timeout=0,entry_timeout=0,attr_timeout=0,use_ino,readdir_ino\
      ,default_permissions,allow_other,big_writes,max_read=65536,max_write=65536

Filesystem, mounting but not working

Each and every partition with datafiles need 10% of free space, lessfs will freeze all writes to the partition for as long as the free space isn't aviable.

Version 1.0.4 has new config parameters:

MIN_SPACE_FREE=2
MIN_SPACE_CLEAN=5
CLEAN_PROGRAM=/tmp/clean.sh

MIN_SPACE_FREE defines minimum space left (in percent?) on the device, MIN_SPACE_CLEAN defines the amount of space left that triggers the CLEAN_PROGRAM

Bucket sizes

Usually when performance drops after reaching a certain database size this indicates that the bucketsize selected in /etc/lessfs.cfg is to low. The default 'lessfs.cfg' example included with the code only allows the database to grow up to: 2GB after which performance degrades. To store 50GB the bucketsize should be at least : 27262976

2MB bucketsize equals 1000000 database entries.
To store 50GB :
50GB = 50*1024*1024*1024 = 53687091200 bytes
53687091200/4096 (4k blocksize) = 13107200 entries in the databases.
13107200/1000000 (divide by 10^6 entries)
13 *2 (10^6 entries requires 2MB) = 26
26 * 1024*1024 = 27262976 -> This is the bucketsize you need for 50GB with a 4k blocksize.

(from http://sourceforge.net/projects/lessfs/forums/forum/932600/topic/3304815)

If you ever need to change the _BS settings, just unmount lessfs, change the settings, mount lessfs and run defrag.

Files to put on different spindles

For max performance compile with –with-sha3 and put the FILEBLOCK and BLOCKUSAGE databases each on there own raid one disk set. The faster these disks are the better. BLOCKDATA can be on raid5/6. (from http://www.lessfs.com/wordpress/?p=295)

Log parser

grep 'lessfs\[8118\]' /var/log/messages.log | grep -Eo \
  'size .* bytes \: lzo compressed bytes .* and .* duplicate blocks' |\
  awk ' { SUM += $2 ; SUMCP += $8 ; DUP += $10 } \
  END { print "total size " SUM/1024/1024/1024 "GiB, compressed size " \
  SUMCP/1024/1024/1024 "GiB (ratio 1 : " SUM/SUMCP ") with " DUP\
  " duplicate blocks (" DUP*64/1024/1024 "GiB deduplicated)" }' 

change lessfs\[8118\] to the PID of lessfs you want to monitor and DUP*64 to the block size in kiB you use in it.

Configuring Samba Domain/File server.

What we want

basically, everything that Samba has to offer:

guide: https://help.ubuntu.com/community/SettingUpSambaPDC