Difference between revisions of "ArchAudio"

From ArchWiki
Jump to: navigation, search
m (Do not use -Sy when installing packages)
(rewrite)
Line 1: Line 1:
 
[[Category:Audio/Video (English)]]
 
[[Category:Audio/Video (English)]]
  
= Introduction =
+
== Introduction ==
''This page contains information related to a third-party initiative. You may be looking for [[Pro Audio]] and not even know it!''
+
''This page contains information related to a [[Getting Involved#Community projects|third-party initiative]]. You may be looking for [[Pro Audio]] and not even know it!''
  
If you came here from another page, or were referred, you are probably interested in the [http://archaudio.org/ ArchAudio] [http://archaudio.org/packages/ repositories].
+
If you've come here from another page, or were referred, you are probably interested in the [http://archaudio.org/ ArchAudio] [http://archaudio.org/packages/ repositories].
  
Read on if you are looking to contribute to the project, or are just plain bored.
+
Read on if you're looking to contribute to the project, or are just plain bored.
  
= Packaging =
+
== Packaging ==
 
Prerequisite(s):
 
Prerequisite(s):
  
* [[ABS]]
+
* [[Creating Packages]]
 
* [[Arch Packaging Standards]]
 
* [[Arch Packaging Standards]]
  
== Getting Started ==
+
=== Getting Started ===
 
Take a look [http://archaudio.org/participate/ here].
 
Take a look [http://archaudio.org/participate/ here].
  
Anyone and everyone keen on contributing source-based packages can start immediately, except for the unfortunate fact that we cannot automatically authenticate registered members to allow commits/check-ins. That is why we only request that you take the time to log on to our IRC channel, look for the user '''jonkristian''', and send him a password hash. You can also PM the admin on the forums.
+
Anyone and everyone keen on contributing to the buildscripts ([[PKGBUILD|PKGBUILDs]] and related files) can start immediately. You only have to take the time to get in touch with either '''schivmeister''' or '''jonkristian''' with a ''password hash''. The fastest way is to log on to IRC (#archaudio@Freenode) and look for them, but a more straightforward approach is to simply e-mail them directly:
  
=== Password Hashing ===
+
printf 'schiv#%s.fun\n' archaudio | sed -e 's/#/@/' -e 's/fun/org/'
As we talked about above, this is needed for anyone wanting to have some kind of write access to our subversion repository:
+
printf 'jon$%s.fun\n' archaudio | sed -e 's/\$/@/' -e 's/fun/org/'
  
~$ htpasswd -cs $filename $username # you need to install apache for this tool
+
==== Password Hashing ====
 +
As we mentioned above, this is needed for anyone wanting to have write access to the Subversion repository:
  
Or you can use the following web application: http://aspirine.org/htpasswd_en.html
+
http://aspirine.org/htpasswd_en.html
  
Simply enter a username and password, change the algorithm to SHA-1, then click on "encrypt password". The output is what we want.
+
Simply enter a username and password, select either SHA-1 or MD5 (up to you), then click on "encrypt password". The output is what we want.
  
=== Buildscript Contributor ===
+
Or if you prefer an offline method:
Since you would only have write access to the relevant parent directory, we recommended checking out with the full path:
+
  
  ~$ svn co https://svn.archaudio.org/trunk/buildscripts archaudio-sources --username=johndoe
+
  htpasswd -cs hashForArchAudio.txt $preferred_username # you need to install 'apache' for this tool
  
=== Subversion Maintenance ===
+
In this example, the file 'hashForArchAudio.txt' will be created with your new [http://phpsec.org/articles/2005/password-hashing.html password hash]. Either send the file itself or copy the hash (the whole line). Don't worry, you're only sharing the ''cryptographic hash'' of your password - and not the password itself - so that we can authenticate you.
It is always a good idea to make the following sanity checks a routine:
+
  
~$ svn stat
+
==== Buildscript Contributor ====
 +
Simply checkout the repository via HTTPS with your username:
  
  ~$ svn merge --dry-run -r BASE:HEAD . # basically a more intuitive '''svn stat -u'''
+
  svn co https://svn.archaudio.org/trunk archaudio --username=$your_username
  
And if you really cannot find out how to use SVN (man, wiki, google):
+
==== Binary Contributor ====
 +
''We suggest that you be well-acquainted with Arch Linux packaging before applying to become a (binary) packager. If you maintain packages in [[AUR]] you are on the right path.''
  
~$ svn add $somedir $somefile # to add to the repo (recursive)
+
The only difference here is that you have to do a second, ''non-recursive'' checkout for the private '''packages''' area, where the binaries are contained and pushed. Although Subversion will not track this directory, we're going to keep it under the main checkout for the sake of consistency. We've already made the necessary ammendments so that SVN does not display the '?' symbol as the directory's status:
  
~$ svn del $somedir $somefile # to delete
 
 
''Warning: If you remove something without the '''svn''' command, Subversion will complain that it is missing since it is not aware of the removal.''
 
 
~$ svn mv $oldname $newname # same warning applies; you '''cannot''' forget the '''svn''' before the ''mv!''
 
 
~$ svn revert $somedir $somefile # well..revert your changes
 
 
~$ svn ci -m "$yourmessage" # to finally commit all changes
 
 
=== Binary Contributor ===
 
We suggest that you be well-acquainted with Arch Linux packaging. If you maintain packages in [[AUR]] you are on the right path.
 
 
You may want to checkout ''sparse'' directories so that you need not download all the uploaded binaries:
 
 
~$ svn co https://svn.archaudio.org/trunk/ ArchAudio --depth empty --username=johndoe
 
 
And then:
 
 
 
  (
 
  (
  cd ArchAudio
+
echo -n 'Username: '
  svn up devel buildscripts
+
read your_username
  svn up packages --depth immediates
+
svn co https://svn.archaudio.org/trunk archaudio --username $your_username
  cd packages
+
cd archaudio
  for i in experimental stable testing; do
+
svn co https://svn.archaudio.org/trunk/packages --depth empty --username $your_username
    (cd $i && svn up i686 --depth empty) # or x86_64 depending on which you have permission to
+
cd packages
  done
+
svn up stable testing experimental --set-depth immediates
 
  ) # if you copy-pasted you have to press ENTER now
 
  ) # if you copy-pasted you have to press ENTER now
  
After that you can, at any time, view what is available:
+
So you would navigate to the proper repository and architecture directory, copy the (binary) package that you built, add it to SVN, and finally commit the addition. Optinally, you may clear out the directory again to maintain the 'sparseness':
  
  cd ArchAudio/packages/testing/i686
+
  cd archaudio/packages/testing/i686
  svn ls
+
cp $package .
 +
svn add $package
 +
svn ci -m "Added binary package $package"
 +
  svn up $package --set-depth exclude
  
There is no need to checkout anything here at all, unless you want to do removal. In that case, simply download the file only:
+
Old packages will be automatically deleted on the server, so you need not worry about removing them yourself. Be careful to add real binary packages and not symlinks or any other file that may deceive you.
  
svn ls | grep $unwantedpkg
+
=== Subversion Maintenance ===
svn up $unwantedpkgfullname
+
It is always a good idea to make the following sanity checks a routine:
svn rm $unwantedpkgfullname
+
svn ci -m "Deleted obsolete package. Please make sure it doesn't exist in repo."
+
svn up --set-depth empty # in case there were multiple pkgs with the name eg. jack & jack2
+
  
Now for the interesting part - committing new packages and maintaining the sparseness of the working copy. Say, for example, you have already built a few packages and have copied it here in [testing/i686] (either manually or by makepkg's ''$PKGDEST''). You just need to add them to the server and then revert back to sparse mode:
+
svn stat
  
  svn add *
+
  svn merge --dry-run -r BASE:HEAD . # basically a more intuitive '''svn stat -u'''
svn ci -m "Added foobar, foobox, toolbar, toolbox, xbox."
+
svn up --set-depth empty
+
  
=== Build Environment ===
+
And if you really cannot find out how to use SVN (man, wiki, google):
It is essential that your environment remains sane throughout all builds. As such, we recommend [[DeveloperWiki:Building_in_a_Clean_Chroot|chrooting]]. You may have to:
+
  
  ~# pacman -S devtools aufs2-util
+
  svn add $somedir $somefile # to add (recursive)
~# modprobe aufs
+
  
If you find yourself building often (ArchAudio or not), add '''aufs''' to your list of startup daemons. You may also need to patch '''makechrootpkg''' to make use of ''aufs'' instead of ''unionfs'':
+
svn del $somedir $somefile # to delete
  
--- /usr/sbin/makechrootpkg.old 2008-11-28 18:57:18.000000000 +0800
+
''Warning: If you remove something without the '''svn''' command, Subversion will complain that it is missing since it's not aware of the removal.''
+++ /usr/sbin/makechrootpkg 2009-06-22 01:03:09.903654612 +0800
+
@@ -106,14 +106,14 @@
+
+
  uniondir="$chrootdir/union"
+
  echo "building union chroot"
+
-grep -Fq unionfs /proc/filesystems
+
+grep -Fq aufs /proc/filesystems
+
  if [ $? -ne 0 ]; then
+
-    modprobe -q unionfs
+
+    modprobe -q aufs
+
      if [ $? -ne 0 ]; then
+
-        echo "ERROR: No unionfs available. Abandon ship!" && exit 1
+
+        echo "ERROR: No aufs available. Abandon ship!" && exit 1
+
      fi
+
  fi
+
-mount -t unionfs none -o "dirs=$chrootdir/rw=rw:$chrootdir/root=ro" "$uniondir"
+
+mount -t aufs none -o "dirs=$chrootdir/rw=rw:$chrootdir/root=ro" "$uniondir"
+
  trap 'cleanup' 0 1 2 15
+
 
+
  if [ -n "$install_pkg" ]; then
+
  
=== Packaging Guidelines ===
+
svn mv $oldname $newname # same warning applies; you '''cannot forget''' the '''svn''' before the ''mv''!
We will try to have a proper outline in our subversion '''docs''' directory, but generally:
+
  
* Use tools such as ''namcap'' and those from ''pacman-contrib'' as complementary solutions
+
svn revert $somedir $somefile # revert some changes (selective)
* Look out for other useful development utilities in the Arch Linux MLs or Forums
+
* Most often you have to rely on yourself and "manual labour" to determine correct dependencies
+
* Always ensure a binary installs and runs as expected
+
* Try to keep in touch with upstream developers of packages you regularly update
+
* Always check upstream for critical changes if an update does not go smoothly
+
  
== Naming Conventions ==
+
svn revert -R . # revert ALL changes (recursive)
Sometimes, software release names are more suited for developers and can be confusing for users. For instance, LV2 is the successor to LADSPA, but it is not really an "SDK" per se. Since all that is provided (and needed) is a core set of headers, the tarball is released as "lv2core". Often, such a naming can imply that there is more to the software than just those core specifications, like, perhaps an "lv2full".
+
  
As for "jack-audio-connection-kit", the fact that it is so long already qualifies it for renaming. We do not want to be expanding recursive acronyms, now, do we?
+
svn ci -m "$yourmessage" # to finally commit all changes
  
Currently, the following are upstream software (release tarballs) and their respective package names in our repository:
+
=== Guidelines ===
 
+
* Use tools such as ''namcap'' and those from ''pacman-contrib'' to help yourself
* lv2core: lv2
+
* Always ensure a package installs and runs as expected before distributing your changes
* jack-audio-connection-kit: jack
+
* Try to keep in touch with upstream developers of packages you regularly update
* jack-1.9.2: jack2
+
* Always check the upstream changelog if an update does not go smoothly
* jackdmp: jack2-svn
+
 
+
== Realtime Kernel ==
+
Specific prerequisite(s):
+
 
+
* [[Kernel Patches and Patchsets]]
+
* [[Kernel Module Package Guidelines]]
+
 
+
As long as there is no user demand for another, we will roll just one kernel. This will be called '''kernel26rt''', and will be indefinitely based on the stock Arch Linux kernel, ''kernel26''.
+
 
+
In the event that you want to start from scratch, you can just:
+
+
~$ cp -r /var/abs/core/kernel26 .
+
 
+
Since the realtime patch can move really fast, and provided that '''Linux Audio''' itself is an experimental paradigm, the buildscripts should be as flexible as possible (contrast this to "portable") to allow for quick user updates. If you take a look at our PKGBUILD, you will notice a number of custom variables:
+
 
+
...
+
_realkernel=      # actual basekernel eg. for rc builds
+
_kernelpatch=
+
_rtpatch=
+
...
+
+
# If sources are rc, old rc, old rt, rc & old rt, old rc & old rt:
+
# rc, rc-old, rt-old, rc-rt-old, all-old or blank for default
+
_source=
+
+
...
+
[ "$_source" = "rc" ] && _rc=testing/
+
[ "$_source" = "rc-old" ] && _rc=testing/v$_realkernel/
+
[ "$_source" = "rt-old" ] && _rt=older/
+
...
+
...
+
 
+
Simply said, this amount of "tainting" is necessary to retain the simplicity; making the PKGBUILD flexible without compromising portability. On kernel updates, you have to compare the two PKGBUILDs for differences and reflect those needed onto ours. Remember - we want to maintain as much compatibility as possible. One of the final steps usually involves checking out what patches ''kernel26'' includes (eg. AUFS, which is important):
+
 
+
~$ git clone http://projects.archlinux.org/git/linux-2.6-ARCH.git
+
 
+
Read '''PATCHCFG''' and reproduce the patching lines for our PKGBUILD. For example:
+
 
+
# add aufs2 support, in reference to:
+
# http://aufs.sourceforge.net
+
aufs2-20090611.patch%1
+
 
+
translates to:
+
 
+
source=(...
+
        ...
+
        aufs2-20090611.patch)
+
+
build() {
+
        ...
+
        patch -Np1 -i ../aufs2-20090611.patch
+
        ...
+
 
+
The files are available under the '''patches''' directory. But of course, if only life were so simple! The patches used in both packages will obviously be different if they are version-dependent, especially in this case where the aufs patch is for '''*30''', whereas the version of kernel26rt is '''*29'''. Sometimes, you may even have to custom-roll your own patchset (grab upstream patch and diff on rt-patched kernel)! Anyway, for aufs you can take a look at their [http://aufs.sourceforge.net/ website]:
+
 
+
o aufs2-standalone tree
+
$ git clone http://git.c3sl.ufpr.br/pub/scm/aufs/aufs2-standalone.git \
+
aufs2-standalone.git
+
$ cd aufs2-standalone.git
+
$ git checkout origin/aufs2-xx # for instance, aufs2-27 for linux-2.6.27
+
              # aufs2 (no -xx) for the latest -rc version.
+
 
+
And grab the ''kbuild'' patch, resulting in:
+
 
+
~$ patch -Np1 -i ../aufs2-kbuild.patch
+
 
+
Hopefully the order should not matter, but a general rule to follow is that you should apply the realtime patch last.
+

Revision as of 18:27, 30 July 2011


Introduction

This page contains information related to a third-party initiative. You may be looking for Pro Audio and not even know it!

If you've come here from another page, or were referred, you are probably interested in the ArchAudio repositories.

Read on if you're looking to contribute to the project, or are just plain bored.

Packaging

Prerequisite(s):

Getting Started

Take a look here.

Anyone and everyone keen on contributing to the buildscripts (PKGBUILDs and related files) can start immediately. You only have to take the time to get in touch with either schivmeister or jonkristian with a password hash. The fastest way is to log on to IRC (#archaudio@Freenode) and look for them, but a more straightforward approach is to simply e-mail them directly:

printf 'schiv#%s.fun\n' archaudio | sed -e 's/#/@/' -e 's/fun/org/'
printf 'jon$%s.fun\n' archaudio | sed -e 's/\$/@/' -e 's/fun/org/'

Password Hashing

As we mentioned above, this is needed for anyone wanting to have write access to the Subversion repository:

http://aspirine.org/htpasswd_en.html

Simply enter a username and password, select either SHA-1 or MD5 (up to you), then click on "encrypt password". The output is what we want.

Or if you prefer an offline method:

htpasswd -cs hashForArchAudio.txt $preferred_username # you need to install 'apache' for this tool

In this example, the file 'hashForArchAudio.txt' will be created with your new password hash. Either send the file itself or copy the hash (the whole line). Don't worry, you're only sharing the cryptographic hash of your password - and not the password itself - so that we can authenticate you.

Buildscript Contributor

Simply checkout the repository via HTTPS with your username:

svn co https://svn.archaudio.org/trunk archaudio --username=$your_username

Binary Contributor

We suggest that you be well-acquainted with Arch Linux packaging before applying to become a (binary) packager. If you maintain packages in AUR you are on the right path.

The only difference here is that you have to do a second, non-recursive checkout for the private packages area, where the binaries are contained and pushed. Although Subversion will not track this directory, we're going to keep it under the main checkout for the sake of consistency. We've already made the necessary ammendments so that SVN does not display the '?' symbol as the directory's status:

(
echo -n 'Username: '
read your_username
svn co https://svn.archaudio.org/trunk archaudio --username $your_username
cd archaudio
svn co https://svn.archaudio.org/trunk/packages --depth empty --username $your_username
cd packages
svn up stable testing experimental --set-depth immediates
) # if you copy-pasted you have to press ENTER now

So you would navigate to the proper repository and architecture directory, copy the (binary) package that you built, add it to SVN, and finally commit the addition. Optinally, you may clear out the directory again to maintain the 'sparseness':

cd archaudio/packages/testing/i686
cp $package .
svn add $package
svn ci -m "Added binary package $package"
svn up $package --set-depth exclude

Old packages will be automatically deleted on the server, so you need not worry about removing them yourself. Be careful to add real binary packages and not symlinks or any other file that may deceive you.

Subversion Maintenance

It is always a good idea to make the following sanity checks a routine:

svn stat
svn merge --dry-run -r BASE:HEAD . # basically a more intuitive svn stat -u

And if you really cannot find out how to use SVN (man, wiki, google):

svn add $somedir $somefile # to add (recursive)
svn del $somedir $somefile # to delete

Warning: If you remove something without the svn command, Subversion will complain that it is missing since it's not aware of the removal.

svn mv $oldname $newname # same warning applies; you cannot forget the svn before the mv!
svn revert $somedir $somefile # revert some changes (selective)
svn revert -R . # revert ALL changes (recursive)
svn ci -m "$yourmessage" # to finally commit all changes

Guidelines

  • Use tools such as namcap and those from pacman-contrib to help yourself
  • Always ensure a package installs and runs as expected before distributing your changes
  • Try to keep in touch with upstream developers of packages you regularly update
  • Always check the upstream changelog if an update does not go smoothly