Custom local repository
From ArchWiki
| i18n |
|---|
| English |
| Русский |
| 简体中文 |
Contents |
[edit] New Way (repo-add)
Pacman 3 introduced a new script named repo-add which makes generating your own repository much easier. Use repo-add --help for more details on its usage.
This script is very easy to run, and also very easy to keep your DB up to date. Simply store all of the built packages you want in your repository in one directory, and execute the following command:
repo-add /path/to/repo.db.tar.gz *.pkg.tar.gz
where 'repo' is the name of your custom repository. The last argument will add all pkg.tar.gz files to your repository, so be careful- if you have multiple versions of a package in your directory, it is unclear which one will take precedence and end up in the repository.
To add a new package (and remove the old if it exists, simply run
repo-add /path/to/repo.db.tar.gz packagetoadd-1.0-1-i686.pkg.tar.gz
If there is a package that you do not want in your repository any longer, read up on repo-remove.
[edit] Old Way (Repository from ABS tree)
This describes how to create your own pacman repository from a personalized ABS tree, containing only those PKGBUILDs that you want to be included in your personal repository. This is useful for creating a local repository or for creating a custom repository for packages that are not included in the official repositories.
Run gensync and read the command line options. In short, the parameters are the root of PKGBUILDs, sorted in subdirectories (ie. like the ABS tree), the intended name and location of the repository database file, and the directory containing the binary packages.
Create an ABS tree to work with. You can do this using the command abs if you want to create a modified copy of the usual abs tree, or you can create a tree manually. Stick to the rule that each PKGBUILD resides in its own directory (whether it is an official PKGBUILD or one you created yourself). If you are modifying an official abs repository, remove any abs directories that you do not want to include in the final repository.
Save all binary packages for the packages you wish to create a repository for to a particular directory (/home/arch/i686/core for example). You can create the binaries using makepkg, or you could download them with pacman, depending on your situation.
run gensync supplying the correct parameters. As an example:
gensync /var/abs /home/arch/i686/core/core.db.tar.gz /home/arch/i686/core
would build a repository for the core repository if it is stored in /home/arch/i686/core. The name of the db.tar.gz file is the name of the repository you will be creating. It is customary to use the same name for the directory that holds the packages.
Use:
tar tzf core.db.tar.gz || less
to verify that the database contains the correct packages.
[edit] Alternative Python way
This script makes a repository from a directory containing packages, without needing ABS. It also lists unsatisfied dependencies (though at the moment it ignores version numbers), and can filter these in a fairly primitive way, to ignore a certain set (e.g. to ignore the packages in /var/lib/pacman/local). A later version of this script, as used in larch/pacin, may be available at http://four.fsphost.com/gradgrind.
#!/usr/bin/env python
# gen_repo.py - build a repository db file from a set of packages
#
# Author: gradgrind <mt.42-at-web.de>
#
# This file is part of the larch project.
#
# larch is free software; you can redistribute it and/or modify
# it under the terms of the GNU General Public License as published by
# the Free Software Foundation; either version 2 of the License, or
# (at your option) any later version.
#
# larch is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
# GNU General Public License for more details.
#
# You should have received a copy of the GNU General Public License
# along with larch; if not, write to the Free Software Foundation, Inc.,
# 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA
#
#----------------------------------------------------------------------------
#
import os
import os.path
import shutil
import sys
import tarfile
import md5
from types import *
import re
# Regex to remove version comparison from package dependency
onlyname = re.compile("([^=><]+).*")
def create_db(dbname, packagesdir, dep_ignore_list):
os.chdir(packagesdir)
# Open tar archive for db
dbtar = tarfile.open(dbname + ".db.tar.gz", "w:gz")
# Get a list of packages
packages = filter(lambda s: s.endswith(".pkg.tar.gz"), os.listdir("."))
packages.sort()
# Make a dict for keeping track of dependencies
dep_dict = {}
if os.path.isdir(dbname): shutil.rmtree(dbname)
os.mkdir(dbname, 0755)
for p in packages:
pkg_dict = get_pkg_info(p)
pkg_name = pkg_dict["pkgname"]
pkg_dbname = pkg_name + "-" + pkg_dict["pkgver"]
pkg_dir = os.path.join(dbname, pkg_dbname)
os.mkdir(pkg_dir, 0755)
mkdesc(pkg_dir, pkg_dict)
mkdepends(pkg_dir, pkg_dict)
# Add dependency info to dependency dict
for d in pkg_dict["depend"]:
# But also need to cater for versioning!!!
# I will just ignore it here ...
dm = onlyname.match(d)
if not dm:
if d:
print "DEBUG: package %s, dependency = '%s'" % (pkg_name, d)
continue
d = dm.group(1)
if not dep_dict.has_key(d):
dep_dict[d] = [False]
dep_dict[d].append(pkg_name)
# Mark packages provided by this one
for p in (pkg_dict["provides"] + [pkg_name]):
if dep_dict.has_key(p):
dep_dict[p][0] = True
else:
dep_dict[p] = [True]
dbtar.add(pkg_dir, pkg_dbname)
# Mark packages in ignore list
for p in dep_ignore_list:
if dep_dict.has_key(p):
dep_dict[p][0] = True
# Close db tar archive
dbtar.close()
# Remove directory structure
shutil.rmtree(dbname)
# Now display unstaisfied dependencies
# Should add the possibility of declaring a list of packages
# available (e.g. the base set, or all those on the live CD ..."
print "-------------\nUnsatisfied dependencies:"
for d, r in dep_dict.items():
if not r[0]:
print " ", d, "- needed by: ",
for p in r[1:]:
print p, " ",
print ""
def get_pkg_info(pkg):
tf = tarfile.open(pkg, "r:gz")
pkginfo = tf.extractfile(".PKGINFO")
pkg_dict = {# the first ones go to 'desc'
"pkgname" : None,
"pkgver" : None,
"pkgdesc" : None,
# from here they are optional, and can occur more than once
"group" : [],
"replaces" : [],
# the rest go to 'depends'
"depend" : [],
"conflict" : [],
"provides" : [],
}
while True:
l = pkginfo.readline().strip()
if not l: break
if l[0] == "#": continue
split3 = l.split(None, 2)
while len(split3) < 3: split3.append("")
key, eq, value = split3
if not pkg_dict.has_key(key): continue
val = pkg_dict[key]
if val == None:
pkg_dict[key] = value
continue
if not isinstance(val, ListType):
print "Unexpected situation ...\n key [oldvalue] <- newvalue"
print key, "[%s]" % val, "<-", value
sys.exit(1)
pkg_dict[key].append(value)
pkginfo.close()
pkg_dict["md5sum"] = md5sum(pkg)
pkg_dict["csize"] = str(os.path.getsize(pkg))
return pkg_dict
def mkdesc(pkg_dir, pkg_dict):
f = open(os.path.join(pkg_dir, "desc"), "w")
f.write("%NAME%\n" + pkg_dict["pkgname"] + "\n\n")
f.write("%VERSION%\n" + pkg_dict["pkgver"] + "\n\n")
f.write("%DESC%\n" + pkg_dict["pkgdesc"] + "\n\n")
f.write("%CSIZE%\n" + pkg_dict["csize"] + "\n\n")
f.write("%MD5SUM%\n" + pkg_dict["md5sum"] + "\n\n")
groups = pkg_dict["group"]
if groups:
f.write("%GROUPS%\n")
for g in groups:
f.write(g + "\n")
f.write("\n")
replaces = pkg_dict["replaces"]
if replaces:
f.write("%REPLACES%\n")
for r in replaces:
f.write(r + "\n")
f.write("\n")
f.close()
def mkdepends(pkg_dir, pkg_dict):
f = open(os.path.join(pkg_dir, "depends"), "w")
depends = pkg_dict["depend"]
if depends:
f.write("%DEPENDS%\n")
for d in depends:
f.write(d + "\n")
f.write("\n")
conflicts = pkg_dict["conflict"]
if conflicts:
f.write("%CONFLICTS%\n")
for c in conflicts:
f.write(c + "\n")
f.write("\n")
provides = pkg_dict["provides"]
if provides:
f.write("%PROVIDES%\n")
for p in provides:
f.write(p + "\n")
f.write("\n")
def md5sum(filepath):
f = file(filepath, 'rb')
m = md5.new()
while True:
d = f.read(8192)
if not d:
break
m.update(d)
f.close()
return m.hexdigest()
def cat(path):
"""Python version of 'cat'"""
fp = open(path, "r")
op = ""
for l in fp:
op += l
fp.close()
return op
def usage():
print """
genrepo.py package-dir [repo-name] [-- ignore-list]
Generate a pacman db file for the packages in package-dir.
If repo-name is given, this will be used as the name for the repository,
otherwise the name of the directory containing the packages will be used.
All dependencies of the packages in the repository will be listed to
standard output, but a list of packages not to be included in this list
can be specified:
ignore-list should be either a file containing the names of packages
not to be listed as dependencies (separated by space or newline), or a
directory containing 'package directories', like /var/abs/base or
/var/lib/pacman/local
"""
sys.exit(1)
if __name__ == "__main__":
if os.getuid() != 0:
print "Must be root to run this"
sys.exit(1)
if len(sys.argv) < 2:
usage()
pkgdir = sys.argv[1]
if (len(sys.argv) == 2) or (sys.argv[2] == "--"):
dbname = os.path.basename(os.path.abspath(pkgdir))
i = 2
else:
dbname = sys.argv[2]
i = 3
if len(sys.argv) == i:
ignore_list = []
elif (len(sys.argv) == i+2) and (sys.argv[i] == "--"):
ignore_list = sys.argv[i+1]
else:
usage()
if not os.path.isdir(pkgdir):
print "\n1st argument must be a directory"
sys.exit(1)
print "\nCreating pacman database (%s.db.tar.gz) file in %s" % (dbname, pkgdir)
i = raw_input("Do you want to do this? [Y/n] ").strip()
if i and (i[0] not in "Yy"):
print " -> Not creating package db"
sys.exit(0)
if ignore_list:
# Get list of packages to be ignored in dependency list
if os.path.isfile(ignore_list):
# A simple file containing the names of packages to ignore
# separated by space or newline.
ignore_list = cat(ignore_list).split()
elif os.path.isdir(ignore_list):
# A directory containing packages or package-directories (like in abs)
l = os.listdir(ignore_list)
# See if there are packages in this directory
lp = filter(lambda s: s.endswith(".pkg.tar.gz"), l)
if lp:
l = map(lambda s: s.replace(".pkg.tar.gz", ""), lp)
re1 = re.compile("(.+)-[^-]+?-[0-9]+")
ignore_list = []
for f in l:
m = re1.match(f)
if m:
ignore_list.append(m.group(1))
else:
print "!!! Invalid ignore-list"
usage()
create_db(dbname, pkgdir, ignore_list)
[edit] Final note
Once you have made a local repository, add the repository to your pacman.conf. The name of the db.tar.gz file is the repository name. You can reference it directly using a file:// url, or you can access it via ftp using ftp://localhost/path/to/directory.
note: I had to rename a package in my local repository with a name ending in '-i686.pkg.tar.gz' for the package to be installed correctly.
If you can and are willing, add your user-repository to our list of unofficial user repositories, so that all other users can find and install your packages.