Difference between revisions of "Nonfree applications package guidelines"

From ArchWiki
Jump to navigation Jump to search
m (→‎File placement: More grammar cleanups)
(→‎Unpacking: iso and AppImage unpacking)
 
(52 intermediate revisions by 23 users not shown)
Line 1: Line 1:
[[Category: Package development (English)]]
+
[[Category:Arch package guidelines]]
{{i18n|Nonfree Applications Package Guidelines}}
+
[[ja:ノンフリーアプリケーションパッケージガイドライン]]
{{Package Guidelines}}
+
[[pt:Nonfree applications package guidelines]]
 +
{{Package guidelines}}
  
 
{{Expansion|cover about encrypting archives, symlinking etc.}}
 
{{Expansion|cover about encrypting archives, symlinking etc.}}
Line 9: Line 10:
  
 
== Rationale ==
 
== Rationale ==
 +
 
There are multiple reasons for packaging even non-packageable software:
 
There are multiple reasons for packaging even non-packageable software:
 +
 
* Simplification of installation/removal process
 
* Simplification of installation/removal process
 
:This is applicable even to the simplest of apps, which consist of a single script to be installed into {{Ic|/usr/bin}}. Instead of issuing:
 
:This is applicable even to the simplest of apps, which consist of a single script to be installed into {{Ic|/usr/bin}}. Instead of issuing:
 +
 
:{{bc|$ chmod +x ''filename''}}
 
:{{bc|$ chmod +x ''filename''}}
 +
:{{bc|$ chown root:root ''filename''}}
 
:{{bc|# cp ''filename'' /usr/bin/}}
 
:{{bc|# cp ''filename'' /usr/bin/}}
 
:you can type just
 
:you can type just
:{{bc|# makepkg -i}}
+
:{{bc|$ makepkg -i}}
 
:Most non-free applications are obviously much more complicated, but the burden of downloading an archive/installer from a homepage (often full of advertising), unpacking/decrypting it, hand-writing stereotypical launcher scripts and doing other similar tasks can be effectively lightened by a well-written packaging script.
 
:Most non-free applications are obviously much more complicated, but the burden of downloading an archive/installer from a homepage (often full of advertising), unpacking/decrypting it, hand-writing stereotypical launcher scripts and doing other similar tasks can be effectively lightened by a well-written packaging script.
 +
 
* Utilizing pacman capabilities
 
* Utilizing pacman capabilities
 
:The ability to track state, perform automatic updates of any installed piece of software, determine ownership of every single file, and store compressed packages in a well-organized cache is what makes GNU/Linux distributions so powerful.
 
:The ability to track state, perform automatic updates of any installed piece of software, determine ownership of every single file, and store compressed packages in a well-organized cache is what makes GNU/Linux distributions so powerful.
 +
 
* Sharing code and knowledge
 
* Sharing code and knowledge
:It is simpler to apply tweaks, fix bugs and seek/provide help in a single public place like AUR versus submitting patches to proprietary developers who may have ceased support or asking vague questions on general purpose forums.
+
:It is simpler to apply tweaks, fix bugs and seek/provide help in a single public place like the AUR versus submitting patches to proprietary developers who may have ceased support or asking vague questions on general purpose forums.
  
 
== Common rules ==
 
== Common rules ==
 +
 
=== Avoid nonfree software when possible ===
 
=== Avoid nonfree software when possible ===
Yes, it's better to leave this guide and spend some time searching (or maybe even creating) alternatives to an application you wanted to package because  
+
 
# Packaging ill software is a mess which is generally against [[The Arch Way]]
+
Yes, it's better to leave this guide and spend some time searching (or maybe even creating) alternatives to an application you wanted to package because:
# It is better to support FOSS developers who generally care about users more
+
 
 +
# It is better to support software that is owned by us all than software that is owned by a company
 +
# It is better to support software that is actively maintained
 +
# It is better to support software that can be fixed if just one person out of millions cares enough
  
 
=== Use open source variants where possible ===
 
=== Use open source variants where possible ===
Many commercial games ([[Common Applications/Games/Reimplemented|some are listed in this Wiki]]) have open source engines and many old games can be played with emulators such as [[Wikipedia:ScummVM|ScummVM]]. Using open source engines together with the original game assets gives users access to bug fixes and eliminates several issues caused by binary packages.
+
 
 +
Many commercial games ([[List of Applications/Games|some are listed in this Wiki]]) have open source engines and many old games can be played with emulators such as [[Wikipedia:ScummVM|ScummVM]]. Using open source engines together with the original game assets gives users access to bug fixes and eliminates several issues caused by binary packages.
  
 
=== Keep it simple ===
 
=== Keep it simple ===
 +
 
If the packaging of some program requires more effort and hacks than buying and using the original version - do the simplest thing, it is Arch!
 
If the packaging of some program requires more effort and hacks than buying and using the original version - do the simplest thing, it is Arch!
  
== Package Naming ==
+
== Package naming ==
Before choosing name on your own search in AUR for existing version of software you want to package. Try to use established naming conversion (e.g. do not create something like {{AUR|gish-hb}} when there are already {{AUR|aquaria-hib-hg}}, {{AUR|penumbra-overture-hib}} and {{AUR|uplink-hib}}). Use suffix {{Ic|-bin}} '''always''' unless you are sure there will never be source-based package – it's creator would have to ask you (or in worst case TUs) to orphan existing package for him and you both will end up with PKGBUILDs cluttered with additional {{Ic|replases}} and {{Ic|conflicts}}.
+
 
 +
Before choosing a name on your own, search in the AUR for existing versions of the software you want to package. Try to use established naming conventions (e.g. do not create something like {{AUR|gish-hb}}{{Broken package link|{{aur-mirror|gish-hb}}}} when there are already packages for {{AUR|aquaria-hib}}, {{AUR|penumbra-overture-hib}} and {{AUR|uplink-hib}}). Use the suffix {{ic|-bin}} '''always''' unless you are sure there will never be a source-based package—its creator would have to ask you (or in the worst case a TU) to orphan the existing package for him and you both will end up with PKGBUILDs cluttered with additional {{ic|replaces}} and {{ic|conflicts}}.
  
 
== File placement ==
 
== File placement ==
Again, analyze existing packages (if present) and decide whether or not you want to conflict with them. Do not place things under {{Ic|/opt}} unless you want to use some ugly hacks like giving ownership {{Ic|root:games}} to the package directory (so users in group {{Ic|games}} running the game can write files in the game's own folder).
+
 
 +
Again, analyze existing packages (if present) and decide whether or not you want to conflict with them. Do not place things under {{ic|/opt}} unless you want to use some ugly hacks like giving ownership {{ic|root:games}} to the package directory (so users in group {{ic|games}} running the game can write files in the game's own folder).
  
 
== Missing files ==
 
== Missing files ==
For most commercial games there is no way to (legally) download game files, which is preferable way to get them for normal packages. Even when it is possible to download files after providing password (like with all [[Wikipedia:Humble Indie Bundle|Humble Indie Bundle]] games) asking user for this password and downloading somewhere in {{Ic|build}} function is not recommended for variety of reasons (for example user can have no Internet access but have all files downloaded and stored locally). Following options should be considered:
 
  
* '''There is only one way to obtain files'''
+
For most commercial games there is no way to (legally) download game files, which is the preferable way to get them for normal packages. Even when it is possible to download files after providing a password (like with all [[Wikipedia:Humble Indie Bundle|Humble Indie Bundle]] games) asking user for this password and downloading somewhere in {{ic|build}} function is not recommended for a variety of reasons (for example, the user may have no Internet access but have all files downloaded and stored locally).
:* Software is distributed in archive/installer
+
 
:add required file to {{Ic|sources}} array:
+
The subsections below provide recommendations for a few situations you may encounter.
:{{Bc|1=sources=(... "''originalname''::'''file://'''''originalname''")}}
+
 
:This way link to file in AUR web interface will look different from names of files, included in source tarball.
+
=== Files can only be obtained in a distributed archive/installer ===
:Add following comment on package page:
+
 
:{{bc|Need archive/installer is required for work.}}
+
The software is only available via that archive/installer file, which must be obtained in order to get the missing files.
:and explain in details in PKGBUILD source.
+
 
 +
Add the required archive/installer to the {{ic|source}} array, renaming the source filename so the source's link in the AUR web interface looks different from names of files included in the source tarball:
 +
 
 +
{{bc|1=sources=(... "''originalname''::'''local:'''//''originalname''")}}
 +
 
 +
Also add a pinned comment like the one below to the package page in the AUR, and explain the details in the PKGBUILD:
 +
 
 +
{{bc|Need archive/installer to work.}}
 +
 
 +
==== Scheme to choose ====
 +
In case you use the '''local://''' scheme in a source array, makepkg behaves as though no scheme were specified, and the file must be manually placed in the same directory as the PKGBUILD.
 +
 
 +
In case you use the '''file://''' scheme, you can additionally specify DLAGENTS for the file protocol, so it may be obtained in a special way. See examples [[#Custom DLAGENTS|below]].
 +
 
 +
However, there are still no clear rules which of these schemes you should use.
 +
 
 +
=== Files can only be obtained in an distributed compact-disk or other type of optical disk media ===
 +
 
 +
The software is only available via an optical disk media (e.g. CD, DVD, Bluray etc.), which must be inserted into the optical disk drive in order get the missing files.
 +
 
 +
Add an installer script and an {{ic|.install}} file to the package contents, like {{AUR|tsukihime-en}}{{Broken package link|{{aur-mirror|tsukihime-en}}}}.
  
:* Software is distributed on compact-disk
+
=== Files can be obtained from several ways ===
:add installer script and {{Ic|.install}} file to package contents like in package {{AUR|tsukihime-en}}.
 
  
* '''There are several ways to obtain files'''
+
Copying files from disk, downloading from Net or getting from archive during the {{ic|build}} phase may look like a good idea but it is not recommended because it limits the user's possibilities and makes package installation interactive (which is generally discouraged and just annoying). Again, a good installer script and {{ic|.install}} file can work instead.
Copying files from disk / downloading from Net / getting from archive during {{Ic|build}} phase may look like good idea but it is not recommended because it limits user's possibilities and makes package installation interactive (which is generally discouraged and just annoying). Again good installer script and {{Ic|.install}} file can do work.
+
 
 +
Few examples of various strategies for obtaining files required for package:
  
Few examples of various strategies of obtaining files required for package:
 
 
* {{AUR|worldofgoo}} – dependency on user-provided file
 
* {{AUR|worldofgoo}} – dependency on user-provided file
* {{AUR|umineko-en}} – combining files from freely available patch and user-provided compact-disk
+
* {{AUR|umineko-en}}{{Broken package link|{{aur-mirror|umineko-en}}}} – combining files from freely available patch and user-provided compact-disk
* {{AUR|worldofgoo-demo}} – autonomic fetching installer during build phase
+
* {{AUR|worldofgoo-demo}}{{Broken package link|{{aur-mirror|worldofgoo-demo}}}} – autonomic fetching installer during build phase
* {{AUR|ut2004-anthology}} – searching for disk via mountpoints
+
* {{AUR|ut2004-anthology}}{{Broken package link|{{aur-mirror|ut2004-anthology}}}} – searching for disk via mountpoints
  
 
== Advanced topics ==
 
== Advanced topics ==
 +
 +
=== Custom DLAGENTS ===
 +
 +
Some software authors aggressively protect their software from automatic downloading: ban certain "User-Agent" strings, create temporary links to files etc. You can still conveniently download these files by using {{ic|DLAGENTS}} variable in the PKGBUILD (see {{man|5|makepkg.conf}}). This is used by some packages in [[official repositories]], for example {{Pkg|ttf-baekmuk}}.
 +
 +
Please pay attention, if you want to have a customized user-agent string, if the latter contains spaces, parentheses, or slashes in it (or actually anything that can break parsing), this will not work. There's no workaround, this is the nature of arrays in bash and the way DLAGENTS was designed to be consumed in makepkg. The following example will thus '''not work''':
 +
 +
DLAGENTS=("http::/usr/bin/curl -A 'Mozilla/4.0 (compatible; MSIE 8.0; Windows NT 6.1)' -fLC - --retry 3 --retry-delay 3 -o %o %u")
 +
 +
Shorten it to the following which is working:
 +
 +
DLAGENTS=("http::/usr/bin/curl -A 'Mozilla' -fLC - --retry 3 --retry-delay 3 -o %o %u")
 +
 +
And the following allows to extract temporary link to file from download page:
 +
 +
DLAGENTS=("http::/usr/bin/wget -r -np -nd -H %u")
 +
 +
In order to download temporary links to files or get past an interactive download, it is possible to analyze the HTTP request used to create the final download link, and then create a DLAGENTS that emulates this using curl. See for example {{AUR|blackmagic-decklink-sdk}} or {{AUR|jlink-software-and-documentation}}.
 +
 +
Alternatively, the DLAGENTS can be used to provide a more informative error message to the user when a file is missing. See for example {{AUR|ttf-ms-win10}}.
 +
 +
=== Unpacking ===
 +
 +
Many proprietary programs are shipped in nasty installers which sometimes do not even run in Wine. The following tools may be of some help:
 +
 +
* {{Pkg|unzip}} and {{Pkg|unrar}} unpack executable SFX archives, based on this formats
 +
* {{Pkg|cabextract}} can unpack most {{ic|.cab}} files (including ones with {{ic|.exe}} extension)
 +
* {{Pkg|unshield}} can extract CAB files from InstallShield installers
 +
* {{Pkg|p7zip}} unpacks not only many archive formats but also [[Wikipedia:NSIS|NSIS]]-based {{ic|.exe}} installers
 +
** it even can extract single sections from common PE ({{ic|.exe}} & {{ic|.dll}}) files!
 +
* {{Pkg|upx}} is sometimes used to compress above-listed executables and can be used for decompressing them as well
 +
* {{Pkg|innoextract}} can unpack {{ic|.exe}} installers created with [[Wikipedia:Inno Setup|Inno Setup]] (used for example by GOG.com games)
 +
* {{Pkg|libarchive}} contains ''bsdtar'', which can extract {{ic|.iso}} images and {{ic|.AppImage}} files (which are actually hybrid self-executable iso9660)
 +
In order to determine exact type of file run {{ic|file ''file_of_unknown_type''}}.
 +
 
=== Getting icons for .desktop files ===
 
=== Getting icons for .desktop files ===
Proprietary programs often have no separate icon files, so there is nothing to use in [[.desktop]] file creation. Happily .ico files can be easily extracted from executables with programs from {{Pkg|icotools}} package. You can even do it on fly during {{Ic|build}} phase (example can be found in {{AUR|tsukihime-en}}).
+
 
 +
Proprietary software often have no separate icon files, so there is nothing to use in [[.desktop]] file creation. Happily {{ic|.ico}} files can be easily extracted from executables using programs from the {{Pkg|icoutils}} package. You can even do it on the fly during the {{ic|build}} phase, for example:
 +
 
 +
$ wrestool -x --output=''icon.ico'' -t14 ''executable.exe''

Latest revision as of 09:33, 24 January 2019

Package creation guidelines

32-bitCLRCrossEclipseElectronFree PascalGNOMEGoHaskellJavaKDEKernelLispMinGWNode.jsNonfreeOCamlPerlPHPPythonRRubyRustVCSWebWine

Tango-view-fullscreen.pngThis article or section needs expansion.Tango-view-fullscreen.png

Reason: cover about encrypting archives, symlinking etc. (Discuss in Talk:Nonfree applications package guidelines#)

For many applications (most of which are Windows ones) there are neither sources nor tarballs available. Many of such applications can not be freely distributed because of license restrictions and/or lack of legal ways to obtain installer for no fee. Such software obviously can not be included into the official repositories but due to nature of AUR it is still possible to privately build packages for it, manageable with pacman.

Note: All information here is package-agnostic, for information specific to the most typical nonfree software see Wine PKGBUILD guidelines.

Rationale

There are multiple reasons for packaging even non-packageable software:

  • Simplification of installation/removal process
This is applicable even to the simplest of apps, which consist of a single script to be installed into /usr/bin. Instead of issuing:
$ chmod +x filename
$ chown root:root filename
# cp filename /usr/bin/
you can type just
$ makepkg -i
Most non-free applications are obviously much more complicated, but the burden of downloading an archive/installer from a homepage (often full of advertising), unpacking/decrypting it, hand-writing stereotypical launcher scripts and doing other similar tasks can be effectively lightened by a well-written packaging script.
  • Utilizing pacman capabilities
The ability to track state, perform automatic updates of any installed piece of software, determine ownership of every single file, and store compressed packages in a well-organized cache is what makes GNU/Linux distributions so powerful.
  • Sharing code and knowledge
It is simpler to apply tweaks, fix bugs and seek/provide help in a single public place like the AUR versus submitting patches to proprietary developers who may have ceased support or asking vague questions on general purpose forums.

Common rules

Avoid nonfree software when possible

Yes, it's better to leave this guide and spend some time searching (or maybe even creating) alternatives to an application you wanted to package because:

  1. It is better to support software that is owned by us all than software that is owned by a company
  2. It is better to support software that is actively maintained
  3. It is better to support software that can be fixed if just one person out of millions cares enough

Use open source variants where possible

Many commercial games (some are listed in this Wiki) have open source engines and many old games can be played with emulators such as ScummVM. Using open source engines together with the original game assets gives users access to bug fixes and eliminates several issues caused by binary packages.

Keep it simple

If the packaging of some program requires more effort and hacks than buying and using the original version - do the simplest thing, it is Arch!

Package naming

Before choosing a name on your own, search in the AUR for existing versions of the software you want to package. Try to use established naming conventions (e.g. do not create something like gish-hbAUR[broken link: archived in aur-mirror] when there are already packages for aquaria-hibAUR, penumbra-overture-hibAUR and uplink-hibAUR). Use the suffix -bin always unless you are sure there will never be a source-based package—its creator would have to ask you (or in the worst case a TU) to orphan the existing package for him and you both will end up with PKGBUILDs cluttered with additional replaces and conflicts.

File placement

Again, analyze existing packages (if present) and decide whether or not you want to conflict with them. Do not place things under /opt unless you want to use some ugly hacks like giving ownership root:games to the package directory (so users in group games running the game can write files in the game's own folder).

Missing files

For most commercial games there is no way to (legally) download game files, which is the preferable way to get them for normal packages. Even when it is possible to download files after providing a password (like with all Humble Indie Bundle games) asking user for this password and downloading somewhere in build function is not recommended for a variety of reasons (for example, the user may have no Internet access but have all files downloaded and stored locally).

The subsections below provide recommendations for a few situations you may encounter.

Files can only be obtained in a distributed archive/installer

The software is only available via that archive/installer file, which must be obtained in order to get the missing files.

Add the required archive/installer to the source array, renaming the source filename so the source's link in the AUR web interface looks different from names of files included in the source tarball:

sources=(... "originalname::local://originalname")

Also add a pinned comment like the one below to the package page in the AUR, and explain the details in the PKGBUILD:

Need archive/installer to work.

Scheme to choose

In case you use the local:// scheme in a source array, makepkg behaves as though no scheme were specified, and the file must be manually placed in the same directory as the PKGBUILD.

In case you use the file:// scheme, you can additionally specify DLAGENTS for the file protocol, so it may be obtained in a special way. See examples below.

However, there are still no clear rules which of these schemes you should use.

Files can only be obtained in an distributed compact-disk or other type of optical disk media

The software is only available via an optical disk media (e.g. CD, DVD, Bluray etc.), which must be inserted into the optical disk drive in order get the missing files.

Add an installer script and an .install file to the package contents, like tsukihime-enAUR[broken link: archived in aur-mirror].

Files can be obtained from several ways

Copying files from disk, downloading from Net or getting from archive during the build phase may look like a good idea but it is not recommended because it limits the user's possibilities and makes package installation interactive (which is generally discouraged and just annoying). Again, a good installer script and .install file can work instead.

Few examples of various strategies for obtaining files required for package:

Advanced topics

Custom DLAGENTS

Some software authors aggressively protect their software from automatic downloading: ban certain "User-Agent" strings, create temporary links to files etc. You can still conveniently download these files by using DLAGENTS variable in the PKGBUILD (see makepkg.conf(5)). This is used by some packages in official repositories, for example ttf-baekmuk.

Please pay attention, if you want to have a customized user-agent string, if the latter contains spaces, parentheses, or slashes in it (or actually anything that can break parsing), this will not work. There's no workaround, this is the nature of arrays in bash and the way DLAGENTS was designed to be consumed in makepkg. The following example will thus not work:

DLAGENTS=("http::/usr/bin/curl -A 'Mozilla/4.0 (compatible; MSIE 8.0; Windows NT 6.1)' -fLC - --retry 3 --retry-delay 3 -o %o %u")

Shorten it to the following which is working:

DLAGENTS=("http::/usr/bin/curl -A 'Mozilla' -fLC - --retry 3 --retry-delay 3 -o %o %u")

And the following allows to extract temporary link to file from download page:

DLAGENTS=("http::/usr/bin/wget -r -np -nd -H %u")

In order to download temporary links to files or get past an interactive download, it is possible to analyze the HTTP request used to create the final download link, and then create a DLAGENTS that emulates this using curl. See for example blackmagic-decklink-sdkAUR or jlink-software-and-documentationAUR.

Alternatively, the DLAGENTS can be used to provide a more informative error message to the user when a file is missing. See for example ttf-ms-win10AUR.

Unpacking

Many proprietary programs are shipped in nasty installers which sometimes do not even run in Wine. The following tools may be of some help:

  • unzip and unrar unpack executable SFX archives, based on this formats
  • cabextract can unpack most .cab files (including ones with .exe extension)
  • unshield can extract CAB files from InstallShield installers
  • p7zip unpacks not only many archive formats but also NSIS-based .exe installers
    • it even can extract single sections from common PE (.exe & .dll) files!
  • upx is sometimes used to compress above-listed executables and can be used for decompressing them as well
  • innoextract can unpack .exe installers created with Inno Setup (used for example by GOG.com games)
  • libarchive contains bsdtar, which can extract .iso images and .AppImage files (which are actually hybrid self-executable iso9660)

In order to determine exact type of file run file file_of_unknown_type.

Getting icons for .desktop files

Proprietary software often have no separate icon files, so there is nothing to use in .desktop file creation. Happily .ico files can be easily extracted from executables using programs from the icoutils package. You can even do it on the fly during the build phase, for example:

$ wrestool -x --output=icon.ico -t14 executable.exe