Difference between revisions of "JHBuild"

From ArchWiki
Jump to: navigation, search
(Python issues: Add note about jhbuild package fixing Python-related issues)
(Troubleshooting: List modules known to require python2)
 
(87 intermediate revisions by 14 users not shown)
Line 1: Line 1:
 
[[Category:Development]]
 
[[Category:Development]]
JHBuild is a tool that allows you to automatically download and compile "modules" (i.e. source code packages). It can pull modules from a variety of sources (CVS, Subversion, Git, Bazaar, tarballs...) and handle dependencies. You can also choose which specific modules you want to build, instead of building the whole project.
+
[[ja:JHBuild]]
 +
JHBuild is a tool that allows you to automatically download and compile "modules" (i.e. source code packages). It can pull modules from a variety of sources (CVS, Subversion, Git, Bazaar, tarballs, etc.) and handle dependencies. You can also choose which specific modules you want to build, instead of building the whole project.
  
 
JHBuild was originally written for building [[GNOME]], but has since been extended to be usable with other projects.
 
JHBuild was originally written for building [[GNOME]], but has since been extended to be usable with other projects.
  
==Installation==
+
== Installation ==
  
JHBuild is available in the [[Arch User Repository]]. Install either {{AUR|jhbuild}} or {{AUR|jhbuild-git}}, the development version.
+
Install the {{AUR|jhbuild}} package, which usually provides the stable version.
  
==Configuration==
+
Alternatively, you may want to install {{AUR|jhbuild-git}} for latest JHBuild version in GNOME repository.
  
The configuration file for JHBuild is {{ic|~/.jhbuildrc}}. It uses [[Python]] syntax to set configuration variables. Here is the sample file provided for building GNOME:
+
== Configuration ==
  
# -*- mode: python -*-
+
JHBuild gets its configuration from a system-wide configuration file installed with the package, the {{Ic|defaults.jhbuildrc}}, and from an optional user configuration file jhbuildrc, in {{ic|~/.config/jhbuildrc}} (if it exists).
# -*- coding: utf-8 -*-
 
 
# edit this file to match your settings and copy it to ~/.jhbuildrc
 
 
# if you have a GNOME git account, uncomment this line
 
# repos['git.gnome.org'] = 'ssh://user@git.gnome.org/git/'
 
 
# what module set should be used.  The default can be found in
 
# jhbuild/defaults.jhbuildrc, but can be any file in the modulesets directory
 
# or a URL of a module set file on a web server.
 
# moduleset = 'gnome-apps-3.2'
 
#
 
# A list of the modules to build.  Defaults to the GNOME core and tested apps.
 
# modules = [ 'meta-gnome-core', 'meta-gnome-apps-tested' ]
 
 
# Or to build the old GNOME 2.32:
 
# moduleset = 'gnome2/gnome-2.32'
 
# modules = ['meta-gnome-desktop']
 
 
# what directory should the source be checked out to?
 
checkoutroot = os.path.expanduser('~/checkout/gnome')
 
 
# the prefix to configure/install modules to (must have write access)
 
prefix = '/opt/gnome'
 
 
# custom CFLAGS / environment pieces for the build
 
# os.environ['CFLAGS'] = '-Wall -g -O0'
 
 
# extra arguments to pass to all autogen.sh scripts
 
# to speed up builds of GNOME, try '--disable-static --disable-gtk-doc'
 
#autogenargs=''
 
 
# On multiprocessor systems setting makeargs to '-j2' may improve compilation
 
# time. Be aware that not all modules compile correctly with '-j2'.
 
# Set makeargs to 'V=1' for verbose build output.
 
#makeargs = '-j2'
 
  
You should edit at least ''modules'' to the desired modules to be built. A reference for most configuration variables is available at [http://developer.gnome.org/jhbuild/unstable/config-reference.html.en GNOME JHBuild Manual].
+
These files use [[Python]] syntax to set configuration variables.
  
==Usage==
+
The variables currently accepted can be found in [https://developer.gnome.org/jhbuild/stable/ JHBuild Manual].
  
===Installing Prerequisites===
+
=== Default JHBuild configuration ===
  
JHBuild can check if the required tools are installed by running ''sanitycheck'':
+
It can be found at {{Ic|/usr/lib/python2.7/site-packages/jhbuild/defaults.jhbuildrc}}.
 +
 
 +
{{Ic|defaults.jhbuildrc}} provides default values for many configuration options, e.g. moduleset, modules, autoargs, module_autoargs, and others.
 +
 
 +
It is provided with the package, so please avoid modifying it.
 +
 
 +
The values set in {{Ic|defaults.jhbuildrc}} should be enough for JHBuild to work. Nevertheless, take a look at its value to decide what you want to use the default, and what you want to customize in a personal jhbuild configuration file.
 +
 
 +
=== User configuration ===
 +
 
 +
{{Ic|~/.config/jhbuildrc}} is very useful for setting personal values in order to overlap what is currently set in {{Ic|defaults.jhbuildrc}} (e.g.: want to set a different moduleset), or to set values not already set in {{Ic|defaults.jhbuildrc}} (e.g.: your own module_autoargs for a specific module).
 +
 
 +
You may find a very extended sample of a configuration file in {{Ic|/usr/share/jhbuild/examples/sample.jhbuildrc}}
 +
 
 +
See below a few examples for {{Ic|~/.config/jhbuildrc}} contents:
 +
 
 +
* Enable a wide-most moduleset, and also force GTK3 for modules that would use GTK2 by default
 +
{{bc|<nowiki>
 +
moduleset = ['gnome-world']
 +
autogenargs = '--with-gtk3'
 +
</nowiki>}}
 +
 
 +
* Or you may want to enable documentation build, even though it will slowdown module compilation
 +
{{bc|<nowiki>
 +
autogenargs = '--enable-gtk-doc'
 +
</nowiki>}}
 +
 
 +
* Or you want some debug output from make command
 +
{{bc|<nowiki>
 +
makeargs = 'V=1'
 +
</nowiki>}}
 +
 
 +
* Or you found out that a module needs a specific setting (this is just an example; itstool is already specified)
 +
{{bc|<nowiki>
 +
module_autogenargs = {
 +
      'itstool': autogenargs + ' PYTHON=/usr/bin/python2'
 +
}
 +
</nowiki>}}
 +
 
 +
== Usage ==
 +
 
 +
This topic provides some information and examples on how to use some JHBuild commands, but without intending to exhaust the subject. For a detailed information on each of JHBuild commands, please refer to [https://developer.gnome.org/jhbuild/stable/ JHBuild Manual], learn from each command's help output or even read JHBuild's [https://git.gnome.org/browse/jhbuild/ source code].
 +
 
 +
JHBuild provides a general {{Ic|--help}} which lists all the commands available, and also a help message for each sub-command, e.g. {{Ic|jhbuild sysdeps --help}}.
 +
 
 +
=== Checking and installing prerequisites ===
 +
 
 +
''sysdeps'' can be used to get a detailed list of which dependencies you have installed and which ones you should install. In order to get this information, just run:
 +
 
 +
$ jhbuild sysdeps
 +
 
 +
Alternatively, ''sanitycheck'' may also be used to report missing tools that are required (e.g. mozjs38). If nothing is reported (and exit code is 0), then you're good to go! Similiar to the above command, just run:
  
 
  $ jhbuild sanitycheck
 
  $ jhbuild sanitycheck
  
If any errors are shown, missing packages may be installed from repositories or running the ''bootstrap'' command, which tries to download, build and install the build prerequisites:
+
If some tools are reported to be missing, you can — besides installing correspondent packages with pacman, if available — run ''bootstrap'' command, which will download, build and install the named module inside your jhbuild tree:
  
 
  $ jhbuild bootstrap
 
  $ jhbuild bootstrap
  
===Building===
+
{{note|All dependencies informed by ''sysdeps'' should be already listed as dependency {{AUR|jhbuild}} by now. If you run ''sysdeps'' and find out a dependency that is not be listed, please report it in the comments of {{AUR|jhbuild}}}}
 +
 
 +
=== Updating modules ===
 +
 
 +
It is possible to simply update the source code of the modules without building them, making it possible build another time without having to wait fetching the source code. There are three ways of simply updating modules:
 +
 
 +
''update'', without any arguments, will update all modules available in the moduleset/modules set in configuration file
 +
 
 +
$ jhbuild update
 +
 
 +
''update'', with one or more modules as arguments, will update '''all''' modules that the named modules depends on. e.g.:
 +
 
 +
$ jhbuild update gedit
 +
 
 +
''updateone'', with one or more modules, will update '''only''' the named module(s). e.g.:
 +
 
 +
$ jhbuild updateonly gedit
 +
 
 +
=== Building modules ===
 +
 
 +
This action will run the whole build process: it will update the source code (unless it is already up-to-date), configure & build, and install it in the proper directory.
 +
 
 +
Just like ''update'', There are three ways of building modules in JHBuild:
  
To build all the modules selected in the configuration file, just run the ''build'' command:
+
''build'', without any arguments, will build all modules available in the moduleset/modules set in configuration file
  
 
  $ jhbuild build
 
  $ jhbuild build
  
JHBuild will download, configure, compile and install each of the modules.
+
''build'', with one or more modules as arguments, will build '''all''' modules that the named modules depends on. e.g.:
 +
 
 +
$ jhbuild build gedit
 +
 
 +
''buildone'', with one or more modules, will build '''only''' the named module(s). e.g.:
 +
 
 +
$ jhbuild buildone gedit
 +
 
 +
{{note|Please notice that since ''buildone'' don't also build the latest development version of a module's dependencies, it may fail due to dependency not being installed or being out-of-date; in this case, you should run ''build'' for the named module}}
 +
 
 +
=== Running modules ===
 +
 
 +
After a successfully installed application in JHBuild, use ''run'' to start the module you just built. e.g.:
 +
 
 +
$ jhbuild run gedit
 +
 
 +
=== Creating dependency graph ===
 +
 
 +
JHBuild can output graph contents which can by piped into {{Pkg|graphviz}} in order to generate e.g. a PNG or PostScript file.
 +
 
 +
To generate a PNG file of e.g. gedit, run:
 +
 
 +
$ jhbuild dot gedit | dot -Tpng > dependencies.png
 +
 
 +
== Troubleshooting ==
 +
 
 +
{{Note|If you encounter an issue that is not documented below, please report it in a comment on the {{AUR|jhbuild}} package.}}
 +
 
 +
=== Python issues ===
 +
 
 +
A module that depends on {{Pkg|python2}} may fail to build as softwares usually expect the binary filename of python 2.x to be {{Ic|/usr/bin/python}} and python 3.x to be {{Ic|/usr/bin/python3}}, which is not the case in Arch Linux: python 2.x is {{Ic|/usr/bin/python2}} and python 3.x is {{Ic|/usr/bin/python}}.
 +
 
 +
For cases like that, force the modules to run {{Ic|/usr/bin/python2}} using the one or more methods below:
 +
 
 +
* set ''module_autoargs'' with ''PYTHON=/usr/bin/python2'' for this specific module in {{Ic|~/.config/jhbuildrc}}, as mentioned in the above Configuration section — this will run autogen.sh or configure with this value for variable PYTHON
  
If an error occurs at any stage, JHBuild will present a menu asking what to do. The choices include dropping to a shell to fix the error, rerunning the build from various stages, giving up on the module, or ignore the error and continue. Often, dropping to a shell and checking makefiles and configuration files can be helpful. If you face a build error, for example, you can try to manually {{ic|make}} and check errors on the shell.
+
* if the ''configure'' or ''Makefile'' doesn't parse PYTHON variable, one approach is to manually find all lines in ''configure''/''Makefile'' that run python binary and rename python -> python2 — this will hard code python2 in the module's source code.
  
Giving up on a module will cause any modules depending on it to fail.
+
* if only the above workarounds still don't work for you, consider editing module's .py files in order to replace ''python'' with ''python2'' when the first line matches ''#!/usr/bin/env python'' or ''#!/usr/bin/python''
  
==Troubleshooting==
+
{{note|All edits and patches that are manually applied to the source code will be lost when you wipe the directory and checkout (download) it again, so it is not exactly a permanent solution}}
  
===Python issues===
+
{{note|If a module's configure/build process misuse PYTHON variable, or doesn't use at all, consider reporting it to the module's upstream and/or providing patch for JHBuild upstream}}
  
{{Note|Users of the {{AUR|jhbuild}} package should not encounter Python-related issues; if you do, please add a comment describing your issue to the {{AUR|jhbuild}} page.}}
+
==== Modules known to require python2 exclusively ====
  
Projects which use ''Python 2'' may encounter errors when building, because they refer to paths like {{ic|/usr/bin/python}}, which points to ''Python 3'' on Arch. Building [[GNOME]] with a default configuration may cause this error.
+
{{AUR|jhbuild}} is already patched to use ''python2'' for modules that exclusively require this Python version, and therefore no action is required if you're using packaged JHBuild.
  
To fix this issue, you can point the correct ''Python 2'' path to JHBuild by appending the following to {{ic|~/.jhbuildrc}}:
+
They are:
 +
* itstool
 +
* telepathy-mission-control
 +
* WebKit
  
autogenargs='PYTHON=/usr/bin/python2'
+
=== pkg-config issues ===
  
Packages will then be configured to use Python 2 instead of Python 3.
+
If you have a malformatted .pc file on your PKG_CONFIG_PATH, JHBuild will not be able to detect all the (valid) .pc files you have installed and will complain that the .pc files are missing. Look at the output of {{ic|jhbuild sysdeps}}—there should be a message about the problematic .pc files.
  
'''If you are building from scratch on your own''', it may be necessary to run autogen.sh with the following:
+
== Packages needed to build specific modules ==
  
PYTHON=/usr/bin/python2 ./autogen.sh
+
* gitg requires {{Pkg|gtkspell3}}
 +
* gtk-vnc requires {{Pkg|perl-text-csv}}
 +
* latexila requires {{AUR|lcov}}
 +
* pango requires {{AUR|libpthread-stubs}}
 +
* totem-pl-parser requires {{Pkg|libgcrypt15}}
 +
* xf86-video-intel requires {{Pkg|xorg-server-devel}}
 +
* xwayland requires {{Pkg|xtrans}}, {{Pkg|xcmiscproto}}, and {{Pkg|bigreqsproto}}
 +
* zeitgeist requires {{Pkg|python2-rdflib}}
 +
* wireless-tools requires {{Pkg|wireless_tools}}
 +
* xorg-macros requires {{Pkg|xorg-util-macros}}
  
It was also necessary for me to update the shebang line in ~/.local/bin/jhbuild
+
== Building JHBuild from scratch ==
  
#!/usr/bin/python2
+
If you do not want to use the {{AUR|jhbuild}} package, and instead you want build JHBuild from scratch on your own, there are a few things you should pay attention too.
  
Once that is finished, run
+
* Make sure to install all the dependencies required by JHBuild and its target modules. Refer to [https://wiki.gnome.org/Projects/Jhbuild/Dependencies/ArchLinux list of dependencies for Arch Linux];
  
jhbuild sanitycheck
+
* JHBuild itself depends on [[Python]] version 2, so make sure to run JHBuild's ''autogen.sh'' with the following command line:
  
That will tell you what dependencies you need. In my case, I had to:
+
$ PYTHON=/usr/bin/python2 ./autogen.sh
  
pacman -S docbook-xsl
+
* Likewise, some modules may still depend on python2. Make sure to read the above topic 'Python issues'.
pacman -S cvs
 
  
I had to switch the symlink "/usr/bin/python" to point to python2.7 to get "jhbuild build" to run correctly. If there is a better way to do this,
+
* For detailed information downloading and building the source code of JHBuild, check [https://wiki.gnome.org/HowDoI/JhbuildJHBuild's How Do I at GNOME wiki].
please update this to reflect that. It was difficult for me to track down Arch Users that had built jhbuild from scratch so I figure these
 
notes are better than none and may save some time. The folks in #gnome-shell on irc.gnome.org will help in any way they can.
 
  
==External links==
+
== See also ==
  
[http://developer.gnome.org/jhbuild/ GNOME JHBuild Manual]
+
* [https://wiki.gnome.org/Projects/Jhbuild JHBuild homepage in GNOME Wiki]
 +
* [https://wiki.gnome.org/Newcomers/BuildGnome JHBuild for GNOME newcomers]
 +
* [https://wiki.gnome.org/HowDoI/Jhbuild JHBuild for experienced GNOME contributors]
 +
* [https://developer.gnome.org/jhbuild/stable/ JHBuild Manual]
 +
* [https://git.gnome.org/browse/jhbuild/ JHBuild Source Code]

Latest revision as of 12:14, 24 February 2017

JHBuild is a tool that allows you to automatically download and compile "modules" (i.e. source code packages). It can pull modules from a variety of sources (CVS, Subversion, Git, Bazaar, tarballs, etc.) and handle dependencies. You can also choose which specific modules you want to build, instead of building the whole project.

JHBuild was originally written for building GNOME, but has since been extended to be usable with other projects.

Installation

Install the jhbuildAUR package, which usually provides the stable version.

Alternatively, you may want to install jhbuild-gitAUR for latest JHBuild version in GNOME repository.

Configuration

JHBuild gets its configuration from a system-wide configuration file installed with the package, the defaults.jhbuildrc, and from an optional user configuration file jhbuildrc, in ~/.config/jhbuildrc (if it exists).

These files use Python syntax to set configuration variables.

The variables currently accepted can be found in JHBuild Manual.

Default JHBuild configuration

It can be found at /usr/lib/python2.7/site-packages/jhbuild/defaults.jhbuildrc.

defaults.jhbuildrc provides default values for many configuration options, e.g. moduleset, modules, autoargs, module_autoargs, and others.

It is provided with the package, so please avoid modifying it.

The values set in defaults.jhbuildrc should be enough for JHBuild to work. Nevertheless, take a look at its value to decide what you want to use the default, and what you want to customize in a personal jhbuild configuration file.

User configuration

~/.config/jhbuildrc is very useful for setting personal values in order to overlap what is currently set in defaults.jhbuildrc (e.g.: want to set a different moduleset), or to set values not already set in defaults.jhbuildrc (e.g.: your own module_autoargs for a specific module).

You may find a very extended sample of a configuration file in /usr/share/jhbuild/examples/sample.jhbuildrc

See below a few examples for ~/.config/jhbuildrc contents:

  • Enable a wide-most moduleset, and also force GTK3 for modules that would use GTK2 by default
 moduleset = ['gnome-world']
 autogenargs = '--with-gtk3'
  • Or you may want to enable documentation build, even though it will slowdown module compilation
 autogenargs = '--enable-gtk-doc'
  • Or you want some debug output from make command
 makeargs = 'V=1'
  • Or you found out that a module needs a specific setting (this is just an example; itstool is already specified)
 module_autogenargs = {
      'itstool': autogenargs + ' PYTHON=/usr/bin/python2'
 }

Usage

This topic provides some information and examples on how to use some JHBuild commands, but without intending to exhaust the subject. For a detailed information on each of JHBuild commands, please refer to JHBuild Manual, learn from each command's help output or even read JHBuild's source code.

JHBuild provides a general --help which lists all the commands available, and also a help message for each sub-command, e.g. jhbuild sysdeps --help.

Checking and installing prerequisites

sysdeps can be used to get a detailed list of which dependencies you have installed and which ones you should install. In order to get this information, just run:

$ jhbuild sysdeps

Alternatively, sanitycheck may also be used to report missing tools that are required (e.g. mozjs38). If nothing is reported (and exit code is 0), then you're good to go! Similiar to the above command, just run:

$ jhbuild sanitycheck

If some tools are reported to be missing, you can — besides installing correspondent packages with pacman, if available — run bootstrap command, which will download, build and install the named module inside your jhbuild tree:

$ jhbuild bootstrap
Note: All dependencies informed by sysdeps should be already listed as dependency jhbuildAUR by now. If you run sysdeps and find out a dependency that is not be listed, please report it in the comments of jhbuildAUR

Updating modules

It is possible to simply update the source code of the modules without building them, making it possible build another time without having to wait fetching the source code. There are three ways of simply updating modules:

update, without any arguments, will update all modules available in the moduleset/modules set in configuration file

$ jhbuild update

update, with one or more modules as arguments, will update all modules that the named modules depends on. e.g.:

$ jhbuild update gedit

updateone, with one or more modules, will update only the named module(s). e.g.:

$ jhbuild updateonly gedit

Building modules

This action will run the whole build process: it will update the source code (unless it is already up-to-date), configure & build, and install it in the proper directory.

Just like update, There are three ways of building modules in JHBuild:

build, without any arguments, will build all modules available in the moduleset/modules set in configuration file

$ jhbuild build

build, with one or more modules as arguments, will build all modules that the named modules depends on. e.g.:

$ jhbuild build gedit

buildone, with one or more modules, will build only the named module(s). e.g.:

$ jhbuild buildone gedit
Note: Please notice that since buildone don't also build the latest development version of a module's dependencies, it may fail due to dependency not being installed or being out-of-date; in this case, you should run build for the named module

Running modules

After a successfully installed application in JHBuild, use run to start the module you just built. e.g.:

$ jhbuild run gedit

Creating dependency graph

JHBuild can output graph contents which can by piped into graphviz in order to generate e.g. a PNG or PostScript file.

To generate a PNG file of e.g. gedit, run:

$ jhbuild dot gedit | dot -Tpng > dependencies.png

Troubleshooting

Note: If you encounter an issue that is not documented below, please report it in a comment on the jhbuildAUR package.

Python issues

A module that depends on python2 may fail to build as softwares usually expect the binary filename of python 2.x to be /usr/bin/python and python 3.x to be /usr/bin/python3, which is not the case in Arch Linux: python 2.x is /usr/bin/python2 and python 3.x is /usr/bin/python.

For cases like that, force the modules to run /usr/bin/python2 using the one or more methods below:

  • set module_autoargs with PYTHON=/usr/bin/python2 for this specific module in ~/.config/jhbuildrc, as mentioned in the above Configuration section — this will run autogen.sh or configure with this value for variable PYTHON
  • if the configure or Makefile doesn't parse PYTHON variable, one approach is to manually find all lines in configure/Makefile that run python binary and rename python -> python2 — this will hard code python2 in the module's source code.
  • if only the above workarounds still don't work for you, consider editing module's .py files in order to replace python with python2 when the first line matches #!/usr/bin/env python or #!/usr/bin/python
Note: All edits and patches that are manually applied to the source code will be lost when you wipe the directory and checkout (download) it again, so it is not exactly a permanent solution
Note: If a module's configure/build process misuse PYTHON variable, or doesn't use at all, consider reporting it to the module's upstream and/or providing patch for JHBuild upstream

Modules known to require python2 exclusively

jhbuildAUR is already patched to use python2 for modules that exclusively require this Python version, and therefore no action is required if you're using packaged JHBuild.

They are:

  • itstool
  • telepathy-mission-control
  • WebKit

pkg-config issues

If you have a malformatted .pc file on your PKG_CONFIG_PATH, JHBuild will not be able to detect all the (valid) .pc files you have installed and will complain that the .pc files are missing. Look at the output of jhbuild sysdeps—there should be a message about the problematic .pc files.

Packages needed to build specific modules

Building JHBuild from scratch

If you do not want to use the jhbuildAUR package, and instead you want build JHBuild from scratch on your own, there are a few things you should pay attention too.

  • JHBuild itself depends on Python version 2, so make sure to run JHBuild's autogen.sh with the following command line:
$ PYTHON=/usr/bin/python2 ./autogen.sh
  • Likewise, some modules may still depend on python2. Make sure to read the above topic 'Python issues'.

See also