Distcc 是一个将 C、C++、Objective C 或 Objective C++ 等程序的编译任务分发到网络中多个主机的程序。distcc 的结果和本地编译一模一样，安装后使用方便，通常比本地编译快很多。
- 1 名词定义
- 2 开始
- 3 配置
- 4 编译
- 5 监视进度
- 6 "Cross Compiling" with distcc
- 7 Troubleshooting
- 接受 master 的编译请求，一个系统可以包含一个或多个从机。
/etc/conf.d/distccd，修改唯一没有被注释掉的那一行，设置正确的 IP 地址或整个子网。
DISTCC_ARGS="--user nobody --allow 192.168.0.0/24"
CIDR Utility Tool可以帮助进行地址转换，其它设置请参考 .
- 确保 BUILDENV 中的 distcc 没有被禁用(前面没有感叹号)
- 编辑DISTCC_HOSTS ，加入可以使用的编译服务器的 IP 地址 + 反斜杠 + 线程数，不同 IP 地址用空格隔开，按照处理器性能排列
- 修改 MAKEFLAGS 中的 N 为所有使用线程的和。在下面的示例为 5+3+3=11. 如果超过这个值，超出的线程将会被 distcc阻塞。
BUILDENV=(distcc fakeroot color !ccache check !sign) MAKEFLAGS="-j11" DISTCC_HOSTS="192.168.0.2/5 192.168.0.3/3 192.168.0.4/3"
非 makepkg 编译
The minimal configuration for distcc on the master includes the setting of the available slaves. This can either be done by setting the addresses in the environment variable
DISTCC_HOSTS or in either of the configuration files
Example for setting the slave address using
$ export DISTCC_HOSTS="192.168.0.3,lzo,cpp 192.168.0.4,lzo,cpp"
Example for setting the slave addresses in the hosts configuration file:
Instead of explicitly listing the server addresses one can also use the avahi zeroconf mode. To use this mode
+zeroconf must be in place instead of the server addresses and the distcc daemons on the slaves have to be started using the
--zeroconf option. Note that this option does not support the pump mode!
The examples add the following options to the address:
lzo: Enables LZO compression for this TCP or SSH host (slave).
cpp: Enables distcc-pump mode for this host (slave). Note: the build command must be wrapped in the pump script in order to start the include server.
A description for the pump mode can be found here: HOW DISTCC-PUMP MODE WORKS and distcc's pump mode: A New Design for Distributed C/C++ Compilation
To use distcc-pump mode for a slave, users must start the compilation using the pump script otherwise the compilation will fail.
然后正常运行 makepkg 即可。
非 makepkg 编译
To compile a source file using the distcc pump mode, use the following command:
$ pump distcc g++ -c hello_world.cpp
In this case the pump script will execute distcc which in turn calls g++ with "-c hello_world.cpp" as parameter.
To compile a Makefile project, first find out which variables are set by the compiler. For example in gzip-1.6, one can find the following line in the Makefile:
CC = gcc -std=gnu99. Normally the variables are called
CC for C projects and
CXX for C++ projects. To compile the project using distcc it would look like this:
$ wget ftp://ftp.gnu.org/pub/gnu/gzip/gzip-1.6.tar.xz $ tar xf gzip-1.6.tar.xz $ cd gzip-1.6 $ ./configure $ pump make -j2 CC="distcc gcc -std=gnu99"
This example would compile gzip using distcc's pump mode with two compile threads. For the correct
-j setting have a look at What -j level to use?
ships with a cli monitor
distccmon-text and a gtk monitor
distccmon-gnome one can use to check on compilation status.
The cli monitor can run continuously by appending a space followed by integer to the command which corresponds to the number of sec to wait for a repeat query:
$ distccmon-text 3 29291 Preprocess probe_64.c 192.168.0.2 30954 Compile apic_noop.c 192.168.0.2 30932 Preprocess kfifo.c 192.168.0.2 30919 Compile blk-core.c 192.168.0.2 30969 Compile i915_gem_debug.c 192.168.0.2 30444 Compile block_dev.c 192.168.0.3 30904 Compile compat.c 192.168.0.3 30891 Compile hugetlb.c 192.168.0.3 30458 Compile catalog.c 192.168.0.4 30496 Compile ulpqueue.c 192.168.0.4 30506 Compile alloc.c 192.168.0.4
"Cross Compiling" with distcc
32-bit x86 (i686)
There are currently two methods from which to select to have the ability of distcc distribution of tasks over a cluster building i686 packages from a native x86_64 environment. Neither is ideal, but to date, there are the only two methods documented on the wiki.
An ideal setup is one that uses the unmodified ARCH packages for distccd running only once one each node regardless of building from the native environment or from within a chroot AND one that works with makepkg. Again, this Utopian setup is not currently known.
A discussion thread has been started on the topic; feel free to contribute.
Chroot method (preferred)
Assuming the user has a 32-bit chroot setup and configured on each node of the distcc cluster, the strategy is to have two separate instances of distccd running on different ports on each node -- one runs in the native x86_64 environment and the other in the x86 chroot on a modified port. Start makepkg via a command invoking makepkg.
Add port numbers to DISTCC_HOSTS on the i686 chroot
Append the port number defined eariler (3692) to each of the hosts in
/opt/arch32/etc/makepkg.conf as follows:
DISTCC_HOSTS="192.168.1.101/5:3692 192.168.1.102/5:3692 192.168.1.103/3:3692"
Invoke makepkg from the Native Environment
Set upon the native x86_64 environment. Invoke makepkg to build an i686 package from the native x86_64 environment, simply by:
$ schroot -p -- makepkg -src
Multilib GCC method (not recommended)
Arch Linux ARM
When building on an Arch Linux ARM device, the developers highly recommend using the official project toolchains which should be installed on the x86_64 slave machine(s). Rather than manually managing these, the AUR provides all four toolchains as well as simple systemd service units:
Setup on the slave machine containing the toolchain is identical to #Slaves[broken link: invalid section] except that the name of the configuration file matches that of the respective package. For example,
A systemd service unit is provided for each respective package. For example,
- EmbToolkit: Tool for creating cross compilation tool chain; supports ARM and MIPS architectures; supports building of an LLVM based tool chain
- crosstool-ng: Similar to EmbToolkit; supports more architectures (see website for more information)
- Linaro: Provides tool chains for ARM development
EmbToolkit provides a nice graphical configuration menu (
make xconfig) for configuring the tool chain.
journalctl to find out what was going wrong:
$ journalctl $(which distccd) -e --since "5 min ago"
Make sure that the tool chain works for the user account under which the distcc daemon process gets started (default is nobody). The following will test if the tool chain works for user nobody. In
/etc/passwd change the login for the nobody user to the following:
$ cat /etc/passwd
... nobody:x:99:99:nobody:/:/bin/bash ...
Then cd into the directory containing the cross compiler binaries and try to execute the compiler:
# su nobody $ ./gcc --version bash: ./gcc: Permission denied
Users experiencing this error should make sure that groups permissions as described in #Other architectures are correctly in setup.
Make sure to change back
/etc/passwd to its original state after these modifications.
Alternatively, use sudo without changing the shell in /etc/passwd.
# sudo -u nobody gcc --version
修改 $HOME/.distcc 位置以限制硬盘使用
$HOME/.distcc 保存中间结果。在内存 /tmp 中创建 .distcc 并链接到 $HOME 可以避免磁盘读写。
$ export DISTCC_DIR=/tmp/distcc
/var/log/messages.log，也可以放到一个单独的文件，例如通过 /tmp 放到 RAM 中。同时降低日志级别也可以只显示错误信息。LEVEL 可以是任意 syslog 级别，例如 critical, error, warning, notice, info, 或 debug.
/etc/conf.d/distccd 的 DISTCC_ARGS 中。
DISTCC_ARGS="--user nobody --allow 192.168.0.0/24 --log-level error --log-file /tmp/distccd.log"