Debugging

From ArchWiki

This article or section needs expansion.

Reason: This article might as well be about debugging in general, so that other useful tools like ltrace can be added here. (Discuss in Talk:Debugging)

This page is mainly about how to gather more information in connection with bug reports. Even though the word "debug" is used, it is not intended as a guide for how to debug programs while developing.

Check availability of a core dump

A core dump is a file containing a process's address space (memory) when the process terminates unexpectedly. If the application is compiled in a debug-friendly way, the "core" file can be used to find out where things went wrong.

The location of core dumps may vary depending on the operating system configuration. See core dump to find whether generation of core dump files is enabled on your system and where do they go.

Segmentation faults

There are several techniques that can be used to figure out what went wrong. Put your detective hat on.

Gdb

gdb is an ancient and well tested application for debugging applications. See Debugging/Getting traces#Getting the trace for more instructions how to use it to obtain a trace. While running from gdb, you might have to wait for the segfault. Afterwards, post the trace to a pastebin service and include the URL in your bug report.

If you have a "core" file, it can be used together with gdb to get a backtrace:

$ gdb appname core
bt full

Valgrind

Assuming you have an unstripped binary without inlined functions, it is usually a good idea to also run that program through valgrind. valgrind is a tool that emulates a CPU and usually shows where things go wrong or provide info in addition to gdb.

$ valgrind appname

it will provide a lot of helpful debug output if there is a crash. Consider -v and --leak-check=full to get even more info.

Alternatively, use:

$ valgrind --tool=callgrind appname

and run the output through kcachegrind to graphically explore the functions the program uses. If a program hangs, this makes it easier to pinpoint the location of the error.

Missing files or libraries

Strace

strace finds out in detail what an application is actually doing. If an application tries to open a file that is not there, it can be discovered by strace.

For finding which files a program named appname tries to open:

$ strace -eopen appname
Tip: If you wish to grep the output from strace, you can try: strace -o /dev/stdout appname | grep string.

LD_DEBUG

Setting LD_DEBUG=files gives another overview of what files an application is looking for. For an application named appname:

$ LD_DEBUG=files appname > appname.log 2>&1

The output will end up in appname.log.

For more information, see ld-linux(8).

Readelf

If you get no such file or directory when running an application, try the following command:

$ readelf -a /usr/bin/appname | grep interp

(replace /usr/bin/appname with the location of your executable)

Make sure the interpreter in question (like /lib/ld-linux-x86-64.so.2) actually exists. Install ld-lsb if need be.

If it is not written in C or C++, but perhaps in Python

Use file on the executable to get more information:

$ file /usr/bin/appname

If it says ELF, it is a binary executable and is usually written in C or C++. If it says Python script, you know you are dealing with an application written in Python.

If it is a shell script, open up the shell script in a text editor and see (usually at the bottom of the file) if you can find the name of the real application (ELF file). You can then temporarily put "gdb" right in the shellscript, before the name of the executable, for debugging purposes. See the sections about gdb further up. Prefix the command with gdb --args if the executable in question needs arguments as well.

For pure shell scripts, you can also use bash -x script_name or bash -xv script_name.

For Python applications, the output will often say which file and line number the crash occurred at. If you are proficient with Python, you can try to fix this and include the fix in the bug report.

Report the bug

First check if the bug in question is a packaging bug. If the bug is introduced due to how Arch Linux packages this application, report it to https://gitlab.archlinux.org/groups/archlinux/packaging/-/issues. This also includes issues with libraries or dependencies (e.g if one of them is not built with a specific feature that is needed). Inspect the PKGBUILD of the package, which is possible with the Arch build system, to see how it gets packaged. See Bug reporting guidelines#Upstream or Arch? for more information.

If the bug is not related to Arch Linux and is reproducible anywhere else, only report it to upstream. Arch Linux can not magically fix upstream bugs. Reporting it to the Arch bugtracker would not help and might even be counterproductive because it tends to waste time of the bug wranglers.

See also