User:NetSysFire/General troubleshooting

From ArchWiki

General troubleshooting needs an overhaul. However, it is difficult to work with it in the current state and may need a complete rewrite. Because of the many aspects of troubleshooting and debugging, I believe the best approach is to put it together piece by piece and eventually re-assemble the page with new and better information. This page serves as the intermediate collection of said pieces.

Hardware issues

Certain hardware issues can be verified in the BIOS. Some firmwares, especially those on Laptops may also provide a hardware diagnostic mode.

  • Broken keyboard keys - If even the BIOS (most have some sort of text entry, at least when setting a password) does not recognize a keypress, it is likely physically broken.
  • Battery - Laptop firmwares can usually display information about them, especially in their diagnostic mode. If it is still unrecognized or wrong, likely a physical issue.

Another way is to try and reproduce on the Arch Linux ISO, which works independently from any installed operating systems and can thus rule out miscellaneous breakage from e.g messing with configuration files, broken cache and similar.

Cold boots

If a problem that should be solved by a reboot, e.g a broken interaction between driver and hardware leaving it in a broken state, persists a reboot, try a cold boot. This forces the firmware to properly reinitialize it and can fix some issues. An example for this can be seen in HP 255 G10#Wireless, where a kernel module causes a broken state in the wireless card, likely due to a firmware bug, that will not be fixed by a regular reboot.

Generic application troubleshooting

Most issues are ordinary software issues and can thus be easily debugged.

  1. First, run said application in the terminal or otherwise inspect its output for any clues to what is happening.
  2. If there are no or not enough logs, try increasing the verbosity. Most applications have arguments such as --verbose, --debug or similar. Check their man page or --help.
    1. In extreme cases, such as the application freezing or crashing without supplying useful logs, e.g because it happens at a very early stage, try inspecting its behavior with strace. Only the very end of the plentiful output is usually relevant, so feel free to skip to the bottom first.
    2. Cases like the above may be caused by a corrupted cache or other damaged, usually binary, files that are read very early.
  3. Check the journal, especially when the application does not output enough logs. You may skip right to the last entries since these are usually the most relevant. Do not forget to run it with superuser privileges or you may only see the log entries for your particular user.

Also, try searching the application's issue trackers. It may save you from a lot of potential troubleshooting if the issue turns out to be a known issue, maybe even with an already existing workaround.

General network troubleshooting

There are multiple aspects to having a working internet connection.[1] First, determine if there is connectivity to the gateway:

$ ping _gateway
Note: _gateway is provided by nss-myhostname(8). Should it not resolve automatically for whatever reason, obtain the address via ip route show default.

If everything looks in order, try pinging a non-local host:

$ ping 1.1

If this succeeds, try checking for DNS problems:

$ ping archlinux.org
$ dig archlinux.org

Most routers will propagate themselves as a DNS server via DHCP, so if the resolved result looks odd, try a different nameserver:

$ dig archlinux.org @1.1
Tip: If everything fails, try restarting your router, which is usually a generic ISP-supplied one. These devices are powered by black magic and lots of duct tape and rebooting them can fix strange issues.[2]

Using strace

strace is a powerful debugging tool to display all syscalls the application makes.

It is generally better to let strace write to a file and then inspect it with your editor of choice, as there will be a lot of output. Additionally, some editors (at least Vim) have syntax highlighting, making the trace vastly easier to read.

$ strace -o trace.strace application

A common use of strace is to find out which paths the application tries to read and write to, making it ideal to e.g debug a mysterious crash caused by cache corruption.