Difference between revisions of "General troubleshooting"

From ArchWiki
Jump to: navigation, search
(Session permissions: polkit)
(Spellcheck is marking all of my text as incorrect!: mv to aspell)
 
(80 intermediate revisions by 21 users not shown)
Line 1: Line 1:
 
[[Category:System administration]]
 
[[Category:System administration]]
 
[[Category:System recovery]]
 
[[Category:System recovery]]
{{stub}}
+
[[Category:Getting and installing Arch]]
 +
[[es:General troubleshooting]]
 +
[[ja:一般的なトラブルシューティング]]
 +
[[ru:General troubleshooting]]
 +
{{Related articles start}}
 +
{{Related|Reporting bug guidelines}}
 +
{{Related|Step-by-step debugging guide}}
 +
{{Related|Debug - Getting Traces}}
 +
{{Related|Boot debugging}}
 +
{{Related|IRC Collaborative Debugging}}
 +
{{Related articles end}}
  
 
This article explains some methods for general troubleshooting. For application specific issues, please reference the particular wiki page for that program.
 
This article explains some methods for general troubleshooting. For application specific issues, please reference the particular wiki page for that program.
  
== Attention To Detail ==
+
== General procedures ==
In order to resolve an issue that you're having with [[Main_Page|Arch Linux]], it is ''absolutely crucial'' to have a firm understanding of how that specific system functions. How it works, and what does it need to run without error? If you cannot comfortably answer these question then it is strongly advised that you review the [[Main_Page|Archwiki]] article for the application/service that you are having troubles with.Once you feel like you've understood the specific system, it will be easier for you to pin-point the problem. Saying, ''"Program X doesn't work"'' is unacceptable. Precision is key.
+
  
The following gives a number of questions for you to ask yourself whenever dealing with a malfunctioning system. Under each question there are notes explaining how you should be answering each question, followed by some light examples on how to easily gather data output and what tools can be used to review logs and the journal.
+
=== Attention to detail ===
  
== Questions / Checklist ==
+
In order to resolve an issue that you are having, it is ''absolutely crucial'' to have a firm basic understanding of how that specific subsystem functions. How does it work, and what does it need to run without error? If you cannot comfortably answer these question then you would best review the [[Table of contents|Archwiki]] article for the subsystem that you are having trouble with. Once you feel like you've understood it, it will be easier for you to pinpoint the cause of the problem.
;1. What <u>is</u> the issue(s)?:Be ''<u>as precise as possible</u>''. This will help you not get confused and/or side-tracked when looking up specific information.
+
;2. Are there error messages? (if any):Copy and paste <u>''full outputs''</u> that contain '''error messages''' related to your issue into a separate file, such as {{ic|$HOME/issue.log}}. For example, to forward the output of the following [[mkinitcpio]] command to {{ic|$HOME/issue.log}}:
+
$ mkinitcpio -p linux >> $HOME/issue.log''
+
;3. Can you reproduce the issue?:If so, give ''exact'' '''step-by-step''' instructions/commands needed to do so.
+
;4. When did you first encounter these issues and what was changed between then and when the system was operating without error?:If it occurred right after an update then, list '''<u>all packages that were updated</u>'''. Include ''version numbers'', also, paste the entire update from [[pacman]].log ({{ic|/var/log/pacman.log}}). Also take note of the statuses of ''any'' service(s) needed to support the malfunctioning application(s) using [[systemd]]'s systemctl tools. For example, to forward the output of the following [[Systemd#Basic_systemctl_usage|systemd]] command to {{ic|$HOME/issue.log}}:
+
$ systemctl status dhcpcd@eth0.service >> $HOME/issue.log
+
{{Note|Using {{ic|'''>>'''}} will ensure any previous text in {{ic|$HOME/issue.log}} will not be overwritten.}}
+
  
== Remember ==
+
=== Questions/checklist ===
; When attempting to resolve an issue, '''never''' approach it as:Application '''X''' does not work.
+
; Instead, look at it in its entirety:Application '''X''' produces '''Y''' error(s) when performing '''Z''' tasks under conditions '''A''' and '''B''
+
  
== Additional Support ==
+
The following gives a number of questions for you whenever dealing with a malfunctioning system. Under each question there are notes explaining how you should be answering each question, followed by some light examples on how to easily gather data output and what tools can be used to review logs and the journal.
With all the information in front of you. You should have a good idea as to what is going on with the system.
+
And you can now start working on a proper fix.
+
  
If you require any additional support, it can be found at irc.freenode.net #archlinux
+
# What is the issue(s)?
 +
#: Be ''as precise as possible''. This will help you not get confused and/or side-tracked when looking up specific information.
 +
# Are there error messages? (if any)
 +
#: Copy and paste ''full outputs'' that contain '''error messages''' related to your issue into a separate file, such as {{ic|$HOME/issue.log}}. For example, to forward the output of the following [[mkinitcpio]] command to {{ic|$HOME/issue.log}}:
 +
#: {{bc|$ mkinitcpio -p linux >> $HOME/issue.log''}}
 +
# Can you reproduce the issue?
 +
#: If so, give ''exact'' '''step-by-step''' instructions/commands needed to do so.
 +
# When did you first encounter these issues and what was changed between then and when the system was operating without error?
 +
#:If it occurred right after an update then, list '''all packages that were updated'''. Include ''version numbers'', also, paste the entire update from [[pacman]].log ({{ic|/var/log/pacman.log}}). Also take note of the statuses of ''any'' service(s) needed to support the malfunctioning application(s) using [[systemd]]'s systemctl tools. For example, to forward the output of the following [[Systemd#Basic_systemctl_usage|systemd]] command to {{ic|$HOME/issue.log}}:
 +
#: {{bc|$ systemctl status dhcpcd@eth0.service >> $HOME/issue.log}}
 +
#: {{Note|Using {{ic|'''>>'''}} will ensure any previous text in {{ic|$HOME/issue.log}} will not be overwritten.}}
  
==Session permissions==
+
=== Be more specific ===
{{Note|You must be using [[systemd]] as your init system for local sessions to work - which is required for polkit permissions and ACLs for various devices (see {{ic|/usr/lib/udev/rules.d/70-uaccess.rules}})}}
+
 
 +
When attempting to resolve an issue, '''never''' approach it as:
 +
 
 +
''Application X does not work.''
 +
 
 +
Instead, look at it in its entirety:
 +
 
 +
''Application X produces Y error(s) when performing Z tasks under conditions A and B.
 +
 
 +
=== Additional support ===
 +
 
 +
With all the information in front of you you should have a good idea as to what is going on with the system and you can now start working on a proper fix.
 +
 
 +
If you require any additional support, it can be found on [https://bbs.archlinux.org the forums] or IRC at irc.freenode.net #archlinux See [[IRC channels]] for other options.
 +
 
 +
When asking for support post the '''complete''' output/logs, not just what you think are the significant sections. Sources of information include:
 +
 
 +
* Full output of any command involved - don't just select what you think is relevant.
 +
* Output from systemd's {{ic|journalctl}}. For more extensive output, use the {{ic|1=systemd.log_level=debug}} boot parameter.
 +
* Log files (have a look in {{ic|/var/log}})
 +
* Relevant configuration files
 +
* Drivers involved
 +
* Versions of packages involved
 +
* Kernel: {{ic|dmesg}}. For a boot problem, at least the last 10 lines displayed, preferably more
 +
* Networking: Exact output of commands involved, and any configuration files
 +
* Xorg: {{ic|/var/log/Xorg.0.log}}, and prior logs if you have overwritten the problematic one
 +
* Pacman: If a recent upgrade broke something, look in {{ic|/var/log/pacman.log}}
 +
 
 +
One of the better ways to post this information is to use an online pastebin. You can [[install]] the {{pkg|pbpst}} or {{pkg|gist}} package to automatically upload information. For example, to upload the content of your systemd journal from this boot you would do:
 +
 
 +
# journalctl -xb | pbpst -S
 +
 
 +
A link will then be output that you can paste to the forum or IRC.
 +
 
 +
Additionally, before posting your question, you may wish to review [http://www.catb.org/esr/faqs/smart-questions.html how to ask smart questions]. See also [[Code of conduct]].
 +
 
 +
== Boot problems ==
 +
 
 +
See [[Boot debugging]] to retrieve additional information.
 +
 
 +
=== Blank screen with Intel video ===
 +
 
 +
This is most likely due to a problem with [[kernel mode setting]]. Try [[Kernel mode setting#Disabling modesetting|disabling modesetting]] or changing the [[Intel#KMS Issue: console is limited to small area|video port]].
 +
 
 +
=== Stuck while loading the kernel ===
 +
 
 +
Try disabling ACPI by adding the {{ic|1=acpi=off}} kernel parameter.
 +
 
 +
=== Unbootable system ===
 +
 
 +
If your system will not boot at all, simply boot from a [https://www.archlinux.org/download/ live image] and [[change root]] to log into the system and fix the issue.
 +
 
 +
=== Debugging kernel modules ===
 +
 
 +
See [[Kernel modules#Obtaining information]].
 +
 
 +
=== Debugging hardware ===
 +
 
 +
See [[udev#Debug output]].
 +
 
 +
== Package management ==
 +
 
 +
See [[Pacman#Troubleshooting]] for general topics, and [[pacman/Package signing#Troubleshooting]] for issues with PGP keys.
 +
 
 +
== fuser ==
 +
 
 +
{{Expansion|Write an example how to use it.}}
 +
 
 +
''fuser'' is a command-line utility for identifying processes using resources such as files, filesystems and TCP/UDP ports.
 +
 +
''fuser'' is provided by the {{Pkg|psmisc}} package, which should be already installed as part of the {{Grp|base}} group.
 +
 
 +
== Session permissions ==
 +
 
 +
{{Note|You must be using [[systemd]] as your init system for local sessions to work.[https://www.archlinux.org/news/d-bus-now-launches-user-buses/] It is required for polkit permissions and ACLs for various devices (see {{ic|/usr/lib/udev/rules.d/70-uaccess.rules}} and  [http://enotty.pipebreaker.pl/2012/05/23/linux-automatic-user-acl-management/])}}
  
 
First, make sure you have a valid local session within X:
 
First, make sure you have a valid local session within X:
Line 36: Line 118:
 
  $ loginctl show-session $XDG_SESSION_ID
 
  $ loginctl show-session $XDG_SESSION_ID
  
This should contain {{ic|1=Remote=no}} and {{ic|1=Active=yes}} in the output. See [[xinitrc#Preserving the session]] for troubleshooting if it does not.
+
This should contain {{ic|1=Remote=no}} and {{ic|1=Active=yes}} in the output. If it does not, make sure that X runs on the same tty where the login occurred. This is required in order to preserve the logind session.
  
A dbus session should also be started along with X, in a way that exports a single {{ic|DBUS_SESSION_BUS_ADDRESS}} for every application in your session. If you use a desktop environment this will be handled for you, otherwise you can copy the code from {{ic|/etc/skel/.xinitrc}} that runs files in {{ic|/etc/X11/xinit/xinitrc.d}} to your {{ic|~/.xinitrc}}, and avoid using {{ic|dbus-launch}} or {{ic|ck-launch-session}}.
+
A D-Bus session should also be started along with X. See [[D-Bus#Starting the user session]] for more information on this.
  
Some polkit actions require further authentication, even with a local session. A polkit authentication agent needs to be running for this to work. There are two alternatives in the repositories:
+
Basic [[polkit]] actions do not require further set-up. Some polkit actions require further authentication, even with a local session. A polkit authentication agent needs to be running for this to work. See [[polkit#Authentication agents]] for more information on this.
  
* {{pkg|polkit-gnome}}, which provides {{ic|/usr/lib/polkit-gnome/polkit-gnome-authentication-agent-1}}
+
== error while loading shared libraries ==
* {{pkg|polkit-kde}}, which provides {{ic|/usr/lib/kde4/libexec/polkit-kde-authentication-agent-1}}
+
  
==Single user mode==
+
{{Accuracy|Or the program needs to be rebuilt after a [[System_maintenance#Partial_upgrades_are_unsupported|soname bump]].}}
If you cannot boot due to errors caused by a daemon, display manager or Xorg, you may be able use the single user [[Runlevels|runlevel]]:
+
 
{{poor writing}}
+
If, while using a program, you get an error similar to:
#Boot to single-user mode by appending {{ic|1}} or {{ic|s}} to the kernel line in GRUB.
+
 
#Then disable the [[systemd]] service that is causing the problem.  
+
error while loading shared libraries: libusb-0.1.so.4: cannot open shared object file: No such file or directory
#Change to the multi-user mode systemd [[Systemd#Targets|target]].
+
 
#Then try to track down the issue by running the service manually.
+
Use [[pacman]] or [[pkgfile]] to search for the package that owns the missing library:
 +
 
 +
{{hc|$ pacman -Fs libusb-0.1.so.4|
 +
extra/libusb-compat 0.1.5-1
 +
    usr/lib/libusb-0.1.so.4
 +
}}
 +
 
 +
In this case, the {{Pkg|libusb-compat}} package needs to be [[installed]].
 +
 
 +
The error could also mean that the package that you used to install your program does not list the library as a dependency in its [[PKGBUILD]]: if it is an official package, [[report a bug]]; if it is an [[AUR]] package, report it to the maintainer using its page in the AUR website.
 +
 
 +
== file: could not find any magic files! ==
 +
 
 +
{{Style|See [[Help:Style]] and related articles.}}
  
==file: could not find any magic files!==
 
{{Poor writing}}
 
 
''Example:'' After an every-day routine update or following the installation of a package you are given the following error:
 
''Example:'' After an every-day routine update or following the installation of a package you are given the following error:
 
  # file: could not find any magic files!
 
  # file: could not find any magic files!
This will most likely leave your system crippled. And, any attempts made to recompile/reinstall the package(s) responsible for the breakage will fail. Also, any attempts made to try to rebuild the [[mkinitcpio|initramfs]] will result in the following:
+
This will most likely leave your system crippled. And, any attempts made to recompile/reinstall the package(s) responsible for the breakage will fail. Also, any attempts made to try to rebuild the [[initramfs]] will result in the following:
 
  # mkinitcpio -p linux
 
  # mkinitcpio -p linux
 
  ==> Building image from preset: 'default'
 
  ==> Building image from preset: 'default'
Line 68: Line 160:
 
  @==> ERROR: invalid kernel specifier: `/boot/vmlinuz-linux'
 
  @==> ERROR: invalid kernel specifier: `/boot/vmlinuz-linux'
  
===Solution===
 
 
Typically a previously installed application had placed a configuration file within {{ic|/etc/ld.so.conf.d/}} or it had made changes to {{ic|/etc/ld.so.conf}} which are now invalid.
 
Typically a previously installed application had placed a configuration file within {{ic|/etc/ld.so.conf.d/}} or it had made changes to {{ic|/etc/ld.so.conf}} which are now invalid.
 
#Boot into the Arch Linux Live CD / Installation Media.
 
#Boot into the Arch Linux Live CD / Installation Media.
#Mount your root ({{ic|'''/'''}}) partition to {{ic|/mnt}} and using [[Change_Root#Change_root|arch-chroot]], [[Change_Root|chroot]] into your system.
+
#Mount your root ({{ic|'''/'''}}) partition to {{ic|/mnt}} and using [[Change root#Change_root|arch-chroot]], [[chroot]] into your system.
{{Note|[[Change_Root#Change_root|arch-chroot]] leaves mounting the {{ic|/boot}} partition up to the user.}}
+
{{Note|[[Change root#Change_root|arch-chroot]] leaves mounting the {{ic|/boot}} partition up to the user.}}
 
#Examine {{ic|/etc/ld.so.conf}} and remove any invalid lines found.
 
#Examine {{ic|/etc/ld.so.conf}} and remove any invalid lines found.
 
#Examine the files located inside the directory {{ic|/etc/ld.so.conf.d/}} and remove all invalid files.
 
#Examine the files located inside the directory {{ic|/etc/ld.so.conf.d/}} and remove all invalid files.
Line 81: Line 172:
 
  # pacman -S <package>
 
  # pacman -S <package>
  
==See also==
+
== See also ==
*[[IRC Collaborative Debugging]]
+
 
*[[Step By Step Debugging Guide]]
+
* [http://www.tuxradar.com/content/how-fix-most-common-linux-problems Fix the Most Common Problems]
*[http://www.maximumpc.com/article/features/linux_troubleshooting_guide_fix_most_common_problems Fix the Most Common Problems]
+
* [https://www.reddit.com/r/archlinux/comments/tjjwr/archlinux_a_howto_in_troubleshooting_for_newcomers/ A how-to in troubleshooting for newcomers]

Latest revision as of 02:18, 24 July 2016

This article explains some methods for general troubleshooting. For application specific issues, please reference the particular wiki page for that program.

General procedures

Attention to detail

In order to resolve an issue that you are having, it is absolutely crucial to have a firm basic understanding of how that specific subsystem functions. How does it work, and what does it need to run without error? If you cannot comfortably answer these question then you would best review the Archwiki article for the subsystem that you are having trouble with. Once you feel like you've understood it, it will be easier for you to pinpoint the cause of the problem.

Questions/checklist

The following gives a number of questions for you whenever dealing with a malfunctioning system. Under each question there are notes explaining how you should be answering each question, followed by some light examples on how to easily gather data output and what tools can be used to review logs and the journal.

  1. What is the issue(s)?
    Be as precise as possible. This will help you not get confused and/or side-tracked when looking up specific information.
  2. Are there error messages? (if any)
    Copy and paste full outputs that contain error messages related to your issue into a separate file, such as $HOME/issue.log. For example, to forward the output of the following mkinitcpio command to $HOME/issue.log:
    $ mkinitcpio -p linux >> $HOME/issue.log
  3. Can you reproduce the issue?
    If so, give exact step-by-step instructions/commands needed to do so.
  4. When did you first encounter these issues and what was changed between then and when the system was operating without error?
    If it occurred right after an update then, list all packages that were updated. Include version numbers, also, paste the entire update from pacman.log (/var/log/pacman.log). Also take note of the statuses of any service(s) needed to support the malfunctioning application(s) using systemd's systemctl tools. For example, to forward the output of the following systemd command to $HOME/issue.log:
    $ systemctl status dhcpcd@eth0.service >> $HOME/issue.log
    Note: Using >> will ensure any previous text in $HOME/issue.log will not be overwritten.

Be more specific

When attempting to resolve an issue, never approach it as:

Application X does not work.

Instead, look at it in its entirety:

Application X produces Y error(s) when performing Z tasks under conditions A and B.

Additional support

With all the information in front of you you should have a good idea as to what is going on with the system and you can now start working on a proper fix.

If you require any additional support, it can be found on the forums or IRC at irc.freenode.net #archlinux See IRC channels for other options.

When asking for support post the complete output/logs, not just what you think are the significant sections. Sources of information include:

  • Full output of any command involved - don't just select what you think is relevant.
  • Output from systemd's journalctl. For more extensive output, use the systemd.log_level=debug boot parameter.
  • Log files (have a look in /var/log)
  • Relevant configuration files
  • Drivers involved
  • Versions of packages involved
  • Kernel: dmesg. For a boot problem, at least the last 10 lines displayed, preferably more
  • Networking: Exact output of commands involved, and any configuration files
  • Xorg: /var/log/Xorg.0.log, and prior logs if you have overwritten the problematic one
  • Pacman: If a recent upgrade broke something, look in /var/log/pacman.log

One of the better ways to post this information is to use an online pastebin. You can install the pbpst or gist package to automatically upload information. For example, to upload the content of your systemd journal from this boot you would do:

# journalctl -xb | pbpst -S

A link will then be output that you can paste to the forum or IRC.

Additionally, before posting your question, you may wish to review how to ask smart questions. See also Code of conduct.

Boot problems

See Boot debugging to retrieve additional information.

Blank screen with Intel video

This is most likely due to a problem with kernel mode setting. Try disabling modesetting or changing the video port.

Stuck while loading the kernel

Try disabling ACPI by adding the acpi=off kernel parameter.

Unbootable system

If your system will not boot at all, simply boot from a live image and change root to log into the system and fix the issue.

Debugging kernel modules

See Kernel modules#Obtaining information.

Debugging hardware

See udev#Debug output.

Package management

See Pacman#Troubleshooting for general topics, and pacman/Package signing#Troubleshooting for issues with PGP keys.

fuser

Tango-view-fullscreen.pngThis article or section needs expansion.Tango-view-fullscreen.png

Reason: Write an example how to use it. (Discuss in Talk:General troubleshooting#)

fuser is a command-line utility for identifying processes using resources such as files, filesystems and TCP/UDP ports.

fuser is provided by the psmisc package, which should be already installed as part of the base group.

Session permissions

Note: You must be using systemd as your init system for local sessions to work.[1] It is required for polkit permissions and ACLs for various devices (see /usr/lib/udev/rules.d/70-uaccess.rules and [2])

First, make sure you have a valid local session within X:

$ loginctl show-session $XDG_SESSION_ID

This should contain Remote=no and Active=yes in the output. If it does not, make sure that X runs on the same tty where the login occurred. This is required in order to preserve the logind session.

A D-Bus session should also be started along with X. See D-Bus#Starting the user session for more information on this.

Basic polkit actions do not require further set-up. Some polkit actions require further authentication, even with a local session. A polkit authentication agent needs to be running for this to work. See polkit#Authentication agents for more information on this.

error while loading shared libraries

Tango-inaccurate.pngThe factual accuracy of this article or section is disputed.Tango-inaccurate.png

Reason: Or the program needs to be rebuilt after a soname bump. (Discuss in Talk:General troubleshooting#)

If, while using a program, you get an error similar to:

error while loading shared libraries: libusb-0.1.so.4: cannot open shared object file: No such file or directory

Use pacman or pkgfile to search for the package that owns the missing library:

$ pacman -Fs libusb-0.1.so.4
extra/libusb-compat 0.1.5-1
    usr/lib/libusb-0.1.so.4

In this case, the libusb-compat package needs to be installed.

The error could also mean that the package that you used to install your program does not list the library as a dependency in its PKGBUILD: if it is an official package, report a bug; if it is an AUR package, report it to the maintainer using its page in the AUR website.

file: could not find any magic files!

Tango-edit-clear.pngThis article or section needs language, wiki syntax or style improvements.Tango-edit-clear.png

Reason: See Help:Style and related articles. (Discuss in Talk:General troubleshooting#)

Example: After an every-day routine update or following the installation of a package you are given the following error:

# file: could not find any magic files!

This will most likely leave your system crippled. And, any attempts made to recompile/reinstall the package(s) responsible for the breakage will fail. Also, any attempts made to try to rebuild the initramfs will result in the following:

# mkinitcpio -p linux
==> Building image from preset: 'default'
 -> -k /boot/vmlinuz-linux -c /etc/mkinitcpio.conf -g /boot/initramfs-linux.img
file: could not find any magic files!
==> ERROR: invalid kernel specifier: `/boot/vmlinuz-linux'
==> Building image from preset: 'fallback'
 -> -k /boot/vmlinuz-linux -c /etc/mkinitcpio.conf -g /boot/initramfs-linux-fallback.img -S autodetect
file: could not find any magic files!
@==> ERROR: invalid kernel specifier: `/boot/vmlinuz-linux'

Typically a previously installed application had placed a configuration file within /etc/ld.so.conf.d/ or it had made changes to /etc/ld.so.conf which are now invalid.

  1. Boot into the Arch Linux Live CD / Installation Media.
  2. Mount your root (/) partition to /mnt and using arch-chroot, chroot into your system.
Note: arch-chroot leaves mounting the /boot partition up to the user.
  1. Examine /etc/ld.so.conf and remove any invalid lines found.
  2. Examine the files located inside the directory /etc/ld.so.conf.d/ and remove all invalid files.
  3. Rebuild the initramfs.
# mkinitcpio -p linux
  1. Reboot back to your installed system.
  2. Once booted, reinstall the package that was responsible for leaving your system inoperable using:
# pacman -S <package>

See also