https://wiki.archlinux.org/api.php?action=feedcontributions&user=Nicman23&feedformat=atomArchWiki - User contributions [en]2024-03-29T10:44:31ZUser contributionsMediaWiki 1.41.0https://wiki.archlinux.org/index.php?title=PipeWire/Examples&diff=651801PipeWire/Examples2021-02-10T13:09:04Z<p>Nicman23: added info for controlling headphones / front volume individually; added script that does that</p>
<hr />
<div>[[Category:Multimedia]]<br />
{{Related articles start}}<br />
{{Related|PipeWire}}<br />
{{Related|PulseAudio/Examples}}<br />
{{Related articles end}}<br />
==Surround sound systems==<br />
=== Splitting front/rear ===<br />
<br />
When [[PipeWire#Audio|using PipeWire as a PulseAudio/JACK replacement]], you can set up Pipewire to replicate the [[PulseAudio/Examples#Splitting_front/rear|Pulseaudio example for splitting front/rear]]. Doing this allows you to send audio streams using Pulseaudio to a separate sink for speakers or headphones. <br />
<br />
Connect speakers to the line-out port and headphones to the rear port. In {{ic|pavucontrol}} set the soundcard used to Analog Surround 4.0 Output. Then using the following commands, make new sinks for the speakers and for the headphones, link the speakers to the front channels and link the headphones to the rear channels:<br />
<br />
pactl load-module module-null-sink sink_name=speakers object.linger=1 media.class=Audio/Sink channel_map=FL,FR<br />
pactl load-module module-null-sink sink_name=headphones object.linger=1 media.class=Audio/Sink channel_map=RL,RR<br />
<br />
{{ic|1=object.linger=1}} keeps the sinks alive after the creating client disconnects. You can name {{ic|sink_name}} whatever you want.<br />
<br />
In order to unload module, you can use {{ic|pw-cli destroy ID}}, where {{ic|ID}} is output of {{ic|pactl load-module}} command. Unloading modules through {{ic|pactl unload-module}} is not currently supported.<br />
<br />
Using {{ic|jack_connect}}, connect the monitors of the new sinks to the sound card's playback ports. Find out the name of the channels by running {{ic|jack_lsp -c}}. <br />
<br />
pw-jack jack_connect speakers:monitor_0 HDA\ ATI\ SB:playback_FL<br />
pw-jack jack_connect speakers:monitor_1 HDA\ ATI\ SB:playback_FR<br />
pw-jack jack_connect headphones:monitor_0 HDA\ ATI\ SB:playback_RL<br />
pw-jack jack_connect headphones:monitor_1 HDA\ ATI\ SB:playback_RR<br />
<br />
{{tip|Add the above commands to a script and [[autostart]] it to automate the process. Be sure to replace {{ic|HDA\ ATI\ SB}} with the name of your sound card.}}<br />
<br />
To individually control the volumes, one option is to use alsa utilities (such as amixer) to control Front and Rear/Surround (alsa naming) channels. A script to automatically do that depending on what is your currently default pulseaudio sink can be found [https://gist.github.com/nicman23/a2a80200e1d5d1f65b75846c5f2387b7 here].</div>Nicman23https://wiki.archlinux.org/index.php?title=PCI_passthrough_via_OVMF&diff=650563PCI passthrough via OVMF2021-02-02T20:28:36Z<p>Nicman23: mention that virsh alloc might not work if ram is used for vm cache and add workaround.</p>
<hr />
<div>[[Category:Virtualization]]<br />
[[ja:OVMF による PCI パススルー]]<br />
[[zh-hans:PCI passthrough via OVMF]]<br />
{{Related articles start}}<br />
{{Related|Intel GVT-g}}<br />
{{Related articles end}}<br />
<br />
The Open Virtual Machine Firmware ([https://github.com/tianocore/tianocore.github.io/wiki/OVMF OVMF]) is a project to enable UEFI support for virtual machines. Starting with Linux 3.9 and recent versions of [[QEMU]], it is now possible to passthrough a graphics card, offering the VM native graphics performance which is useful for graphic-intensive tasks.<br />
<br />
Provided you have a desktop computer with a spare GPU you can dedicate to the host (be it an integrated GPU or an old OEM card, the brands do not even need to match) and that your hardware supports it (see [[#Prerequisites]]), it is possible to have a VM of any OS with its own dedicated GPU and near-native performance. For more information on techniques see the background [https://www.linux-kvm.org/images/b/b3/01x09b-VFIOandYou-small.pdf presentation (pdf)]. <br />
<br />
== Prerequisites ==<br />
<br />
A VGA Passthrough relies on a number of technologies that are not ubiquitous as of today and might not be available on your hardware. You will not be able to do this on your machine unless the following requirements are met :<br />
<br />
* Your CPU must support hardware virtualization (for kvm) and IOMMU (for the passthrough itself)<br />
** [https://ark.intel.com/Search/FeatureFilter?productType=873&0_VTD=True List of compatible Intel CPUs (Intel VT-x and Intel VT-d)]<br />
** All AMD CPUs from the Bulldozer generation and up (including Zen) should be compatible.<br />
*** CPUs from the K10 generation (2007) do not have an IOMMU, so you '''need''' to have a motherboard with a [https://support.amd.com/TechDocs/43403.pdf#page=18 890FX] or [https://support.amd.com/TechDocs/48691.pdf#page=21 990FX] chipset to make it work, as those have their own IOMMU.<br />
* Your motherboard must also support IOMMU<br />
** Both the chipset and the BIOS must support it. It is not always easy to tell at a glance whether or not this is the case, but there is a fairly comprehensive list on the matter on the [https://wiki.xen.org/wiki/VTd_HowTo Xen wiki] as well as [[Wikipedia:List of IOMMU-supporting hardware]].<br />
* Your guest GPU ROM must support UEFI.<br />
** If you can find [https://www.techpowerup.com/vgabios/ any ROM in this list] that applies to your specific GPU and is said to support UEFI, you are generally in the clear. All GPUs from 2012 and later should support this, as Microsoft made UEFI a requirement for devices to be marketed as compatible with Windows 8.<br />
<br />
You will probably want to have a spare monitor or one with multiple input ports connected to different GPUs (the passthrough GPU will not display anything if there is no screen plugged in and using a VNC or Spice connection will not help your performance), as well as a mouse and a keyboard you can pass to your VM. If anything goes wrong, you will at least have a way to control your host machine this way.<br />
<br />
== Setting up IOMMU ==<br />
<br />
{{Note|<br />
* IOMMU is a generic name for Intel VT-d and AMD-Vi.<br />
* VT-d stands for ''Intel Virtualization Technology for Directed I/O'' and should not be confused with VT-x ''Intel Virtualization Technology''. VT-x allows one hardware platform to function as multiple “virtual” platforms while VT-d improves security and reliability of the systems and also improves performance of I/O devices in virtualized environments.<br />
}}<br />
<br />
Using IOMMU opens to features like PCI passthrough and memory protection from faulty or malicious devices, see [[Wikipedia:Input-output memory management unit#Advantages]] and [https://www.quora.com/Memory-Management-computer-programming/Could-you-explain-IOMMU-in-plain-English Memory Management (computer programming): Could you explain IOMMU in plain English?].<br />
<br />
=== Enabling IOMMU ===<br />
<br />
Ensure that AMD-Vi/Intel VT-d is supported by the CPU and enabled in the BIOS settings. Both normally show up alongside other CPU features (meaning they could be in an overclocking-related menu) either with their actual names ("VT-d" or "AMD-Vi") or in more ambiguous terms such as "Virtualization technology", which may or may not be explained in the manual.<br />
<br />
Enable IOMMU support by setting the correct [[kernel parameter]] depending on the type of CPU in use:<br />
<br />
* For Intel CPUs (VT-d) set {{ic|1=intel_iommu=on}} <br />
* For AMD CPUs (AMD-Vi) set {{ic|1=amd_iommu=on}}<br />
<br />
You should also append the {{ic|1=iommu=pt}} parameter. This will prevent Linux from touching devices which cannot be passed through.<br />
<br />
After rebooting, check dmesg to confirm that IOMMU has been correctly enabled:<br />
<br />
{{hc|dmesg {{!}} grep -i -e DMAR -e IOMMU|<br />
[ 0.000000] ACPI: DMAR 0x00000000BDCB1CB0 0000B8 (v01 INTEL BDW 00000001 INTL 00000001)<br />
[ 0.000000] Intel-IOMMU: enabled<br />
[ 0.028879] dmar: IOMMU 0: reg_base_addr fed90000 ver 1:0 cap c0000020660462 ecap f0101a<br />
[ 0.028883] dmar: IOMMU 1: reg_base_addr fed91000 ver 1:0 cap d2008c20660462 ecap f010da<br />
[ 0.028950] IOAPIC id 8 under DRHD base 0xfed91000 IOMMU 1<br />
[ 0.536212] DMAR: No ATSR found<br />
[ 0.536229] IOMMU 0 0xfed90000: using Queued invalidation<br />
[ 0.536230] IOMMU 1 0xfed91000: using Queued invalidation<br />
[ 0.536231] IOMMU: Setting RMRR:<br />
[ 0.536241] IOMMU: Setting identity map for device 0000:00:02.0 [0xbf000000 - 0xcf1fffff]<br />
[ 0.537490] IOMMU: Setting identity map for device 0000:00:14.0 [0xbdea8000 - 0xbdeb6fff]<br />
[ 0.537512] IOMMU: Setting identity map for device 0000:00:1a.0 [0xbdea8000 - 0xbdeb6fff]<br />
[ 0.537530] IOMMU: Setting identity map for device 0000:00:1d.0 [0xbdea8000 - 0xbdeb6fff]<br />
[ 0.537543] IOMMU: Prepare 0-16MiB unity mapping for LPC<br />
[ 0.537549] IOMMU: Setting identity map for device 0000:00:1f.0 [0x0 - 0xffffff]<br />
[ 2.182790] [drm] DMAR active, disabling use of stolen memory<br />
}}<br />
<br />
=== Ensuring that the groups are valid ===<br />
<br />
The following script should allow you to see how your various PCI devices are mapped to IOMMU groups. If it does not return anything, you either have not enabled IOMMU support properly or your hardware does not support it.<br />
<br />
#!/bin/bash<br />
shopt -s nullglob<br />
for g in `find /sys/kernel/iommu_groups/* -maxdepth 0 -type d | sort -V`; do<br />
echo "IOMMU Group ${g##*/}:"<br />
for d in $g/devices/*; do<br />
echo -e "\t$(lspci -nns ${d##*/})"<br />
done;<br />
done;<br />
<br />
Example output:<br />
<br />
IOMMU Group 1:<br />
00:01.0 PCI bridge: Intel Corporation Xeon E3-1200 v2/3rd Gen Core processor PCI Express Root Port [8086:0151] (rev 09)<br />
IOMMU Group 2:<br />
00:14.0 USB controller: Intel Corporation 7 Series/C210 Series Chipset Family USB xHCI Host Controller [8086:0e31] (rev 04)<br />
IOMMU Group 4:<br />
00:1a.0 USB controller: Intel Corporation 7 Series/C210 Series Chipset Family USB Enhanced Host Controller #2 [8086:0e2d] (rev 04)<br />
IOMMU Group 10:<br />
00:1d.0 USB controller: Intel Corporation 7 Series/C210 Series Chipset Family USB Enhanced Host Controller #1 [8086:0e26] (rev 04)<br />
IOMMU Group 13:<br />
06:00.0 VGA compatible controller: NVIDIA Corporation GM204 [GeForce GTX 970] [10de:13c2] (rev a1)<br />
06:00.1 Audio device: NVIDIA Corporation GM204 High Definition Audio Controller [10de:0fbb] (rev a1)<br />
<br />
An IOMMU group is the smallest set of physical devices that can be passed to a virtual machine. For instance, in the example above, both the GPU in 06:00.0 and its audio controller in 6:00.1 belong to IOMMU group 13 and can only be passed together. The frontal USB controller, however, has its own group (group 2) which is separate from both the USB expansion controller (group 10) and the rear USB controller (group 4), meaning that [[#USB controller|any of them could be passed to a VM without affecting the others]].<br />
<br />
=== Gotchas ===<br />
<br />
==== Plugging your guest GPU in an unisolated CPU-based PCIe slot ====<br />
<br />
Not all PCI-E slots are the same. Most motherboards have PCIe slots provided by both the CPU and the PCH. Depending on your CPU, it is possible that your processor-based PCIe slot does not support isolation properly, in which case the PCI slot itself will appear to be grouped with the device that is connected to it.<br />
<br />
IOMMU Group 1:<br />
00:01.0 PCI bridge: Intel Corporation Xeon E3-1200 v2/3rd Gen Core processor PCI Express Root Port (rev 09)<br />
01:00.0 VGA compatible controller: NVIDIA Corporation GM107 [GeForce GTX 750] (rev a2)<br />
01:00.1 Audio device: NVIDIA Corporation Device 0fbc (rev a1)<br />
<br />
This is fine so long as only your guest GPU is included in here, such as above. Depending on what is plugged in to your other PCIe slots and whether they are allocated to your CPU or your PCH, you may find yourself with additional devices within the same group, which would force you to pass those as well. If you are ok with passing everything that is in there to your VM, you are free to continue. Otherwise, you will either need to try and plug your GPU in your other PCIe slots (if you have any) and see if those provide isolation from the rest or to install the ACS override patch, which comes with its own drawbacks. See [[#Bypassing the IOMMU groups (ACS override patch)]] for more information.<br />
<br />
{{Note|If they are grouped with other devices in this manner, pci root ports and bridges should neither be bound to vfio at boot, nor be added to the VM.}}<br />
<br />
== Isolating the GPU ==<br />
<br />
In order to assign a device to a virtual machine, this device and all those sharing the same IOMMU group must have their driver replaced by a stub driver or a VFIO driver in order to prevent the host machine from interacting with them. In the case of most devices, this can be done on the fly right before the VM starts.<br />
<br />
However, due to their size and complexity, GPU drivers do not tend to support dynamic rebinding very well, so you cannot just have some GPU you use on the host be transparently passed to a VM without having both drivers conflict with each other. Because of this, it is generally advised to bind those placeholder drivers manually before starting the VM, in order to stop other drivers from attempting to claim it.<br />
<br />
The following section details how to configure a GPU so those placeholder drivers are bound early during the boot process, which makes said device inactive until a VM claims it or the driver is switched back. This is the preferred method, considering it has less caveats than switching drivers once the system is fully online.<br />
<br />
{{Warning|Once you reboot after this procedure, whatever GPU you have configured will no longer be usable on the host until you reverse the manipulation. Make sure the GPU you intend to use on the host is properly configured before doing this - your motherboard should be set to display using the host GPU.}}<br />
<br />
Starting with Linux 4.1, the kernel includes vfio-pci. This is a VFIO driver, meaning it fulfills the same role as pci-stub did, but it can also control devices to an extent, such as by switching them into their D3 state when they are not in use.<br />
<br />
=== Binding vfio-pci via device ID ===<br />
<br />
Vfio-pci normally targets PCI devices by ID, meaning you only need to specify the IDs of the devices you intend to passthrough. For the following IOMMU group, you would want to bind vfio-pci with {{ic|10de:13c2}} and {{ic|10de:0fbb}}, which will be used as example values for the rest of this section.<br />
<br />
IOMMU Group 13:<br />
06:00.0 VGA compatible controller: NVIDIA Corporation GM204 [GeForce GTX 970] [10de:13c2] (rev a1)<br />
06:00.1 Audio device: NVIDIA Corporation GM204 High Definition Audio Controller [10de:0fbb] (rev a1)}}<br />
<br />
{{Note|<br />
* You cannot specify which device to isolate using vendor-device ID pairs if the host GPU and the guest GPU share the same pair (i.e : if both are the same model). If this is your case, read [[#Using identical guest and host GPUs]] instead.<br />
* If, as noted in [[#Plugging your guest GPU in an unisolated CPU-based PCIe slot]], your pci root port is part of your IOMMU group, you '''must not''' pass its ID to {{ic|vfio-pci}}, as it needs to remain attached to the host to function properly. Any other device within that group, however, should be left for {{ic|vfio-pci}} to bind with.<br />
* Binding the audio device ({{ic|10de:0fbb}} in above's example) is optional. Libvirt is able to unbind it from the {{ic|snd_hda_intel}} driver on its own.<br />
}}<br />
<br />
Two methods exist for providing the device IDs. Specifying them via [[kernel parameters]] has the advantage of being able to easily edit, remove, or undo any breaking changes via your boot loader:<br />
<br />
vfio-pci.ids=10de:13c2,10de:0fbb<br />
<br />
Alternatively, the IDs may be added to a modprobe conf file. Since these conf files are embedded in the initramfs image, any changes require regenerating a new image each time:<br />
<br />
{{hc|/etc/modprobe.d/vfio.conf|2=<br />
options vfio-pci ids=10de:13c2,10de:0fbb<br />
}}<br />
<br />
=== Loading vfio-pci early ===<br />
<br />
Since Arch's {{Pkg|linux}} has vfio-pci built as a module, we need to force it to load early before the graphics drivers have a chance to bind to the card. To ensure that, add {{ic|vfio_pci}}, {{ic|vfio}}, {{ic|vfio_iommu_type1}}, and {{ic|vfio_virqfd}} to [[mkinitcpio]]:<br />
<br />
{{hc|/etc/mkinitcpio.conf|2=<br />
MODULES=(... vfio_pci vfio vfio_iommu_type1 vfio_virqfd ...)<br />
}}<br />
<br />
{{Note|If you also have another driver loaded this way for [[Kernel mode setting#Early KMS start|early modesetting]] (such as {{ic|nouveau}}, {{ic|radeon}}, {{ic|amdgpu}}, {{ic|i915}}, etc.), all of the aforementioned VFIO modules must precede it.}}<br />
<br />
Also, ensure that the modconf hook is included in the HOOKS list of {{ic|mkinitcpio.conf}}:<br />
<br />
{{hc|/etc/mkinitcpio.conf|2=<br />
HOOKS=(... modconf ...)<br />
}}<br />
<br />
Since new modules have been added to the initramfs configuration, you must [[regenerate the initramfs]].<br />
<br />
=== Verifying that the configuration worked ===<br />
<br />
Reboot and verify that vfio-pci has loaded properly and that it is now bound to the right devices.<br />
<br />
{{hc|$ dmesg {{!}} grep -i vfio|<br />
[ 0.329224] VFIO - User Level meta-driver version: 0.3<br />
[ 0.341372] vfio_pci: add [10de:13c2[ffff:ffff]] class 0x000000/00000000<br />
[ 0.354704] vfio_pci: add [10de:0fbb[ffff:ffff]] class 0x000000/00000000<br />
[ 2.061326] vfio-pci 0000:06:00.0: enabling device (0100 -> 0103)<br />
}}<br />
<br />
It is not necessary for all devices (or even expected device) from {{ic|vfio.conf}} to be in dmesg output. Sometimes a device does not appear in output at boot but actually is able to be visible and operatable in guest VM.<br />
<br />
{{hc|$ lspci -nnk -d 10de:13c2|<br />
06:00.0 VGA compatible controller: NVIDIA Corporation GM204 [GeForce GTX 970] [10de:13c2] (rev a1)<br />
Kernel driver in use: vfio-pci<br />
Kernel modules: nouveau nvidia<br />
}}<br />
<br />
{{hc|$ lspci -nnk -d 10de:0fbb|<br />
06:00.1 Audio device: NVIDIA Corporation GM204 High Definition Audio Controller [10de:0fbb] (rev a1)<br />
Kernel driver in use: vfio-pci<br />
Kernel modules: snd_hda_intel<br />
}}<br />
<br />
== Setting up an OVMF-based guest VM ==<br />
<br />
OVMF is an open-source UEFI firmware for QEMU virtual machines. While it is possible to use SeaBIOS to get similar results to an actual PCI passthrough, the setup process is different and it is generally preferable to use the EFI method if your hardware supports it.<br />
<br />
=== Configuring libvirt ===<br />
<br />
[[Libvirt]] is a wrapper for a number of virtualization utilities that greatly simplifies the configuration and deployment process of virtual machines. In the case of KVM and QEMU, the frontend it provides allows us to avoid dealing with the permissions for QEMU and make it easier to add and remove various devices on a live VM. Its status as a wrapper, however, means that it might not always support all of the latest qemu features, which could end up requiring the use of a wrapper script to provide some extra arguments to QEMU.<br />
<br />
Install {{Pkg|qemu}}, {{Pkg|libvirt}}, {{Pkg|edk2-ovmf}}, and {{Pkg|virt-manager}}.<br />
<br />
You can now [[enable]] and [[start]] {{ic|libvirtd.service}} and its logging component {{ic|virtlogd.socket}}.<br />
<br />
You may also need to [https://wiki.libvirt.org/page/Networking#NAT_forwarding_.28aka_.22virtual_networks.22.29 activate the default libvirt network].<br />
<br />
{{Note|The default libvirt network will only be listed if the virsh command is run as root.}}<br />
<br />
=== Setting up the guest OS ===<br />
<br />
The process of setting up a VM using {{ic|virt-manager}} is mostly self-explanatory, as most of the process comes with fairly comprehensive on-screen instructions.<br />
<br />
However, you should pay special attention to the following steps:<br />
<br />
* When the VM creation wizard asks you to name your VM (final step before clicking "Finish"), check the "Customize before install" checkbox.<br />
* In the "Overview" section, [https://i.imgur.com/73r2ctM.png set your firmware to "UEFI"]. If the option is grayed out, make sure that:<br />
** Your hypervisor is running as a system session and not a user session. This can be verified [https://i.ibb.co/N1XZCdp/Deepin-Screenshot-select-area-20190125113216.png by clicking, then hovering] over the session in virt-manager. If you are accidentally running it as a user session, you must open a new connection by clicking "File" > "Add Connection..", then select the option from the drop-down menu station "QEMU/KVM" and not "QEMU/KVM user session".<br />
* In the "CPUs" section, change your CPU model to "host-passthrough". If it is not in the list, you will have to either type it by hand or by using {{ic|virt-xml ''vmname'' --edit --cpu host-passthrough}}. This will ensure that your CPU is detected properly, since it causes libvirt to expose your CPU capabilities exactly as they are instead of only those it recognizes (which is the preferred default behavior to make CPU behavior easier to reproduce). Without it, some applications may complain about your CPU being of an unknown model.<br />
* If you want to minimize IO overhead, it's easier to setup [[#Virtio disk]] before installing<br />
<br />
The rest of the installation process will take place as normal using a standard QXL video adapter running in a window. At this point, there is no need to install additional drivers for the rest of the virtual devices, since most of them will be removed later on. Once the guest OS is done installing, simply turn off the virtual machine. It is possible you will be dropped into the UEFI menu instead of starting the installation upon powering your VM for the first time. Sometimes the correct ISO file was not automatically detected and you will need to manually specify the drive to boot. By typing exit and navigating to "boot manager" you will enter a menu that allows you to choose between devices.<br />
<br />
=== Attaching the PCI devices ===<br />
<br />
With the installation done, it is now possible to edit the hardware details in libvirt and remove virtual integration devices, such as the spice channel and virtual display, the QXL video adapter, the emulated mouse and keyboard and the USB tablet device. Since that leaves you with no input devices, you may want to bind a few USB host devices to your VM as well, but remember to '''leave at least one mouse and/or keyboard assigned to your host''' in case something goes wrong with the guest. At this point, it also becomes possible to attach the PCI device that was isolated earlier; simply click on "Add Hardware" and select the PCI Host Devices you want to passthrough. If everything went well, the screen plugged into your GPU should show the OVMF splash screen and your VM should start up normally. From there, you can setup the drivers for the rest of your VM.<br />
<br />
=== Video card driver virtualisation detection ===<br />
<br />
Video card drivers by both Nvidia and AMD incorporate very basic virtual machine detection. Should this detection mechanism trigger, current drivers by both manufacturers will refuse to run (resulting in a generic exit code 43 in case of Nvidia and a black screen in case of AMD).<br />
<br />
It is therefore required to modify the reported Hyper-V vendor ID:<br />
<br />
{{hc|$ virsh edit ''vmname''|2=<br />
...<br />
<features><br />
...<br />
<hyperv><br />
...<br />
<vendor_id state='on' value='randomid'/><br />
...<br />
</hyperv><br />
...<br />
</features><br />
...<br />
}}<br />
<br />
In addition to above's modification, Nvidia guest drivers also require hiding the KVM CPU leaf:<br />
<br />
{{hc|$ virsh edit ''vmname''|2=<br />
...<br />
<features><br />
...<br />
<kvm><br />
<hidden state='on'/><br />
</kvm><br />
...<br />
</features><br />
...<br />
}}<br />
<br />
As the KVM CPU leaf remains unused under Windows, no performance loss is incurred.<br />
<br />
Note that this does not equate 'hiding' the virtual machine from Windows or any drivers/programs running in the VM.<br />
<br />
=== Passing keyboard/mouse via Evdev ===<br />
<br />
If you do not have a spare mouse or keyboard to dedicate to your guest, and you do not want to suffer from the video overhead of Spice, you can setup evdev to share them between your Linux host and your virtual machine.<br />
<br />
{{Note|Press both left and right '''Ctrl''' keys at the same time to swap control between the host and the guest.}}<br />
<br />
First, modify the libvirt configuration. If you encounter {{ic|failed to get domain ''vmname''}} error, add {{ic|-c qemu:///system}} arguments to {{ic|virsh}}.<br />
<br />
{{hc|$ virsh edit ''vmname''|2=<br />
<domain type='kvm'><br />
}}<br />
<br />
to<br />
<br />
{{hc|$ virsh edit ''vmname''|2=<br />
<domain type='kvm' xmlns:qemu='<nowiki>http://libvirt.org/schemas/domain/qemu/1.0</nowiki>'><br />
}}<br />
<br />
Next, find your keyboard and mouse devices in {{ic|/dev/input/by-id/}}. You may find multiple devices associated to your mouse or keyboard, so try {{ic|cat /dev/input/by-id/''device_id''}} and either hit some keys on the keyboard or wiggle your mouse to see if input comes through, if so you have got the right device. Now add those devices to your configuration right before the closing </domain> tag:<br />
<br />
{{hc|$ virsh edit ''vmname''|<nowiki><br />
...<br />
<qemu:commandline><br />
<qemu:arg value='-object'/><br />
<qemu:arg value='input-linux,id=mouse1,evdev=/dev/input/by-id/MOUSE_NAME'/><br />
<qemu:arg value='-object'/><br />
<qemu:arg value='input-linux,id=kbd1,evdev=/dev/input/by-id/KEYBOARD_NAME,grab_all=on,repeat=on'/><br />
</qemu:commandline><br />
...<br />
</nowiki>}}<br />
<br />
Replace {{ic|MOUSE_NAME}} and {{ic|KEYBOARD_NAME}} with your device id. You will also need to include these devices in your qemu config, and setting the user and group to one that has access to your input devices:<br />
<br />
{{hc|/etc/libvirt/qemu.conf|<nowiki><br />
...<br />
user = "<your_user>"<br />
group = "kvm"<br />
...<br />
cgroup_device_acl = [<br />
"/dev/input/by-id/KEYBOARD_NAME",<br />
"/dev/input/by-id/MOUSE_NAME",<br />
"/dev/null", "/dev/full", "/dev/zero",<br />
"/dev/random", "/dev/urandom",<br />
"/dev/ptmx", "/dev/kvm", "/dev/kqemu",<br />
"/dev/rtc","/dev/hpet", "/dev/sev"<br />
]<br />
...<br />
</nowiki>}}<br />
<br />
Then ensure that the user you provided has access to the {{ic|kvm}} and {{ic|input}} [[user group]]s. [[Restart]] {{ic|libvirtd.service}}. Now you can startup the guest OS and test swapping control of your mouse and keyboard between the host and guest by pressing both the left and right control keys at the same time.<br />
<br />
You may also consider switching from PS/2 to Virtio inputs in your configurations. Add these two devices:<br />
<br />
{{hc|$ virsh edit ''vmname''|<nowiki><br />
...<br />
<input type='mouse' bus='virtio'/><br />
<input type='keyboard' bus='virtio'/><br />
...<br />
</nowiki>}}<br />
<br />
The virtio input devices will not actually be used until the guest drivers are installed. QEMU will continue to send key events to the PS2 devices until it detects the virtio input driver initialization. Note that the PS2 devices cannot be removed as they are an internal function of the emulated Q35/440FX chipsets.<br />
<br />
=== Gotchas ===<br />
<br />
==== Using a non-EFI image on an OVMF-based VM ====<br />
<br />
The OVMF firmware does not support booting off non-EFI mediums. If the installation process drops you in a UEFI shell right after booting, you may have an invalid EFI boot media. Try using an alternate Linux/Windows image to determine if you have an invalid media.<br />
<br />
== Performance tuning ==<br />
<br />
Most use cases for PCI passthroughs relate to performance-intensive domains such as video games and GPU-accelerated tasks. While a PCI passthrough on its own is a step towards reaching native performance, there are still a few ajustments on the host and guest to get the most out of your VM.<br />
<br />
=== CPU pinning ===<br />
<br />
The default behavior for KVM guests is to run operations coming from the guest as a number of threads representing virtual processors. Those threads are managed by the Linux scheduler like any other thread and are dispatched to any available CPU cores based on niceness and priority queues. As such, the local CPU cache benefits (L1/L2) are lost each time the host scheduler reschedules the virtual CPU thread on a different physical CPU. This can noticeably harm performance on the guest. CPU pinning aims to resolve this by limiting which physical CPUs the virtual CPUs are allowed to run on. The ideal setup is a one to one mapping such that the virtual CPU cores match physical CPU cores while taking hyperthreading/SMT into account.<br />
<br />
{{Note|For certain users enabling CPU pinning may introduce stuttering and short hangs, especially with the MuQSS scheduler (present in linux-ck and linux-zen kernels). You might want to try disabling pinning first if you experience similar issues, which effectively trades maximum performance for responsiveness at all times.}}<br />
<br />
==== CPU topology ====<br />
<br />
Most modern CPUs support hardware multitasking, also known as hyper-threading on Intel CPUs or SMT on AMD CPUs. Hyper-threading/SMT is simply a very efficient way of running two threads on one CPU core at any given time. You will want to take into consideration that the CPU pinning you choose will greatly depend on what you do with your host while your VM is running.<br />
<br />
To find the topology for your CPU run {{ic|1=lscpu -e}}:<br />
<br />
{{Note|Pay special attention to the 4th column '''"CORE"''' as this shows the association of the Physical/Logical CPU cores}}<br />
<br />
{{ic|lscpu -e}} on a 6c/12t Ryzen 5 1600:<br />
<br />
CPU NODE SOCKET CORE L1d:L1i:L2:L3 ONLINE MAXMHZ MINMHZ<br />
0 0 0 0 0:0:0:0 yes 3800.0000 1550.0000<br />
1 0 0 0 0:0:0:0 yes 3800.0000 1550.0000<br />
2 0 0 1 1:1:1:0 yes 3800.0000 1550.0000<br />
3 0 0 1 1:1:1:0 yes 3800.0000 1550.0000<br />
4 0 0 2 2:2:2:0 yes 3800.0000 1550.0000<br />
5 0 0 2 2:2:2:0 yes 3800.0000 1550.0000<br />
6 0 0 3 3:3:3:1 yes 3800.0000 1550.0000<br />
7 0 0 3 3:3:3:1 yes 3800.0000 1550.0000<br />
8 0 0 4 4:4:4:1 yes 3800.0000 1550.0000<br />
9 0 0 4 4:4:4:1 yes 3800.0000 1550.0000<br />
10 0 0 5 5:5:5:1 yes 3800.0000 1550.0000<br />
11 0 0 5 5:5:5:1 yes 3800.0000 1550.0000<br />
<br />
{{Note|Ryzen 3000 ComboPi AGESA changes topology to match Intel example, even on prior generation CPUs. Above valid only on older AGESA. }}<br />
<br />
{{ic|lscpu -e}} on a 6c/12t Intel 8700k:<br />
<br />
CPU NODE SOCKET CORE L1d:L1i:L2:L3 ONLINE MAXMHZ MINMHZ<br />
0 0 0 0 0:0:0:0 yes 4600.0000 800.0000<br />
1 0 0 1 1:1:1:0 yes 4600.0000 800.0000<br />
2 0 0 2 2:2:2:0 yes 4600.0000 800.0000<br />
3 0 0 3 3:3:3:0 yes 4600.0000 800.0000<br />
4 0 0 4 4:4:4:0 yes 4600.0000 800.0000<br />
5 0 0 5 5:5:5:0 yes 4600.0000 800.0000<br />
6 0 0 0 0:0:0:0 yes 4600.0000 800.0000<br />
7 0 0 1 1:1:1:0 yes 4600.0000 800.0000<br />
8 0 0 2 2:2:2:0 yes 4600.0000 800.0000<br />
9 0 0 3 3:3:3:0 yes 4600.0000 800.0000<br />
10 0 0 4 4:4:4:0 yes 4600.0000 800.0000<br />
11 0 0 5 5:5:5:0 yes 4600.0000 800.0000<br />
<br />
As we see above, with AMD '''Core 0''' is sequential with '''CPU 0 & 1''', whereas Intel places '''Core 0''' on '''CPU 0 & 6'''.<br />
<br />
{{Tip|You can view your systems topology in diagram form, which may help some users. If you have {{Pkg|hwloc}} installed, run {{ic|lstopo}} to generate a helpful image of your CPU/Thread groupings.}}<br />
<br />
If you do not need all cores for the guest, it would then be preferable to leave at the very least one core for the host. Choosing which cores one to use for the host or guest should be based on the specific hardware characteristics of your CPU, however '''Core 0''' is a good choice for the host in most cases. If any cores are reserved for the host, it is recommended to pin the emulator and iothreads, if used, to the host cores rather than the VCPUs. This may improve performance and reduce latency for the guest since those threads will not pollute the cache or contend for scheduling with the guest VCPU threads. If all cores are passed to the guest, there is no need or benefit to pinning the emulator or iothreads.<br />
<br />
==== XML examples ====<br />
<br />
{{Note|Do not use the '''iothread''' lines from the XML examples shown below if you have not added an '''iothread''' to your disk controller. '''iothread''''s only work on '''virtio-scsi''' or '''virtio-blk''' devices.}}<br />
<br />
===== 4c/1t CPU w/o Hyperthreading Example =====<br />
<br />
{{hc|$ virsh edit ''vmname''|<nowiki><br />
...<br />
<vcpu placement='static'>4</vcpu><br />
<cputune><br />
<vcpupin vcpu='0' cpuset='0'/><br />
<vcpupin vcpu='1' cpuset='1'/><br />
<vcpupin vcpu='2' cpuset='2'/><br />
<vcpupin vcpu='3' cpuset='3'/><br />
</cputune><br />
...<br />
</nowiki>}}<br />
<br />
===== 4c/2t Intel/AMD CPU example (after ComboPI AGESA bios update) =====<br />
<br />
{{hc|$ virsh edit ''vmname''|<nowiki><br />
...<br />
<vcpu placement='static'>8</vcpu><br />
<iothreads>1</iothreads><br />
<cputune><br />
<vcpupin vcpu='0' cpuset='2'/><br />
<vcpupin vcpu='1' cpuset='8'/><br />
<vcpupin vcpu='2' cpuset='3'/><br />
<vcpupin vcpu='3' cpuset='9'/><br />
<vcpupin vcpu='4' cpuset='4'/><br />
<vcpupin vcpu='5' cpuset='10'/><br />
<vcpupin vcpu='6' cpuset='5'/><br />
<vcpupin vcpu='7' cpuset='11'/><br />
<emulatorpin cpuset='0,6'/><br />
<iothreadpin iothread='1' cpuset='0,6'/><br />
</cputune><br />
...<br />
<cpu mode='host-passthrough'><br />
<topology sockets='1' cores='4' threads='2'/><br />
</cpu><br />
...<br />
</nowiki>}}<br />
<br />
===== 4c/2t AMD CPU example (Before ComboPi AGESA bios update) =====<br />
<br />
{{hc|$ virsh edit ''vmname''|<nowiki><br />
...<br />
<vcpu placement='static'>8</vcpu><br />
<iothreads>1</iothreads><br />
<cputune><br />
<vcpupin vcpu='0' cpuset='2'/><br />
<vcpupin vcpu='1' cpuset='3'/><br />
<vcpupin vcpu='2' cpuset='4'/><br />
<vcpupin vcpu='3' cpuset='5'/><br />
<vcpupin vcpu='4' cpuset='6'/><br />
<vcpupin vcpu='5' cpuset='7'/><br />
<vcpupin vcpu='6' cpuset='8'/><br />
<vcpupin vcpu='7' cpuset='9'/><br />
<emulatorpin cpuset='0-1'/><br />
<iothreadpin iothread='1' cpuset='0-1'/><br />
</cputune><br />
...<br />
<cpu mode='host-passthrough'><br />
<topology sockets='1' cores='4' threads='2'/><br />
</cpu><br />
...<br />
</nowiki>}}<br />
<br />
{{Note|If further CPU isolation is needed, consider using the '''isolcpus''' kernel command-line parameter on the unused physical/logical cores.}}<br />
<br />
If you do not intend to be doing any computation-heavy work on the host (or even anything at all) at the same time as you would on the VM, you may want to pin your VM threads across all of your cores, so that the VM can fully take advantage of the spare CPU time the host has available. Be aware that pinning all physical and logical cores of your CPU could induce latency in the guest VM.<br />
<br />
=== Huge memory pages ===<br />
<br />
When dealing with applications that require large amounts of memory, memory latency can become a problem since the more memory pages are being used, the more likely it is that this application will attempt to access information across multiple memory "pages", which is the base unit for memory allocation. Resolving the actual address of the memory page takes multiple steps, and so CPUs normally cache information on recently used memory pages to make subsequent uses on the same pages faster. Applications using large amounts of memory run into a problem where, for instance, a virtual machine uses 4 GiB of memory divided into 4 KiB pages (which is the default size for normal pages) for a total of 1.04 million pages, meaning that such cache misses can become extremely frequent and greatly increase memory latency. Huge pages exist to mitigate this issue by giving larger individual pages to those applications, increasing the odds that multiple operations will target the same page in succession.<br />
<br />
==== Transparent huge pages ====<br />
<br />
QEMU will use 2MiB sized transparent huge pages automatically without any explicit configuration in QEMU or Libvirt, subject to some important caveats. When using VFIO the pages are locked in at boot time and transparent huge pages are allocated up front when the VM first boots. If the kernel memory is highly fragmented, or the VM is using a majority of the remaining free memory, it is likely that the kernel will not have enough 2MiB pages to fully satisfy the allocation. In such a case, it silently fails by using a mix of 2MiB and 4KiB pages. Since the pages are locked in VFIO mode, the kernel will not be able to convert those 4KiB pages to huge after the VM starts either. The number of available 2MiB huge pages available to THP is the same as via the [[#Dynamic huge pages]] mechanism described in the following sections.<br />
<br />
To check how much memory THP is using globally:<br />
<br />
{{hc|$ grep AnonHugePages /proc/meminfo|<br />
AnonHugePages: 8091648 kB<br />
}}<br />
<br />
To check a specific QEMU instance. QEMU's PID must be substituted in the grep command:<br />
<br />
{{hc|$ grep -P 'AnonHugePages:\s+(?!0)\d+' /proc/[PID]/smaps|<br />
AnonHugePages: 8087552 kB<br />
}}<br />
<br />
In this example, the VM was allocated 8388608KiB of memory, but only 8087552KiB was available via THP. The remaining 301056KiB are allocated as 4KiB pages. Aside from manually checking, there is no indication when partial allocations occur. As such, THP's effectiveness is very much dependent on the host system's memory fragmentation at the time of VM startup. If this trade off is unacceptable or strict guarantees are required, [[#Static huge pages]] is recommended.<br />
<br />
Arch kernels have THP compiled in and enabled by default with {{ic|1=/sys/kernel/mm/transparent_hugepage/enabled}} set to {{ic|1=madvise}} mode.<br />
<br />
Also when trying to alloc pages for virsh (ie {{ic|1=virsh allocpages 2M 2048}}) you need free *NOT* available ram. If your system has been up for a while, the kernel will use the free ram for caches. A simple fix is dropping the vm caches:<br />
<br />
{{ic|# echo 1 > /proc/sys/vm/drop_caches}}<br />
<br />
==== Static huge pages ====<br />
<br />
While transparent huge pages should work in the vast majority of cases, they can also be allocated statically during boot. This should only be needed to make use 1 GiB hugepages on machines that support it, since transparent huge pages normally only go up to 2 MiB.<br />
<br />
{{Warning|Static huge pages lock down the allocated amount of memory, making it unavailable for applications that are not configured to use them. Allocating 4 GiBs worth of huge pages on a machine with 8 GiB of memory will only leave you with 4 GiB of available memory on the host '''even when the VM is not running'''.}}<br />
<br />
To allocate huge pages at boot, one must simply specify the desired amount on their kernel command line with {{ic|1=hugepages=''x''}}. For instance, reserving 1024 pages with {{ic|1=hugepages=1024}} and the default size of 2048 KiB per huge page creates 2 GiB worth of memory for the virtual machine to use.<br />
<br />
If supported by CPU page size could be set manually. 1 GiB huge page support could be verified by {{ic|grep pdpe1gb /proc/cpuinfo}}. Setting 1 GiB huge page size via kernel parameters : {{ic|1=default_hugepagesz=1G hugepagesz=1G hugepages=X}}.<br />
<br />
Also, since static huge pages can only be used by applications that specifically request it, you must add this section in your libvirt domain configuration to allow kvm to benefit from them :<br />
<br />
{{hc|$ virsh edit ''vmname''|<nowiki><br />
...<br />
<memoryBacking><br />
<hugepages/><br />
</memoryBacking><br />
...<br />
</nowiki>}}<br />
<br />
==== Dynamic huge pages ====<br />
<br />
{{Accuracy|Need futher testing if this variant as effective as static one}}<br />
<br />
Hugepages could be allocated manually via {{ic|vm.nr_overcommit_hugepages}} [[sysctl]] parameter.<br />
<br />
{{hc|/etc/sysctl.d/10-kvm.conf|2=<br />
vm.nr_hugepages = 0<br />
vm.nr_overcommit_hugepages = ''num''<br />
}}<br />
<br />
Where {{ic|''num''}} - is the number of huge pages, which default size if 2 MiB.<br />
Pages will be automatically allocated, and freed after VM stops.<br />
<br />
More manual way:<br />
<br />
# echo ''num'' > /sys/kernel/mm/hugepages/hugepages-2048kB/nr_hugepages<br />
# echo ''num'' > /sys/kernel/mm/hugepages/hugepages-1048576kB/nr_hugepages<br />
<br />
For 2 MiB and 1 GiB page size respectively.<br />
And they should be manually freed in the same way.<br />
<br />
It is hardly recommended to drop caches, compact memory and wait couple of seconds before starting VM, as there could be not enough free contiguous memory for required huge pages blocks. Especially after some uptime of the host system.<br />
<br />
# echo 3 > /proc/sys/vm/drop_caches<br />
# echo 1 > /proc/sys/vm/compact_memory<br />
<br />
Theoretically, 1 GiB pages works as 2 MiB. But practically - no guaranteed way was found to get contiguous 1 GiB memory blocks. Each consequent request of 1 GiB blocks lead to lesser and lesser dynamically allocated count.<br />
<br />
=== CPU frequency governor ===<br />
<br />
Depending on the way your [[CPU frequency scaling|CPU governor]] is configured, the VM threads may not hit the CPU load thresholds for the frequency to ramp up. Indeed, KVM cannot actually change the CPU frequency on its own, which can be a problem if it does not scale up with vCPU usage as it would result in underwhelming performance. An easy way to see if it behaves correctly is to check if the frequency reported by {{ic|watch lscpu}} goes up when running a CPU-intensive task on the guest. If you are indeed experiencing stutter and the frequency does not go up to reach its reported maximum, it may be due to [https://lime-technology.com/forum/index.php?topic=46664.msg447678#msg447678 cpu scaling being controlled by the host OS]. In this case, try setting all cores to maximum frequency to see if this improves performance. Note that if you are using a modern intel chip with the default pstate driver, cpupower commands will be [[CPU frequency scaling#CPU frequency driver|ineffective]], so monitor {{ic|/proc/cpuinfo}} to make sure your cpu is actually at max frequency.<br />
<br />
=== Isolating pinned CPUs ===<br />
<br />
CPU pinning by itself will not prevent other host processes from running on the pinned CPUs. Properly isolating the pinned CPUs can reduce latency in the guest VM.<br />
<br />
==== With isolcpus kernel parameter ====<br />
<br />
In this example, let us assume you are using CPUs 4-7.<br />
Use the [[kernel parameters]] {{ic|isolcpus nohz_full rcu_nocbs}} to completely isolate the CPUs from the kernel. For example:<br />
<br />
isolcpus=4-7 nohz_full=4-7 rcu_nocbs=4-7<br />
<br />
Then, run {{ic|qemu-system-x86_64}} with taskset and chrt:<br />
<br />
# chrt -r 1 taskset -c 4-7 qemu-system-x86_64 ...<br />
<br />
The {{ic|chrt}} command will ensure that the task scheduler will round-robin distribute work (otherwise it will all stay on the first cpu). For {{ic|taskset}}, the CPU numbers can be comma- and/or dash-separated, like "0,1,2,3" or "0-4" or "1,7-8,10" etc.<br />
<br />
See [https://www.removeddit.com/r/VFIO/comments/6vgtpx/high_dpc_latency_and_audio_stuttering_on_windows/dm0sfto/ this Removeddit mirror of a Reddit thread] for more info. ([https://www.reddit.com/r/VFIO/comments/6vgtpx/high_dpc_latency_and_audio_stuttering_on_windows/dm0sfto/ The original thread] is worthless because of deleted comments.)<br />
<br />
==== Dynamically isolating CPUs ====<br />
<br />
The isolcpus kernel parameter will permanently reserve CPU cores, even when the guest is not running. A more flexible alternative is to use the cset tool from {{AUR|cpuset-git}} to dynamically isolate CPUs when starting the guest.<br />
<br />
See this [https://www.redhat.com/archives/vfio-users/2016-September/msg00072.html vfio-users post] for more info, as well as this [https://rokups.github.io/#!pages/gaming-vm-performance.md blog post] and this [https://github.com/PassthroughPOST/VFIO-Tools/blob/master/libvirt_hooks/hooks/cset.sh script] for working examples.<br />
<br />
=== Improving performance on AMD CPUs ===<br />
<br />
Starting with QEMU 3.1 the TOPOEXT cpuid flag is disabled by default. In order to use hyperthreading (SMT) on AMD CPUs you need to manually enable it:<br />
<br />
<cpu mode='host-passthrough' check='none'><br />
<topology sockets='1' cores='4' threads='2'/><br />
<feature policy='require' name='topoext'/><br />
</cpu><br />
<br />
commit: https://git.qemu.org/?p=qemu.git;a=commit;h=7210a02c58572b2686a3a8d610c6628f87864aed<br />
<br />
=== Virtio disk ===<br />
<br />
{{Merge|QEMU#Installing virtio drivers|Off-topic.|section=Moving virtio disk section to QEMU}}<br />
<br />
The default disk types are SATA or IDE emulation out of the box. These controllers offer maximum compatibility but are not suited for efficient virtualization. Two accelerated models exist: {{ic|1=virtio-scsi}} for SCSI emulation and passthrough, or {{ic|1=virtio-blk}} for a more basic block device emulation.<br />
<br />
==== Drivers ====<br />
<br />
* Linux guests should support these out of the box on any modern kernel<br />
* macOS has rudimentary {{ic|1=virtio-blk}} support starting in Mojave via {{ic|1=AppleVirtIO.kext}} but performance is poor and TRIM does not appear to work. SATA is still suggested<br />
* Windows needs the [https://docs.fedoraproject.org/en-US/quick-docs/creating-windows-virtual-machines-using-virtio-drivers/ Windows virtio drivers]. {{ic|1=virtio-scsi}} uses the {{ic|1=vioscsi}} driver. {{ic|1=virtio-blk}} uses the {{ic|1=viostor}} driver<br />
* Windows can be installed directly onto these disks by selecting 'load driver' on the installer disk selection menu. The windows iso and virtio driver iso should both be attached as regular SATA/IDE cdroms during the installation process<br />
* To switch boot disks to virtio on an existing Windows installation:<br />
** {{ic|1=virtio-blk}}: Add a temporary disk with bus {{ic|1=virtio}}, boot windows & load the driver for the disk, then shutdown and switch the boot disk disk bus to {{ic|1=virtio}}<br />
** {{ic|1=virtio-scsi}}: Add a scsi controller with model {{ic|1=virtio}}, boot windows & load the driver for the controller, then shutdown and switch the boot disk bus to {{ic|1=scsi}} (not virtio)<br />
<br />
==== Considerations ====<br />
<br />
* {{ic|1=virtio-scsi}} TRIM support is mature, all versions should support it. Traditionally, {{ic|1=virtio-scsi}} has been the preferred approach for this reason<br />
* {{ic|1=virtio-blk}} TRIM support is new, this requires requires qemu 4.0+, guest linux kernel 5.0+, guest windows drivers 0.1.173+<br />
* Thin provisioning works by enabling TRIM on a sparse image file: {{ic|1=discard='unmap'}}. Unused blocks will be freed and the disk usage will drop (works on both raw and qcow2). Actual on-disk size of a sparse image file may be checked with {{ic|1=du /path/to/disk.img}}<br />
* Thin provisioning can also work with block storage such as zfs zvols or thin lvm<br />
* Virt queue count will influence the number of threads inside the guest kernel used for IO processing, suggest using {{ic|1=queues='4'}} or more<br />
* Native mode ({{ic|1=io='native'}}) uses a single threaded model based on linux AIO, is a bit more CPU efficient but may have lower peak performance and does not allow host side caching to be used<br />
* Threaded mode ({{ic|1=io='threads'}}) will spawn dozens of threads on demand as the disk is used. This is less efficient but may perform better if there are enough host cores available to run them, and allows for host side caching to be used<br />
* Modern versions of libvirt will group the dynamic worker threads created when using threaded mode in with the iothread=1 cgroup for pinning purposes. Very old versions of libvirt left these in the emulator cgroup<br />
<br />
==== IO threads ====<br />
<br />
An IO thread is a dedicated thread for processing disk events, rather than using the main qemu emulator loop. This should not be confused with the worker threads spawned on demand with {{ic|1=io='threads'}}.<br />
<br />
* You can only use one iothread per disk controller. The thread must be assigned to a specific controller with {{ic|1=iothread='X'}} in the {{ic|1=<driver>}} tag. Furthermore, extra & unassigned iothreads will not be used and do nothing<br />
* In the case of {{ic|1=virtio-scsi}}, there is one controller for multiple scsi disks. The iothread is assigned on the controller: {{ic|1=<controller><driver iothread='X'>}}<br />
* In the case of {{ic|1=virtio-blk}}, each disk has its own controller. The iothread is assigned in the driver tag under the disk itself: {{ic|1=<disk><driver iothread='X'>}}<br />
* Since emulated disks incur a significant amount of CPU overhead, that can lead to vcpu stuttering under high disk load (especially high random IOPS). In this case it helps to pin the IO to different core(s) than your vcpus with {{ic|1=<iothreadpin>}}<br />
<br />
==== Examples with libvirt ====<br />
<br />
virtio-scsi + iothread + worker threads + host side writeback caching + full disk block device backend:<br />
<domain><br />
<devices><br />
<disk type='block' device='disk'><br />
<driver name='qemu' type='raw' cache='writeback' io='threads' discard='unmap'/><br />
<source dev='/dev/disk/by-id/ata-Samsung_SSD_840_EVO_1TB_S1D9NSAF206396F'/><br />
<target dev='sda' bus='scsi'/><br />
</disk><br />
<controller type='scsi' index='0' model='virtio-scsi'><br />
<driver iothread='1' queues='8'/><br />
</controller><br />
<br />
virtio-blk + iothread + native aio + no host caching + raw sparse image backend:<br />
<domain><br />
<devices><br />
<disk type='file' device='disk'><br />
<driver name='qemu' type='raw' cache='none' io='native' discard='unmap' iothread='1' queues='8'/><br />
<source file='/var/lib/libvirt/images/pool/win10.img'/><br />
<target dev='vda' bus='virtio'/><br />
</disk><br />
<br />
Creating the iothreads:<br />
<domain><br />
<iothreads>1</iothreads><br />
<br />
Pinning iothreads:<br />
<domain><br />
<cputune><br />
<iothreadpin iothread='1' cpuset='0-1,6-7'/><br />
<br />
=== Virtio network ===<br />
<br />
The default NIC models rtl8139 or e1000 can be a bottleneck for gigabit+ speeds and have a significant amount of CPU overhead compared to {{ic|1=virtio-net}}.<br />
<br />
* Select {{ic|1=virtio}} as the model for the NIC with libvirt or use the {{ic|1=virtio-net-pci}} device in bare qemu<br />
* Windows needs the {{ic|1=NetKVM}} driver from [https://docs.fedoraproject.org/en-US/quick-docs/creating-windows-virtual-machines-using-virtio-drivers/ Windows virtio drivers]<br />
* Virtio uses vhost-net by default for in-kernel packet processing without exiting to userspace<br />
* Multiqueue can enabled for a further speedup with multiple connections but typically will not boost single stream speeds. For libvirt add {{ic|1=<driver queues='8'/>}} under the interface tag<br />
* Zero copy transmit may also be enabled on macvtap by setting the module parameter {{ic|1=vhost_net.experimental_zcopytx=1}} but this may actually have worse performance, see [https://github.com/torvalds/linux/commit/098eadce3c622c07b328d0a43dda379b38cf7c5e commit]<br />
<br />
Libvirt example with a bridge:<br />
<br />
<interface type='bridge'><br />
<mac address="52:54:00:6d:6e:2e"/><br />
<source bridge='br0'/><br />
<model type='virtio'/><br />
<driver queues='8'/><br />
</interface><br />
<br />
=== Further tuning ===<br />
<br />
More specialized VM tuning tips are available at [https://access.redhat.com/documentation/en-US/Red_Hat_Enterprise_Linux/7/html-single/Virtualization_Tuning_and_Optimization_Guide/index.html Red Hat's Virtualization Tuning and Optimization Guide].<br />
<br />
== Special procedures ==<br />
<br />
Certain setups require specific configuration tweaks in order to work properly. If you are having problems getting your host or your VM to work properly, see if your system matches one of the cases below and try adjusting your configuration accordingly.<br />
<br />
=== Using identical guest and host GPUs ===<br />
<br />
{{Expansion|A number of users have been having issues with this, it should probably be adressed by the article.|Talk:PCI passthrough via OVMF#Additionnal sections}}<br />
<br />
Due to how vfio-pci uses your vendor and device id pair to identify which device they need to bind to at boot, if you have two GPUs sharing such an ID pair you will not be able to get your passthrough driver to bind with just one of them. This sort of setup makes it necessary to use a script, so that whichever driver you are using is instead assigned by pci bus address using the {{ic|driver_override}} mechanism.<br />
<br />
==== Script variants ====<br />
<br />
===== Passthrough all GPUs but the boot GPU =====<br />
<br />
Here, we will make a script to bind vfio-pci to all GPUs but the boot gpu. Create the script {{ic|/usr/local/bin/vfio-pci-override.sh}}:<br />
<br />
{{bc|<nowiki><br />
#!/bin/sh<br />
<br />
for i in /sys/bus/pci/devices/*/boot_vga; do<br />
if [ $(cat "$i") -eq 0 ]; then<br />
GPU="${i%/boot_vga}"<br />
AUDIO="$(echo "$GPU" | sed -e "s/0$/1/")"<br />
USB="$(echo "$GPU" | sed -e "s/0$/2/")"<br />
echo "vfio-pci" > "$GPU/driver_override"<br />
if [ -d "$AUDIO" ]; then<br />
echo "vfio-pci" > "$AUDIO/driver_override"<br />
fi<br />
if [ -d "$USB" ]; then<br />
echo "vfio-pci" > "$USB/driver_override"<br />
fi<br />
fi<br />
done<br />
<br />
modprobe -i vfio-pci<br />
</nowiki>}}<br />
<br />
===== Passthrough selected GPU =====<br />
<br />
In this case we manually specify the GPU to bind.<br />
<br />
{{bc|<nowiki><br />
#!/bin/sh<br />
<br />
DEVS="0000:03:00.0 0000:03:00.1"<br />
<br />
if [ ! -z "$(ls -A /sys/class/iommu)" ]; then<br />
for DEV in $DEVS; do<br />
echo "vfio-pci" > /sys/bus/pci/devices/$DEV/driver_override<br />
done<br />
fi<br />
<br />
modprobe -i vfio-pci<br />
</nowiki>}}<br />
<br />
==== Script installation ====<br />
<br />
Edit {{ic|/etc/mkinitcpio.conf}}:<br />
<br />
# Add {{ic|modconf}} to the [[mkinitcpio#HOOKS|HOOKS]] array and {{ic|/usr/local/bin/vfio-pci-override.sh}} to the [[mkinitcpio#BINARIES and FILES|FILES]] array.<br />
<br />
Edit {{ic|/etc/modprobe.d/vfio.conf}}:<br />
<br />
# Add the following line: {{ic|install vfio-pci /usr/local/bin/vfio-pci-override.sh}}<br />
# [[Regenerate the initramfs]] and reboot.<br />
<br />
=== Passing the boot GPU to the guest ===<br />
<br />
{{Expansion|This is related to VBIOS issues and should be moved into a separate section regarding VBIOS compatibility.|section=UEFI (OVMF) Compatibility in VBIOS}}<br />
<br />
The GPU marked as {{ic|boot_vga}} is a special case when it comes to doing PCI passthroughs, since the BIOS needs to use it in order to display things like boot messages or the BIOS configuration menu. To do that, it makes [https://www.redhat.com/archives/vfio-users/2016-May/msg00224.html a copy of the VGA boot ROM which can then be freely modified]. This modified copy is the version the system gets to see, which the passthrough driver may reject as invalid. As such, it is generally recommended to change the boot GPU in the BIOS configuration so the host GPU is used instead or, if that is not possible, to swap the host and guest cards in the machine itself.<br />
<br />
=== Using Looking Glass to stream guest screen to the host ===<br />
<br />
It is possible to make VM share the monitor, and optionally a keyboard and a mouse with a help of [https://looking-glass.hostfission.com/ Looking Glass].<br />
<br />
==== Adding IVSHMEM Device to VM ====<br />
<br />
Looking glass works by creating a shared memory buffer between a host and a guest. This is a lot faster than streaming frames via localhost, but requires additional setup. <br />
<br />
With your VM turned off open the machine configuration<br />
<br />
{{hc|$ virsh edit ''vmname''|<nowiki><br />
...<br />
<devices><br />
...<br />
<shmem name='looking-glass'><br />
<model type='ivshmem-plain'/><br />
<size unit='M'>32</size><br />
</shmem><br />
</devices><br />
...<br />
</nowiki>}}<br />
<br />
You should replace 32 with your own calculated value based on what resolution you are going to pass through. It can be calculated like this:<br />
<br />
width x height x 4 x 2 = total bytes<br />
total bytes / 1024 / 1024 = total mebibytes + 2<br />
<br />
For example, in case of 1920x1080<br />
<br />
1920 x 1080 x 4 x 2 = 16,588,800 bytes<br />
16,588,800 / 1024 / 1024 = 15.82 MiB + 2 = 17.82<br />
<br />
The result must be '''rounded up''' to the nearest power of two, and since 17.82 is bigger than 16 we should choose 32.<br />
<br />
Next create a configuration file to create the shared memory file on boot<br />
<br />
{{hc|/etc/tmpfiles.d/10-looking-glass.conf|2=<br />
f /dev/shm/looking-glass 0660 '''user''' kvm -<br />
}}<br />
<br />
Replace user with your username.<br />
<br />
Ask systemd-tmpfiles to create the shared memory file now without waiting to next boot<br />
<br />
# systemd-tmpfiles --create /etc/tmpfiles.d/10-looking-glass.conf<br />
<br />
==== Installing the IVSHMEM Host to Windows guest ====<br />
<br />
Currently Windows would not notify users about a new IVSHMEM device, it would silently install a dummy driver. To actually enable the device you have to go into device manager and update the driver for the device under the "System Devices" node for '''"PCI standard RAM Controller"'''. Download the signed driver [https://fedorapeople.org/groups/virt/virtio-win/direct-downloads/upstream-virtio/ from Red Hat].<br />
<br />
Once the driver is installed you must download a matching [https://github.com/gnif/LookingGlass/releases looking-glass-host] that matches the client you installed from AUR and start it on your guest. In order to run it you would also need to install Microsoft Visual C++ Redistributable from [https://www.visualstudio.com/downloads/ Microsoft]<br />
It is also possible to make it start automatically on VM boot by editing the {{ic|HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Windows\CurrentVersion\Run}} registry and adding a path to the downloaded executable.<br />
<br />
==== Getting a client ====<br />
<br />
Looking glass client can be installed from AUR using {{AUR|looking-glass}} or {{AUR|looking-glass-git}} packages.<br />
<br />
You can start it once the VM is set up and running<br />
<br />
$ looking-glass-client<br />
<br />
If you do not want to use Spice to control the guest mouse and keyboard you can disable the Spice server.<br />
<br />
$ looking-glass-client -s<br />
<br />
Additionally you may want to start Looking Glass Client as full screen, otherwise the image may be scaled down resulting in poor image fidelity.<br />
<br />
$ looking-glass-client -F<br />
<br />
Launch with the {{ic|--help}} option for further information.<br />
<br />
=== Swap peripherals to and from the Host ===<br />
<br />
Looking Glass includes a Spice client in order to control mouse movement on the Windows guest. However this may have too much latency for certain applications, such as gaming. An alternative method is passing through specific USB devices for minimal latency. This allows for switching the devices between host and guest.<br />
<br />
First create a .xml file for the device(s) you wish to pass-through, which libvirt will use to identify the device.<br />
<br />
{{hc|~/.VFIOinput/input_1.xml|2=<br />
<hostdev mode='subsystem' type='usb' managed='no'><br />
<source><br />
<vendor id='0x[Before Colon]'/><br />
<product id='0x[After Colon]'/><br />
</source><br />
</hostdev><br />
}}<br />
<br />
Replace [Before/After Colon] with the contents of the 'lsusb' command, specific to the device you want to pass-through.<br />
<br />
For instance my mouse is {{ic|Bus 005 Device 002: ID 1532:0037 Razer USA, Ltd}} so I would replace {{ic|vendor id}} with 1532, and {{ic|product id}} with 1037.<br />
<br />
Repeat this process for any additional USB devices you want to pass-through. If your mouse / keyboard has multiple entries in {{ic|lsusb}}, perhaps if it is wireless, then create additional xml files for each.<br />
<br />
{{Note|Do not forget to change the path & name of the script(s) above and below to match your user and specific system.}}<br />
<br />
Next a bash script file is needed to tell libvirt what to attach/detach the USB devices to the guest.<br />
<br />
{{hc|~/.VFIOinput/input_attach.sh|2=<br />
#!/bin/bash<br />
<br />
virsh attach-device [VM-Name] [USBdevice]<br />
}}<br />
<br />
Replace [VM-Name] with the name of your virtual machine, which can be seen under virt-manager. Additionally replace [USBdevice] with the '''full''' path to the .xml file for the device you wish to pass-through. Add additional lines for more than 1 device. For example here is my script:<br />
<br />
{{hc|~/.VFIOinput/input_attach.sh|2=<br />
#!/bin/bash<br />
<br />
virsh attach-device win10 /home/$USER/.VFIOinput/input_mouse.xml<br />
virsh attach-device win10 /home/$USER/.VFIOinput/input_keyboard.xml<br />
}}<br />
<br />
Next duplicate the script file and replace {{ic|attach-device}} with {{ic|detach-device}}. Ensure both scripts are executable with {{ic|chmod +x $script.sh}}<br />
<br />
This 2 script files can now be executed to attach or detach your USB devices from the host to the guest VM. It is important to note that they may need to be executed as root. To run the script from the Windows VM, one possibility is using [[PuTTY]] to [[SSH]] into the host, and execute the script. On Windows PuTTY comes with plink.exe which can execute singular commands over SSH before then logging out, instead of opening a SSH terminal, all in the background.<br />
<br />
{{hc|detach_devices.bat|2=<br />
"C:\Program Files\PuTTY\plink.exe" root@$HOST_IP -pw $ROOTPASSWORD /home/$USER/.VFIOinput/input_detach.sh<br />
}}<br />
<br />
Replace {{ic|$HOST_IP}} with the Host [[Network configuration#IP addresses|IP Address]] and $ROOTPASSWORD with the root password.<br />
<br />
{{warning|This method is insecure if somebody has access to your VM, since they could open the file and read your password. It is advisable to use [[SSH keys]] instead!}}<br />
<br />
You may also want to execute the script files using key binds. On Windows one option is [https://autohotkey.com/ Autohotkey], and on the Host [[Xbindkeys]]. Because of the need to run the scripts as root, you may also need to use [[Polkit]] or [[Sudo]] which can both be used to authenticate specific executables as able to run as root without needing a password.<br />
<br />
=== Bypassing the IOMMU groups (ACS override patch) ===<br />
<br />
If you find your PCI devices grouped among others that you do not wish to pass through, you may be able to seperate them using Alex Williamson's ACS override patch. Make sure you understand [https://vfio.blogspot.com/2014/08/iommu-groups-inside-and-out.html the potential risk] of doing so. <br />
<br />
You will need a kernel with the patch applied. The easiest method to acquiring this is through the {{AUR|linux-vfio}} package. <br />
<br />
In addition, the ACS override patch needs to be enabled with kernel command line options. The patch file adds the following documentation:<br />
<br />
pcie_acs_override =<br />
[PCIE] Override missing PCIe ACS support for:<br />
downstream<br />
All downstream ports - full ACS capabilties<br />
multifunction<br />
All multifunction devices - multifunction ACS subset<br />
id:nnnn:nnnn<br />
Specfic device - full ACS capabilities<br />
Specified as vid:did (vendor/device ID) in hex<br />
<br />
The option {{ic|1=pcie_acs_override=downstream,multifunction}} should break up as many devices as possible.<br />
<br />
After installation and configuration, reconfigure your [[Kernel parameters|bootloader kernel parameters]] to load the new kernel with the {{ic|1=pcie_acs_override=}} option enabled.<br />
<br />
== Plain QEMU without libvirt ==<br />
<br />
Instead of setting up a virtual machine with the help of libvirt, plain QEMU commands with custom parameters can be used for running the VM intended to be used with PCI passthrough. This is desirable for some use cases like scripted setups, where the flexibility for usage with other scripts is needed.<br />
<br />
To achieve this after [[#Setting up IOMMU]] and [[#Isolating the GPU]], follow the [[QEMU]] article to setup the virtualized environment, [[QEMU#Enabling KVM|enable KVM]] on it and use the flag {{ic|1=-device vfio-pci,host=07:00.0}} replacing the identifier (07:00.0) with your actual device's ID that you used for the GPU isolation earlier.<br />
<br />
For utilizing the OVMF firmware, make sure the {{Pkg|edk2-ovmf}} package is installed, copy the UEFI variables from {{ic|/usr/share/edk2-ovmf/x64/OVMF_VARS.fd}} to temporary location like {{ic|/tmp/MY_VARS.fd}} and finally specify the OVMF paths by appending the following parameters to the QEMU command (order matters):<br />
<br />
* {{ic|1=-drive if=pflash,format=raw,readonly,file=/usr/share/edk2-ovmf/x64/OVMF_CODE.fd}} for the actual OVMF firmware binary, note the readonly option<br />
* {{ic|1=-drive if=pflash,format=raw,file=/tmp/MY_VARS.fd}} for the variables<br />
<br />
{{Note|<br />
* Make sure that {{ic|1=OVMF_CODE.fd}} is given as a command line parameter before {{ic|1=MY_VARS.fd}}. The boot sequence will fail otherwise.<br />
* QEMU's default SeaBIOS can be used instead of OVMF, but it is not recommended as it can cause issues with passthrough setups.<br />
}}<br />
<br />
It is recommended to study the QEMU article for ways to enhance the performance by using the [[QEMU#Installing virtio drivers|virtio drivers]] and other further configurations for the setup.<br />
<br />
You also might have to use the {{ic|1=-cpu host,kvm=off}} parameter to forward the host's CPU model info to the VM and fool the virtualization detection used by Nvidia's and possibly other manufacturers' device drivers trying to block the full hardware usage inside a virtualized system.<br />
<br />
== Passing through other devices ==<br />
<br />
=== USB controller ===<br />
<br />
If your motherboard has multiple USB controllers mapped to multiple groups, it is possible to pass those instead of USB devices. Passing an actual controller over an individual USB device provides the following advantages : <br />
<br />
* If a device disconnects or changes ID over the course of an given operation (such as a phone undergoing an update), the VM will not suddenly stop seeing it.<br />
* Any USB port managed by this controller is directly handled by the VM and can have its devices unplugged, replugged and changed without having to notify the hypervisor.<br />
* Libvirt will not complain if one of the USB devices you usually pass to the guest is missing when starting the VM.<br />
<br />
Unlike with GPUs, drivers for most USB controllers do not require any specific configuration to work on a VM and control can normally be passed back and forth between the host and guest systems with no side effects.<br />
<br />
{{Warning|Make sure your USB controller supports resetting: [[#Passing through a device that does not support resetting]]}}<br />
<br />
You can find out which PCI devices correspond to which controller and how various ports and devices are assigned to each one of them using this command :<br />
<br />
{{hc|$ <nowiki>for usb_ctrl in /sys/bus/pci/devices/*/usb*; do pci_path=${usb_ctrl%/*}; iommu_group=$(readlink $pci_path/iommu_group); echo "Bus $(cat $usb_ctrl/busnum) --> ${pci_path##*/} (IOMMU group ${iommu_group##*/})"; lsusb -s ${usb_ctrl#*/usb}:; echo; done</nowiki>|<br />
Bus 1 --> 0000:00:1a.0 (IOMMU group 4)<br />
Bus 001 Device 004: ID 04f2:b217 Chicony Electronics Co., Ltd Lenovo Integrated Camera (0.3MP)<br />
Bus 001 Device 007: ID 0a5c:21e6 Broadcom Corp. BCM20702 Bluetooth 4.0 [ThinkPad]<br />
Bus 001 Device 008: ID 0781:5530 SanDisk Corp. Cruzer<br />
Bus 001 Device 002: ID 8087:0024 Intel Corp. Integrated Rate Matching Hub<br />
Bus 001 Device 001: ID 1d6b:0002 Linux Foundation 2.0 root hub<br />
<br />
Bus 2 --> 0000:00:1d.0 (IOMMU group 9)<br />
Bus 002 Device 006: ID 0451:e012 Texas Instruments, Inc. TI-Nspire Calculator<br />
Bus 002 Device 002: ID 8087:0024 Intel Corp. Integrated Rate Matching Hub<br />
Bus 002 Device 001: ID 1d6b:0002 Linux Foundation 2.0 root hub<br />
}}<br />
<br />
This laptop has 3 USB ports managed by 2 USB controllers, each with their own IOMMU group. In this example, Bus 001 manages a single USB port (with a SanDisk USB pendrive plugged into it so it appears on the list), but also a number of internal devices, such as the internal webcam and the bluetooth card. Bus 002, on the other hand, does not apprear to manage anything except for the calculator that is plugged into it. The third port is empty, which is why it does not show up on the list, but is actually managed by Bus 002.<br />
<br />
Once you have identified which controller manages which ports by plugging various devices into them and decided which one you want to passthrough, simply add it to the list of PCI host devices controlled by the VM in your guest configuration. No other configuration should be needed.<br />
<br />
{{Note|If your USB controller does not support resetting, is not in an isolated group, or is otherwise unable to be passed through then it may still be possible to accomplish similar results through [[udev]] rules. See [https://github.com/olavmrk/usb-libvirt-hotplug] which allows any device connected to specified USB ports to be automatically attached to a virtual machine.}}<br />
<br />
=== Passing VM audio to host via PulseAudio ===<br />
<br />
It is possible to route the virtual machine's audio to the host as an application using libvirt. This has the advantage of multiple audio streams being routable to one host output, and working with audio output devices that do not support passthrough. [[PulseAudio]] is required for this to work.<br />
<br />
First, remove the comment from the {{ic|1=#user = ""}} line. Then add your username in the quotations. This tells QEMU which user's pulseaudio stream to route through.<br />
<br />
{{hc|/etc/libvirt/qemu.conf|2=<br />
user = "example"<br />
}}<br />
<br />
Next, modify the libvirt configuration<br />
<br />
{{hc|$ virsh edit ''vmname''|2=<br />
<domain type='kvm'><br />
}}<br />
<br />
to<br />
<br />
{{hc|$ virsh edit ''vmname''|2=<br />
<domain type='kvm' xmlns:qemu='<nowiki>http://libvirt.org/schemas/domain/qemu/1.0</nowiki>'><br />
}}<br />
<br />
Then set the QEMU PulseAudio variables at the bottom of the libvirt xml file<br />
<br />
{{hc|$ virsh edit ''vmname''|<br />
</devices><br />
</domain><br />
}}<br />
<br />
to<br />
<br />
{{hc|$ virsh edit ''vmname''|<nowiki><br />
</devices><br />
<qemu:commandline><br />
<qemu:arg value="-audiodev"/><br />
<qemu:arg value="pa,id=snd0,server=/run/user/1000/pulse/native"/><br />
</qemu:commandline><br />
</domain><br />
</nowiki>}}<br />
<br />
You can also use the /tmp directory if you have multiple users accessing PulseAudio<br />
<br />
{{hc|$ virsh edit ''vmname''|<nowiki><br />
</devices><br />
<qemu:commandline><br />
<qemu:arg value="-audiodev"/><br />
<qemu:arg value="pa,id=snd0,server=unix:/tmp/pulse-socket"/><br />
</qemu:commandline><br />
</domain><br />
</nowiki>}}<br />
<br />
You may want to change 1000 under the user directory to your current user uid (which can be found by running the {{ic|id}} command. Remember to save the file and exit it without ending the process before continuing, otherwise the changes will not register. If you get the message {{ic|Domain ''vmname'' XML configuration edited.}} after exiting, it means that your changes have been applied. Virtual Machine audio will now be routed through the host as an application. A application such as {{Pkg|pavucontrol}} can be used to control the output device.<br />
<br />
==== QEMU 4.0/4.2+ audio changes ====<br />
<br />
As of QEMU 4.0 and made official with [https://www.kraxel.org/blog/2020/01/qemu-sound-audiodev/ QEMU 4.2] the {{ic|1=-audiodev}} parameter is now to be used with PulseAudio sound passthrough, any previous environmental variables you have defined in your XML '''need''' to be removed e.g. {{ic|qemu:env}} to properly use this.<br />
<br />
You will also need to change the chipset accordingly with each QEMU update to how your VM is set up, i.e. {{ic|pc-q35-4.2}} or {{ic|pc-i440fx-4.2}} to take advantage of the changes in QEMU:<br />
<br />
{{hc|$ virsh edit ''vmname''|2=<br />
<os><br />
<type arch='x86_64' machine='pc-q35-4.2'>hvm</type><br />
...<br />
</os><br />
...<br />
<os><br />
<type arch='x86_64' machine='pc-i440fx-4.2'>hvm</type><br />
...<br />
</os><br />
}}<br />
<br />
The latest audiodev changes in QEMU also suggest using the ICH9 instead of ICH6 or AC97 for audio emulation to take advantage of this newer code, it is recommended to use the parameters as shown below because Libvirt currently does not define the {{ic|1=id=}} parameter correctly:<br />
<br />
{{hc|$ virsh edit ''vmname''|<nowiki><br />
<qemu:arg value="-device"/><br />
<qemu:arg value="ich9-intel-hda,bus=pcie.0,addr=0x1b"/><br />
<qemu:arg value="-device"/><br />
<qemu:arg value="hda-micro,audiodev=hda"/><br />
<qemu:arg value="-audiodev"/><br />
<qemu:arg value="pa,id=hda,server=unix:/tmp/pulse-socket"/><br />
</nowiki>}}<br />
<br />
QEMU 4.2 also has 5.1 channel audio support via {{ic|usb-audio}} emulation, this can be enabled as shown below:<br />
<br />
{{hc|$ virsh edit ''vmname''|<nowiki><br />
<qemu:arg value="-device"/><br />
<qemu:arg value="usb-audio,audiodev=usb,multi=on"/><br />
<qemu:arg value="-audiodev"/><br />
<qemu:arg value="pa,id=usb,server=unix:/tmp/pulse-socket,out.mixing-engine=off"/><br />
</nowiki>}}<br />
<br />
{{Note|1=<nowiki></nowiki><br />
* You can have multiple audio backends, by simply specifying {{ic|1=-audiodev}} multiple times in your XML and by assigning them different ids. This can be useful for a use case of having two identical backends. With PulseAudio each backend is a separate stream and can be routed to different output devices on the host (using a pulse mixer app like {{Pkg|pavucontrol}} or {{Pkg|pulsemixer}}).<br />
* USB 3 emulation is needed in Libvirt/QEMU to enable the {{ic|usb-audio}} parameter<br />
* It is recommended to enable MSI interrupts with a tool such as [https://forums.guru3d.com/threads/windows-line-based-vs-message-signaled-based-interrupts-msi-tool.378044/ this] on the ICH9 audio device to mitigate any crackling, stuttering, speedup, or no audio at all after VM restart<br />
* The {{ic|1=-audiodev=hda}} parameter can be changed to your liking, but as shown above, the names must match each other or the VM will not start<br />
* If audio is crackling/stuttering/speedup etc. is still present you may want to adjust parameters such as {{ic|buffer-length}} and {{ic|timer-period}}, more information on these parameters and more can be found in the {{man|1|qemu}} manual<br />
* Some audio chipsets such as [https://bugzilla.kernel.org/show_bug.cgi?id=195303 Realtek alc1220] may also have issues out of the box so do consider this when using any audio emulation with QEMU<br />
* Improper pinning or heavy host usage without using [[#Isolating pinned CPUs|isolcpus]] can also influence sound bugs, especially while gaming in a VM<br />
}}<br />
<br />
=== Passing VM audio to host via Scream ===<br />
<br />
It is possible to pass VM audio through a bridged network such as the one provided by Libvirt or by adding a IVSHMEM device to the host by using a application called [https://github.com/duncanthrax/scream Scream]. This section will only cover using PulseAudio as a receiver on the host.<br />
See the project page for more details and instructions on other methods.<br />
<br />
==== Using Scream with a bridged network ====<br />
<br />
{{Note|<br />
* This is the ''preferred'' way to use this, although results may vary per user<br />
* It is recommend to use the [[#Virtio network]] adapter while using Scream, other virtual adapters provided by QEMU such as '''e1000e''' may lead to poor performance<br />
}}<br />
<br />
To use scream via your network you will want to find your bridge name via {{ic|1=ip -a}}, in most cases it will be called '''br0''' or '''virbr0'''. Below is a example of the command needed to start the Scream application:<br />
<br />
$ scream -o pulse -i virbr0 &<br />
<br />
{{Warning| This will not work with a '''macvtap bridge''' as that does not allow host to guest communication, also make sure you have the proper firewall ports open for it to communicate with the VM}}<br />
<br />
==== Adding the IVSHMEM device to use Scream with IVSHMEM ====<br />
<br />
With the VM turned off, edit the machine configuration<br />
<br />
{{hc|$ virsh edit ''vmname''|<nowiki><br />
...<br />
<devices><br />
...<br />
<shmem name='scream-ivshmem'><br />
<model type='ivshmem-plain'/><br />
<size unit='M'>2</size><br />
</shmem><br />
</devices><br />
...<br />
</nowiki>}}<br />
<br />
In the above configuration, the size of the IVSHMEM device is 2MB (the recommended amount). Change this as needed.<br />
<br />
Now refer to [[#Adding IVSHMEM Device to VM]] to configure the host to create the shared memory file on boot, replacing {{ic|looking-glass}} with {{ic|scream-ivshmem}}.<br />
<br />
===== Configuring the Windows guest for IVSHMEM =====<br />
<br />
The correct driver must be installed for the IVSHMEM device on the guest. <br />
See [[#Installing the IVSHMEM Host to Windows guest]]. Ignore the part about {{ic|looking-glass-host}}.<br />
<br />
Install the [https://github.com/duncanthrax/scream/releases Scream] virtual audio driver on the guest. <br />
If you have secure boot enabled for your VM, you may need to disable it. <br />
<br />
Using the registry editor, set the DWORD {{ic|HKLM\SYSTEM\CurrentControlSet\Services\Scream\Options\UseIVSHMEM}} to the size of the IVSHMEM device in MB.<br />
Note that scream identifies its IVSHMEM device using its size, so make sure there is only one device of that size.<br />
<br />
====== Configuring the host ======<br />
<br />
Install {{AUR|scream}}.<br />
<br />
Create the systemd user service file to control the reciever<br />
<br />
{{hc|~/.config/systemd/user/scream-ivshmem-pulse.service|<nowiki><br />
[Unit]<br />
Description=Scream IVSHMEM pulse reciever<br />
After=pulseaudio.service<br />
Wants=pulseaudio.service<br />
<br />
[Service]<br />
Type=simple<br />
ExecStartPre=/usr/bin/truncate -s 0 /dev/shm/scream-ivshmem<br />
ExecStartPre=/usr/bin/dd if=/dev/zero of=/dev/shm/scream-ivshmem bs=1M count=2<br />
ExecStart=/usr/bin/scream -m /dev/shm/scream-ivshmem<br />
<br />
[Install]<br />
WantedBy=default.target<br />
</nowiki>}}<br />
<br />
Edit {{ic|1=count=2}} with the size of the IVSHMEM device in MB.<br />
<br />
Now start the service with <br />
$ systemctl start --user scream-ivshmem-pulse<br />
<br />
To have it automatically start on next login, enable the service<br />
$ systemctl enable --user scream-ivshmem-pulse<br />
<br />
=== Physical disk/partition ===<br />
<br />
Raw and qcow2 especially can have noticeable overhead for heavy IO. A whole disk or a partition may be used directly to bypass the filesystem and improve I/O performance. If you wish to dual boot the guest OS natively you would need to pass the entire disk without any partitioning. It is suggested to use /dev/disk/by- paths to refer to the disk since /dev/sdX entries can change between boots. To find out which disk/partition is associated with the one you would like to pass:<br />
<br />
{{hc|$ ls -l /dev/disk/by-id|<br />
ata-ST1000LM002-9VQ14L_Z0501SZ9 -> ../../sdd<br />
}}<br />
<br />
See [[#Virtio disk]] on how to add these with libvirt XML. You can also add the disk with Virt-Manager's '''Add Hardware''' menu and then type the disk you want in the '''Select or create custom storage''' box, e.g. '''/dev/disk/by-id/ata-ST1000LM002-9VQ14L_Z0501SZ9'''<br />
<br />
=== Gotchas ===<br />
<br />
==== Passing through a device that does not support resetting ====<br />
<br />
When the VM shuts down, all devices used by the guest are deinitialized by its OS in preparation for shutdown. In this state, those devices are no longer functional and must then be power-cycled before they can resume normal operation. Linux can handle this power-cycling on its own, but when a device has no known reset methods, it remains in this disabled state and becomes unavailable. Since Libvirt and Qemu both expect all host PCI devices to be ready to reattach to the host before completely stopping the VM, when encountering a device that will not reset, they will hang in a "Shutting down" state where they will not be able to be restarted until the host system has been rebooted. It is therefore recommended to only pass through PCI devices which the kernel is able to reset, as evidenced by the presence of a {{ic|reset}} file in the PCI device sysfs node, such as {{ic|/sys/bus/pci/devices/0000:00:1a.0/reset}}.<br />
<br />
The following bash command shows which devices can and cannot be reset.<br />
<br />
{{hc|<nowiki>for iommu_group in $(find /sys/kernel/iommu_groups/ -maxdepth 1 -mindepth 1 -type d);do echo "IOMMU group $(basename "$iommu_group")"; for device in $(\ls -1 "$iommu_group"/devices/); do if [[ -e "$iommu_group"/devices/"$device"/reset ]]; then echo -n "[RESET]"; fi; echo -n $'\t';lspci -nns "$device"; done; done</nowiki>|<br />
IOMMU group 0<br />
00:00.0 Host bridge [0600]: Intel Corporation Xeon E3-1200 v2/Ivy Bridge DRAM Controller [8086:0158] (rev 09)<br />
IOMMU group 1<br />
00:01.0 PCI bridge [0604]: Intel Corporation Xeon E3-1200 v2/3rd Gen Core processor PCI Express Root Port [8086:0151] (rev 09)<br />
01:00.0 VGA compatible controller [0300]: NVIDIA Corporation GK208 [GeForce GT 720] [10de:1288] (rev a1)<br />
01:00.1 Audio device [0403]: NVIDIA Corporation GK208 HDMI/DP Audio Controller [10de:0e0f] (rev a1)<br />
IOMMU group 2<br />
00:14.0 USB controller [0c03]: Intel Corporation 7 Series/C210 Series Chipset Family USB xHCI Host Controller [8086:1e31] (rev 04)<br />
IOMMU group 4<br />
[RESET] 00:1a.0 USB controller [0c03]: Intel Corporation 7 Series/C210 Series Chipset Family USB Enhanced Host Controller #2 [8086:1e2d] (rev 04)<br />
IOMMU group 5<br />
[RESET] 00:1b.0 Audio device [0403]: Intel Corporation 7 Series/C210 Series Chipset Family High Definition Audio Controller [8086:1e20] (rev 04)<br />
IOMMU group 10<br />
[RESET] 00:1d.0 USB controller [0c03]: Intel Corporation 7 Series/C210 Series Chipset Family USB Enhanced Host Controller #1 [8086:1e26] (rev 04)<br />
IOMMU group 13<br />
06:00.0 VGA compatible controller [0300]: NVIDIA Corporation GM204 [GeForce GTX 970] [10de:13c2] (rev a1)<br />
06:00.1 Audio device [0403]: NVIDIA Corporation GM204 High Definition Audio Controller [10de:0fbb] (rev a1)<br />
}}<br />
<br />
This signals that the xHCI USB controller in 00:14.0 cannot be reset and will therefore stop the VM from shutting down properly, while the integrated sound card in 00:1b.0 and the other two controllers in 00:1a.0 and 00:1d.0 do not share this problem and can be passed without issue.<br />
<br />
== Complete setups and examples ==<br />
<br />
For many reasons users may seek to see [[PCI_passthrough_via_OVMF/Examples|complete passthrough setup examples]].<br />
<br />
These examples offer a supplement to existing hardware compatibility lists. Additionally, if you have trouble configuring a certain mechanism in your setup, you might find these examples very valuable. Users there have described their setups in detail, and some have provided examples of their configuration files as well. <br />
<br />
We encourage those who successfully build their system from this resource to help improve it by contributing their builds. Due to the many different hardware manufacturers involved, the seemingly significant lack of sufficient documentation, as well as other issues due to the nature of this process, community contributions are necessary.<br />
<br />
== Troubleshooting ==<br />
<br />
If your issue is not mentioned below, you may want to browse [[QEMU#Troubleshooting]].<br />
<br />
=== QEMU 4.0: Unable to load graphics drivers/BSOD/Graphics stutter after driver install using Q35 ===<br />
<br />
Starting with QEMU 4.0, the Q35 machine type changes the default {{ic|<nowiki>kernel_irqchip</nowiki>}} from {{ic|off}} to {{ic|split}} which breaks some guest devices, such as nVidia graphics (the driver fails to load / black screen / code 43 / graphics stutters, usually when mouse moving). Switch to full KVM mode instead by adding {{ic|<nowiki><ioapic driver='kvm'/></nowiki>}} under libvirt's {{ic|<nowiki><features></nowiki>}} tag in your VM configuration or by adding {{ic|<nowiki>kernel_irqchip=on</nowiki>}} in the {{ic|-machine}} QEMU arg.<br />
<br />
=== QEMU 5.0: host-passthrough with kernel version 5.5 to 5.8.1 when using Zen 2 processors: Windows 10 BSOD loop 'KERNEL SECURITY CHECK FAILURE' ===<br />
<br />
{{Note|As of kernel version 5.8.2, disabling STIBP is not required anymore.}}<br />
<br />
Starting with QEMU 5.0 virtual machines running on Zen 2 and newer kernels than 5.4 will cause a BSOD loop of: 'KERNEL SECURITY CHECK FAILURE'. This can be fixed by either updating to kernel version 5.8.2 or higher, or disabling STIBP:<br />
<cpu mode='host-passthrough' ...><br />
...<br />
<feature policy='disable' name='amd-stibp'/><br />
...<br />
</cpu><br />
This requires libvirt 6.5 or higher. On older versions, several workarounds exist:<br />
* Switch CPU mode from {{ic|host-passthrough}} to {{ic|host-model}}. This only works on libvirt 6.4 or lower.<br />
* Manually patch {{Pkg|qemu}} in order to revert [https://github.com/qemu/qemu/commit/143c30d4d346831a09e59e9af45afdca0331e819 this] commit.<br />
* On qemu commandline, add {{ic|1=amd-stibp=off}} to the cpu flags string. This can also be invoked through libvirt via a {{ic|<qemu:commandline>}} entry.<br />
<br />
=== "Error 43: Driver failed to load" with mobile (Optimus/max-q) nvidia GPUs ===<br />
This error occurs because the Nvidia driver wants to check the status of the power supply. If no battery is present, the driver does not work. Whether Libvirt or Quemu, by default none of them provide the possibility to simulate a battery.<br />
You can however create and add a custom acpi table file to the virtual machine which will do the work.<br />
<br />
First you have to create the custom acpi table file by pasting the following base64 string [https://base64.guru/converter/decode/file here] and save the result file as SSDT1.dat:<br />
{{bc|<nowiki><br />
U1NEVKEAAAAB9EJPQ0hTAEJYUENTU0RUAQAAAElOVEwYEBkgoA8AFVwuX1NCX1BDSTAGABBMBi5f<br />
U0JfUENJMFuCTwVCQVQwCF9ISUQMQdAMCghfVUlEABQJX1NUQQCkCh8UK19CSUYApBIjDQELcBcL<br />
cBcBC9A5C1gCCywBCjwKPA0ADQANTElPTgANABQSX0JTVACkEgoEAAALcBcL0Dk=<br />
</nowiki>}}<br />
<br />
Next you must add the processed file to the virtual machine:<br />
{{bc|<nowiki><br />
<qemu:commandline><br />
<qemu:arg value="-acpitable"/><br />
<qemu:arg value="file=/path/to/your/SSDT1.dat"/><br />
</qemu:commandline><br />
</nowiki>}}<br />
<br />
[https://www.reddit.com/r/VFIO/comments/ebo2uk/nvidia_geforce_rtx_2060_mobile_success_qemu_ovmf/ Source]<br />
<br />
=== "BAR 3: cannot reserve [mem]" error in dmesg after starting VM ===<br />
<br />
{{Expansion|This error is actually related to the boot_vgs issue and should be merged together with everything else concerning GPU ROMs.|section=UEFI (OVMF) Compatibility in VBIOS}}<br />
<br />
With respect to [https://www.linuxquestions.org/questions/linux-kernel-70/kernel-fails-to-assign-memory-to-pcie-device-4175487043/ this article]:<br />
<br />
If you still have code 43 check dmesg for memory reservation errors after starting up VM, if you have similar it could be the case:<br />
<br />
vfio-pci 0000:09:00.0: BAR 3: cannot reserve [mem 0xf0000000-0xf1ffffff 64bit pref]<br />
<br />
Find out a PCI Bridge your graphic card is connected to. This will give actual hierarchy of devices:<br />
<br />
$ lspci -t<br />
<br />
Before starting VM run following lines replacing IDs with actual from previous output.<br />
<br />
# echo 1 > /sys/bus/pci/devices/0000\:00\:03.1/remove<br />
# echo 1 > /sys/bus/pci/rescan<br />
<br />
{{Note|Probably setting [[kernel parameter]] {{ic|1=video=efifb:off}} is required as well. [https://pve.proxmox.com/wiki/Pci_passthrough#BAR_3:_can.27t_reserve_.5Bmem.5D_error Source]}}<br />
<br />
In addition try adding kernel parameter {{ic|1=pci=realloc}} which also [https://github.com/Dunedan/mbp-2016-linux/issues/60#issuecomment-396311301 helps with hotplugging issues].<br />
<br />
=== UEFI (OVMF) compatibility in VBIOS ===<br />
<br />
{{Remove|Flashing you guest GPU for the purpose of a GPU passthrough is '''never''' good advice. A full section should be dedicated to VBIOS compatibility.|section= UEFI (OVMF) Compatibility in VBIOS}}<br />
<br />
With respect to [https://pve.proxmox.com/wiki/Pci_passthrough#How_to_known_if_card_is_UEFI_.28ovmf.29_compatible this article]:<br />
<br />
Error 43 can be caused by the GPU's VBIOS without UEFI support. To check whenever your VBIOS supports it, you will have to use {{ic|rom-parser}}:<br />
<br />
$ git clone https://github.com/awilliam/rom-parser<br />
$ cd rom-parser && make<br />
<br />
Dump the GPU VBIOS:<br />
<br />
# echo 1 > /sys/bus/pci/devices/0000:01:00.0/rom<br />
# cat /sys/bus/pci/devices/0000:01:00.0/rom > /tmp/image.rom<br />
# echo 0 > /sys/bus/pci/devices/0000:01:00.0/rom<br />
<br />
And test it for compatibility:<br />
<br />
{{hc|$ ./rom-parser /tmp/image.rom|<br />
Valid ROM signature found @600h, PCIR offset 190h<br />
PCIR: type 0 (x86 PC-AT), vendor: 10de, device: 1184, class: 030000<br />
PCIR: revision 0, vendor revision: 1<br />
Valid ROM signature found @fa00h, PCIR offset 1ch<br />
PCIR: type 3 (EFI), vendor: 10de, device: 1184, class: 030000<br />
PCIR: revision 3, vendor revision: 0<br />
EFI: Signature Valid, Subsystem: Boot, Machine: X64<br />
Last image<br />
}}<br />
<br />
To be UEFI compatible, you need a "type 3 (EFI)" in the result. If it is not there, try updating your GPU VBIOS. GPU manufacturers often share VBIOS upgrades on their support pages. A large database of known compatible and working VBIOSes (along with their UEFI compatibility status!) is available on [https://www.techpowerup.com/vgabios/ TechPowerUp].<br />
<br />
Updated VBIOS can be used in the VM without flashing. To load it in QEMU:<br />
<br />
-device vfio-pci,host=07:00.0,......,romfile=/path/to/your/gpu/bios.bin \<br />
<br />
And in libvirt:<br />
<br />
{{bc|1=<br />
<hostdev><br />
...<br />
<rom file='/path/to/your/gpu/bios.bin'/><br />
...<br />
</hostdev><br />
}}<br />
<br />
One should compare VBIOS versions between host and guest systems using [https://www.techpowerup.com/download/nvidia-nvflash/ nvflash] (Linux versions under ''Show more versions'') or <br />
[https://www.techpowerup.com/download/techpowerup-gpu-z/ GPU-Z] (in Windows guest). To check the currently loaded VBIOS:<br />
<br />
{{hc|$ ./nvflash --version|<br />
...<br />
Version : 80.04.XX.00.97<br />
...<br />
UEFI Support : No<br />
UEFI Version : N/A<br />
UEFI Variant Id : N/A ( Unknown )<br />
UEFI Signer(s) : Unsigned<br />
...<br />
}}<br />
<br />
And to check a given VBIOS file:<br />
<br />
{{hc|$ ./nvflash --version NV299MH.rom|<br />
...<br />
Version : 80.04.XX.00.95<br />
...<br />
UEFI Support : Yes<br />
UEFI Version : 0x10022 (Jul 2 2013 @ 16377903 )<br />
UEFI Variant Id : 0x0000000000000004 ( GK1xx )<br />
UEFI Signer(s) : Microsoft Corporation UEFI CA 2011<br />
...<br />
}}<br />
<br />
If the external ROM did not work as it should in the guest, you will have to flash the newer VBIOS image to the GPU. In some cases it is possible to create your own VBIOS image with UEFI support using [https://www.win-raid.com/t892f16-AMD-and-Nvidia-GOP-update-No-requests-DIY.html GOPUpd] tool, however this is risky and may result in GPU brick.<br />
<br />
{{Warning|Failure during flashing may "brick" your GPU - recovery may be possible, but rarely easy and often requires additional hardware. '''DO NOT''' flash VBIOS images for other GPU models (different boards may use different VBIOSes, clocks, fan configuration). If it breaks, you get to keep all the pieces.}}<br />
<br />
In order to avoid the irreparable damage to your graphics adapter it is necessary to unload the NVIDIA kernel driver first:<br />
<br />
# modprobe -r nvidia_modeset nvidia <br />
<br />
Flashing the VBIOS can be done with:<br />
<br />
# ./nvflash romfile.bin<br />
<br />
{{Warning|'''DO NOT''' interrupt the flashing process, even if it looks like it is stuck. Flashing should take about a minute on most GPUs, but may take longer.}}<br />
<br />
=== Slowed down audio pumped through HDMI on the video card ===<br />
<br />
For some users VM's audio slows down/starts stuttering/becomes demonic after a while when it is pumped through HDMI on the video card. This usually also slows down graphics.<br />
A possible solution consists of enabling MSI (Message Signaled-Based Interrupts) instead of the default (Line-Based Interrupts).<br />
<br />
In order to check whether MSI is supported or enabled, run the following command as root:<br />
<br />
# lspci -vs $device | grep 'MSI:'<br />
<br />
where `$device` is the card's address (e.g. `01:00.0`).<br />
<br />
The output should be similar to:<br />
<br />
Capabilities: [60] MSI: Enable'''-''' Count=1/1 Maskable- 64bit+<br />
<br />
A {{ic|-}} after {{ic|Enable}} means MSI is supported, but not used by the VM, while a {{ic|+}} says that the VM is using it.<br />
<br />
The procedure to enable it is quite complex, instructions and an overview of the setting can be found [https://forums.guru3d.com/showthread.php?t=378044 here].<br />
<br />
On a linux guest you can use modinfo to see if there is option to enable MSI (for example: "modinfo snd_hda_intel |grep msi"). If there is, one can enable it by adding the relevant option to a custom omdprobe file - in "/etc/modprobe.d/snd-hda-intel.conf" inserting "options snd-hda-intel enable_msi=1"<br />
<br />
Other hints can be found on the [https://lime-technology.com/wiki/index.php/UnRAID_6/VM_Guest_Support#Enable_MSI_for_Interrupts_to_Fix_HDMI_Audio_Support lime-technology's wiki], or on this article on [https://vfio.blogspot.it/2014/09/vfio-interrupts-and-how-to-coax-windows.html VFIO tips and tricks].<br />
<br />
A UI tool called [https://github.com/CHEF-KOCH/MSI-utility MSI Utility (FOSS Version 2)] works with Windows 10 64-bit and simplifies the process.<br />
<br />
In order to fix the issues enabling MSI on the 0 function of a nVidia card ({{ic|01:00.0 VGA compatible controller: NVIDIA Corporation GM206 [GeForce GTX 960] (rev a1) (prog-if 00 [VGA controller])}}) was not enough; it will also be required to enable it on the other function ({{ic|01:00.1 Audio device: NVIDIA Corporation Device 0fba (rev a1)}}) to fix the issue.<br />
<br />
=== No HDMI audio output on host when intel_iommu is enabled ===<br />
<br />
If after enabling {{ic|intel_iommu}} the HDMI output device of Intel GPU becomes unusable on the host then setting the option {{ic|igfx_off}} (i.e. {{ic|1=intel_iommu=on,igfx_off}}) might bring the audio back, please read [https://www.kernel.org/doc/html/latest/x86/intel-iommu.html#graphics-problems intel-iommu.html] for details about setting {{ic|igfx_off}}.<br />
<br />
=== X does not start after enabling vfio_pci ===<br />
<br />
This is related to the host GPU being detected as a secondary GPU, which causes X to fail/crash when it tries to load a driver for the guest GPU. To circumvent this, a Xorg configuration file specifying the BusID for the host GPU is required. The correct BusID can be acquired from lspci or the Xorg log. [https://www.redhat.com/archives/vfio-users/2016-August/msg00025.html Source →]<br />
<br />
{{hc|/etc/X11/xorg.conf.d/10-intel.conf|<nowiki><br />
Section "Device"<br />
Identifier "Intel GPU"<br />
Driver "modesetting"<br />
BusID "PCI:0:2:0"<br />
EndSection<br />
</nowiki>}}<br />
<br />
=== Chromium ignores integrated graphics for acceleration ===<br />
<br />
Chromium and friends will try to detect as many GPUs as they can in the system and pick which one is preferred (usually discrete NVIDIA/AMD graphics). It tries to pick a GPU by looking at PCI devices, not OpenGL renderers available in the system - the result is that Chromium may ignore the integrated GPU available for rendering and try to use the dedicated GPU bound to the {{ic|vfio-pci}} driver, and unusable on the host system, regardless of whenever a guest VM is running or not. This results in software rendering being used (leading to higher CPU load, which may also result in choppy video playback, scrolling and general un-smoothness).<br />
<br />
This can be fixed by [[Chromium/Tips and tricks#Forcing specific GPU|explicitly telling Chromium which GPU you want to use]].<br />
<br />
=== VM only uses one core ===<br />
<br />
For some users, even if IOMMU is enabled and the core count is set to more than 1, the VM still only uses one CPU core and thread. To solve this enable "Manually set CPU topology" in {{ic|virt-manager}} and set it to the desirable amount of CPU sockets, cores and threads. Keep in mind that "Threads" refers to the thread count per CPU, not the total count.<br />
<br />
=== Passthrough seems to work but no output is displayed ===<br />
<br />
Make sure if you are using virt-manager that UEFI firmware is selected for your virtual machine. Also, make sure you have passed the correct device to the VM.<br />
<br />
=== Host lockup after VM shutdown ===<br />
<br />
This issue seems to primarily affect users running a Windows 10 guest and usually after the VM has been run for a prolonged period of time: the host will experience multiple CPU core lockups (see [https://bbs.archlinux.org/viewtopic.php?id=206050&p=2]). To fix this try enabling Message Signal Interrupts on the GPU passed through to the guest. A good guide for how to do this can be found in [https://forums.guru3d.com/threads/windows-line-based-vs-message-signaled-based-interrupts.378044/]. You can also download this application for windows here [https://github.com/TechtonicSoftware/MSIInturruptEnabler] that should make the process easier.<br />
<br />
=== Host lockup if guest is left running during sleep ===<br />
<br />
VFIO-enabled virtual machines tend to become unstable if left running through a sleep/wakeup cycle and have been known to cause the host machine to lockup when an attempt is then made to shut them down. In order to avoid this, one can simply prevent the host from going into sleep while the guest is running using the following libvirt hook script and systemd unit. The hook file needs executable permissions to work.<br />
<br />
{{hc|/etc/libvirt/hooks/qemu|<nowiki><br />
#!/bin/bash<br />
<br />
OBJECT="$1"<br />
OPERATION="$2"<br />
SUBOPERATION="$3"<br />
EXTRA_ARG="$4"<br />
<br />
case "$OPERATION" in<br />
"prepare")<br />
systemctl start libvirt-nosleep@"$OBJECT"<br />
;;<br />
"release")<br />
systemctl stop libvirt-nosleep@"$OBJECT"<br />
;;<br />
esac<br />
</nowiki>}}<br />
<br />
{{hc|/etc/systemd/system/libvirt-nosleep@.service|<nowiki><br />
[Unit]<br />
Description=Preventing sleep while libvirt domain "%i" is running<br />
<br />
[Service]<br />
Type=simple<br />
ExecStart=/usr/bin/systemd-inhibit --what=sleep --why="Libvirt domain \"%i\" is running" --who=%U --mode=block sleep infinity<br />
</nowiki>}}<br />
<br />
=== Cannot boot after upgrading ovmf ===<br />
<br />
If you cannot boot after upgrading from {{Pkg|ovmf}}{{Broken package link|replaced by {{Pkg|edk2-ovmf}}}} version 1:r23112.018432f0ce-1 then you need to remove the old {{ic|*VARS.fd}} file in {{ic|/var/lib/libvirt/qemu/nvram/}}:<br />
<br />
# mv /var/lib/libvirt/qemu/nvram/vmname_VARS.fd /var/lib/libvirt/qemu/nvram/vmname_VARS.fd.old<br />
<br />
See {{Bug|57825}} for further details.<br />
<br />
=== QEMU via cli pulseaudio stuttering/delay ===<br />
<br />
Using following flags for the audio device and chipset might help if you are running into the stuttering/delay audio issues when running QEMU via cli:<br />
<br />
qemu-system-x86_64 \<br />
-machine pc-i440fx-3.0 \<br />
-device hda-micro \<br />
-soundhw hda \<br />
-...<br />
<br />
As noted in [[#QEMU 3.0 audio changes|QEMU 3.0 audio changes]]{{Broken section link}} the specified chipset will include a series of audio patches.<br />
<br />
Setting {{ic|QEMU_AUDIO_TIMER_PERIOD}} to values higher than 100 might also help (did not test value lower than 100).<br />
<br />
=== Bluescreen at boot since Windows 10 1803 ===<br />
<br />
Since Windows 10 1803 there is a problem when you are using "host-passthrough" as cpu model. The machine cannot boot and is either boot looping or you get a bluescreen.<br />
You can workaround this by:<br />
<br />
# echo 1 > /sys/module/kvm/parameters/ignore_msrs<br />
<br />
To make it permanently you can create a modprobe file {{ic|kvm.conf}}:<br />
<br />
options kvm ignore_msrs=1<br />
<br />
To prevent clogging up dmesg with "ignored rdmsr" messages you can additionally add:<br />
<br />
options kvm report_ignored_msrs=0<br />
<br />
=== AMD Ryzen / BIOS updates (AGESA) yields "Error: internal error: Unknown PCI header type ‘127’" ===<br />
<br />
AMD users have been experiencing breakage of their KVM setups after updating the BIOS on their motherboard. There is a kernel [https://clbin.com/VCiYJ patch], (see [[Kernel/Arch Build System]] for instruction on compiling kernels with custom patches) that can resolve the issue as of now (7/28/19), but this is not the first time AMD has made an error of this very nature, so take this into account if you are considering updating your BIOS in the future as a VFIO user.<br />
<br />
=== AMD GPU not resetting properly yielding "Error: internal error: Unknown PCI header type ‘127’" (Separate issue from the one above) ===<br />
Passing through an AMD GPU may result into a problem known as the "AMD reset bug". Upon power cycling the guest, the GPU does not properly reset its state which causes the device to malfunction until the host is also rebooted. This is usually paired with a "code 43" driver error in a Windows guest, and the message "Error: internal error: Unknown PCI header type '127'" in the libvirt log on the host.<br />
<br />
In the past, this meant having to use work-arounds to manually reset the GPU, or resorting to the use of kernel patches that were unlikely to land in upstream. Currently, the recommended solution that does not require patching of the kernel is to install {{AUR|vendor-reset-git}} or {{AUR|vendor-reset-dkms-git}} and making sure the 'vendor-reset' kernel module is loaded before booting the guest.<br />
<br />
{{Note| Make sure you do not have any of the AMD reset bug kernel patches installed if you are using {{AUR|vendor-reset-git}} or {{AUR|vendor-reset-dkms-git}}.}}<br />
<br />
=== Host crashes when hotplugging Nvidia card with USB ===<br />
<br />
If attempting to hotplug an Nvidia card with a USB port, you may have to blacklist the {{ic|i2c_nvidia_gpu}} driver. Do this by adding the line {{ic|blacklist i2c_nvidia_gpu}} to {{ic|/etc/modprobe.d/blacklist.conf}}.<br />
=== Host unable to boot and stuck in black screen after enabling vfio ===<br />
<br />
If debug kernel messages during boot are enabled by adding with the {{ic|debug ignore_loglevel}} [[kernel parameters]], you may see boot stuck with the last message similar to<br />
<br />
vfio-pci 0000:01:00.0: vgaarb: changed VGA decodes: olddecodes=io+mem,decodes=io+mem:owns=none<br />
<br />
This can be resolved by disconnecting the passthroughed GPU with your moniter. You may reconnect the passthroughed GPU to a moniter after host is booted.<br />
<br />
=== AER errors when passing through PCIe USB hub ===<br />
<br />
In some cases passing through a PCIe USB hub, such as one connected to the guest GPU, might fail with AER errors similar to the following:<br />
<br />
kernel: pcieport 0000:00:01.1: AER: Uncorrected (Non-Fatal) error received: 0000:00:01.1<br />
kernel: pcieport 0000:00:01.1: AER: PCIe Bus Error: severity=Uncorrected (Non-Fatal), type=Transaction Layer, (Requester ID)<br />
kernel: pcieport 0000:00:01.1: AER: device [8086:1905] error status/mask=00100000/00000000<br />
kernel: pcieport 0000:00:01.1: AER: [20] UnsupReq (First)<br />
kernel: pcieport 0000:00:01.1: AER: TLP Header: 00000000 00000000 00000000 00000000<br />
kernel: pcieport 0000:00:01.1: AER: device recovery successful<br />
<br />
=== Reserved Memory Region Reporting (RMRR) Conflict ===<br />
<br />
If you run into an issue passing through a device because of the BIOS's usage of RMRR, like the error below.<br />
<br />
vfio-pci 0000:01:00.1: Device is ineligible for IOMMU domain attach due to platform RMRR requirement. Contact your platform vendor.<br />
<br />
You can try the patches here: https://github.com/kiler129/relax-intel-rmrr<br />
<br />
== See also ==<br />
<br />
* [https://bbs.archlinux.org/viewtopic.php?id=162768 Discussion on Arch Linux forums] | [https://archive.is/kZYMt Archived link]<br />
* [https://docs.google.com/spreadsheet/ccc?key=0Aryg5nO-kBebdFozaW9tUWdVd2VHM0lvck95TUlpMlE User contributed hardware compatibility list]<br />
* [https://pastebin.com/rcnUZCv7 Example script] from https://www.youtube.com/watch?v=37D2bRsthfI<br />
* [https://vfio.blogspot.com/ Complete tutorial for PCI passthrough]<br />
* [https://www.redhat.com/archives/vfio-users/ VFIO users mailing list]<br />
* [ircs://chat.freenode.net/vfio-users #vfio-users on freenode]<br />
* [https://www.youtube.com/watch?v=aLeWg11ZBn0 YouTube: Level1Linux - GPU Passthrough for Virtualization with Ryzen]<br />
* [https://www.reddit.com/r/VFIO /r/VFIO: A subreddit focused on vfio]<br />
* [https://github.com/intel/gvt-linux/wiki/GVTd_Setup_Guide GVT-d: passthrough of an entire integrated GPU]</div>Nicman23https://wiki.archlinux.org/index.php?title=PulseAudio&diff=648640PulseAudio2021-01-10T16:26:28Z<p>Nicman23: bad formatting of the previous edit</p>
<hr />
<div>[[Category:Sound]]<br />
[[cs:PulseAudio]]<br />
[[de:Pulseaudio]]<br />
[[es:PulseAudio]]<br />
[[fr:PulseAudio]]<br />
[[it:PulseAudio]]<br />
[[ja:PulseAudio]]<br />
[[pt:PulseAudio]]<br />
[[ru:PulseAudio]]<br />
[[zh-hans:PulseAudio]]<br />
{{Related articles start}}<br />
{{Related|PulseAudio/Examples}}<br />
{{Related|PulseAudio/Troubleshooting}}<br />
{{Related articles end}}<br />
[[Wikipedia:PulseAudio|PulseAudio]] is a general purpose sound server intended to run as a middleware between your applications and your hardware devices, either using [[ALSA]] or [[OSS]]. It also offers easy network streaming across local devices using [[Avahi]] if enabled. While its main purpose is to ease audio configuration, its modular design allows more advanced users to configure the daemon precisely to best suit their needs.<br />
<br />
{{Note|Some confusion may occur between [[ALSA]] and PulseAudio. ALSA includes a Linux kernel component with sound card drivers, as well as a userspace component, {{ic|libasound}}.[http://www.alsa-project.org/main/index.php/Download] PulseAudio builds only on the kernel component, but offers compatibility with {{ic|libasound}} through {{Pkg|pulseaudio-alsa}}.[https://www.freedesktop.org/wiki/Software/PulseAudio/FAQ/#index14h3]}}<br />
<br />
== Installation ==<br />
<br />
[[Install]] the {{Pkg|pulseaudio}} package.<br />
<br />
Some PulseAudio modules are not included in the main package and must be installed separately if needed:<br />
<br />
* {{Pkg|pulseaudio-alsa}} for PulseAudio to manage [[ALSA]] as well, see [[#ALSA]]<br />
* {{Pkg|pulseaudio-bluetooth}} for [[bluetooth]] support (Bluez), see [[bluetooth headset]] page<br />
* {{Pkg|pulseaudio-equalizer}} for equalizer sink (qpaeq)<br />
* {{Pkg|pulseaudio-jack}} for [[JACK]] sink, source and jackdbus detection<br />
* {{Pkg|pulseaudio-lirc}} for infrared volume control with [[LIRC]]<br />
* {{Pkg|pulseaudio-zeroconf}} for Zeroconf ([[Avahi]]/DNS-SD) support<br />
<br />
=== Front-ends ===<br />
<br />
There are a number of front-ends available for controlling the PulseAudio daemon:<br />
<br />
==== Console ====<br />
<br />
* {{App|ncpamixer|Ncurses mixer for PulseAudio inspired by pavucontrol.|https://github.com/fulhax/ncpamixer|{{AUR|ncpamixer}}}}<br />
* {{App|pacmixer|Alsamixer alike for PulseAudio.|https://github.com/KenjiTakahashi/pacmixer|{{AUR|pacmixer}}}}<br />
* {{App|PAmix|Ncurses PulseAudio mixer similar to pavucontrol.|https://github.com/patroclos/PAmix|{{AUR|pamix-git}}}}<br />
* {{App|pamixer|PulseAudio command line mixer.|https://github.com/cdemoulins/pamixer|{{Pkg|pamixer}}}}<br />
* {{App|pavolume|Simple command-line volume control for PulseAudio with libnotify messages.|https://github.com/sseemayer/pavolume|{{AUR|pavolume-git}}}}<br />
* {{App|Ponymix|Command line mixer for PulseAudio.|https://github.com/falconindy/ponymix|{{Pkg|ponymix}}}}<br />
* {{App|pulseaudio-ctl|Control PulseAudio volume from the shell or mapped to keyboard shortcuts.|https://github.com/graysky2/pulseaudio-ctl|{{AUR|pulseaudio-ctl}}}}<br />
* {{App|pulsemixer|CLI and curses mixer for PulseAudio|https://github.com/GeorgeFilipkin/pulsemixer|{{Pkg|pulsemixer}}}}<br />
<br />
==== Graphical ====<br />
<br />
* {{App|KMix|[[KDE]] volume control application supporting several platforms including PulseAudio, system tray applet configurable.|https://kde.org/applications/en/multimedia/org.kde.kmix|{{pkg|kmix}}}}<br />
* {{App|MicTray|Lightweight system tray application which lets you control the microphone state and volume using PulseAudio.|https://github.com/Junker/MicTray|{{AUR|mictray}}}}<br />
* {{App|pa-applet|System tray applet for PulseAudio with volume bar.|https://github.com/fernandotcl/pa-applet|{{AUR|pa-applet-git}}}}<br />
* {{App|pasystray|System tray applet for PulseAudio.|https://github.com/christophgysin/pasystray|{{Pkg|pasystray}}}}<br />
* {{App|plasma-pa|[[KDE]] Plasma applet for audio volume management using PulseAudio|https://invent.kde.org/plasma/plasma-pa|{{Pkg|plasma-pa}}}}<br />
* {{App|PulseAudio Equalizer|LADSPA based multiband equalizer for PulseAudio.|https://github.com/pulseaudio-equalizer-ladspa/equalizer|{{Pkg|pulseaudio-equalizer-ladspa}}}}<br />
* {{App|PulseAudio Graph Control|Electron-based volume and graph control for PulseAudio.|https://github.com/futpib/pagraphcontrol#readme|{{AUR|pagraphcontrol-git}}}}<br />
* {{App|PulseAudio Manager|Simple GTK frontend for PulseAudio.|http://0pointer.de/lennart/projects/paman/|{{AUR|paman}}}}<br />
* {{App|PulseAudio Preferences|Simple GTK configuration dialog for PulseAudio.|https://freedesktop.org/software/pulseaudio/paprefs/|{{Pkg|paprefs}}}}<br />
* {{App|PulseAudio Volume Control|Simple GTK volume control tool ("mixer") for PulseAudio.|https://freedesktop.org/software/pulseaudio/pavucontrol/|{{Pkg|pavucontrol}}}}<br />
* {{App|PulseAudio Volume Control (Qt)|Mixer for PulseAudio (Qt port of pavucontrol).|https://github.com/lxqt/pavucontrol-qt|{{Pkg|pavucontrol-qt}}}}<br />
* {{App|PulseAudio Volume Control (Sandsmark)|Lightweight fork of the LXQt's pavucontrol, with missing features from pavucontrol implemented, bug fixes and unnecessary dependencies removed.|https://github.com/sandsmark/pavucontrol-qt|{{AUR|pavucontrol-qt-sandsmark-git}}}}<br />
* {{App|PulseAudio Volume Meter|Simple GTK volume meter for PulseAudio.|http://0pointer.de/lennart/projects/pavumeter/|{{AUR|pavumeter}}}}<br />
* {{App|PulseEffects|Audio effects for PulseAudio applications.|https://github.com/wwmm/pulseeffects|{{Pkg|pulseeffects}}}}<br />
* {{App|Volctl|Per-application system tray applet volume control for PulseAudio.|https://buzz.github.io/volctl/|{{AUR|volctl}}}}<br />
* {{App|Xfce PulseAudio Panel Plugin|PulseAudio plugin for [[Xfce]]4 panel.|https://goodies.xfce.org/projects/panel-plugins/xfce4-pulseaudio-plugin|{{Pkg|xfce4-pulseaudio-plugin}}}}<br />
<br />
== Configuration ==<br />
{{Merge|PulseAudio/Configuration|Configuration should stay in the main article, so the linked page should be merged here.|section=Abandoned draft}}<br />
<br />
By default, PulseAudio is configured to automatically detect all sound cards and manage them. It takes control of all detected ALSA devices and redirects all audio streams to itself, making the PulseAudio daemon the central configuration point. The daemon should work mostly out of the box, only requiring a few minor tweaks. <br />
<br />
While PulseAudio usually runs fine out of the box and requires only minimal configuration, advanced users can change almost every aspect of the daemon by either altering the default configuration file to disable modules or writing your own from scratch.<br />
<br />
PulseAudio runs as a server daemon that can run either system-wide or on per-user basis using a client/server architecture. The daemon by itself does nothing without its '''modules''' except to provide an API and host dynamically loaded modules. The audio routing and processing tasks are all handled by various modules, including PulseAudio's native protocol itself (provided by [https://www.freedesktop.org/wiki/Software/PulseAudio/Documentation/User/Modules/#index22h3 module-native-protocol-unix]). Clients reach the server through one of many protocol modules that will accept audio from external sources, route it through PulseAudio and eventually have it go out through a final other module. The output module does not have to be an actual sound output: it can dump the stream into a file, stream it to a broadcasting server such as [[Icecast]], or even just discard it.<br />
<br />
You can find a detailed list of all available modules at [https://www.freedesktop.org/wiki/Software/PulseAudio/Documentation/User/Modules/ Pulseaudio Loadable Modules]. To enable them you can just add a line {{ic|load-module <module-name-from-list>}} to {{ic|~/.config/pulse/default.pa}}.<br />
<br />
=== Configuration files ===<br />
<br />
PulseAudio will first look for configuration files in the home directory {{ic|~/.config/pulse}}, and if they are not found, the system-wide configuration from {{ic|/etc/pulse}} will be applied.<br />
<br />
{{Tip|<br />
* It is strongly suggested not to edit system-wide configuration files, but rather edit user ones. Create the {{ic|~/.config/pulse}} directory, then copy the system configuration files into it and edit according to your need.<br />
* Make sure you keep user configuration in sync with changes to the packaged files in {{ic|/etc/pulse/}}. Otherwise, PulseAudio may refuse to start due to configuration errors.<br />
* There is usually no need to add your user to the {{ic|audio}} group, as PulseAudio uses [[udev]] and ''logind'' to give access dynamically to the currently "active" user. Exceptions would include running the machine headless so that there is no currently "active" user.}}<br />
<br />
==== daemon.conf ====<br />
<br />
This is the main configuration file to configure the daemon itself. It defines base settings like the default sample rates used by modules, resampling methods, realtime scheduling and various other settings related to the server process. These can not be changed at runtime without restarting the PulseAudio daemon. The defaults are sensible for most users, see the {{man|5|pulse-daemon.conf}} manpage for additional information. Boolean options accepts any of these: {{ic|true}}, {{ic|yes}}, {{ic|on}} and {{ic|1}} as well as {{ic|false}}, {{ic|no}}, {{ic|off}} and {{ic|0}}.<br />
<br />
{{Note|PulseAudio does not perform tilde expansion on paths in this file. Use absolute paths for any files.}}<br />
<br />
{| class="wikitable"<br />
! Option || Description<br />
|+<br />
| daemonize || Controls whether the server will daemonize itself and return. Set to {{ic|no}} when debugging so you can see the debugging information on the terminal.<br />
|+<br />
| resample-method || Which resampler to use when audio with incompatible sample rates needs to be passed between modules (e.g. playback of 96kHz audio on hardware which only supports 48kHz). The available resamplers can be listed with {{ic|$ pulseaudio --dump-resample-methods}}. Choose the best tradeoff between CPU usage and audio quality for the present use-case. {{Tip|In some cases PulseAudio will generate a high CPU load. This can happen when multiple streams are resampled (individually). If this is a common use-case in a workflow, it should be considered to create an additional sink at a matching sample rate which can then be fed into the main sink, resampling only once.}}<br />
|+<br />
| avoid-resampling || With {{ic|1=avoid-resampling = yes}}, PulseAudio automatically configures the hardware to the sample rate which the application uses, if the hardware supports this sample rate (needs [https://www.freedesktop.org/wiki/Software/PulseAudio/Notes/11.0/ PA 11] or higher)<br />
{{Warning|Enabling this feature might cause audio distortion, therefore it is disabled by default, see the [https://www.freedesktop.org/wiki/Software/PulseAudio/Notes/11.0/ release notes] for more information.}}<br />
|+<br />
| enable-remixing || When the input and output have a different channel count (for example, outputting a 6 channel movie into a stereo sink), pulse can either remix all the channels (default, {{ic|yes}}) or just trivially map the channels by their name (left goes to left, right to right, all others ignored) when {{ic|no}}<br />
|+<br />
| system-instance || If set to {{ic|yes}}, run the daemon as a [https://www.freedesktop.org/wiki/Software/PulseAudio/Documentation/User/SystemWide/ system-wide] instance. [https://www.freedesktop.org/wiki/Software/PulseAudio/Documentation/User/WhatIsWrongWithSystemWide/ Highly discouraged] as it can introduce security issues. Useful on [[Xorg multiseat|Multiseat]] systems, or headless systems that have no real local users. Defaults to {{ic|no}}.<br />
|+<br />
| flat-volumes ||{{ic|flat-volumes}} scales the device-volume with the volume of the "loudest" application. For example, raising the VoIP call volume will raise the hardware volume and adjust the music-player volume so it stays where it was, without having to lower the volume of the music-player manually. Defaults to {{ic|yes}} upstream, but to {{ic|no}} within Arch. {{Note|The default behavior upstream can sometimes be confusing and some applications, unaware of this feature, can set their volume to 100% at startup, potentially blowing your speakers or your ears. This is why Arch defaults to the classic (ALSA) behavior by setting this to {{ic|no}}.}}<br />
|+<br />
| realtime-scheduling || If your [[kernel]] supports realtime scheduling (for instance, [[Realtime kernel]] or [[Linux-ck]]), set this to {{ic|yes}} to ensure PulseAudio can deliver low-latency glitch-free playback. You can adjust {{ic|realtime-priority}} as well to have it use the correct priority, especially when [[JACK]] is also running on the system.<br />
|+<br />
| nice-level || Since PulseAudio runs in userspace and involves inter-process communication, audio can be subject to dropouts if the daemon does not have enough CPU time to process the audio. The default usually is enough, but can be tweaked to give pulse the wanted priority over (or below) other applications.<br />
|+<br />
| exit-idle-time || If you want to run PulseAudio only when needed and use ALSA otherwise, you can set a delay in seconds after which the daemon will automatically shutdown after all clients are disconnected. Set it to -1 to disable this feature.<br />
|+<br />
| log-level || When debugging, you may want to increase the logging level of the daemon to see exactly why a specific module fails to load. High logging levels will sometimes print useful information such as detected minimum latency for the system, which can then be used to tweak {{ic|default-fragments}} and {{ic|default-fragment-size-msec}}.<br />
|+<br />
| default-sample-format || This usually does not need to be changed, but if your sound card's native format is different, performance and quality can be improved by setting the right format here.<br />
|+<br />
| default-sample-rate || The default sample rate user by pulse unless overriden at module level. Change this if your sound card does not support 44100Hz or if you wish to upsample all audio. See previous note about CPU usage.<br />
|+<br />
| alternate-sample-rate || To fix a common limitation where movies at 48000Hz were needlessly downsampled to 44100Hz, some modules support changing their sample rate dynamically to avoid resampling when possible. See manual for more in-depth information. This usually does not need to be changed.<br />
|+<br />
| default-channels || The default number of channels when not specified. Usually do not need any change as you can configure more channels on per-module basis.<br />
|+<br />
| default-fragments || Audio samples are split into multiple fragments of {{ic|default-fragment-size-msec}} each. The larger the buffer is, the less likely audio will skip when the system is overloaded. On the downside this will increase the overall latency. Increase this value if you have issues.<br />
|+<br />
| default-fragment-size-msec || The size in milliseconds of each fragment. This is the amount of data that will be processed at once by the daemon.<br />
|}<br />
<br />
==== default.pa ====<br />
<br />
This file is a startup script and is used to configure modules. It is actually parsed and read after the daemon has finished initializing and additional commands can be sent at runtime using {{ic|$ pactl}} or {{ic|$ pacmd}}. The startup script can also be provided on the command line by starting PulseAudio in a terminal using {{ic|$ pulseaudio -nC}}. This will make the daemon load the CLI module and will accept the configuration directly from the command line, and output resulting information or error messages on the same terminal. This can be useful when debugging the daemon or just to test various modules before setting them permanently on disk. The manual page is quite self-explanatory, consult {{man|5|pulse-cli-syntax}} for the details of the syntax.<br />
<br />
{{tip|<br />
* Rather than being a complete copy, {{ic|~/.config/pulse/default.pa}} can start with the line {{ic|.include /etc/pulse/default.pa}} and then just override the defaults.<br />
* Run {{ic|<nowiki>$ pacmd list-sinks|egrep -i 'index:|name:'</nowiki>}} to list available sinks. The present default sink is marked with an asterisk.<br />
* Edit {{ic|~/.config/pulse/default.pa}} to insert/alter the set-default-sink command using the sink's name as the numbering cannot be guaranteed repeatable.<br />
}}<br />
<br />
==== client.conf ====<br />
This is the configuration file read by every PulseAudio client application. It is used to configure runtime options for individual clients. It can be used to set and configure the default sink and source statically as well as allowing (or disallowing) clients to automatically start the server if not currently running.<br />
<br />
=== Configuration command ===<br />
<br />
The main command to configure a server during runtime is {{ic|$ pacmd}}. Run {{ic|$ pacmd --help}} for a list options, or just run {{ic|$ pacmd}} to enter the shell interactive mode and {{ic|Ctrl+d}} to exit. All modifications will immediately be applied.<br />
<br />
Once your new settings have been tested and meet your needs, edit the {{ic|default.pa}} accordingly to make the change persistent. See [[PulseAudio/Examples]] for some basic settings.<br />
<br />
{{Tip|leave the {{ic|load-module module-default-device-restore}} line in the {{ic|default.pa}} file untouched. It will allow you to restart the server in its default state, thus dismissing any wrong setting.}}<br />
<br />
It is important to understand that the "sources" (processes, capture devices) and "sinks" (sound cards, servers, other processes) accessible and selectable through PulseAudio depend upon the current hardware "Profile" selected. These "Profiles" are those ALSA "pcms" listed by the command {{ic|aplay -L}}, and more specifically by the command {{ic|pacmd list-cards}}, which will include a line "index:", a list beginning "profiles:", and a line "active profile: <...>" in the output, among other things. "Profiles" correspond to different card input/output configurations, notably the number of available input/output channels.<br />
<br />
The "active profile" can be set with the command {{ic|pacmd set-card-profile INDEX PROFILE}}, with ''no'' comma separating INDEX and PROFILE, where INDEX is just the number on the line "index:" and a PROFILE name is everything shown from the beginning of any line under "profile:" to just ''before'' the colon and first space, as shown by the command {{ic|pacmd list-cards}}. For instance, {{ic|pacmd set-card-profile 0 output:analog-stereo+input:analog-stereo}}.<br />
<br />
It may be easier to select a "Profile" with a graphical tool like {{ic|pavucontrol}}, under the "Configuration" tab, or KDE System Settings, "Multimedia/Audio and Video Settings", under the "Audio Hardware Setup" tab. Each audio "Card", which are those devices listed by the command {{ic|aplay -l}}, or again by the command {{ic|pacmd list-cards}}, will have its own selectable "Profile". When a "Profile" has been selected, the then available "sources" and "sinks" can be seen by using the commands {{ic|pacmd list-sources}} and {{ic|pacmd list-sinks}}. Note that the "index" of the available sources and sinks will change each time a card profile is changed.<br />
<br />
The selected "Profile" can be an issue for some applications, especially the Adobe Flash players, typically {{ic|/usr/lib/mozilla/plugins/libflashplayer.so}} and {{ic|/usr/lib/PepperFlash/libpepflashplayer.so}}. Often, these Flash players will only work when one of the Stereo profiles is selected, and otherwise, will play video with no sound, or will simply "crash". When all else fails, you might try selecting a different profile.<br />
<br />
Of course, when configuring some variation of Surround Sound in PulseAudio, the appropriate Surround profile will have to be selected, before Surround Sound will work, or in order to do things like remap the speaker channels.<br />
<br />
== Running ==<br />
<br />
PulseAudio on Arch has {{ic|pulseaudio.socket}} enabled by default for the [[systemd/User]] instance. This means that PulseAudio will automatically start when needed.<br />
<br />
{{Note|<br />
* To disable {{ic|pulseaudio.socket}}, make sure that {{ic|$XDG_CONFIG_HOME/systemd/user/}} exists and run {{ic|systemctl --user mask pulseaudio.socket}}.<br />
* Many [[desktop environments]] support [[XDG Autostart]]. In those desktop environments, PulseAudio will be launched automatically regardless of the socket activation status.<br />
}}<br />
<br />
For more information, see [https://www.freedesktop.org/wiki/Software/PulseAudio/Documentation/User/Running/ PulseAudio: Running].<br />
<br />
== Stopping ==<br />
$ systemctl --user stop pulseaudio.socket<br />
$ systemctl --user stop pulseaudio.service<br />
<br />
== Back-end configuration ==<br />
<br />
=== ALSA ===<br />
<br />
If you have applications that do not support PulseAudio explicitly but rely on ALSA, these applications will try to access the sound card directly via ALSA and will therefore bypass PulseAudio. PulseAudio will thus not have access to the sound card any more. As a result, all applications relying on PulseAudio will not be working any more, leading to [[PulseAudio/Troubleshooting#The only device shown is "dummy output" or newly connected cards are not detected|this issue]]. To prevent this, you will need to install the {{Pkg|pulseaudio-alsa}} package. It contains the necessary {{ic|/etc/asound.conf}} for configuring ALSA to use PulseAudio. Also make sure that {{ic|~/.asoundrc}} does not exist, as it would override the {{ic|/etc/asound.conf}} file.<br />
<br />
Also install {{Pkg|lib32-libpulse}} and {{Pkg|lib32-alsa-plugins}} if you run a x86_64 system and want to have sound for 32-bit [[multilib]] programs like [[Wine]] and [[Steam]].<br />
<br />
To prevent applications from using ALSA's OSS emulation and bypassing PulseAudio (thereby preventing other applications from playing sound), make sure the module {{ic|snd_pcm_oss}} is not being loaded at boot. If it is currently loaded ({{ic|<nowiki>lsmod | grep oss</nowiki>}}), disable it by executing:<br />
# rmmod snd_pcm_oss<br />
<br />
==== Enable DTS via ALSA ====<br />
<br />
To enable PulseAudio DTS (Digital Theater System) via ALSA install {{aur|dcaenc}} package and enable it:<br />
<br />
{{hc|/etc/asound.conf|2=<br />
<confdir:pcm/dca.conf><br />
}}<br />
<br />
Finally restart PulseAudio. If experience volume issues with your DTS device and/or PulseAudio, you may fix it by looking for more setting option at [https://github.com/darealshinji/dcaenc dcaenc's Github].<br />
<br />
==== Expose PulseAudio sources, sinks and mixers to ALSA ====<br />
Although {{Pkg|pulseaudio-alsa}} contains the necessary configuration file to allow ALSA applications to use PulseAudio's default device, ALSA's {{ic|pulse}} plugin is more versatile than that:<br />
<br />
{{hc|~/.asoundrc (or /etc/asound.conf)|2=<br />
# Create an alsa input/output using specific PulseAudio sources/sinks<br />
pcm.pulse-example1 {<br />
type pulse<br />
device "my-combined-sink" # name of a source or sink<br />
fallback "pulse-example2" # if combined not available<br />
}<br />
<br />
pcm.pulse-example2 {<br />
type pulse<br />
device "other-sound-card" # name of a source or sink<br />
# example: device "alsa_output.pci-0000_00_1b.0.analog-stereo"<br />
}<br />
<br />
# Create an alsa mixer using specific PulseAudio sources/sinks<br />
# these can be tested with "alsamixer -D pulse-example3"<br />
ctl.pulse-example3 {<br />
type pulse<br />
device "my-output" # name of source or sink to control<br />
<br />
# example: always control the laptop speakers:<br />
# device "alsa_output.pci-0000_00_1b.0.analog-stereo"<br />
fallback "pulse-example4" # supports fallback too<br />
}<br />
<br />
# Mixers also can control a specific source and sink, separately:<br />
ctl.pulse-example4 {<br />
type pulse<br />
sink "my-usb-headphones"<br />
source "my-internal-mic"<br />
<br />
# example: output to HDMI, record using internal<br />
sink "alsa_output.pci-0000_01_00.1.hdmi-stereo-extra1"<br />
source "alsa_input.pci-0000_00_1b.0.analog-stereo"<br />
}<br />
<br />
# These can override the default mixer (example: for pnmixer integration)<br />
ctl.!default {<br />
type pulse<br />
sink "alsa_output.pci-0000_01_00.1.hdmi-stereo-extra1"<br />
source "alsa_input.pci-0000_00_1b.0.analog-stereo"<br />
}<br />
}}<br />
<br />
The [http://git.alsa-project.org/?p=alsa-plugins.git;a=tree;f=pulse;hb=HEAD source code] can be read to know all available options.<br />
<br />
==== ALSA/dmix without grabbing hardware device ====<br />
<br />
{{Note|This section describes alternative configuration, which is generally '''not''' recommended.}}<br />
<br />
You may want to use ALSA directly in most of your applications while still being able to use applications which require PulseAudio at the same time. The following steps allow you to make PulseAudio use dmix instead of grabbing ALSA hardware device.<br />
<br />
* Remove package {{Pkg|pulseaudio-alsa}}, which provides compatibility layer between ALSA applications and PulseAudio. After this your ALSA apps will use ALSA directly without being hooked by Pulse.<br />
<br />
* Edit {{ic|/etc/pulse/default.pa}}.<br />
<br />
:Find and uncomment lines which load back-end drivers. Add '''device''' parameters as follows. Then find and comment lines which load autodetect modules.<br />
<br />
load-module module-alsa-sink '''device=dmix'''<br />
load-module module-alsa-source '''device=dsnoop'''<br />
# load-module module-udev-detect<br />
# load-module module-detect<br />
<br />
* ''Optional:'' If you use {{Pkg|kmix}} you may want to control ALSA volume instead of PulseAudio volume:<br />
<br />
$ echo export KMIX_PULSEAUDIO_DISABLE=1 > ~/.kde4/env/kmix_disable_pulse.sh<br />
$ chmod +x ~/.kde4/env/kmix_disable_pulse.sh<br />
<br />
* Now, reboot your computer and try running ALSA and PulseAudio applications at the same time. They both should produce sound simultaneously.<br />
:Use {{Pkg|pavucontrol}} to control PulseAudio volume if needed.<br />
<br />
=== OSS ===<br />
<br />
There are multiple ways of making OSS-only programs output to PulseAudio:<br />
<br />
==== ossp ====<br />
<br />
Install {{Pkg|ossp}} package and start {{ic|osspd.service}}.<br />
<br />
==== padsp wrapper ====<br />
<br />
Programs using OSS can work with PulseAudio by starting it with padsp (included with PulseAudio):<br />
<br />
$ padsp OSSprogram<br />
<br />
A few examples:<br />
<br />
$ padsp aumix<br />
$ padsp sox foo.wav -t ossdsp /dev/dsp<br />
<br />
You can also add a custom wrapper script like this: <br />
<br />
{{hc|/usr/local/bin/OSSProgram|<nowiki><br />
#!/bin/sh<br />
exec padsp /usr/bin/OSSprogram "$@"<br />
</nowiki>}}<br />
<br />
Make sure {{ic|/usr/local/bin}} comes before {{ic|/usr/bin}} in your '''PATH'''.<br />
<br />
{{Warning|<nowiki>This does not work when the module-udev-detect has the option tsched=0.</nowiki>}}<br />
<br />
=== GStreamer ===<br />
<br />
Install {{Pkg|gst-plugins-good}}, or {{AUR|gstreamer0.10-good-plugins}} if your intended program has a legacy [[GStreamer]] implementation.<br />
<br />
=== OpenAL ===<br />
<br />
OpenAL Soft should use PulseAudio by default, but can be explicitly configured to do so: {{hc|/etc/openal/alsoft.conf|2=drivers=pulse,alsa}}<br />
<br />
By default, OpenAL does not allow pulseaudio to move audio streams to a different device. To change this, add the allow-moves option:<br />
<br />
{{hc|/etc/openal/alsoft.conf|2=<br />
[pulse]<br />
allow-moves=true<br />
}}<br />
<br />
=== libao ===<br />
<br />
Edit the libao configuration file:<br />
{{hc|/etc/libao.conf|2=default_driver=pulse}}<br />
Be sure to remove the {{ic|1=dev=default}} option of the alsa driver or adjust it to specify a specific Pulse sink name or number. <br />
<br />
{{Note|You could possibly also keep the libao standard of outputting to the ''alsa'' driver and its default device if you install {{pkg|pulseaudio-alsa}} since the ALSA default device then '''is''' PulseAudio.}}<br />
<br />
== Audio post-processing ==<br />
<br />
=== PulseEffects ===<br />
<br />
PulseEffects is a GTK advanced utility for applying audio effects to applications output or to microphone before sending its audio stream to a recording tool. The user has full control over which effects to apply and their order. To install it, choose between {{Pkg|pulseeffects}} or {{AUR|pulseeffects-git}}.<br />
<br />
Effects for applications output are limiter, auto gain, expander, compressor, multiband compressor, equalizer, bass enhancer, exciter, crystalizer, reverberation, crossfeed, filter (lowpass, highpass, bandpass and bandreject modes), stereo tools, loudness, maximizer, pitch, gate, multiband gate, deesser and convolver. Effects applicable to input streams are gate, multiband gate, WebRTC, limiter, compressor, multiband compressor, equalizer, reverberation, pitch, filter (lowpass, highpass, bandpass and bandreject modes) and deesser.<br />
<br />
If PulseEffects plugins are greyed out after installing plugins, trying to start the daemon produces an error, or no devices are shown in the ''Settings > PulseAudio'' tab, consider clearing the cache as shown in [https://github.com/wwmm/pulseeffects/issues/488#issuecomment-484101349].<br />
<br />
A collection of PulseEffects presets can be found in [https://github.com/wwmm/pulseeffects/wiki/Community-presets community presets].<br />
<br />
=== Equalization ===<br />
If you want to use a different equalizer rather that the one integrated in [[#PulseEffects]], there are the following options. <br />
<br />
==== LADSPA module ====<br />
<br />
Install {{Pkg|pulseaudio-equalizer-ladspa}}, an equalizer based on LADSPA {{Pkg|swh-plugins}}. Launch {{ic|pulseaudio-equalizer-gtk}} GUI and tweak the parameters to match your expectations.<br />
<br />
==== Integrated module ==== <br />
<br />
PulseAudio has an integrated 10-band equalizer system. In order to use it, install {{Pkg|pulseaudio-equalizer}} and read the following instructions.<br />
<br />
{{Warning|PulseAudio equalizer module is considered [https://lists.freedesktop.org/archives/pulseaudio-discuss/2014-March/020174.html unstable and might be removed from PulseAudio].}}<br />
<br />
Load the equalizer sink and dbus-protocol module<br />
<br />
$ pactl load-module module-equalizer-sink<br />
$ pactl load-module module-dbus-protocol<br />
<br />
To start the GUI, run {{ic|qpaeq}}.<br />
<br />
{{Note|If qpaeq has no effect, install {{pkg|pavucontrol}} and change "ALSA Playback on" to "FFT based equalizer on ..." while the media player is running.}}<br />
<br />
To load the equalizer and dbus module on every boot, edit the {{ic|/etc/pulse/default.pa}} or {{ic|~/.config/pulse/default.pa}} file with your favorite editor and append the following lines:<br />
<br />
### Load the integrated PulseAudio equalizer and D-Bus module<br />
load-module module-equalizer-sink<br />
load-module module-dbus-protocol<br />
<br />
{{Note|The equalizer sink needs to be loaded after the master sink is already available.}}<br />
<br />
=== Dynamic Range Compression ===<br />
[[wikipedia:Dynamic_range_compression|Dynamic range compression]] can be done with [[#PulseEffects]]. Anyway PulseEffects might introduce much overhead and latency to audio stream, so if you only need a compression effect and a minor load on the system, other options are available using a [https://www.freedesktop.org/wiki/Software/PulseAudio/Documentation/User/Modules/#index47h3 module-ladspa-sink].<br />
<br />
==== Steve Harris plugin ====<br />
Steve Harris LADSPA is a set of plugins containing various compression modules. Install {{Pkg|swh-plugins}} and edit the configuration as the following<br />
<br />
{{hc|~/.config/pulse/default.pa|2=<br />
.include /etc/pulse/default.pa<br />
<br />
set-default-sink your_card_sink_name<br />
<br />
load-module module-ladspa-sink sink_name=shw_sc4 sink_master=your_card_sink_name plugin=sc4_1882 label=sc4 control=,,,,,,,,<br />
set-default-sink shw_sc4<br />
}}<br />
<br />
You have to specify your card sink name, get it from {{ic|pacmd list-sinks}}. In order to apply the changes, stop and restart Pulseaudio. The above configuration has empty control options using the default values. <br />
<br />
To tweak the module with custom control parameters, fill them respecting the right order.<br />
<br />
{| class="wikitable"<br />
! Control option || Description<br />
|+<br />
| RMS/peak (0/1) || The blanace between the RMS and peak envelope followers. RMS is generally better for subtle, musical compression and peak is better for heavier, fast compression and percussion.<br />
|+<br />
| Attack time (ms) || The attack time in milliseconds.<br />
|+<br />
| Release time (ms) || The release time in milliseconds.<br />
|+<br />
| Threshold level (dB) || The point at which the compressor will start to kick in.<br />
|+<br />
| Ratio (1:n) || The gain reduction ratio used when the signal level exceeds the threshold. 1 means no compression; higher values stronger compression. <br />
|+<br />
| Knee radius (dB) || The distance from the threshold where the knee curve starts.<br />
|+<br />
| Makeup gain (dB) || Controls the gain of the makeup input signal in dB's.<br />
|+<br />
| Amplitude (dB) || The level of the input signal, in decibels.<br />
|+<br />
| Gain reduction (dB) || The degree of gain reduction applied to the input signal, in decibels.<br />
|}<br />
<br />
Other plugins can be found in [http://plugin.org.uk/ladspa-swh/docs/ladspa-swh.html Steve Harris' LADSPA Plugin Documentation].<br />
<br />
==== Calf plugin ====<br />
For a more professional compressor, you can use the one developed by [https://calf-studio-gear.org/ Calf Studio Gear]. Install {{AUR|calf-ladspa}} and edit the configuration as the following<br />
<br />
{{hc|~/.config/pulse/default.pa|2=<br />
.include /etc/pulse/default.pa<br />
<br />
set-default-sink your_card_sink_name<br />
<br />
load-module module-ladspa-sink sink_name=calf_comp_x2 sink_master=your_card_sink_name plugin=veal label=Compressor control=,,,,,,,,,,<br />
set-default-sink calf_comp_x2<br />
}}<br />
<br />
The plugin has 11 control options. If you want to insert custom values, read the following table and do not forget to specify them in the right order. <br />
<br />
{| class="wikitable"<br />
! Control option || Default || Min || Max || Type || Info<br />
|+<br />
| Bypass || 0 || 0 || 1 || Bool || <br />
|+<br />
| Level in || 1 || 0.015625 || 64 || Float db || <br />
|+<br />
| Threshold || 0.125 || 0.000976563 || 1 || Float dbFs || For example, to set -18 db, the right value is 10^(-18/20) = 0.158<br />
|+<br />
| Ratio || 2 || 1 || 20 || Float || <br />
|+<br />
| Attack || 20 || 0.01 || 2000 || Float ms || <br />
|+<br />
| Release || 250 || 0.01 || 2000 || Float ms || <br />
|+<br />
| Makeup || 1 || 1 || 64 || Float db || <br />
|+<br />
| Knee || 2.828427125 || 1 || 8 || Float db || <br />
|+<br />
| RMS/Peak || 0 || 0 || 1 || Bool || 0 = RMS; 1 = Peak<br />
|+<br />
| Stereo Link || 0 || 0 || 1 || Bool || 0 = Average; 1 = Max<br />
|+<br />
| Mix || 1 || 0 || 1 || Float || Percentage<br />
|+<br />
| colspan="6" | To understand the meaning of every single option, read the [https://calf-studio-gear.org/doc/Compressor.html Calf Compressor Documentation].<br />
|}<br />
<br />
=== Microphone Echo-Cancellation and Noise Reduction ===<br />
<br />
See [[PulseAudio/Troubleshooting#Enable Echo/Noise-Cancellation]].<br />
<br />
== Applications ==<br />
<br />
=== QEMU ===<br />
<br />
Refer to [[QEMU#Host]] for a detailed guide on how to configure pulseaudio within [[QEMU]].<br />
<br />
=== AlsaMixer.app ===<br />
<br />
Make {{AUR|alsamixer.app}} dockapp for the {{AUR|windowmaker}} use pulseaudio, e.g.<br />
$ AlsaMixer.app --device pulse<br />
<br />
Here is a two examples where the first one is for ALSA and the other one is for pulseaudio. You can run multiple instances of it. Use the {{ic|-w}} option to choose which of the control buttons to bind to the mouse wheel.<br />
<br />
# AlsaMixer.app -3 Mic -1 Master -2 PCM --card 0 -w 1<br />
# AlsaMixer.app --device pulse -1 Capture -2 Master -w 2<br />
<br />
{{Note|It can use only those output sinks that set as default.}}<br />
<br />
=== XMMS2 ===<br />
<br />
Make it switch to pulseaudio output<br />
<br />
$ nyxmms2 server config output.plugin pulse<br />
<br />
and to alsa<br />
<br />
$ nyxmms2 server config output.plugin alsa<br />
<br />
To make xmms2 use a different output sink, e.g.<br />
<br />
$ nyxmms2 server config pulse.sink alsa_output.pci-0000_04_01.0.analog-stereo.monitor<br />
<br />
See also the official guide [https://xmms2.org/wiki/Using_the_application].<br />
<br />
=== KDE Plasma Workspaces and Qt4 ===<br />
<br />
PulseAudio will automatically be used by KDE/Qt4 applications. It is supported by default in the KDE sound mixer. For more information see the [https://www.freedesktop.org/wiki/Software/PulseAudio/Desktops/KDE/ KDE page in the PulseAudio wiki].<br />
<br />
One useful tidbit from that page is that {{ic|load-module module-device-manager}} should be loaded. This usually happens automatically at login through the script {{ic|/usr/bin/start-pulseaudio-x11}}; if you find that the module is not loaded automatically you can consider adding it manually to {{ic|/etc/pulse/default.pa}}. See [[#Switch on connect]] for possible conflicts with the {{ic|module-switch-on-connect}}.<br />
<br />
If the phonon-gstreamer backend is used for [[Phonon]], GStreamer should also be configured as described in [[#GStreamer]].<br />
<br />
=== Audacious ===<br />
<br />
[[Audacious]] natively supports PulseAudio. In order to use it, set Audacious Preferences -> Audio -> Current output plugin to 'PulseAudio Output Plugin'.<br />
<br />
=== Music Player Daemon (MPD) ===<br />
<br />
[http://mpd.wikia.com/wiki/PulseAudio Configure] [[MPD]] to use PulseAudio. See also [[MPD/Tips and Tricks#PulseAudio]].<br />
<br />
=== MPlayer ===<br />
<br />
[[MPlayer]] natively supports PulseAudio output with the {{ic|-ao pulse}} option. It can also be configured to default to PulseAudio output, in {{ic|~/.mplayer/config}} for per-user, or {{ic|/etc/mplayer/mplayer.conf}} for system-wide:<br />
<br />
{{hc|/etc/mplayer/mplayer.conf|2=<br />
ao=pulse<br />
}}<br />
<br />
=== mpv ===<br />
<br />
[[mpv]] supports PulseAudio same as written for [[#MPlayer]]. Configuration in {{ic|~/.config/mpv/mpv.conf}} per-user, or {{ic|/etc/mpv/mpv.conf}} system-wide.<br />
<br />
=== guvcview ===<br />
<br />
{{Pkg|guvcview}} when using the PulseAudio input from a [[Webcam]] may have the audio input suspended resulting in no audio being recorded. You can check this by executing:<br />
<br />
$ pactl list sources<br />
<br />
If the audio source is "suspended" then modifying the following line in {{ic|/etc/pulse/default.pa}} and changing:<br />
<br />
load-module module-suspend-on-idle<br />
<br />
to<br />
<br />
#load-module module-suspend-on-idle<br />
<br />
And then either restarting PulseAudio or your computer will only idle the input source instead of suspending it. guvcview will then correctly record audio from the device.<br />
<br />
== Networked audio ==<br />
{{Expansion|please allow for some time for me to port the most important information here -- [[User:Nodiscc|Nodiscc]]|Talk:PulseAudio#Networked_audio}}<br />
<br />
Play sound through the outputs of another computer on the network. This method streams raw PCM audio over the network, which can use pretty much network bandwidth (around 1.4 Mb/s for CD-quality sound).[https://www.freedesktop.org/wiki/Software/PulseAudio/Documentation/User/Network/]<br />
<br />
=== Basic setup with direct connection ===<br />
<br />
==== On the server ====<br />
<br />
Edit {{ic|~/.config/pulse/default.pa}} or {{ic|/etc/pulse/default.pa}} (or {{ic|/etc/pulse/system.pa}} if PulseAudio is started in system mode) and add the following line:<br />
<br />
load-module module-native-protocol-tcp auth-ip-acl=127.0.0.1;172.16.0.0/16<br />
<br />
Here only client from the IPs or IPs range specified here can stream sound.<br />
<br />
To allow access from everywhere:<br />
<br />
load-module module-native-protocol-tcp auth-anonymous=true<br />
<br />
{{Note| If {{ic|auth-ip-acl}} neither {{ic|auth-anonymous}} are specified, authentification is done via {{ic|~/.config/pulse/cookie}} which must be the same on clients and server.}}<br />
<br />
By default PulseAudio listens on port {{ic|tcp/4713}} for incoming connections, you may need to open this port in your [[firewall]].<br />
<br />
==== On the client ====<br />
<br />
Edit {{ic|~/.config/pulse/client.conf}} or {{ic|/etc/pulse/client.conf}}, to respectively apply this directive to one user or to all, and add :<br />
<br />
default-server = ''server-address''<br />
<br />
''server-address'' can be a simple domain-name or IPv4, for more see [https://www.freedesktop.org/wiki/Software/PulseAudio/Documentation/User/ServerStrings/ the documentation]<br />
<br />
It is also possible to set the server address in the environment variable {{ic|$PULSE_SERVER}}.<br />
<br />
== Tips and tricks ==<br />
<br />
{{Merge|PulseAudio/Examples|Same topic.}}<br />
<br />
=== Keyboard volume control ===<br />
<br />
See [[Keyboard shortcuts#Xorg]] to bind the following commands to your volume keys: {{ic|XF86AudioRaiseVolume}}, {{ic|XF86AudioLowerVolume}} and {{ic|XF86AudioMute}}.<br />
<br />
First find out which sink corresponds to the audio output you would like to control.<br />
To list available sinks:<br />
<br />
$ pactl list sinks short<br />
<br />
Suppose sink 0 is to be used, to raise the volume:<br />
sh -c "pactl set-sink-mute 0 false ; pactl set-sink-volume 0 +5%"<br />
<br />
To lower the volume:<br />
<br />
$ sh -c "pactl set-sink-mute 0 false ; pactl set-sink-volume 0 -5%"<br />
<br />
To mute/unmute the volume:<br />
<br />
$ pactl set-sink-mute 0 toggle<br />
<br />
To mute/unmute the microphone:<br />
<br />
$ pactl set-source-mute 1 toggle<br />
<br />
{{Tip|<br />
* To have keyboard shortcuts operate always on the default sink, specify {{ic|@DEFAULT_SINK@}} as the sink number, for example {{ic|pactl set-sink-mute @DEFAULT_SINK@ toggle}}.<br />
* For more advanced control, such as limiting the maximum volume, consider using one of the [[#Console|console front-ends]].<br />
}}<br />
<br />
=== Play sound from a non-interactive shell (systemd service, cron) ===<br />
<br />
Set {{ic|XDG_RUNTIME_DIR}} before the command (replace {{ic|''user_id''}} with the ID of the user running PulseAudio):<br />
<br />
$ XDG_RUNTIME_DIR=/run/user/''user_id'' paplay /usr/share/sounds/freedesktop/stereo/complete.oga<br />
<br />
Or use {{ic|machinectl}}:<br />
<br />
# machinectl shell .host --uid=''user_id'' /usr/bin/paplay /usr/share/sounds/freedesktop/stereo/complete.oga<br />
<br />
=== X11 Bell Events ===<br />
<br />
To get pulseaudio to handle X11 bell events, run the following commands after the X11 session has been started:<br />
<br />
$ pactl upload-sample /usr/share/sounds/freedesktop/stereo/bell.oga x11-bell<br />
$ pactl load-module module-x11-bell sample=x11-bell display=$DISPLAY<br />
<br />
Or use configuration files {{ic|/etc/pulse/default.pa}} or {{ic|~/.config/pulse/default.pa}}:<br />
<br />
{{hc|~/.config/pulse/default.pa|2=<br />
.include /etc/pulse/default.pa<br />
<br />
# audible bell<br />
load-sample-lazy x11-bell /usr/share/sounds/freedesktop/stereo/bell.oga<br />
load-module module-x11-bell sample=x11-bell<br />
}}<br />
<br />
To adjust the volume of the X11 bell, run the following command:<br />
<br />
$ xset b 100<br />
<br />
100 is a percentage. This requires the {{Pkg|xorg-xset}} package. See [[Autostarting]] for a way to run these commands automatically when the X11 session is started.<br />
<br />
=== Switch on connect ===<br />
<br />
This is a default enabled module used to switch the output sound to the newly connected device. For example, if you plug in a USB headset, the output will be switched to that. If you unplug it, the output will be set back to the last device. This used to be quite buggy but got a lot of attention in PulseAudio 8.0 and should work quite well now. <br />
<br />
{{Note|It does not work with some devices, see [[PulseAudio/Troubleshooting#ALSA channels mute when headphones are plugged/unplugged improperly]].}}<br />
<br />
{{Accuracy|Editing {{ic|/usr/bin/start-pulseaudio-x11}} will not survive package upgrade. The offending module can be unloaded in the config before loading {{ic|module-switch-on-connect}}, see [[Talk:Bluetooth_headset#GDMs_pulseaudio_instance_captures_bluetooth_headset]].}}<br />
<br />
On KDE/Plasma5 you should furthermore disable module-device-manager. As soon as Plasma5 is started it loads (via start-pulseaudio-x11) the module module-device-manager for pulseaudio to manage the devices. But that module apparently conflicts with module-switch-on-connect. Therefore you should disable that module by editing /bin/start-pulseaudio-x11 and commenting the lines for KDE. Simply logout and login again and in order to renew your pulseaudio session. On connect switching should now work properly.<br />
<br />
=== Script for switching analog outputs ===<br />
<br />
Some sound cards present the option of multiple analog outputs, being switchable through using Pulseaudio profiles. But switching manually can become a chore, so you can use the following commands to switch it:<br />
<br />
$ pactl set-sink-port 'number of the card' 'port'<br />
<br />
This will set the default output to whatever port you chose.<br />
Example:<br />
<br />
$ pactl set-sink-port 0 "analog-output;output-speaker" <br />
<br />
The values can be easily obtained using:<br />
<br />
$ pactl list<br />
<br />
Current output can be obtained through:<br />
<br />
$ pactl list sinks | grep "active profile"| cut -d ' ' -f 3-<br />
<br />
This process can be automated through a simple script. This script then can be given a shortcut by the user:<br />
<br />
{{hc|~/pa.sh (or anything the user wants)|<nowiki><br />
#!/bin/bash<br />
# This script uses kdialog notification to warn the user of the currently swapped to profile. User could adapt it to their needs or change it.<br />
<br />
CURRENT_PROFILE=$(pactl list sinks | grep "active profile"| cut -d ' ' -f 3-)<br />
<br />
if [ "$CURRENT_PROFILE" = "analog-output;output-speaker" ] ; then<br />
pactl set-sink-port 0 "analog-output;output-headphones-1"<br />
kdialog --title "Pulseaudio" --passivepopup "Headphone" 2 & <br />
else <br />
pactl set-sink-port 0 "analog-output;output-speaker" <br />
kdialog --title "Pulseaudio" --passivepopup "Speaker" 2 &<br />
fi<br />
</nowiki>}}<br />
<br />
This script is intended to swap between two profiles. First checking the current profile then swapping it. Users are required to change the field 'active profile' according to the language pactl reports. Users might need to change the number of the card and the output to fit their machine.<br />
<br />
=== Disable muting media on entering voice call (module-role-cork) ===<br />
When entering a voice call (e.g. in Microsoft Teams, maybe others too) any media applications might be muted. To disable this behaviour you can simply disable this module by hashing it out in pulseaudio configuration:<br />
<br />
{{hc|/etc/pulse/default.pa |<nowiki><br />
#load-module module-role-cork<br />
</nowiki>}}<br />
<br />
<br />
=== Disable auto switching headset to HSP/HFP ===<br />
When using a bluetooth headset that supports multiple profiles, some applications switch to HSP/HFP profile automatically. If this behaviour is undesired you can disable this by appending the auto_switch=false parameter to the bluetooth-policy module:<br />
<br />
{{hc|/etc/pulse/default.pa |<nowiki><br />
load-module module-bluetooth-policy auto_switch=false<br />
</nowiki>}}<br />
<br />
== Troubleshooting ==<br />
<br />
See [[PulseAudio/Troubleshooting]].<br />
<br />
== See also ==<br />
<br />
* [https://www.freedesktop.org/wiki/Software/PulseAudio/ PulseAudio official website], including documentation<br />
* [https://gavv.github.io/articles/pulseaudio-under-the-hood/ Pulseaudio under the hood]</div>Nicman23https://wiki.archlinux.org/index.php?title=PulseAudio&diff=648639PulseAudio2021-01-10T16:21:27Z<p>Nicman23: i was losing my mind and verified this on 2 different machines. turns out that factorio did not have audio / crashed on exit with padsp and tsched=0</p>
<hr />
<div>[[Category:Sound]]<br />
[[cs:PulseAudio]]<br />
[[de:Pulseaudio]]<br />
[[es:PulseAudio]]<br />
[[fr:PulseAudio]]<br />
[[it:PulseAudio]]<br />
[[ja:PulseAudio]]<br />
[[pt:PulseAudio]]<br />
[[ru:PulseAudio]]<br />
[[zh-hans:PulseAudio]]<br />
{{Related articles start}}<br />
{{Related|PulseAudio/Examples}}<br />
{{Related|PulseAudio/Troubleshooting}}<br />
{{Related articles end}}<br />
[[Wikipedia:PulseAudio|PulseAudio]] is a general purpose sound server intended to run as a middleware between your applications and your hardware devices, either using [[ALSA]] or [[OSS]]. It also offers easy network streaming across local devices using [[Avahi]] if enabled. While its main purpose is to ease audio configuration, its modular design allows more advanced users to configure the daemon precisely to best suit their needs.<br />
<br />
{{Note|Some confusion may occur between [[ALSA]] and PulseAudio. ALSA includes a Linux kernel component with sound card drivers, as well as a userspace component, {{ic|libasound}}.[http://www.alsa-project.org/main/index.php/Download] PulseAudio builds only on the kernel component, but offers compatibility with {{ic|libasound}} through {{Pkg|pulseaudio-alsa}}.[https://www.freedesktop.org/wiki/Software/PulseAudio/FAQ/#index14h3]}}<br />
<br />
== Installation ==<br />
<br />
[[Install]] the {{Pkg|pulseaudio}} package.<br />
<br />
Some PulseAudio modules are not included in the main package and must be installed separately if needed:<br />
<br />
* {{Pkg|pulseaudio-alsa}} for PulseAudio to manage [[ALSA]] as well, see [[#ALSA]]<br />
* {{Pkg|pulseaudio-bluetooth}} for [[bluetooth]] support (Bluez), see [[bluetooth headset]] page<br />
* {{Pkg|pulseaudio-equalizer}} for equalizer sink (qpaeq)<br />
* {{Pkg|pulseaudio-jack}} for [[JACK]] sink, source and jackdbus detection<br />
* {{Pkg|pulseaudio-lirc}} for infrared volume control with [[LIRC]]<br />
* {{Pkg|pulseaudio-zeroconf}} for Zeroconf ([[Avahi]]/DNS-SD) support<br />
<br />
=== Front-ends ===<br />
<br />
There are a number of front-ends available for controlling the PulseAudio daemon:<br />
<br />
==== Console ====<br />
<br />
* {{App|ncpamixer|Ncurses mixer for PulseAudio inspired by pavucontrol.|https://github.com/fulhax/ncpamixer|{{AUR|ncpamixer}}}}<br />
* {{App|pacmixer|Alsamixer alike for PulseAudio.|https://github.com/KenjiTakahashi/pacmixer|{{AUR|pacmixer}}}}<br />
* {{App|PAmix|Ncurses PulseAudio mixer similar to pavucontrol.|https://github.com/patroclos/PAmix|{{AUR|pamix-git}}}}<br />
* {{App|pamixer|PulseAudio command line mixer.|https://github.com/cdemoulins/pamixer|{{Pkg|pamixer}}}}<br />
* {{App|pavolume|Simple command-line volume control for PulseAudio with libnotify messages.|https://github.com/sseemayer/pavolume|{{AUR|pavolume-git}}}}<br />
* {{App|Ponymix|Command line mixer for PulseAudio.|https://github.com/falconindy/ponymix|{{Pkg|ponymix}}}}<br />
* {{App|pulseaudio-ctl|Control PulseAudio volume from the shell or mapped to keyboard shortcuts.|https://github.com/graysky2/pulseaudio-ctl|{{AUR|pulseaudio-ctl}}}}<br />
* {{App|pulsemixer|CLI and curses mixer for PulseAudio|https://github.com/GeorgeFilipkin/pulsemixer|{{Pkg|pulsemixer}}}}<br />
<br />
==== Graphical ====<br />
<br />
* {{App|KMix|[[KDE]] volume control application supporting several platforms including PulseAudio, system tray applet configurable.|https://kde.org/applications/en/multimedia/org.kde.kmix|{{pkg|kmix}}}}<br />
* {{App|MicTray|Lightweight system tray application which lets you control the microphone state and volume using PulseAudio.|https://github.com/Junker/MicTray|{{AUR|mictray}}}}<br />
* {{App|pa-applet|System tray applet for PulseAudio with volume bar.|https://github.com/fernandotcl/pa-applet|{{AUR|pa-applet-git}}}}<br />
* {{App|pasystray|System tray applet for PulseAudio.|https://github.com/christophgysin/pasystray|{{Pkg|pasystray}}}}<br />
* {{App|plasma-pa|[[KDE]] Plasma applet for audio volume management using PulseAudio|https://invent.kde.org/plasma/plasma-pa|{{Pkg|plasma-pa}}}}<br />
* {{App|PulseAudio Equalizer|LADSPA based multiband equalizer for PulseAudio.|https://github.com/pulseaudio-equalizer-ladspa/equalizer|{{Pkg|pulseaudio-equalizer-ladspa}}}}<br />
* {{App|PulseAudio Graph Control|Electron-based volume and graph control for PulseAudio.|https://github.com/futpib/pagraphcontrol#readme|{{AUR|pagraphcontrol-git}}}}<br />
* {{App|PulseAudio Manager|Simple GTK frontend for PulseAudio.|http://0pointer.de/lennart/projects/paman/|{{AUR|paman}}}}<br />
* {{App|PulseAudio Preferences|Simple GTK configuration dialog for PulseAudio.|https://freedesktop.org/software/pulseaudio/paprefs/|{{Pkg|paprefs}}}}<br />
* {{App|PulseAudio Volume Control|Simple GTK volume control tool ("mixer") for PulseAudio.|https://freedesktop.org/software/pulseaudio/pavucontrol/|{{Pkg|pavucontrol}}}}<br />
* {{App|PulseAudio Volume Control (Qt)|Mixer for PulseAudio (Qt port of pavucontrol).|https://github.com/lxqt/pavucontrol-qt|{{Pkg|pavucontrol-qt}}}}<br />
* {{App|PulseAudio Volume Control (Sandsmark)|Lightweight fork of the LXQt's pavucontrol, with missing features from pavucontrol implemented, bug fixes and unnecessary dependencies removed.|https://github.com/sandsmark/pavucontrol-qt|{{AUR|pavucontrol-qt-sandsmark-git}}}}<br />
* {{App|PulseAudio Volume Meter|Simple GTK volume meter for PulseAudio.|http://0pointer.de/lennart/projects/pavumeter/|{{AUR|pavumeter}}}}<br />
* {{App|PulseEffects|Audio effects for PulseAudio applications.|https://github.com/wwmm/pulseeffects|{{Pkg|pulseeffects}}}}<br />
* {{App|Volctl|Per-application system tray applet volume control for PulseAudio.|https://buzz.github.io/volctl/|{{AUR|volctl}}}}<br />
* {{App|Xfce PulseAudio Panel Plugin|PulseAudio plugin for [[Xfce]]4 panel.|https://goodies.xfce.org/projects/panel-plugins/xfce4-pulseaudio-plugin|{{Pkg|xfce4-pulseaudio-plugin}}}}<br />
<br />
== Configuration ==<br />
{{Merge|PulseAudio/Configuration|Configuration should stay in the main article, so the linked page should be merged here.|section=Abandoned draft}}<br />
<br />
By default, PulseAudio is configured to automatically detect all sound cards and manage them. It takes control of all detected ALSA devices and redirects all audio streams to itself, making the PulseAudio daemon the central configuration point. The daemon should work mostly out of the box, only requiring a few minor tweaks. <br />
<br />
While PulseAudio usually runs fine out of the box and requires only minimal configuration, advanced users can change almost every aspect of the daemon by either altering the default configuration file to disable modules or writing your own from scratch.<br />
<br />
PulseAudio runs as a server daemon that can run either system-wide or on per-user basis using a client/server architecture. The daemon by itself does nothing without its '''modules''' except to provide an API and host dynamically loaded modules. The audio routing and processing tasks are all handled by various modules, including PulseAudio's native protocol itself (provided by [https://www.freedesktop.org/wiki/Software/PulseAudio/Documentation/User/Modules/#index22h3 module-native-protocol-unix]). Clients reach the server through one of many protocol modules that will accept audio from external sources, route it through PulseAudio and eventually have it go out through a final other module. The output module does not have to be an actual sound output: it can dump the stream into a file, stream it to a broadcasting server such as [[Icecast]], or even just discard it.<br />
<br />
You can find a detailed list of all available modules at [https://www.freedesktop.org/wiki/Software/PulseAudio/Documentation/User/Modules/ Pulseaudio Loadable Modules]. To enable them you can just add a line {{ic|load-module <module-name-from-list>}} to {{ic|~/.config/pulse/default.pa}}.<br />
<br />
=== Configuration files ===<br />
<br />
PulseAudio will first look for configuration files in the home directory {{ic|~/.config/pulse}}, and if they are not found, the system-wide configuration from {{ic|/etc/pulse}} will be applied.<br />
<br />
{{Tip|<br />
* It is strongly suggested not to edit system-wide configuration files, but rather edit user ones. Create the {{ic|~/.config/pulse}} directory, then copy the system configuration files into it and edit according to your need.<br />
* Make sure you keep user configuration in sync with changes to the packaged files in {{ic|/etc/pulse/}}. Otherwise, PulseAudio may refuse to start due to configuration errors.<br />
* There is usually no need to add your user to the {{ic|audio}} group, as PulseAudio uses [[udev]] and ''logind'' to give access dynamically to the currently "active" user. Exceptions would include running the machine headless so that there is no currently "active" user.}}<br />
<br />
==== daemon.conf ====<br />
<br />
This is the main configuration file to configure the daemon itself. It defines base settings like the default sample rates used by modules, resampling methods, realtime scheduling and various other settings related to the server process. These can not be changed at runtime without restarting the PulseAudio daemon. The defaults are sensible for most users, see the {{man|5|pulse-daemon.conf}} manpage for additional information. Boolean options accepts any of these: {{ic|true}}, {{ic|yes}}, {{ic|on}} and {{ic|1}} as well as {{ic|false}}, {{ic|no}}, {{ic|off}} and {{ic|0}}.<br />
<br />
{{Note|PulseAudio does not perform tilde expansion on paths in this file. Use absolute paths for any files.}}<br />
<br />
{| class="wikitable"<br />
! Option || Description<br />
|+<br />
| daemonize || Controls whether the server will daemonize itself and return. Set to {{ic|no}} when debugging so you can see the debugging information on the terminal.<br />
|+<br />
| resample-method || Which resampler to use when audio with incompatible sample rates needs to be passed between modules (e.g. playback of 96kHz audio on hardware which only supports 48kHz). The available resamplers can be listed with {{ic|$ pulseaudio --dump-resample-methods}}. Choose the best tradeoff between CPU usage and audio quality for the present use-case. {{Tip|In some cases PulseAudio will generate a high CPU load. This can happen when multiple streams are resampled (individually). If this is a common use-case in a workflow, it should be considered to create an additional sink at a matching sample rate which can then be fed into the main sink, resampling only once.}}<br />
|+<br />
| avoid-resampling || With {{ic|1=avoid-resampling = yes}}, PulseAudio automatically configures the hardware to the sample rate which the application uses, if the hardware supports this sample rate (needs [https://www.freedesktop.org/wiki/Software/PulseAudio/Notes/11.0/ PA 11] or higher)<br />
{{Warning|Enabling this feature might cause audio distortion, therefore it is disabled by default, see the [https://www.freedesktop.org/wiki/Software/PulseAudio/Notes/11.0/ release notes] for more information.}}<br />
|+<br />
| enable-remixing || When the input and output have a different channel count (for example, outputting a 6 channel movie into a stereo sink), pulse can either remix all the channels (default, {{ic|yes}}) or just trivially map the channels by their name (left goes to left, right to right, all others ignored) when {{ic|no}}<br />
|+<br />
| system-instance || If set to {{ic|yes}}, run the daemon as a [https://www.freedesktop.org/wiki/Software/PulseAudio/Documentation/User/SystemWide/ system-wide] instance. [https://www.freedesktop.org/wiki/Software/PulseAudio/Documentation/User/WhatIsWrongWithSystemWide/ Highly discouraged] as it can introduce security issues. Useful on [[Xorg multiseat|Multiseat]] systems, or headless systems that have no real local users. Defaults to {{ic|no}}.<br />
|+<br />
| flat-volumes ||{{ic|flat-volumes}} scales the device-volume with the volume of the "loudest" application. For example, raising the VoIP call volume will raise the hardware volume and adjust the music-player volume so it stays where it was, without having to lower the volume of the music-player manually. Defaults to {{ic|yes}} upstream, but to {{ic|no}} within Arch. {{Note|The default behavior upstream can sometimes be confusing and some applications, unaware of this feature, can set their volume to 100% at startup, potentially blowing your speakers or your ears. This is why Arch defaults to the classic (ALSA) behavior by setting this to {{ic|no}}.}}<br />
|+<br />
| realtime-scheduling || If your [[kernel]] supports realtime scheduling (for instance, [[Realtime kernel]] or [[Linux-ck]]), set this to {{ic|yes}} to ensure PulseAudio can deliver low-latency glitch-free playback. You can adjust {{ic|realtime-priority}} as well to have it use the correct priority, especially when [[JACK]] is also running on the system.<br />
|+<br />
| nice-level || Since PulseAudio runs in userspace and involves inter-process communication, audio can be subject to dropouts if the daemon does not have enough CPU time to process the audio. The default usually is enough, but can be tweaked to give pulse the wanted priority over (or below) other applications.<br />
|+<br />
| exit-idle-time || If you want to run PulseAudio only when needed and use ALSA otherwise, you can set a delay in seconds after which the daemon will automatically shutdown after all clients are disconnected. Set it to -1 to disable this feature.<br />
|+<br />
| log-level || When debugging, you may want to increase the logging level of the daemon to see exactly why a specific module fails to load. High logging levels will sometimes print useful information such as detected minimum latency for the system, which can then be used to tweak {{ic|default-fragments}} and {{ic|default-fragment-size-msec}}.<br />
|+<br />
| default-sample-format || This usually does not need to be changed, but if your sound card's native format is different, performance and quality can be improved by setting the right format here.<br />
|+<br />
| default-sample-rate || The default sample rate user by pulse unless overriden at module level. Change this if your sound card does not support 44100Hz or if you wish to upsample all audio. See previous note about CPU usage.<br />
|+<br />
| alternate-sample-rate || To fix a common limitation where movies at 48000Hz were needlessly downsampled to 44100Hz, some modules support changing their sample rate dynamically to avoid resampling when possible. See manual for more in-depth information. This usually does not need to be changed.<br />
|+<br />
| default-channels || The default number of channels when not specified. Usually do not need any change as you can configure more channels on per-module basis.<br />
|+<br />
| default-fragments || Audio samples are split into multiple fragments of {{ic|default-fragment-size-msec}} each. The larger the buffer is, the less likely audio will skip when the system is overloaded. On the downside this will increase the overall latency. Increase this value if you have issues.<br />
|+<br />
| default-fragment-size-msec || The size in milliseconds of each fragment. This is the amount of data that will be processed at once by the daemon.<br />
|}<br />
<br />
==== default.pa ====<br />
<br />
This file is a startup script and is used to configure modules. It is actually parsed and read after the daemon has finished initializing and additional commands can be sent at runtime using {{ic|$ pactl}} or {{ic|$ pacmd}}. The startup script can also be provided on the command line by starting PulseAudio in a terminal using {{ic|$ pulseaudio -nC}}. This will make the daemon load the CLI module and will accept the configuration directly from the command line, and output resulting information or error messages on the same terminal. This can be useful when debugging the daemon or just to test various modules before setting them permanently on disk. The manual page is quite self-explanatory, consult {{man|5|pulse-cli-syntax}} for the details of the syntax.<br />
<br />
{{tip|<br />
* Rather than being a complete copy, {{ic|~/.config/pulse/default.pa}} can start with the line {{ic|.include /etc/pulse/default.pa}} and then just override the defaults.<br />
* Run {{ic|<nowiki>$ pacmd list-sinks|egrep -i 'index:|name:'</nowiki>}} to list available sinks. The present default sink is marked with an asterisk.<br />
* Edit {{ic|~/.config/pulse/default.pa}} to insert/alter the set-default-sink command using the sink's name as the numbering cannot be guaranteed repeatable.<br />
}}<br />
<br />
==== client.conf ====<br />
This is the configuration file read by every PulseAudio client application. It is used to configure runtime options for individual clients. It can be used to set and configure the default sink and source statically as well as allowing (or disallowing) clients to automatically start the server if not currently running.<br />
<br />
=== Configuration command ===<br />
<br />
The main command to configure a server during runtime is {{ic|$ pacmd}}. Run {{ic|$ pacmd --help}} for a list options, or just run {{ic|$ pacmd}} to enter the shell interactive mode and {{ic|Ctrl+d}} to exit. All modifications will immediately be applied.<br />
<br />
Once your new settings have been tested and meet your needs, edit the {{ic|default.pa}} accordingly to make the change persistent. See [[PulseAudio/Examples]] for some basic settings.<br />
<br />
{{Tip|leave the {{ic|load-module module-default-device-restore}} line in the {{ic|default.pa}} file untouched. It will allow you to restart the server in its default state, thus dismissing any wrong setting.}}<br />
<br />
It is important to understand that the "sources" (processes, capture devices) and "sinks" (sound cards, servers, other processes) accessible and selectable through PulseAudio depend upon the current hardware "Profile" selected. These "Profiles" are those ALSA "pcms" listed by the command {{ic|aplay -L}}, and more specifically by the command {{ic|pacmd list-cards}}, which will include a line "index:", a list beginning "profiles:", and a line "active profile: <...>" in the output, among other things. "Profiles" correspond to different card input/output configurations, notably the number of available input/output channels.<br />
<br />
The "active profile" can be set with the command {{ic|pacmd set-card-profile INDEX PROFILE}}, with ''no'' comma separating INDEX and PROFILE, where INDEX is just the number on the line "index:" and a PROFILE name is everything shown from the beginning of any line under "profile:" to just ''before'' the colon and first space, as shown by the command {{ic|pacmd list-cards}}. For instance, {{ic|pacmd set-card-profile 0 output:analog-stereo+input:analog-stereo}}.<br />
<br />
It may be easier to select a "Profile" with a graphical tool like {{ic|pavucontrol}}, under the "Configuration" tab, or KDE System Settings, "Multimedia/Audio and Video Settings", under the "Audio Hardware Setup" tab. Each audio "Card", which are those devices listed by the command {{ic|aplay -l}}, or again by the command {{ic|pacmd list-cards}}, will have its own selectable "Profile". When a "Profile" has been selected, the then available "sources" and "sinks" can be seen by using the commands {{ic|pacmd list-sources}} and {{ic|pacmd list-sinks}}. Note that the "index" of the available sources and sinks will change each time a card profile is changed.<br />
<br />
The selected "Profile" can be an issue for some applications, especially the Adobe Flash players, typically {{ic|/usr/lib/mozilla/plugins/libflashplayer.so}} and {{ic|/usr/lib/PepperFlash/libpepflashplayer.so}}. Often, these Flash players will only work when one of the Stereo profiles is selected, and otherwise, will play video with no sound, or will simply "crash". When all else fails, you might try selecting a different profile.<br />
<br />
Of course, when configuring some variation of Surround Sound in PulseAudio, the appropriate Surround profile will have to be selected, before Surround Sound will work, or in order to do things like remap the speaker channels.<br />
<br />
== Running ==<br />
<br />
PulseAudio on Arch has {{ic|pulseaudio.socket}} enabled by default for the [[systemd/User]] instance. This means that PulseAudio will automatically start when needed.<br />
<br />
{{Note|<br />
* To disable {{ic|pulseaudio.socket}}, make sure that {{ic|$XDG_CONFIG_HOME/systemd/user/}} exists and run {{ic|systemctl --user mask pulseaudio.socket}}.<br />
* Many [[desktop environments]] support [[XDG Autostart]]. In those desktop environments, PulseAudio will be launched automatically regardless of the socket activation status.<br />
}}<br />
<br />
For more information, see [https://www.freedesktop.org/wiki/Software/PulseAudio/Documentation/User/Running/ PulseAudio: Running].<br />
<br />
== Stopping ==<br />
$ systemctl --user stop pulseaudio.socket<br />
$ systemctl --user stop pulseaudio.service<br />
<br />
== Back-end configuration ==<br />
<br />
=== ALSA ===<br />
<br />
If you have applications that do not support PulseAudio explicitly but rely on ALSA, these applications will try to access the sound card directly via ALSA and will therefore bypass PulseAudio. PulseAudio will thus not have access to the sound card any more. As a result, all applications relying on PulseAudio will not be working any more, leading to [[PulseAudio/Troubleshooting#The only device shown is "dummy output" or newly connected cards are not detected|this issue]]. To prevent this, you will need to install the {{Pkg|pulseaudio-alsa}} package. It contains the necessary {{ic|/etc/asound.conf}} for configuring ALSA to use PulseAudio. Also make sure that {{ic|~/.asoundrc}} does not exist, as it would override the {{ic|/etc/asound.conf}} file.<br />
<br />
Also install {{Pkg|lib32-libpulse}} and {{Pkg|lib32-alsa-plugins}} if you run a x86_64 system and want to have sound for 32-bit [[multilib]] programs like [[Wine]] and [[Steam]].<br />
<br />
To prevent applications from using ALSA's OSS emulation and bypassing PulseAudio (thereby preventing other applications from playing sound), make sure the module {{ic|snd_pcm_oss}} is not being loaded at boot. If it is currently loaded ({{ic|<nowiki>lsmod | grep oss</nowiki>}}), disable it by executing:<br />
# rmmod snd_pcm_oss<br />
<br />
==== Enable DTS via ALSA ====<br />
<br />
To enable PulseAudio DTS (Digital Theater System) via ALSA install {{aur|dcaenc}} package and enable it:<br />
<br />
{{hc|/etc/asound.conf|2=<br />
<confdir:pcm/dca.conf><br />
}}<br />
<br />
Finally restart PulseAudio. If experience volume issues with your DTS device and/or PulseAudio, you may fix it by looking for more setting option at [https://github.com/darealshinji/dcaenc dcaenc's Github].<br />
<br />
==== Expose PulseAudio sources, sinks and mixers to ALSA ====<br />
Although {{Pkg|pulseaudio-alsa}} contains the necessary configuration file to allow ALSA applications to use PulseAudio's default device, ALSA's {{ic|pulse}} plugin is more versatile than that:<br />
<br />
{{hc|~/.asoundrc (or /etc/asound.conf)|2=<br />
# Create an alsa input/output using specific PulseAudio sources/sinks<br />
pcm.pulse-example1 {<br />
type pulse<br />
device "my-combined-sink" # name of a source or sink<br />
fallback "pulse-example2" # if combined not available<br />
}<br />
<br />
pcm.pulse-example2 {<br />
type pulse<br />
device "other-sound-card" # name of a source or sink<br />
# example: device "alsa_output.pci-0000_00_1b.0.analog-stereo"<br />
}<br />
<br />
# Create an alsa mixer using specific PulseAudio sources/sinks<br />
# these can be tested with "alsamixer -D pulse-example3"<br />
ctl.pulse-example3 {<br />
type pulse<br />
device "my-output" # name of source or sink to control<br />
<br />
# example: always control the laptop speakers:<br />
# device "alsa_output.pci-0000_00_1b.0.analog-stereo"<br />
fallback "pulse-example4" # supports fallback too<br />
}<br />
<br />
# Mixers also can control a specific source and sink, separately:<br />
ctl.pulse-example4 {<br />
type pulse<br />
sink "my-usb-headphones"<br />
source "my-internal-mic"<br />
<br />
# example: output to HDMI, record using internal<br />
sink "alsa_output.pci-0000_01_00.1.hdmi-stereo-extra1"<br />
source "alsa_input.pci-0000_00_1b.0.analog-stereo"<br />
}<br />
<br />
# These can override the default mixer (example: for pnmixer integration)<br />
ctl.!default {<br />
type pulse<br />
sink "alsa_output.pci-0000_01_00.1.hdmi-stereo-extra1"<br />
source "alsa_input.pci-0000_00_1b.0.analog-stereo"<br />
}<br />
}}<br />
<br />
The [http://git.alsa-project.org/?p=alsa-plugins.git;a=tree;f=pulse;hb=HEAD source code] can be read to know all available options.<br />
<br />
==== ALSA/dmix without grabbing hardware device ====<br />
<br />
{{Note|This section describes alternative configuration, which is generally '''not''' recommended.}}<br />
<br />
You may want to use ALSA directly in most of your applications while still being able to use applications which require PulseAudio at the same time. The following steps allow you to make PulseAudio use dmix instead of grabbing ALSA hardware device.<br />
<br />
* Remove package {{Pkg|pulseaudio-alsa}}, which provides compatibility layer between ALSA applications and PulseAudio. After this your ALSA apps will use ALSA directly without being hooked by Pulse.<br />
<br />
* Edit {{ic|/etc/pulse/default.pa}}.<br />
<br />
:Find and uncomment lines which load back-end drivers. Add '''device''' parameters as follows. Then find and comment lines which load autodetect modules.<br />
<br />
load-module module-alsa-sink '''device=dmix'''<br />
load-module module-alsa-source '''device=dsnoop'''<br />
# load-module module-udev-detect<br />
# load-module module-detect<br />
<br />
* ''Optional:'' If you use {{Pkg|kmix}} you may want to control ALSA volume instead of PulseAudio volume:<br />
<br />
$ echo export KMIX_PULSEAUDIO_DISABLE=1 > ~/.kde4/env/kmix_disable_pulse.sh<br />
$ chmod +x ~/.kde4/env/kmix_disable_pulse.sh<br />
<br />
* Now, reboot your computer and try running ALSA and PulseAudio applications at the same time. They both should produce sound simultaneously.<br />
:Use {{Pkg|pavucontrol}} to control PulseAudio volume if needed.<br />
<br />
=== OSS ===<br />
<br />
There are multiple ways of making OSS-only programs output to PulseAudio:<br />
<br />
==== ossp ====<br />
<br />
Install {{Pkg|ossp}} package and start {{ic|osspd.service}}.<br />
<br />
==== padsp wrapper ====<br />
<br />
Programs using OSS can work with PulseAudio by starting it with padsp (included with PulseAudio):<br />
<br />
$ padsp OSSprogram<br />
<br />
A few examples:<br />
<br />
$ padsp aumix<br />
$ padsp sox foo.wav -t ossdsp /dev/dsp<br />
<br />
You can also add a custom wrapper script like this: <br />
<br />
{{hc|/usr/local/bin/OSSProgram|<nowiki><br />
#!/bin/sh<br />
exec padsp /usr/bin/OSSprogram "$@"<br />
</nowiki>}}<br />
<br />
Make sure {{ic|/usr/local/bin}} comes before {{ic|/usr/bin}} in your '''PATH'''.<br />
<br />
{{Warning|This does not work when the module-udev-detect has the option tsched=0.}}<br />
<br />
=== GStreamer ===<br />
<br />
Install {{Pkg|gst-plugins-good}}, or {{AUR|gstreamer0.10-good-plugins}} if your intended program has a legacy [[GStreamer]] implementation.<br />
<br />
=== OpenAL ===<br />
<br />
OpenAL Soft should use PulseAudio by default, but can be explicitly configured to do so: {{hc|/etc/openal/alsoft.conf|2=drivers=pulse,alsa}}<br />
<br />
By default, OpenAL does not allow pulseaudio to move audio streams to a different device. To change this, add the allow-moves option:<br />
<br />
{{hc|/etc/openal/alsoft.conf|2=<br />
[pulse]<br />
allow-moves=true<br />
}}<br />
<br />
=== libao ===<br />
<br />
Edit the libao configuration file:<br />
{{hc|/etc/libao.conf|2=default_driver=pulse}}<br />
Be sure to remove the {{ic|1=dev=default}} option of the alsa driver or adjust it to specify a specific Pulse sink name or number. <br />
<br />
{{Note|You could possibly also keep the libao standard of outputting to the ''alsa'' driver and its default device if you install {{pkg|pulseaudio-alsa}} since the ALSA default device then '''is''' PulseAudio.}}<br />
<br />
== Audio post-processing ==<br />
<br />
=== PulseEffects ===<br />
<br />
PulseEffects is a GTK advanced utility for applying audio effects to applications output or to microphone before sending its audio stream to a recording tool. The user has full control over which effects to apply and their order. To install it, choose between {{Pkg|pulseeffects}} or {{AUR|pulseeffects-git}}.<br />
<br />
Effects for applications output are limiter, auto gain, expander, compressor, multiband compressor, equalizer, bass enhancer, exciter, crystalizer, reverberation, crossfeed, filter (lowpass, highpass, bandpass and bandreject modes), stereo tools, loudness, maximizer, pitch, gate, multiband gate, deesser and convolver. Effects applicable to input streams are gate, multiband gate, WebRTC, limiter, compressor, multiband compressor, equalizer, reverberation, pitch, filter (lowpass, highpass, bandpass and bandreject modes) and deesser.<br />
<br />
If PulseEffects plugins are greyed out after installing plugins, trying to start the daemon produces an error, or no devices are shown in the ''Settings > PulseAudio'' tab, consider clearing the cache as shown in [https://github.com/wwmm/pulseeffects/issues/488#issuecomment-484101349].<br />
<br />
A collection of PulseEffects presets can be found in [https://github.com/wwmm/pulseeffects/wiki/Community-presets community presets].<br />
<br />
=== Equalization ===<br />
If you want to use a different equalizer rather that the one integrated in [[#PulseEffects]], there are the following options. <br />
<br />
==== LADSPA module ====<br />
<br />
Install {{Pkg|pulseaudio-equalizer-ladspa}}, an equalizer based on LADSPA {{Pkg|swh-plugins}}. Launch {{ic|pulseaudio-equalizer-gtk}} GUI and tweak the parameters to match your expectations.<br />
<br />
==== Integrated module ==== <br />
<br />
PulseAudio has an integrated 10-band equalizer system. In order to use it, install {{Pkg|pulseaudio-equalizer}} and read the following instructions.<br />
<br />
{{Warning|PulseAudio equalizer module is considered [https://lists.freedesktop.org/archives/pulseaudio-discuss/2014-March/020174.html unstable and might be removed from PulseAudio].}}<br />
<br />
Load the equalizer sink and dbus-protocol module<br />
<br />
$ pactl load-module module-equalizer-sink<br />
$ pactl load-module module-dbus-protocol<br />
<br />
To start the GUI, run {{ic|qpaeq}}.<br />
<br />
{{Note|If qpaeq has no effect, install {{pkg|pavucontrol}} and change "ALSA Playback on" to "FFT based equalizer on ..." while the media player is running.}}<br />
<br />
To load the equalizer and dbus module on every boot, edit the {{ic|/etc/pulse/default.pa}} or {{ic|~/.config/pulse/default.pa}} file with your favorite editor and append the following lines:<br />
<br />
### Load the integrated PulseAudio equalizer and D-Bus module<br />
load-module module-equalizer-sink<br />
load-module module-dbus-protocol<br />
<br />
{{Note|The equalizer sink needs to be loaded after the master sink is already available.}}<br />
<br />
=== Dynamic Range Compression ===<br />
[[wikipedia:Dynamic_range_compression|Dynamic range compression]] can be done with [[#PulseEffects]]. Anyway PulseEffects might introduce much overhead and latency to audio stream, so if you only need a compression effect and a minor load on the system, other options are available using a [https://www.freedesktop.org/wiki/Software/PulseAudio/Documentation/User/Modules/#index47h3 module-ladspa-sink].<br />
<br />
==== Steve Harris plugin ====<br />
Steve Harris LADSPA is a set of plugins containing various compression modules. Install {{Pkg|swh-plugins}} and edit the configuration as the following<br />
<br />
{{hc|~/.config/pulse/default.pa|2=<br />
.include /etc/pulse/default.pa<br />
<br />
set-default-sink your_card_sink_name<br />
<br />
load-module module-ladspa-sink sink_name=shw_sc4 sink_master=your_card_sink_name plugin=sc4_1882 label=sc4 control=,,,,,,,,<br />
set-default-sink shw_sc4<br />
}}<br />
<br />
You have to specify your card sink name, get it from {{ic|pacmd list-sinks}}. In order to apply the changes, stop and restart Pulseaudio. The above configuration has empty control options using the default values. <br />
<br />
To tweak the module with custom control parameters, fill them respecting the right order.<br />
<br />
{| class="wikitable"<br />
! Control option || Description<br />
|+<br />
| RMS/peak (0/1) || The blanace between the RMS and peak envelope followers. RMS is generally better for subtle, musical compression and peak is better for heavier, fast compression and percussion.<br />
|+<br />
| Attack time (ms) || The attack time in milliseconds.<br />
|+<br />
| Release time (ms) || The release time in milliseconds.<br />
|+<br />
| Threshold level (dB) || The point at which the compressor will start to kick in.<br />
|+<br />
| Ratio (1:n) || The gain reduction ratio used when the signal level exceeds the threshold. 1 means no compression; higher values stronger compression. <br />
|+<br />
| Knee radius (dB) || The distance from the threshold where the knee curve starts.<br />
|+<br />
| Makeup gain (dB) || Controls the gain of the makeup input signal in dB's.<br />
|+<br />
| Amplitude (dB) || The level of the input signal, in decibels.<br />
|+<br />
| Gain reduction (dB) || The degree of gain reduction applied to the input signal, in decibels.<br />
|}<br />
<br />
Other plugins can be found in [http://plugin.org.uk/ladspa-swh/docs/ladspa-swh.html Steve Harris' LADSPA Plugin Documentation].<br />
<br />
==== Calf plugin ====<br />
For a more professional compressor, you can use the one developed by [https://calf-studio-gear.org/ Calf Studio Gear]. Install {{AUR|calf-ladspa}} and edit the configuration as the following<br />
<br />
{{hc|~/.config/pulse/default.pa|2=<br />
.include /etc/pulse/default.pa<br />
<br />
set-default-sink your_card_sink_name<br />
<br />
load-module module-ladspa-sink sink_name=calf_comp_x2 sink_master=your_card_sink_name plugin=veal label=Compressor control=,,,,,,,,,,<br />
set-default-sink calf_comp_x2<br />
}}<br />
<br />
The plugin has 11 control options. If you want to insert custom values, read the following table and do not forget to specify them in the right order. <br />
<br />
{| class="wikitable"<br />
! Control option || Default || Min || Max || Type || Info<br />
|+<br />
| Bypass || 0 || 0 || 1 || Bool || <br />
|+<br />
| Level in || 1 || 0.015625 || 64 || Float db || <br />
|+<br />
| Threshold || 0.125 || 0.000976563 || 1 || Float dbFs || For example, to set -18 db, the right value is 10^(-18/20) = 0.158<br />
|+<br />
| Ratio || 2 || 1 || 20 || Float || <br />
|+<br />
| Attack || 20 || 0.01 || 2000 || Float ms || <br />
|+<br />
| Release || 250 || 0.01 || 2000 || Float ms || <br />
|+<br />
| Makeup || 1 || 1 || 64 || Float db || <br />
|+<br />
| Knee || 2.828427125 || 1 || 8 || Float db || <br />
|+<br />
| RMS/Peak || 0 || 0 || 1 || Bool || 0 = RMS; 1 = Peak<br />
|+<br />
| Stereo Link || 0 || 0 || 1 || Bool || 0 = Average; 1 = Max<br />
|+<br />
| Mix || 1 || 0 || 1 || Float || Percentage<br />
|+<br />
| colspan="6" | To understand the meaning of every single option, read the [https://calf-studio-gear.org/doc/Compressor.html Calf Compressor Documentation].<br />
|}<br />
<br />
=== Microphone Echo-Cancellation and Noise Reduction ===<br />
<br />
See [[PulseAudio/Troubleshooting#Enable Echo/Noise-Cancellation]].<br />
<br />
== Applications ==<br />
<br />
=== QEMU ===<br />
<br />
Refer to [[QEMU#Host]] for a detailed guide on how to configure pulseaudio within [[QEMU]].<br />
<br />
=== AlsaMixer.app ===<br />
<br />
Make {{AUR|alsamixer.app}} dockapp for the {{AUR|windowmaker}} use pulseaudio, e.g.<br />
$ AlsaMixer.app --device pulse<br />
<br />
Here is a two examples where the first one is for ALSA and the other one is for pulseaudio. You can run multiple instances of it. Use the {{ic|-w}} option to choose which of the control buttons to bind to the mouse wheel.<br />
<br />
# AlsaMixer.app -3 Mic -1 Master -2 PCM --card 0 -w 1<br />
# AlsaMixer.app --device pulse -1 Capture -2 Master -w 2<br />
<br />
{{Note|It can use only those output sinks that set as default.}}<br />
<br />
=== XMMS2 ===<br />
<br />
Make it switch to pulseaudio output<br />
<br />
$ nyxmms2 server config output.plugin pulse<br />
<br />
and to alsa<br />
<br />
$ nyxmms2 server config output.plugin alsa<br />
<br />
To make xmms2 use a different output sink, e.g.<br />
<br />
$ nyxmms2 server config pulse.sink alsa_output.pci-0000_04_01.0.analog-stereo.monitor<br />
<br />
See also the official guide [https://xmms2.org/wiki/Using_the_application].<br />
<br />
=== KDE Plasma Workspaces and Qt4 ===<br />
<br />
PulseAudio will automatically be used by KDE/Qt4 applications. It is supported by default in the KDE sound mixer. For more information see the [https://www.freedesktop.org/wiki/Software/PulseAudio/Desktops/KDE/ KDE page in the PulseAudio wiki].<br />
<br />
One useful tidbit from that page is that {{ic|load-module module-device-manager}} should be loaded. This usually happens automatically at login through the script {{ic|/usr/bin/start-pulseaudio-x11}}; if you find that the module is not loaded automatically you can consider adding it manually to {{ic|/etc/pulse/default.pa}}. See [[#Switch on connect]] for possible conflicts with the {{ic|module-switch-on-connect}}.<br />
<br />
If the phonon-gstreamer backend is used for [[Phonon]], GStreamer should also be configured as described in [[#GStreamer]].<br />
<br />
=== Audacious ===<br />
<br />
[[Audacious]] natively supports PulseAudio. In order to use it, set Audacious Preferences -> Audio -> Current output plugin to 'PulseAudio Output Plugin'.<br />
<br />
=== Music Player Daemon (MPD) ===<br />
<br />
[http://mpd.wikia.com/wiki/PulseAudio Configure] [[MPD]] to use PulseAudio. See also [[MPD/Tips and Tricks#PulseAudio]].<br />
<br />
=== MPlayer ===<br />
<br />
[[MPlayer]] natively supports PulseAudio output with the {{ic|-ao pulse}} option. It can also be configured to default to PulseAudio output, in {{ic|~/.mplayer/config}} for per-user, or {{ic|/etc/mplayer/mplayer.conf}} for system-wide:<br />
<br />
{{hc|/etc/mplayer/mplayer.conf|2=<br />
ao=pulse<br />
}}<br />
<br />
=== mpv ===<br />
<br />
[[mpv]] supports PulseAudio same as written for [[#MPlayer]]. Configuration in {{ic|~/.config/mpv/mpv.conf}} per-user, or {{ic|/etc/mpv/mpv.conf}} system-wide.<br />
<br />
=== guvcview ===<br />
<br />
{{Pkg|guvcview}} when using the PulseAudio input from a [[Webcam]] may have the audio input suspended resulting in no audio being recorded. You can check this by executing:<br />
<br />
$ pactl list sources<br />
<br />
If the audio source is "suspended" then modifying the following line in {{ic|/etc/pulse/default.pa}} and changing:<br />
<br />
load-module module-suspend-on-idle<br />
<br />
to<br />
<br />
#load-module module-suspend-on-idle<br />
<br />
And then either restarting PulseAudio or your computer will only idle the input source instead of suspending it. guvcview will then correctly record audio from the device.<br />
<br />
== Networked audio ==<br />
{{Expansion|please allow for some time for me to port the most important information here -- [[User:Nodiscc|Nodiscc]]|Talk:PulseAudio#Networked_audio}}<br />
<br />
Play sound through the outputs of another computer on the network. This method streams raw PCM audio over the network, which can use pretty much network bandwidth (around 1.4 Mb/s for CD-quality sound).[https://www.freedesktop.org/wiki/Software/PulseAudio/Documentation/User/Network/]<br />
<br />
=== Basic setup with direct connection ===<br />
<br />
==== On the server ====<br />
<br />
Edit {{ic|~/.config/pulse/default.pa}} or {{ic|/etc/pulse/default.pa}} (or {{ic|/etc/pulse/system.pa}} if PulseAudio is started in system mode) and add the following line:<br />
<br />
load-module module-native-protocol-tcp auth-ip-acl=127.0.0.1;172.16.0.0/16<br />
<br />
Here only client from the IPs or IPs range specified here can stream sound.<br />
<br />
To allow access from everywhere:<br />
<br />
load-module module-native-protocol-tcp auth-anonymous=true<br />
<br />
{{Note| If {{ic|auth-ip-acl}} neither {{ic|auth-anonymous}} are specified, authentification is done via {{ic|~/.config/pulse/cookie}} which must be the same on clients and server.}}<br />
<br />
By default PulseAudio listens on port {{ic|tcp/4713}} for incoming connections, you may need to open this port in your [[firewall]].<br />
<br />
==== On the client ====<br />
<br />
Edit {{ic|~/.config/pulse/client.conf}} or {{ic|/etc/pulse/client.conf}}, to respectively apply this directive to one user or to all, and add :<br />
<br />
default-server = ''server-address''<br />
<br />
''server-address'' can be a simple domain-name or IPv4, for more see [https://www.freedesktop.org/wiki/Software/PulseAudio/Documentation/User/ServerStrings/ the documentation]<br />
<br />
It is also possible to set the server address in the environment variable {{ic|$PULSE_SERVER}}.<br />
<br />
== Tips and tricks ==<br />
<br />
{{Merge|PulseAudio/Examples|Same topic.}}<br />
<br />
=== Keyboard volume control ===<br />
<br />
See [[Keyboard shortcuts#Xorg]] to bind the following commands to your volume keys: {{ic|XF86AudioRaiseVolume}}, {{ic|XF86AudioLowerVolume}} and {{ic|XF86AudioMute}}.<br />
<br />
First find out which sink corresponds to the audio output you would like to control.<br />
To list available sinks:<br />
<br />
$ pactl list sinks short<br />
<br />
Suppose sink 0 is to be used, to raise the volume:<br />
sh -c "pactl set-sink-mute 0 false ; pactl set-sink-volume 0 +5%"<br />
<br />
To lower the volume:<br />
<br />
$ sh -c "pactl set-sink-mute 0 false ; pactl set-sink-volume 0 -5%"<br />
<br />
To mute/unmute the volume:<br />
<br />
$ pactl set-sink-mute 0 toggle<br />
<br />
To mute/unmute the microphone:<br />
<br />
$ pactl set-source-mute 1 toggle<br />
<br />
{{Tip|<br />
* To have keyboard shortcuts operate always on the default sink, specify {{ic|@DEFAULT_SINK@}} as the sink number, for example {{ic|pactl set-sink-mute @DEFAULT_SINK@ toggle}}.<br />
* For more advanced control, such as limiting the maximum volume, consider using one of the [[#Console|console front-ends]].<br />
}}<br />
<br />
=== Play sound from a non-interactive shell (systemd service, cron) ===<br />
<br />
Set {{ic|XDG_RUNTIME_DIR}} before the command (replace {{ic|''user_id''}} with the ID of the user running PulseAudio):<br />
<br />
$ XDG_RUNTIME_DIR=/run/user/''user_id'' paplay /usr/share/sounds/freedesktop/stereo/complete.oga<br />
<br />
Or use {{ic|machinectl}}:<br />
<br />
# machinectl shell .host --uid=''user_id'' /usr/bin/paplay /usr/share/sounds/freedesktop/stereo/complete.oga<br />
<br />
=== X11 Bell Events ===<br />
<br />
To get pulseaudio to handle X11 bell events, run the following commands after the X11 session has been started:<br />
<br />
$ pactl upload-sample /usr/share/sounds/freedesktop/stereo/bell.oga x11-bell<br />
$ pactl load-module module-x11-bell sample=x11-bell display=$DISPLAY<br />
<br />
Or use configuration files {{ic|/etc/pulse/default.pa}} or {{ic|~/.config/pulse/default.pa}}:<br />
<br />
{{hc|~/.config/pulse/default.pa|2=<br />
.include /etc/pulse/default.pa<br />
<br />
# audible bell<br />
load-sample-lazy x11-bell /usr/share/sounds/freedesktop/stereo/bell.oga<br />
load-module module-x11-bell sample=x11-bell<br />
}}<br />
<br />
To adjust the volume of the X11 bell, run the following command:<br />
<br />
$ xset b 100<br />
<br />
100 is a percentage. This requires the {{Pkg|xorg-xset}} package. See [[Autostarting]] for a way to run these commands automatically when the X11 session is started.<br />
<br />
=== Switch on connect ===<br />
<br />
This is a default enabled module used to switch the output sound to the newly connected device. For example, if you plug in a USB headset, the output will be switched to that. If you unplug it, the output will be set back to the last device. This used to be quite buggy but got a lot of attention in PulseAudio 8.0 and should work quite well now. <br />
<br />
{{Note|It does not work with some devices, see [[PulseAudio/Troubleshooting#ALSA channels mute when headphones are plugged/unplugged improperly]].}}<br />
<br />
{{Accuracy|Editing {{ic|/usr/bin/start-pulseaudio-x11}} will not survive package upgrade. The offending module can be unloaded in the config before loading {{ic|module-switch-on-connect}}, see [[Talk:Bluetooth_headset#GDMs_pulseaudio_instance_captures_bluetooth_headset]].}}<br />
<br />
On KDE/Plasma5 you should furthermore disable module-device-manager. As soon as Plasma5 is started it loads (via start-pulseaudio-x11) the module module-device-manager for pulseaudio to manage the devices. But that module apparently conflicts with module-switch-on-connect. Therefore you should disable that module by editing /bin/start-pulseaudio-x11 and commenting the lines for KDE. Simply logout and login again and in order to renew your pulseaudio session. On connect switching should now work properly.<br />
<br />
=== Script for switching analog outputs ===<br />
<br />
Some sound cards present the option of multiple analog outputs, being switchable through using Pulseaudio profiles. But switching manually can become a chore, so you can use the following commands to switch it:<br />
<br />
$ pactl set-sink-port 'number of the card' 'port'<br />
<br />
This will set the default output to whatever port you chose.<br />
Example:<br />
<br />
$ pactl set-sink-port 0 "analog-output;output-speaker" <br />
<br />
The values can be easily obtained using:<br />
<br />
$ pactl list<br />
<br />
Current output can be obtained through:<br />
<br />
$ pactl list sinks | grep "active profile"| cut -d ' ' -f 3-<br />
<br />
This process can be automated through a simple script. This script then can be given a shortcut by the user:<br />
<br />
{{hc|~/pa.sh (or anything the user wants)|<nowiki><br />
#!/bin/bash<br />
# This script uses kdialog notification to warn the user of the currently swapped to profile. User could adapt it to their needs or change it.<br />
<br />
CURRENT_PROFILE=$(pactl list sinks | grep "active profile"| cut -d ' ' -f 3-)<br />
<br />
if [ "$CURRENT_PROFILE" = "analog-output;output-speaker" ] ; then<br />
pactl set-sink-port 0 "analog-output;output-headphones-1"<br />
kdialog --title "Pulseaudio" --passivepopup "Headphone" 2 & <br />
else <br />
pactl set-sink-port 0 "analog-output;output-speaker" <br />
kdialog --title "Pulseaudio" --passivepopup "Speaker" 2 &<br />
fi<br />
</nowiki>}}<br />
<br />
This script is intended to swap between two profiles. First checking the current profile then swapping it. Users are required to change the field 'active profile' according to the language pactl reports. Users might need to change the number of the card and the output to fit their machine.<br />
<br />
=== Disable muting media on entering voice call (module-role-cork) ===<br />
When entering a voice call (e.g. in Microsoft Teams, maybe others too) any media applications might be muted. To disable this behaviour you can simply disable this module by hashing it out in pulseaudio configuration:<br />
<br />
{{hc|/etc/pulse/default.pa |<nowiki><br />
#load-module module-role-cork<br />
</nowiki>}}<br />
<br />
<br />
=== Disable auto switching headset to HSP/HFP ===<br />
When using a bluetooth headset that supports multiple profiles, some applications switch to HSP/HFP profile automatically. If this behaviour is undesired you can disable this by appending the auto_switch=false parameter to the bluetooth-policy module:<br />
<br />
{{hc|/etc/pulse/default.pa |<nowiki><br />
load-module module-bluetooth-policy auto_switch=false<br />
</nowiki>}}<br />
<br />
== Troubleshooting ==<br />
<br />
See [[PulseAudio/Troubleshooting]].<br />
<br />
== See also ==<br />
<br />
* [https://www.freedesktop.org/wiki/Software/PulseAudio/ PulseAudio official website], including documentation<br />
* [https://gavv.github.io/articles/pulseaudio-under-the-hood/ Pulseaudio under the hood]</div>Nicman23https://wiki.archlinux.org/index.php?title=Nouveau&diff=559523Nouveau2018-12-19T15:20:40Z<p>Nicman23: This reduced crashes for me, but did not eradicate them</p>
<hr />
<div>[[Category:Graphics]]<br />
[[Category:X server]]<br />
[[es:Nouveau]]<br />
[[it:Nouveau]]<br />
[[ja:Nouveau]]<br />
[[ru:Nouveau]]<br />
[[zh-hans:Nouveau]]<br />
{{Related articles start}}<br />
{{Related|NVIDIA}}<br />
{{Related|Xorg}}<br />
{{Related|Bumblebee}}<br />
{{Related articles end}}<br />
<br />
This article covers the open-source [https://nouveau.freedesktop.org/ Nouveau] driver for NVIDIA graphics cards. For information about the proprietary driver, see [[NVIDIA]].<br />
<br />
Find your card's [https://nouveau.freedesktop.org/wiki/CodeNames code name] (a more detailed list is available on [[Wikipedia:Comparison of Nvidia Graphics Processing Units|Wikipedia]]), and compare it with the [https://nouveau.freedesktop.org/wiki/FeatureMatrix/ feature matrix] for supported features.<br />
<br />
== Installation ==<br />
<br />
[[Install]] the {{Pkg|mesa}} package, which provides the DRI driver for 3D acceleration.<br />
<br />
* For 32-bit application support, also install the {{Pkg|lib32-mesa}} package from the [[multilib]] repostory.<br />
* For the DDX driver (which provides 2D acceleration in [[Xorg]]), [[install]] the {{Pkg|xf86-video-nouveau}} package.<br />
<br />
Also see [[Hardware video acceleration]].<br />
<br />
== Loading ==<br />
<br />
The Nouveau kernel module should load automatically on system boot. If it does not happen, then:<br />
<br />
* Make sure you do '''not''' have {{ic|nomodeset}} or {{ic|1=vga=}} as a [[kernel parameter]], since Nouveau requires kernel mode-setting.<br />
* Also, check that you do not have Nouveau disabled using any modprobe blacklisting technique within {{ic|/etc/modprobe.d/}} or {{ic|/usr/lib/modprobe.d/}}.<br />
* If all above still fails to load nouveau check dmesg for an opcode error. Add {{ic|1=nouveau.config=NvBios=PRAMIN}} to your [[Kernel parameters]] to prevent module unloading.[https://nouveau.freedesktop.org/wiki/TroubleShooting/#index10h3]<br />
<br />
=== Enable early KMS ===<br />
<br />
{{Tip|If you have problems with the resolution, check [[Kernel mode setting#Forcing modes and EDID]].}}<br />
<br />
[[Kernel mode setting]] (KMS) is required by the Nouveau driver. By default, the KMS is done after the other kernel modules are loaded. You will see the text "Loading modules" and the size of the text may change, possibly with an undesirable flicker. See the [https://nouveau.freedesktop.org/wiki/KernelModeSetting Nouveau KernelModeSetting page] for more details.<br />
<br />
It is also possible to start the KMS as early as possible in the boot process, when the [[initramfs]] is loaded. <br />
<br />
To do this, add {{ic|nouveau}} to the {{ic|MODULES}} array in {{ic|/etc/mkinitcpio.conf}} (module names are separated by spaces): <br />
<br />
MODULES="... nouveau ..."<br />
<br />
If you are using a custom EDID file, you should embed it into initramfs as well:<br />
<br />
{{hc|/etc/mkinitcpio.conf|<br />
2=FILES="/lib/firmware/edid/your_edid.bin"}}<br />
<br />
Re-generate the initial ramdisk image:<br />
<br />
# mkinitcpio -p <kernel preset; e.g. ''linux''><br />
<br />
If you're experiencing troubles with Nouveau leading to rebuild nouveau-drm several times for testing purposes, do not add {{ic|nouveau}} to the initramfs. It is too easy to forget to rebuild the initramfs and it will just make any testing harder. Just use "Late start" until you are confident the system is stable. There might be additional problems with initramfs if you need a custom firmware (generally not advised).<br />
<br />
== Tips and tricks ==<br />
<br />
=== Keep NVIDIA driver installed ===<br />
<br />
If you want to keep the proprietary NVIDIA driver installed (and are not using OpenGL), but want to use the Nouveau driver, comment out nouveau blacklisting in {{ic|/etc/modprobe.d/nouveau_blacklist.conf}} or {{ic|/usr/lib/modprobe.d/nvidia.conf}} modifying it as follows:<br />
<br />
#blacklist nouveau<br />
<br />
And tell Xorg to load nouveau instead of nvidia by creating the file {{ic|/etc/X11/xorg.conf.d/20-nouveau.conf}} with the following content:<br />
<br />
Section "Device"<br />
Identifier "Nvidia card"<br />
Driver "nouveau"<br />
EndSection<br />
<br />
If you already used the NVIDIA driver, and want to test Nouveau without reboot, make sure the 'nvidia' module is no longer loaded:<br />
<br />
# rmmod nvidia<br />
<br />
Then load the 'nouveau' module: <br />
<br />
# modprobe nouveau<br />
<br />
And check that it loaded fine by looking at kernel messages: <br />
<br />
$ dmesg<br />
<br />
=== Installing the latest development packages ===<br />
<br />
To get the latest Nouveau improvements<br />
*{{AUR|linux-git}} PKGBUILD to use the Nouveau tree at https://github.com/skeggsb/linux/, instead of making a package from scratch.<br />
*{{AUR|libdrm-git}}<br />
*{{AUR|lib32-libdrm-git}}<br />
*{{AUR|lib32-mesa-git}}<br />
*{{AUR|mesa-git}}<br />
*{{AUR|xf86-video-nouveau-git}}<br />
<br />
=== Dual head ===<br />
<br />
Nouveau supports the xrandr extension for modesetting and multiple monitors. See the [[xrandr]] page for tutorials.<br />
<br />
Here is a full sample {{ic|/etc/X11/xorg.conf.d/20-nouveau.conf}} above for running 2 monitors in dual head mode. You may prefer to use a graphical tool to configure monitors like GNOME Control Center's Display panel ({{ic|gnome-control-center display}}).<br />
<br />
{{bc|<br />
# the right one<br />
Section "Monitor"<br />
Identifier "NEC"<br />
Option "PreferredMode" "1280x1024_60.00"<br />
EndSection<br />
<br />
# the left one<br />
Section "Monitor"<br />
Identifier "FUS"<br />
Option "PreferredMode" "1280x1024_60.00"<br />
Option "LeftOf" "NEC"<br />
EndSection<br />
<br />
Section "Device"<br />
Identifier "nvidia card"<br />
Driver "nouveau"<br />
Option "Monitor-DVI-I-1" "NEC"<br />
Option "Monitor-DVI-I-2" "FUS"<br />
EndSection<br />
<br />
Section "Screen"<br />
Identifier "screen1"<br />
Monitor "NEC"<br />
DefaultDepth 24<br />
SubSection "Display"<br />
Depth 24<br />
Virtual 2560 2048<br />
EndSubSection<br />
Device "nvidia card"<br />
EndSection<br />
<br />
Section "ServerLayout"<br />
Identifier "layout1"<br />
Screen "screen1"<br />
EndSection}}<br />
<br />
=== Setting console resolution ===<br />
<br />
Use the {{Pkg|fbset}} tool to adjust console resolution.<br />
<br />
You can also pass the resolution to nouveau with the {{ic|1=video=}} kernel line option (see [[KMS]]).<br />
<br />
=== Power management ===<br />
<br />
The lack of proper power management in the nouveau driver is one of the most important causes of performance issues, since most cards will remain in their lower power state with lower clocks during their use. Experimental support for GPU reclocking is available for some cards (see the [https://nouveau.freedesktop.org/wiki/PowerManagement Nouveau PowerManagement page]) and since kernel 4.5 can be controlled through a debugfs interface located at {{ic|/sys/kernel/debug/dri/*/pstate}}.<br />
<br />
For example, to check the available power states and the current setting for the first card in your system, run:<br />
<br />
# cat /sys/kernel/debug/dri/0/pstate<br />
<br />
It's also possible to manually set/force a certain power state by writing to said interface:<br />
<br />
# echo ''pstate'' > /sys/kernel/debug/dri/0/pstate<br />
<br />
{{Warning|The support for reclocking is highly experimental. Manually setting the power state may hang your system, cause corruption or overheat your card.}}<br />
<br />
==== Fan control ====<br />
<br />
If it is implemented for you card you can configure fan control via {{ic|/sys}}.<br />
<br />
$ find /sys -name pwm1_enable<br />
/sys/devices/pci0000:00/0000:00:01.0/0000:01:00.0/hwmon/hwmon1/pwm1_enable<br />
$ readlink /sys/devices/pci0000:00/0000:00:01.0/0000:01:00.0/driver<br />
../../../../bus/pci/drivers/nouveau<br />
<br />
{{ic|pwm1_enable}} can be set to 0, 1 or 2 meaning NONE, MANUAL and AUTO fan control. If set to manual fan control, you can set {{ic|pwm1}} manually, for example to 40 for 40%.<br />
{{Warning|Use at your own risk! Don't overheat your card!}}<br />
<br />
You can also set it by udev rule:<br />
$ cat /etc/udev/rules.d/50-nouveau-hwmon.rules<br />
ACTION=="add", SUBSYSTEM=="hwmon", DRIVERS=="nouveau", ATTR{pwm1_enable}="2"<br />
<br />
Sources:<br />
* http://floppym.blogspot.de/2013/07/fan-control-with-nouveau.html<br />
* https://kalgan.cc/blog/posts/Controlling_nVidia_cards_fans_with_nouveau_in_Debian/<br />
<br />
=== Optimus ===<br />
<br />
You have two solutions to use [[Optimus]] on a laptop (aka hybrid graphics, when you have two GPUs on your laptop): [[bumblebee]] and [[PRIME]]<br />
<br />
=== Vertical Sync ===<br />
Xorg compositors are prone to show issues with Nouveau. Unlike most of them, [[Compton]] offers lots of options to tweak for a smoother and tearing free result. A configuration which is expected to deliver a good result would be the following:<br />
compton -b --paint-on-overlay --unredir-if-possible --backend xr_glx_hybrid --vsync drm --glx-swap-method -1 --glx-no-stencil<br />
{{Tip|Don't forget to turn off compositing of your DE's window manager like KWin when using a different compositor.}}<br />
<br />
== Troubleshooting ==<br />
Add {{ic|1=drm.debug=14}} and {{ic|1=log_buf_len=16M}} to your [[kernel parameters]] to turn on video debugging:<br />
<br />
Create verbose Xorg log:<br />
$ startx -- -logverbose 9 -verbose 9<br />
<br />
View loaded video module parameters and values:<br />
$ modinfo -p video<br />
<br />
===Disable MSI===<br />
If you are still having problems loading the module or starting X server append {{ic|1=nouveau.config=NvMSI=0}} to your [[Kernel parameters]].<br />
<br />
Source: https://bugs.freedesktop.org/show_bug.cgi?id=78441<br />
<br />
===Phantom output issue===<br />
<br />
It is possible for the nouveau driver to detect "phantom" outputs. For example, both VGA-1 and LVDS-1 are shown as connected but only LVDS-1 is present.<br />
<br />
This causes display problems and a corrupted screen.<br />
<br />
The problem can be overcome by disabling the phantom output (VGA-1 in the examples given) on the kernel command line of your boot loader. This can be achieved by appending the following:<br />
<br />
video=VGA-1:d<br />
<br />
Where '''d''' = disable.<br />
<br />
The phantom output can also be disabled in X by adding the following to {{ic|/etc/X11/xorg.conf.d/20-nouveau.conf}}:<br />
<br />
Section "Monitor"<br />
Identifier "VGA-1"<br />
Option "Ignore" "1"<br />
EndSection<br />
<br />
Source: http://gentoo-en.vfose.ru/wiki/Nouveau#Phantom_and_unpopulated_output_connector_issues{{Dead link|2017|06|01}}<br />
<br />
===Random lockups with kernel error messages===<br />
<br />
Specific Nvidia chips with Nouveau may give random system lockups and more commonly throw many kernel messages, seen with ''dmesg''. Try adding the {{ic|1=nouveau.noaccel=1}} [[kernel parameter]]. See [https://fedoraproject.org/wiki/Common_kernel_problems#Systems_with_nVidia_adapters_using_the_nouveau_driver_lock_up_randomly] for more information.<br />
<br />
Also as an alternative you can use QT_XCB_FORCE_SOFTWARE_OPENGL=1 in your environment to disable opengl acceleration in QT applications.<br />
<br />
===Flat Panel Table Invalid===<br />
{{Expansion|Missing references to the bug reports or support threads.}}<br />
<br />
NVIDIA graphics cards with recent chipsets can cause startup issues - this includes X11 being unable to start and lspci freezing indefinitely[https://bugzilla.redhat.com/show_bug.cgi?id=1425253][https://bbs.archlinux.org/viewtopic.php?id=192532][https://stackoverflow.com/questions/28062458/nouveau-error-while-booting-arch][https://bbs.archlinux.org/viewtopic.php?id=207602][https://unix.stackexchange.com/questions/207895/how-do-i-install-antergos-with-a-gtx-970]. <br />
<br />
This can break live distributions/installation media. This can be detected either by running lspci, or checking the systemd journal for the error:<br />
<br />
nouveau E[ DRM]Pointer to flat panel table invalid<br />
<br />
The system may start if the Nouveau driver is disabled by passing the following [[kernel parameters]]:<br />
<br />
modprobe.blacklist=nouveau<br />
<br />
The Nouveau driver can then be loaded using<br />
<br />
modprobe nouveau<br />
<br />
The system should then function correctly.<br />
If you have another Nvidia graphics card, or just want to be safe, you can disable the offending card using:<br />
<br />
$ echo 1 > /sys/bus/pci/devices/[card device id]/remove<br />
<br />
The [[NVIDIA]] proprietary driver currently works correctly (version 381).</div>Nicman23https://wiki.archlinux.org/index.php?title=PCI_passthrough_via_OVMF&diff=540170PCI passthrough via OVMF2018-09-06T11:30:22Z<p>Nicman23: added subreddit</p>
<hr />
<div>[[Category:Virtualization]]<br />
[[ja:OVMF による PCI パススルー]]<br />
[[zh-hans:PCI passthrough via OVMF]]<br />
The Open Virtual Machine Firmware ([https://github.com/tianocore/tianocore.github.io/wiki/OVMF OVMF]) is a project to enable UEFI support for virtual machines. Starting with Linux 3.9 and recent versions of [[QEMU]], it is now possible to passthrough a graphics card, offering the VM native graphics performance which is useful for graphic-intensive tasks.<br />
<br />
Provided you have a desktop computer with a spare GPU you can dedicate to the host (be it an integrated GPU or an old OEM card, the brands do not even need to match) and that your hardware supports it (see [[#Prerequisites]]), it is possible to have a VM of any OS with its own dedicated GPU and near-native performance. For more information on techniques see the background [https://www.linux-kvm.org/images/b/b3/01x09b-VFIOandYou-small.pdf presentation (pdf)]. <br />
<br />
== Prerequisites ==<br />
<br />
A VGA Passthrough relies on a number of technologies that are not ubiquitous as of today and might not be available on your hardware. You will not be able to do this on your machine unless the following requirements are met :<br />
<br />
* Your CPU must support hardware virtualization (for kvm) and IOMMU (for the passthrough itself)<br />
** [https://ark.intel.com/Search/FeatureFilter?productType=processors&VTD=true List of compatible Intel CPUs (Intel VT-x and Intel VT-d)]<br />
** All AMD CPUs from the Bulldozer generation and up (including Zen) should be compatible.<br />
*** CPUs from the K10 generation (2007) do not have an IOMMU, so you '''need''' to have a motherboard with a [https://support.amd.com/TechDocs/43403.pdf#page=18 890FX] or [https://support.amd.com/TechDocs/48691.pdf#page=21 990FX] chipset to make it work, as those have their own IOMMU.<br />
* Your motherboard must also support IOMMU<br />
** Both the chipset and the BIOS must support it. It is not always easy to tell at a glance whether or not this is the case, but there is a fairly comprehensive list on the matter on the [https://wiki.xen.org/wiki/VTd_HowTo Xen wiki] as well as [[Wikipedia:List of IOMMU-supporting hardware]].<br />
* Your guest GPU ROM must support UEFI.<br />
** If you can find [https://www.techpowerup.com/vgabios/ any ROM in this list] that applies to your specific GPU and is said to support UEFI, you are generally in the clear. All GPUs from 2012 and later should support this, as Microsoft made UEFI a requirement for devices to be marketed as compatible with Windows 8.<br />
<br />
You will probably want to have a spare monitor or one with multiple input ports connected to different GPUs (the passthrough GPU will not display anything if there is no screen plugged in and using a VNC or Spice connection will not help your performance), as well as a mouse and a keyboard you can pass to your VM. If anything goes wrong, you will at least have a way to control your host machine this way.<br />
<br />
== Setting up IOMMU ==<br />
<br />
{{Note|<br />
* IOMMU is a generic name for Intel VT-d and AMD-Vi.<br />
* VT-d stands for ''Intel Virtualization Technology for Directed I/O'' and should not be confused with VT-x ''Intel Virtualization Technology''. VT-x allows one hardware platform to function as multiple “virtual” platforms while VT-d improves security and reliability of the systems and also improves performance of I/O devices in virtualized environments.}}<br />
<br />
Using IOMMU opens to features like PCI passthrough and memory protection from faulty or malicious devices, see [[wikipedia:Input-output memory management unit#Advantages]] and [https://www.quora.com/Memory-Management-computer-programming/Could-you-explain-IOMMU-in-plain-English Memory Management (computer programming): Could you explain IOMMU in plain English?].<br />
<br />
=== Enabling IOMMU ===<br />
<br />
Ensure that AMD-Vi/Intel VT-d is supported by the CPU and enabled in the BIOS settings. Both normally show up alongside other CPU features (meaning they could be in an overclocking-related menu) either with their actual names ("VT-d" or "AMD-Vi") or in more ambiguous terms such as "Virtualization technology", which may or may not be explained in the manual.<br />
<br />
Enable IOMMU support by setting the correct [[kernel parameter]] depending on the type of CPU in use:<br />
* For Intel CPUs (VT-d) set {{ic|1=intel_iommu=on}} <br />
* For AMD CPUs (AMD-Vi) set {{ic|1=amd_iommu=on}}<br />
<br />
You should also append the {{ic|1=iommu=pt}} parameter. This will prevent Linux from touching devices which cannot be passed through.<br />
<br />
After rebooting, check dmesg to confirm that IOMMU has been correctly enabled:<br />
<br />
{{hc|dmesg {{!}} grep -e DMAR -e IOMMU|<br />
[ 0.000000] ACPI: DMAR 0x00000000BDCB1CB0 0000B8 (v01 INTEL BDW 00000001 INTL 00000001)<br />
[ 0.000000] Intel-IOMMU: enabled<br />
[ 0.028879] dmar: IOMMU 0: reg_base_addr fed90000 ver 1:0 cap c0000020660462 ecap f0101a<br />
[ 0.028883] dmar: IOMMU 1: reg_base_addr fed91000 ver 1:0 cap d2008c20660462 ecap f010da<br />
[ 0.028950] IOAPIC id 8 under DRHD base 0xfed91000 IOMMU 1<br />
[ 0.536212] DMAR: No ATSR found<br />
[ 0.536229] IOMMU 0 0xfed90000: using Queued invalidation<br />
[ 0.536230] IOMMU 1 0xfed91000: using Queued invalidation<br />
[ 0.536231] IOMMU: Setting RMRR:<br />
[ 0.536241] IOMMU: Setting identity map for device 0000:00:02.0 [0xbf000000 - 0xcf1fffff]<br />
[ 0.537490] IOMMU: Setting identity map for device 0000:00:14.0 [0xbdea8000 - 0xbdeb6fff]<br />
[ 0.537512] IOMMU: Setting identity map for device 0000:00:1a.0 [0xbdea8000 - 0xbdeb6fff]<br />
[ 0.537530] IOMMU: Setting identity map for device 0000:00:1d.0 [0xbdea8000 - 0xbdeb6fff]<br />
[ 0.537543] IOMMU: Prepare 0-16MiB unity mapping for LPC<br />
[ 0.537549] IOMMU: Setting identity map for device 0000:00:1f.0 [0x0 - 0xffffff]<br />
[ 2.182790] [drm] DMAR active, disabling use of stolen memory<br />
}}<br />
<br />
=== Ensuring that the groups are valid ===<br />
<br />
The following script should allow you to see how your various PCI devices are mapped to IOMMU groups. If it does not return anything, you either have not enabled IOMMU support properly or your hardware does not support it.<br />
<br />
#!/bin/bash<br />
shopt -s nullglob<br />
for d in /sys/kernel/iommu_groups/*/devices/*; do <br />
n=${d#*/iommu_groups/*}; n=${n%%/*}<br />
printf 'IOMMU Group %s ' "$n"<br />
lspci -nns "${d##*/}"<br />
done;<br />
<br />
Example output:<br />
<br />
IOMMU Group 0 00:00.0 Host bridge [0600]: Intel Corporation 2nd Generation Core Processor Family DRAM Controller [8086:0104] (rev 09)<br />
IOMMU Group 1 00:16.0 Communication controller [0780]: Intel Corporation 6 Series/C200 Series Chipset Family MEI Controller #1 [8086:1c3a] (rev 04)<br />
IOMMU Group 2 00:19.0 Ethernet controller [0200]: Intel Corporation 82579LM Gigabit Network Connection [8086:1502] (rev 04)<br />
IOMMU Group 3 00:1a.0 USB controller [0c03]: Intel Corporation 6 Series/C200 Series Chipset Family USB Enhanced Host Controller #2 [8086:1c2d] (rev <br />
...<br />
<br />
An IOMMU group is the smallest set of physical devices that can be passed to a virtual machine. For instance, in the example above, both the GPU in 06:00.0 and its audio controller in 6:00.1 belong to IOMMU group 13 and can only be passed together. The frontal USB controller, however, has its own group (group 2) which is separate from both the USB expansion controller (group 10) and the rear USB controller (group 4), meaning that [[#USB controller|any of them could be passed to a VM without affecting the others]].<br />
<br />
=== Gotchas ===<br />
<br />
==== Plugging your guest GPU in an unisolated CPU-based PCIe slot ====<br />
<br />
Not all PCI-E slots are the same. Most motherboards have PCIe slots provided by both the CPU and the PCH. Depending on your CPU, it is possible that your processor-based PCIe slot does not support isolation properly, in which case the PCI slot itself will be appear to be grouped with the device that is connected to it.<br />
<br />
IOMMU Group 1 00:01.0 PCI bridge: Intel Corporation Xeon E3-1200 v2/3rd Gen Core processor PCI Express Root Port (rev 09)<br />
IOMMU Group 1 01:00.0 VGA compatible controller: NVIDIA Corporation GM107 [GeForce GTX 750] (rev a2)<br />
IOMMU Group 1 01:00.1 Audio device: NVIDIA Corporation Device 0fbc (rev a1)<br />
<br />
This is fine so long as only your guest GPU is included in here, such as above. Depending on what is plugged in your other PCIe slots and whether they are allocated to your CPU or your PCH, you may find yourself with additional devices within the same group, which would force you to pass those as well. If you are ok with passing everything that is in there to your VM, you are free to continue. Otherwise, you will either need to try and plug your GPU in your other PCIe slots (if you have any) and see if those provide isolation from the rest or to install the ACS override patch, which comes with its own drawbacks. See [[#Bypassing the IOMMU groups (ACS override patch)]] for more information.<br />
<br />
{{Note|If they are grouped with other devices in this manner, pci root ports and bridges should neither be bound to vfio at boot, nor be added to the VM.}}<br />
<br />
== Isolating the GPU ==<br />
<br />
In order to assign a device to a virtual machine, this device and all those sharing the same IOMMU group must have their driver replaced by a stub driver or a VFIO driver in order to prevent the host machine from interacting with them. In the case of most devices, this can be done on the fly right before the VM starts.<br />
<br />
However, due to their size and complexity, GPU drivers do not tend to support dynamic rebinding very well, so you cannot just have some GPU you use on the host be transparently passed to a VM without having both drivers conflict with each other. Because of this, it is generally advised to bind those placeholder drivers manually before starting the VM, in order to stop other drivers from attempting to claim it.<br />
<br />
The following section details how to configure a GPU so those placeholder drivers are bound early during the boot process, which makes said device inactive until a VM claims it or the driver is switched back. This is the preferred method, considering it has less caveats than switching drivers once the system is fully online.<br />
<br />
{{warning|Once you reboot after this procedure, whatever GPU you have configured will no longer be usable on the host until you reverse the manipulation. Make sure the GPU you intend to use on the host is properly configured before doing this - your motherboard should be set to display using the host GPU.}}<br />
<br />
Starting with Linux 4.1, the kernel includes vfio-pci. This is a VFIO driver, meaning it fulfills the same role as pci-stub did, but it can also control devices to an extent, such as by switching them into their D3 state when they are not in use.<br />
<br />
{{hc|$ modinfo vfio-pci|<br />
filename: /lib/modules/4.4.5-1-ARCH/kernel/drivers/vfio/pci/vfio-pci.ko.gz<br />
description: VFIO PCI - User Level meta-driver<br />
author: Alex Williamson <alex.williamson@redhat.com><br />
...<br />
}}<br />
<br />
Vfio-pci normally targets PCI devices by ID, meaning you only need to specify the IDs of the devices you intend to passthrough. For the following IOMMU group, you would want to bind vfio-pci with {{ic|10de:13c2}} and {{ic|10de:0fbb}}, which will be used as example values for the rest of this section.<br />
<br />
IOMMU Group 13 06:00.0 VGA compatible controller: NVIDIA Corporation GM204 [GeForce GTX 970] [10de:13c2] (rev a1)<br />
IOMMU Group 13 06:00.1 Audio device: NVIDIA Corporation GM204 High Definition Audio Controller [10de:0fbb] (rev a1)}}<br />
<br />
{{Note|You cannot specify which device to isolate using vendor-device ID pairs if the host GPU and the guest GPU share the same pair (i.e : if both are the same model). If this is your case, read [[#Using identical guest and host GPUs]] instead.}}<br />
<br />
You can then add those vendor-device ID pairs to the default parameters passed to vfio-pci whenever it is inserted into the kernel.<br />
<br />
{{hc|/etc/modprobe.d/vfio.conf|2=<br />
options vfio-pci ids=10de:13c2,10de:0fbb<br />
}}<br />
<br />
{{Note|If, as noted in [[#Plugging your guest GPU in an unisolated CPU-based PCIe slot]], your pci root port is part of your IOMMU group, you '''must not''' pass its ID to {{ic|vfio-pci}}, as it needs to remain attached to the host to function properly. Any other device within that group, however, should be left for {{ic|vfio-pci}} to bind with.}}<br />
<br />
This, however, does not guarantee that vfio-pci will be loaded before other graphics drivers. To ensure that, we need to statically bind it in the kernel image alongside with its dependencies. That means adding, in this order, {{ic|vfio_pci}}, {{ic|vfio}}, {{ic|vfio_iommu_type1}}, and {{ic|vfio_virqfd}} to [[mkinitcpio]]:<br />
<br />
{{hc|/etc/mkinitcpio.conf|2=<br />
MODULES=(... vfio_pci vfio vfio_iommu_type1 vfio_virqfd ...)<br />
}}<br />
<br />
{{Note|If you also have another driver loaded this way for [[Kernel mode setting#Early KMS start|early modesetting]] (such as {{ic|nouveau}}, {{ic|radeon}}, {{ic|amdgpu}}, {{ic|i915}}, etc.), all of the aforementioned VFIO modules must precede it.}}<br />
<br />
Also, ensure that the modconf hook is included in the HOOKS list of {{ic|mkinitcpio.conf}}:<br />
<br />
{{hc|/etc/mkinitcpio.conf|2=<br />
HOOKS=(... modconf ...)<br />
}}<br />
<br />
Since new modules have been added to the initramfs configuration, you must [[regenerate the initramfs]]. Should you change the IDs of the devices in {{ic|/etc/modprobe.d/vfio.conf}}, you will also have to regenerate it, as those parameters must be specified in the initramfs to be known during the early boot stages.<br />
<br />
Reboot and verify that vfio-pci has loaded properly and that it is now bound to the right devices.<br />
<br />
{{hc|$ dmesg {{!}} grep -i vfio|<br />
[ 0.329224] VFIO - User Level meta-driver version: 0.3<br />
[ 0.341372] vfio_pci: add [10de:13c2[ffff:ffff]] class 0x000000/00000000<br />
[ 0.354704] vfio_pci: add [10de:0fbb[ffff:ffff]] class 0x000000/00000000<br />
[ 2.061326] vfio-pci 0000:06:00.0: enabling device (0100 -> 0103)<br />
}}<br />
<br />
It is not necessary for all devices (or even expected device) from {{ic|vfio.conf}} to be in dmesg output. Sometimes a device does not appear in output at boot but actually is able to be visible and operatable in guest VM.<br />
<br />
{{hc|$ lspci -nnk -d 10de:13c2|<br />
06:00.0 VGA compatible controller: NVIDIA Corporation GM204 [GeForce GTX 970] [10de:13c2] (rev a1)<br />
Kernel driver in use: vfio-pci<br />
Kernel modules: nouveau nvidia<br />
}}<br />
<br />
{{hc|$ lspci -nnk -d 10de:0fbb|<br />
06:00.1 Audio device: NVIDIA Corporation GM204 High Definition Audio Controller [10de:0fbb] (rev a1)<br />
Kernel driver in use: vfio-pci<br />
Kernel modules: snd_hda_intel<br />
}}<br />
<br />
== Setting up an OVMF-based guest VM ==<br />
<br />
OVMF is an open-source UEFI firmware for QEMU virtual machines. While it's possible to use SeaBIOS to get similar results to an actual PCI passthough, the setup process is different and it is generally preferable to use the EFI method if your hardware supports it.<br />
<br />
=== Configuring libvirt ===<br />
<br />
[[Libvirt]] is a wrapper for a number of virtualization utilities that greatly simplifies the configuration and deployment process of virtual machines. In the case of KVM and QEMU, the frontend it provides allows us to avoid dealing with the permissions for QEMU and make it easier to add and remove various devices on a live VM. Its status as a wrapper, however, means that it might not always support all of the latest qemu features, which could end up requiring the use of a wrapper script to provide some extra arguments to QEMU.<br />
<br />
After installing {{Pkg|qemu}}, {{Pkg|libvirt}}, {{Pkg|ovmf}}, and {{Pkg|virt-manager}}, add the path to your OVMF firmware image and runtime variables template to your libvirt config so {{ic|virt-install}} or {{ic|virt-manager}} can find those later on.<br />
<br />
{{hc|/etc/libvirt/qemu.conf|2=<br />
nvram = [<br />
"/usr/share/ovmf/x64/OVMF_CODE.fd:/usr/share/ovmf/x64/OVMF_VARS.fd"<br />
]<br />
}}<br />
<br />
You can now [[enable]] and [[start]] {{ic|libvirtd.service}} and its logging component {{ic|virtlogd.socket}}.<br />
<br />
=== Setting up the guest OS ===<br />
<br />
The process of setting up a VM using {{ic|virt-manager}} is mostly self-explanatory, as most of the process comes with fairly comprehensive on-screen instructions.<br />
<br />
If using {{ic|virt-manager}}, you have to add your user to the libvirt [[group]] to ensure authentication.<br />
<br />
However, you should pay special attention to the following steps :<br />
<br />
* When the VM creation wizard asks you to name your VM (final step before clicking "Finish"), check the "Customize before install" checkbox.<br />
* In the "Overview" section, [https://i.imgur.com/73r2ctM.png set your firmware to "UEFI"]. If the option is grayed out, make sure that you have correctly specified the location of your firmware in {{ic|/etc/libvirt/qemu.conf}} and restart {{ic|libvirtd.service}}.<br />
* In the "CPUs" section, change your CPU model to "host-passthrough". If it is not in the list, you will have to type it by hand. This will ensure that your CPU is detected properly, since it causes libvirt to expose your CPU capabilities exactly as they are instead of only those it recognizes (which is the preferred default behavior to make CPU behavior easier to reproduce). Without it, some applications may complain about your CPU being of an unknown model.<br />
* If you want to minimize IO overhead, go into "Add Hardware" and add a Controller for SCSI drives of the "VirtIO SCSI" model. You can then change the default IDE disk for a SCSI disk, which will bind to said controller.<br />
** Windows VMs will not recognize those drives by default, so you need to download the ISO containing the drivers from [https://fedorapeople.org/groups/virt/virtio-win/direct-downloads/latest-virtio/ here] and add an IDE (or SATA for Windows 8.1 and newer) CD-ROM storage device linking to said ISO, otherwise you will not be able to get Windows to recognize it during the installation process. When prompted to select a disk to install windows on, load the drivers contained on the CD-ROM under ''vioscsi''.<br />
<br />
The rest of the installation process will take place as normal using a standard QXL video adapter running in a window. At this point, there is no need to install additional drivers for the rest of the virtual devices, since most of them will be removed later on. Once the guest OS is done installing, simply turn off the virtual machine. It is possible you will be dropped into the UEFI menu instead of starting the installation upon powering your VM for the first time. Sometimes the correct ISO file was not automatically detected and you will need to manually specify the drive to boot. By typing exit and navigating to "boot manager" you will enter a menu that allows you to choose between devices.<br />
<br />
=== Attaching the PCI devices ===<br />
<br />
With the installation done, it's now possible to edit the hardware details in libvirt and remove virtual integration devices, such as the spice channel and virtual display, the QXL video adapter, the emulated mouse and keyboard and the USB tablet device. Since that leaves you with no input devices, you may want to bind a few USB host devices to your VM as well, but remember to '''leave at least one mouse and/or keyboard assigned to your host''' in case something goes wrong with the guest. At this point, it also becomes possible to attach the PCI device that was isolated earlier; simply click on "Add Hardware" and select the PCI Host Devices you want to passthrough. If everything went well, the screen plugged into your GPU should show the OVMF splash screen and your VM should start up normally. From there, you can setup the drivers for the rest of your VM.<br />
<br />
=== Gotchas ===<br />
<br />
==== Using a non-EFI image on an OVMF-based VM ====<br />
<br />
The OVMF firmware does not support booting off non-EFI mediums. If the installation process drops you in a UEFI shell right after booting, you may have an invalid EFI boot media. Try using an alternate linux/windows image to determine if you have an invalid media.<br />
<br />
== Performance tuning ==<br />
<br />
Most use cases for PCI passthroughs relate to performance-intensive domains such as video games and GPU-accelerated tasks. While a PCI passthrough on its own is a step towards reaching native performance, there are still a few ajustments on the host and guest to get the most out of your VM.<br />
<br />
=== CPU pinning ===<br />
<br />
The default behavior for KVM guests is to run operations coming from the guest as a number of threads representing virtual processors. Those threads are managed by the Linux scheduler like any other thread and are dispatched to any available CPU cores based on niceness and priority queues. Since switching between threads adds a bit of overhead (because context switching forces the core to change its cache between operations), this can noticeably harm performance on the guest. CPU pinning aims to resolve this as it overrides process scheduling and ensures that the VM threads will always run and only run on those specific cores.<br />
<br />
{{Note|For certain users enabling CPU pinning may introduce stuttering and short hangs, especially with the MuQSS scheduler (present in linux-ck and linux-zen kernels). You might want to try disabling pinning first if you experience similar issues, which effectively trades maximum performance for responsiveness at all times.}}<br />
<br />
==== CPU topology ====<br />
<br />
Most modern CPU's support hardware multitasking, also known as hyper-threading on Intel CPU's or SMT on AMD CPU's. Hyper-threading/SMT is simply a very efficient way of running two threads on one CPU core at any given time. You will want to take into consideration that the CPU pinning you choose will greatly depend on what you do with your host while your VM is running.<br />
<br />
To find the topology for your CPU run {{ic|1=lscpu -e}}:<br />
<br />
{{Note|Pay special attention to the 4th column '''"CORE"''' as this shows the association of the Physical/Logical CPU cores}}<br />
<br />
{{ic|lscpu -e}} on a 6c/12t Ryzen 5 1600:<br />
<br />
CPU NODE SOCKET CORE L1d:L1i:L2:L3 ONLINE MAXMHZ MINMHZ<br />
0 0 0 0 0:0:0:0 yes 3800.0000 1550.0000<br />
1 0 0 0 0:0:0:0 yes 3800.0000 1550.0000<br />
2 0 0 1 1:1:1:0 yes 3800.0000 1550.0000<br />
3 0 0 1 1:1:1:0 yes 3800.0000 1550.0000<br />
4 0 0 2 2:2:2:0 yes 3800.0000 1550.0000<br />
5 0 0 2 2:2:2:0 yes 3800.0000 1550.0000<br />
6 0 0 3 3:3:3:1 yes 3800.0000 1550.0000<br />
7 0 0 3 3:3:3:1 yes 3800.0000 1550.0000<br />
8 0 0 4 4:4:4:1 yes 3800.0000 1550.0000<br />
9 0 0 4 4:4:4:1 yes 3800.0000 1550.0000<br />
10 0 0 5 5:5:5:1 yes 3800.0000 1550.0000<br />
11 0 0 5 5:5:5:1 yes 3800.0000 1550.0000<br />
<br />
{{ic|lscpu -e}} on a 6c/12t Intel 8700k:<br />
<br />
CPU NODE SOCKET CORE L1d:L1i:L2:L3 ONLINE MAXMHZ MINMHZ<br />
0 0 0 0 0:0:0:0 yes 4600.0000 800.0000<br />
1 0 0 1 1:1:1:0 yes 4600.0000 800.0000<br />
2 0 0 2 2:2:2:0 yes 4600.0000 800.0000<br />
3 0 0 3 3:3:3:0 yes 4600.0000 800.0000<br />
4 0 0 4 4:4:4:0 yes 4600.0000 800.0000<br />
5 0 0 5 5:5:5:0 yes 4600.0000 800.0000<br />
6 0 0 0 0:0:0:0 yes 4600.0000 800.0000<br />
7 0 0 1 1:1:1:0 yes 4600.0000 800.0000<br />
8 0 0 2 2:2:2:0 yes 4600.0000 800.0000<br />
9 0 0 3 3:3:3:0 yes 4600.0000 800.0000<br />
10 0 0 4 4:4:4:0 yes 4600.0000 800.0000<br />
11 0 0 5 5:5:5:0 yes 4600.0000 800.0000<br />
<br />
As we see above, with AMD '''Core 0''' is sequential with '''CPU 0 & 1''', whereas Intel places '''Core 0''' on '''CPU 0 & 6'''.<br />
<br />
If you prefer to have the host and guest running intensive tasks at the same time, it would then be preferable to leave at the very least, '''Core 0''' to the host. Next you will pin the amount of physical CPU's and their associated logical CPU's to the guest XML as shown below. It is also recommended to pin the emulator and iothread to '''Core 0''' as this helps eliminate latency within the VM.<br />
<br />
==== XML examples ====<br />
{{Note|If you are not using Virtio storage drivers for your VM, ''do not'' use the '''iothread''' lines from the XML examples that are shown below.}}<br />
===== 4c/1t CPU w/o Hyperthreading Example =====<br />
{{hc|$ virsh edit [vmname]|<nowiki><br />
...<br />
<vcpu placement='static'>4</vcpu><br />
<cputune><br />
<vcpupin vcpu='0' cpuset='0'/><br />
<vcpupin vcpu='1' cpuset='1'/><br />
<vcpupin vcpu='2' cpuset='2'/><br />
<vcpupin vcpu='3' cpuset='3'/><br />
</cputune><br />
...<br />
</nowiki>}}<br />
<br />
===== 4c/2t Intel CPU pinning example =====<br />
{{hc|$ virsh edit [vmname]|<nowiki><br />
...<br />
<vcpu placement='static'>8</vcpu><br />
<iothreads>1</iothreads><br />
<cputune><br />
<vcpupin vcpu='0' cpuset='2'/><br />
<vcpupin vcpu='1' cpuset='8'/><br />
<vcpupin vcpu='2' cpuset='3'/><br />
<vcpupin vcpu='3' cpuset='9'/><br />
<vcpupin vcpu='4' cpuset='4'/><br />
<vcpupin vcpu='5' cpuset='10'/><br />
<vcpupin vcpu='6' cpuset='5'/><br />
<vcpupin vcpu='7' cpuset='11'/><br />
<emulatorpin cpuset='0,6'/><br />
<iothreadpin iothread='1' cpuset='0,6'/><br />
</cputune><br />
...<br />
<topology sockets='1' cores='4' threads='2'/><br />
...<br />
</nowiki>}}<br />
<br />
===== 4c/2t AMD CPU example =====<br />
{{hc|$ virsh edit [vmname]|<nowiki><br />
...<br />
<vcpu placement='static'>8</vcpu><br />
<iothreads>1</iothreads><br />
<cputune><br />
<vcpupin vcpu='0' cpuset='2'/><br />
<vcpupin vcpu='1' cpuset='3'/><br />
<vcpupin vcpu='2' cpuset='4'/><br />
<vcpupin vcpu='3' cpuset='5'/><br />
<vcpupin vcpu='4' cpuset='6'/><br />
<vcpupin vcpu='5' cpuset='7'/><br />
<vcpupin vcpu='6' cpuset='8'/><br />
<vcpupin vcpu='7' cpuset='9'/><br />
<emulatorpin cpuset='0-1'/><br />
<iothreadpin iothread='1' cpuset='0-1'/><br />
</cputune><br />
...<br />
<topology sockets='1' cores='4' threads='2'/><br />
...<br />
</nowiki>}}<br />
<br />
{{Note|If further CPU isolation is needed, consider using the '''isolcpus''' kernel command-line parameter on the unused physical/logical cores.}}<br />
<br />
If you do not intend to be doing any computation-heavy work on the host (or even anything at all) at the same time as you would on the VM, you may want to pin your VM threads across all of your cores, so that the VM can fully take advantage of the spare CPU time the host has available. Be aware that pinning all physical and logical cores of your CPU could induce latency in the guest VM.<br />
<br />
=== Static huge pages ===<br />
<br />
When dealing with applications that require large amounts of memory, memory latency can become a problem since the more memory pages are being used, the more likely it is that this application will attempt to access information across multiple memory "pages", which is the base unit for memory allocation. Resolving the actual address of the memory page takes multiple steps, and so CPUs normally cache information on recently used memory pages to make subsequent uses on the same pages faster. Applications using large amounts of memory run into a problem where, for instance, a virtual machine uses 4GB of memory divided into 4kB pages (which is the default size for normal pages), meaning that such cache misses can become extremely frequent and greatly increase memory latency. Huge pages exist to mitigate this issue by giving larger individual pages to those applications, increasing the odds that multiple operations will target the same page in succession. This is normally handeled with transparent huge pages, which dynamically manages hugepages to keep up with the demand.<br />
<br />
On a VM with a PCI passthrough, however, it is '''not possible''' to benefit from transparent huge pages, as IOMMU requires that the guest's memory be allocated and pinned as soon as the VM starts. It is therefore required to allocate huge pages statically in order to benefit from them. <br />
<br />
{{Warning|Static huge pages lock down the allocated amount of memory, making it unavailable for applications that are not configured to use them. Allocating 4GBs worth of huge pages on a machine with 8GBs of memory will only leave you with 4GBs of available memory on the host '''even when the VM is not running'''.}}<br />
<br />
To allocate huge pages at boot, one must simply specify the desired amount on their kernel command line with {{ic|1=hugepages=''x''}}. For instance, reserving 1024 pages with {{ic|1=hugepages=1024}} and the default size of 2048kB per huge page creates 2GBs worth of memory for the virtual machine to use.<br />
<br />
If supported by CPU page size could be set manually. 1 GB huge page support could be verified by {{ic|<nowiki>grep pdpe1gb /proc/cpuinfo</nowiki>}}. Setting 1 GB huge page size via kernel parameters : {{ic|1=default_hugepagesz=1G hugepagesz=1G hugepages=X transparent_hugepage=never}}.<br />
<br />
Also, since static huge pages can only be used by applications that specifically request it, you must add this section in your libvirt domain configuration to allow kvm to benefit from them :<br />
<br />
{{hc|$ virsh edit [vmname]|<nowiki><br />
...<br />
<memoryBacking><br />
<hugepages/><br />
</memoryBacking><br />
...<br />
</nowiki>}}<br />
<br />
=== Dynamic huge pages ===<br />
<br />
{{Accuracy|Need futher testing if this variant as effective as static one}}<br />
<br />
Hugepages could be allocated dynamically via vm.nr_overcommit_hugepages parameter.<br />
<br />
{{hc|sysctl.d/10-kvm.conf|<nowiki><br />
vm.nr_hugepages = 0<br />
vm.nr_overcommit_hugepages = <num><br />
</nowiki>}}<br />
<br />
Where num - is the number of huge pages, which default size if 2Mb.<br />
Pages will be automatically allocated, and freed after VM stops.<br />
<br />
More manual way <br />
{{hc|manual dynamic huge pages|<nowiki><br />
echo <num> | sudo tee /sys/kernel/mm/hugepages/hugepages-2048kB/nr_hugepages<br />
echo <num> | sudo tee /sys/kernel/mm/hugepages/hugepages-1048576kB/nr_hugepages<br />
</nowiki>}}<br />
<br />
For 2Mb and 1Gb page size respectively.<br />
And they should be manually freed in the same way.<br />
<br />
It is hardly recommended to drop caches, compact memory and wait couple of seconds before starting VM, as there could be not enough free contiguous memory for required huge pages blocks. Especially after some uptime of the host system.<br />
{{hc|organize ram|<nowiki><br />
echo 3 | sudo tee /proc/sys/vm/drop_caches<br />
echo 1 | sudo tee /proc/sys/vm/compact_memory<br />
</nowiki>}}<br />
<br />
Theoretically, 1Gb pages works as 2Mb. But practically - no guaranteed way was found to get contiguous 1Gb memory blocks. Each consequent request of 1Gb blocks lead to lesser and lesser dynamically allocated count.<br />
<br />
=== CPU frequency governor ===<br />
<br />
Depending on the way your [[CPU frequency scaling|CPU governor]] is configured, the VM threads may not hit the CPU load thresholds for the frequency to ramp up. Indeed, KVM cannot actually change the CPU frequency on its own, which can be a problem if it does not scale up with vCPU usage as it would result in underwhelming performance. An easy way to see if it behaves correctly is to check if the frequency reported by {{ic|watch lscpu}} goes up when running a CPU-intensive task on the guest. If you are indeed experiencing stutter and the frequency does not go up to reach its reported maximum, it may be due to [https://lime-technology.com/forum/index.php?topic=46664.msg447678#msg447678 cpu scaling being controlled by the host OS]. In this case, try setting all cores to maximum frequency to see if this improves performance. Note that if you are using a modern intel chip with the default pstate driver, cpupower commands will be [[CPU frequency scaling#CPU frequency driver|ineffective]], so monitor {{ic|/proc/cpuinfo}} to make sure your cpu is actually at max frequency.<br />
<br />
=== CPU pinning with isolcpus ===<br />
<br />
Alternatively, make sure that you have isolated CPUs properly. In this example, let us assume you are using CPUs 4-7.<br />
Use the kernel parameters {{ic|isolcpus nohz_full rcu_nocbs}} to completely isolate the CPUs from the kernel. <br />
<br />
{{hc|sudo vim /etc/defaults/grub|<nowiki><br />
...<br />
GRUB_CMDLINE_LINUX="..your other params.. isolcpus=4-7 nohz_full=4-7 rcu_nocbs=4-7"<br />
...<br />
</nowiki>}}<br />
<br />
Then, run {{ic|qemu-system-x86_64}} with taskset and chrt:<br />
<br />
# chrt -r 1 taskset -c 4-7 qemu-system-x86_64 ...<br />
<br />
The {{ic|chrt}} command will ensure that the task scheduler will round-robin distribute work (otherwise it will all stay on the first cpu). For {{ic|taskset}}, the CPU numbers can be comma- and/or dash-separated, like "0,1,2,3" or "0-4" or "1,7-8,10" etc.<br />
<br />
See [https://www.reddit.com/r/VFIO/comments/6vgtpx/high_dpc_latency_and_audio_stuttering_on_windows/dm0sfto/ this reddit comment] for more info.<br />
<br />
=== Improving performance on AMD CPUs ===<br />
<br />
Previously, Nested Page Tables (NPT) had to be disabled on AMD systems running KVM to improve GPU performance because of a [https://sourceforge.net/p/kvm/bugs/230/ very old bug], but the trade off was decreased CPU performance, including stuttering.<br />
<br />
There is a [https://patchwork.kernel.org/patch/10027525/ kernel patch] that resolves this issue, which was accepted into kernel 4.14-stable and 4.9-stable. If you are running the official {{Pkg|linux}} or {{Pkg|linux-lts}} kernel the patch has already been applied (make sure you are on the latest). If you are running another kernel you might need to manually patch yourself.<br />
<br />
{{Note|Several Ryzen users (see [https://www.reddit.com/r/VFIO/comments/78i3jx/possible_fix_for_the_npt_issue_discussed_on_iommu/ this Reddit thread]) have tested the patch, and can confirm that it works, bringing GPU passthrough performance up to near native quality.}}<br />
<br />
Starting with QEMU 3.1 the TOPOEXT cpuid flag is disabled by default. In order to use hyperthreading(SMT) on AMD CPU's you need to manually enable it:<br />
<br />
<cpu mode='host-passthrough' check='none'><br />
<topology sockets='1' cores='4' threads='2'/><br />
<feature policy='require' name='topoext'/><br />
</cpu><br />
<br />
commit: https://git.qemu.org/?p=qemu.git;a=commit;h=7210a02c58572b2686a3a8d610c6628f87864aed<br />
<br />
=== Further tuning ===<br />
<br />
More specialized VM tuning tips are available at [https://access.redhat.com/documentation/en-US/Red_Hat_Enterprise_Linux/7/html-single/Virtualization_Tuning_and_Optimization_Guide/index.html Red Hat's Virtualization Tuning and Optimization Guide]. This guide may be especially helpful if you are experiencing:<br />
<br />
* Moderate CPU load on the host during downloads/uploads from within the guest: See ''Bridge Zero Copy Transmit'' for a potential fix.<br />
* Guests capping out at certain network speeds during downloads/uploads despite virtio-net being used: See ''Multi-queue virtio-net'' for a potential fix.<br />
* Guests "stuttering" under high I/O, despite the same workload not affecting hosts to the same degree: See ''Multi-queue virtio-scsi'' for a potential fix.<br />
<br />
== Special procedures ==<br />
<br />
Certain setups require specific configuration tweaks in order to work properly. If you are having problems getting your host or your VM to work properly, see if your system matches one of the cases below and try adjusting your configuration accordingly.<br />
<br />
=== Using identical guest and host GPUs ===<br />
<br />
{{Expansion|A number of users have been having issues with this, it should probably be adressed by the article.|Talk:PCI passthrough via OVMF#Additionnal sections}}<br />
<br />
Due to how vfio-pci uses your vendor and device id pair to identify which device they need to bind to at boot, if you have two GPUs sharing such an ID pair you will not be able to get your passthough driver to bind with just one of them. This sort of setup makes it necessary to use a script, so that whichever driver you are using is instead assigned by pci bus address using the {{ic|driver_override}} mechanism.<br />
<br />
==== Script variants ====<br />
<br />
===== Passthrough all GPUs but the boot GPU =====<br />
<br />
Here, we will make a script to bind vfio-pci to all GPUs but the boot gpu. Create the script {{ic|/usr/bin/vfio-pci-override.sh}}:<br />
<br />
{{bc|<nowiki><br />
#!/bin/sh<br />
<br />
for i in /sys/devices/pci*/*/boot_vga; do<br />
if [ $(cat "$i") -eq 0 ]; then<br />
GPU="${i%/boot_vga}"<br />
AUDIO="$(echo "$GPU" | sed -e "s/0$/1/")"<br />
echo "vfio-pci" > "$GPU/driver_override"<br />
if [ -d "$AUDIO" ]; then<br />
echo "vfio-pci" > "$AUDIO/driver_override"<br />
fi<br />
fi<br />
done<br />
<br />
modprobe -i vfio-pci<br />
</nowiki>}}<br />
<br />
===== Passthrough selected GPU =====<br />
<br />
In this case we manually specify the GPU to bind.<br />
<br />
{{bc|<nowiki><br />
#!/bin/sh<br />
<br />
GROUP="0000:00:03.0"<br />
DEVS="0000:03:00.0 0000:03:00.1 ."<br />
<br />
if [ ! -z "$(ls -A /sys/class/iommu)" ]; then<br />
for DEV in $DEVS; do<br />
echo "vfio-pci" > /sys/bus/pci/devices/$GROUP/$DEV/driver_override<br />
done<br />
fi<br />
<br />
modprobe -i vfio-pci<br />
</nowiki>}}<br />
<br />
==== Script installation ====<br />
<br />
Create {{ic|/etc/modprobe.d/vfio.conf}} with the following:<br />
<br />
install vfio-pci /usr/bin/vfio-pci-override.sh<br />
<br />
Edit {{ic|/etc/mkinitcpio.conf}}<br />
<br />
Remove any video drivers from MODULES, and add {{ic|vfio-pci}}, and {{ic|vfio_iommu_type1}}<br />
<br />
MODULES=(ext4 vfat vfio-pci vfio_iommu_type1)<br />
<br />
Add {{ic|/etc/modprobe.d/vfio.conf}} and {{ic|/usr/bin/vfio-pci-override.sh}} to FILES:<br />
<br />
FILES=(/etc/modprobe.d/vfio.conf /usr/bin/vfio-pci-override.sh)<br />
<br />
[[Regenerate the initramfs]] and reboot:<br />
<br />
=== Passing the boot GPU to the guest ===<br />
<br />
{{Expansion|This is related to VBIOS issues and should be moved into a separate section regarding VBIOS compatibility.|section=UEFI (OVMF) Compatibility in VBIOS}}<br />
<br />
The GPU marked as {{ic|boot_vga}} is a special case when it comes to doing PCI passthroughs, since the BIOS needs to use it in order to display things like boot messages or the BIOS configuration menu. To do that, it makes [https://www.redhat.com/archives/vfio-users/2016-May/msg00224.html a copy of the VGA boot ROM which can then be freely modified]. This modified copy is the version the system gets to see, which the passthrough driver may reject as invalid. As such, it is generally recommended to change the boot GPU in the BIOS configuration so the host GPU is used instead or, if that's not possible, to swap the host and guest cards in the machine itself.<br />
<br />
=== Using Looking Glass to stream guest screen to the host ===<br />
<br />
It is possible to make VM share the monitor, and optionally a keyboard and a mouse with a help of [https://looking-glass.hostfission.com/ Looking Glass].<br />
<br />
==== Adding IVSHMEM Device to VM ====<br />
<br />
Looking glass works by creating a shared memory buffer between a host and a guest. This is a lot faster than streaming frames via localhost, but requires additional setup. <br />
<br />
With your VM turned off open the machine configuration<br />
<br />
{{hc|$ virsh edit [vmname]|<nowiki><br />
...<br />
<devices><br />
...<br />
<shmem name='looking-glass'><br />
<model type='ivshmem-plain'/><br />
<size unit='M'>32</size><br />
</shmem><br />
</devices><br />
...<br />
</nowiki>}}<br />
<br />
You should replace 32 with your own calculated value based on what resolution you are going to pass through. It can be calculated like this:<br />
<br />
width x height x 4 x 2 = total bytes<br />
total bytes / 1024 / 1024 = total megabytes + 2<br />
<br />
For example, in case of 1920x1080<br />
<br />
1920 x 1080 x 4 x 2 = 16,588,800 bytes<br />
16,588,800 / 1024 / 1024 = 15.82 MB + 2 = 17.82<br />
<br />
The result must be '''rounded up''' to the nearest power of two, and since 17.82 is bigger than 16 we should choose 32.<br />
<br />
Next create a script to create a shared memory file.<br />
<br />
{{hc|/usr/local/bin/looking-glass-init.sh|2=<br />
#!/bin/sh<br />
<br />
touch /dev/shm/looking-glass<br />
chown '''user''':kvm /dev/shm/looking-glass<br />
chmod 660 /dev/shm/looking-glass<br />
}}<br />
<br />
Replace user with your username.<br />
<br />
Remember to make the script [[executable]].<br />
<br />
Create a [[systemd]] unit to execute this script during boot<br />
<br />
{{hc|/etc/systemd/system/looking-glass-init.service|2=<br />
[Unit]<br />
Description=Create shared memory for looking glass<br />
<br />
[Service]<br />
Type=oneshot<br />
ExecStart=/usr/local/bin/looking-glass-init.sh<br />
<br />
[Install]<br />
WantedBy=multi-user.target<br />
}}<br />
<br />
[[Start]] and [[enable]] {{ic|looking-glass-init.service}}<br />
<br />
==== Installing the IVSHMEM Host to Windows guest ====<br />
<br />
Currently Windows would not notify users about a new IVSHMEM device, it would silently install a dummy driver. To actually enable the device you have to go into device manager and update the driver for the device under the "System Devices" node for '''"PCI standard RAM Controller"'''. Download a signed driver [https://github.com/virtio-win/kvm-guest-drivers-windows/issues/217 from issue 217], as it is not yet available elsewhere.<br />
<br />
Once the driver is installed you must download the latest [https://looking-glass.hostfission.com/downloads looking-glass-host] from the looking glass website and start it on your guest. In order to run it you would also need to install Microsoft Visual C++ Redistributable from [https://www.visualstudio.com/downloads/ Microsoft]<br />
It is also possible to make it start automatically on VM boot by editing the {{ic|HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Windows\CurrentVersion\Run}} registry and adding a path to the downloaded executable.<br />
<br />
==== Getting a client ====<br />
<br />
Looking glass client can be installed from AUR using {{AUR|looking-glass}} or {{AUR|looking-glass-git}} packages.<br />
<br />
You can start it once the VM is set up and running<br />
<br />
$ looking-glass-client<br />
<br />
If you do not want to use Spice to control the guest mouse and keyboard you can disable the Spice server.<br />
<br />
$ looking-glass-client -s<br />
<br />
Additionally you may want to start Looking Glass Client as full screen, otherwise the image may be scaled down resulting in poor image fidelity.<br />
<br />
$ looking-glass-client -F<br />
<br />
Launch with the {{ic|--help}} option for further information.<br />
<br />
=== Swap peripherals to and from the Host ===<br />
<br />
Looking Glass includes a Spice client in order to control mouse movement on the Windows guest. However this may have too much latency for certain applications, such as gaming. An alternative method is passing through specific USB devices for minimal latency. This allows for switching the devices between host and guest.<br />
<br />
First create a .xml file for the device(s) you wish to pass-through, which libvirt will use to identify the device.<br />
<br />
{{hc|/home/$USER/.VFIOinput/input_1.xml|2=<br />
<hostdev mode='subsystem' type='usb' managed='no'><br />
<source><br />
<vendor id='0x[Before Colon]'/><br />
<product id='0x[After Colon]'/><br />
</source><br />
</hostdev><br />
}}<br />
<br />
Replace [Before/After Colon] with the contents of the 'lsusb' command, specific to the device you want to pass-through.<br />
<br />
For instance my mouse is {{ic|Bus 005 Device 002: ID 1532:0037 Razer USA, Ltd}} so I would replace {{ic|vendor id}} with 1532, and {{ic|product id}} with 1037.<br />
<br />
Repeat this process for any additional USB devices you want to pass-through. If your mouse / keyboard has multiple entries in {{ic|lsusb}}, perhaps if it is wireless, then create additional xml files for each.<br />
<br />
{{Note|Do not forget to change the path & name of the script(s) above and below to match your user and specific system.}}<br />
<br />
Next a bash script file is needed to tell libvirt what to attach/detach the USB devices to the guest.<br />
<br />
{{hc|/home/$USER/.VFIOinput/input_attach.sh|2=<br />
#!/bin/bash<br />
<br />
virsh attach-device [VM-Name] [USBdevice]<br />
}}<br />
<br />
Replace [VM-Name] with the name of your virtual machine, which can be seen under virt-manager. Additionally replace [USBdevice] with the '''full''' path to the .xml file for the device you wish to pass-through. Add additional lines for more than 1 device. For example here is my script:<br />
<br />
{{hc|/home/ajmar/.VFIOinput/input_attach.sh|2=<br />
#!/bin/bash<br />
<br />
virsh attach-device win10 /home/ajmar/.VFIOinput/input_mouse.xml<br />
virsh attach-device win10 /home/ajmar/.VFIOinput/input_keyboard.xml<br />
}}<br />
<br />
Next duplicate the script file and replace {{ic|attach-device}} with {{ic|detach-device}}. Ensure both scripts are executable with {{ic|chmod +x $script.sh}}<br />
<br />
This 2 script files can now be executed to attach or detach your USB devices from the host to the guest VM. It is important to note that they may need to be executed as root. To run the script from the Windows VM, one possibility is using [[PuTTY]] to [[SSH]] into the host, and execute the script. On Windows PuTTY comes with plink.exe which can execute singular commands over SSH before then logging out, instead of opening a SSH terminal, all in the background.<br />
<br />
{{hc|detach_devices.bat|2=<br />
"C:\Program Files\PuTTY\plink.exe" root@$HOST_IP -pw $ROOTPASSWORD /home/$USER/.VFIOinput/input_detach.sh<br />
}}<br />
<br />
Replace {{ic|$HOST_IP}} with the Host [[Network configuration#IP addresses|IP Address]] and $ROOTPASSWORD with the root password.<br />
<br />
{{warning|This method is insecure if somebody has access to your VM, since they could open the file and read your password. It is advisable to use [[SSH keys]] instead!}}<br />
<br />
You may also want to execute the script files using key binds. On Windows one option is [https://autohotkey.com/ Autohotkey], and on the Host [[Xbindkeys]]. Because of the need to run the scripts as root, you may also need to use [[Polkit]] which can be used to authenticate specific executables as able to run as root without needing a password.<br />
<br />
=== Bypassing the IOMMU groups (ACS override patch) ===<br />
<br />
If you find your PCI devices grouped among others that you do not wish to pass through, you may be able to seperate them using Alex Williamson's ACS override patch. Make sure you understand [https://vfio.blogspot.com/2014/08/iommu-groups-inside-and-out.html the potential risk] of doing so. <br />
<br />
You will need a kernel with the patch applied. The easiest method to acquiring this is through the {{AUR|linux-vfio}} package. <br />
<br />
In addition, the ACS override patch needs to be enabled with kernel command line options. The patch file adds the following documentation:<br />
<br />
pcie_acs_override =<br />
[PCIE] Override missing PCIe ACS support for:<br />
downstream<br />
All downstream ports - full ACS capabilties<br />
multifunction<br />
All multifunction devices - multifunction ACS subset<br />
id:nnnn:nnnn<br />
Specfic device - full ACS capabilities<br />
Specified as vid:did (vendor/device ID) in hex<br />
<br />
The option {{ic|1=pcie_acs_override=downstream}} is typically sufficient.<br />
<br />
After installation and configuration, reconfigure your [[Kernel parameters|bootloader kernel parameters]] to load the new kernel with the {{ic|1=pcie_acs_override=}} option enabled.<br />
<br />
== Plain QEMU without libvirt ==<br />
<br />
Instead of setting up a virtual machine with the help of libvirt, plain QEMU commands with custom parameters can be used for running the VM intended to be used with PCI passthrough. This is desirable for some use cases like scripted setups, where the flexibility for usage with other scripts is needed.<br />
<br />
To achieve this after [[#Setting up IOMMU]] and [[#Isolating the GPU]], follow the [[QEMU]] article to setup the virtualized environment, [[QEMU#Enabling KVM|enable KVM]] on it and use the flag {{ic|1=-device vfio-pci,host=07:00.0}} replacing the identifier (07:00.0) with your actual device's ID that you used for the GPU isolation earlier.<br />
<br />
For utilizing the OVMF firmware, make sure the {{Pkg|ovmf}} package is installed, copy the UEFI variables from {{ic|/usr/share/ovmf/x64/OVMF_VARS.fd}} to temporary location like {{ic|/tmp/MY_VARS.fd}} and finally specify the OVMF paths by appending the following parameters to the QEMU command (order matters):<br />
<br />
* {{ic|1=-drive if=pflash,format=raw,readonly,file=/usr/share/ovmf/x64/OVMF_CODE.fd}} for the actual OVMF firmware binary, note the readonly option<br />
* {{ic|1=-drive if=pflash,format=raw,file=/tmp/MY_VARS.fd}} for the variables<br />
<br />
{{Note|QEMU's default SeaBIOS can be used instead of OVMF, but it's not recommended as it can cause issues with passthrough setups.}}<br />
<br />
It's recommended to study the QEMU article for ways to enhance the performance by using the [[QEMU#Installing virtio drivers|virtio drivers]] and other further configurations for the setup.<br />
<br />
You also might have to use the {{ic|1=-cpu host,kvm=off}} parameter to forward the host's CPU model info to the VM and fool the virtualization detection used by Nvidia's and possibly other manufacturers' device drivers trying to block the full hardware usage inside a virtualized system.<br />
<br />
== Passing though other devices ==<br />
<br />
=== USB controller ===<br />
<br />
If your motherboard has multiple USB controllers mapped to multiple groups, it is possible to pass those instead of USB devices. Passing an actual controller over an individual USB device provides the following advantages : <br />
<br />
* If a device disconnects or changes ID over the course of an given operation (such as a phone undergoing an update), the VM will not suddenly stop seeing it.<br />
* Any USB port managed by this controller is directly handled by the VM and can have its devices unplugged, replugged and changed without having to notify the hypervisor.<br />
* Libvirt will not complain if one of the USB devices you usually pass to the guest is missing when starting the VM.<br />
<br />
Unlike with GPUs, drivers for most USB controllers do not require any specific configuration to work on a VM and control can normally be passed back and forth between the host and guest systems with no side effects.<br />
<br />
{{Warning|Make sure your USB controller supports resetting: [[#Passing through a device that does not support resetting]]}}<br />
<br />
You can find out which PCI devices correspond to which controller and how various ports and devices are assigned to each one of them using this command :<br />
<br />
{{hc|$ <nowiki>for usb_ctrl in $(find /sys/bus/usb/devices/usb* -maxdepth 0 -type l); do pci_path="$(dirname "$(realpath "${usb_ctrl}")")"; echo "Bus $(cat "${usb_ctrl}/busnum") --> $(basename $pci_path) (IOMMU group $(basename $(realpath $pci_path/iommu_group)))"; lsusb -s "$(cat "${usb_ctrl}/busnum"):"; echo; done</nowiki>|<br />
Bus 1 --> 0000:00:1a.0 (IOMMU group 4)<br />
Bus 001 Device 004: ID 04f2:b217 Chicony Electronics Co., Ltd Lenovo Integrated Camera (0.3MP)<br />
Bus 001 Device 007: ID 0a5c:21e6 Broadcom Corp. BCM20702 Bluetooth 4.0 [ThinkPad]<br />
Bus 001 Device 008: ID 0781:5530 SanDisk Corp. Cruzer<br />
Bus 001 Device 002: ID 8087:0024 Intel Corp. Integrated Rate Matching Hub<br />
Bus 001 Device 001: ID 1d6b:0002 Linux Foundation 2.0 root hub<br />
<br />
Bus 2 --> 0000:00:1d.0 (IOMMU group 9)<br />
Bus 002 Device 006: ID 0451:e012 Texas Instruments, Inc. TI-Nspire Calculator<br />
Bus 002 Device 002: ID 8087:0024 Intel Corp. Integrated Rate Matching Hub<br />
Bus 002 Device 001: ID 1d6b:0002 Linux Foundation 2.0 root hub<br />
}}<br />
<br />
This laptop has 3 USB ports managed by 2 USB controllers, each with their own IOMMU group. In this example, Bus 001 manages a single USB port (with a SanDisk USB pendrive plugged into it so it appears on the list), but also a number of internal devices, such as the internal webcam and the bluetooth card. Bus 002, on the other hand, does not apprear to manage anything except for the calculator that is plugged into it. The third port is empty, which is why it does not show up on the list, but is actually managed by Bus 002.<br />
<br />
Once you have identified which controller manages which ports by plugging various devices into them and decided which one you want to passthrough, simply add it to the list of PCI host devices controlled by the VM in your guest configuration. No other configuration should be needed.<br />
<br />
{{Note|If your USB controller does not support resetting, is not in an isolated group, or is otherwise unable to be passed through then it may still be possible to accomplish similar results through [[udev]] rules. See [https://github.com/olavmrk/usb-libvirt-hotplug] which allows any device connected to specified USB ports to be automatically attached to a virtual machine.}}<br />
<br />
=== Passing VM audio to host via PulseAudio ===<br />
<br />
It is possible to route the virtual machine's audio to the host as an application using libvirt. This has the advantage of multiple audio streams being routable to one host output, and working with audio output devices that do not support passthrough. [[PulseAudio]] is required for this to work.<br />
<br />
First, remove the comment from the {{ic|1=#user = ""}} line. Then add your username in the quotations. This tells QEMU which user's pulseaudio stream to route through.<br />
<br />
{{hc|/etc/libvirt/qemu.conf|2=<br />
user = "example"<br />
}}<br />
<br />
Next, modify the libvirt configuration<br />
<br />
{{hc|$ virsh edit [vmname]|<br />
<domain type<nowiki>=</nowiki>'kvm'><br />
}}<br />
<br />
to<br />
<br />
{{hc|$ virsh edit [vmname]|<br />
<domain type<nowiki>=</nowiki>'kvm' xmlns:qemu<nowiki>='http://libvirt.org/schemas/domain/qemu/1.0'</nowiki>><br />
}}<br />
<br />
Then set the QEMU PulseAudio environment variables at the bottom of the libvirt xml file<br />
<br />
{{hc|$ virsh edit [vmname]|<br />
</devices><br />
</domain><br />
}}<br />
<br />
to<br />
<br />
{{hc|$ virsh edit [vmname]|<br />
</devices><br />
<qemu:commandline><br />
<qemu:env name<nowiki>=</nowiki>'QEMU_AUDIO_DRV' value<nowiki>=</nowiki>'pa'/><br />
<qemu:env name<nowiki>=</nowiki>'QEMU_PA_SERVER' value<nowiki>=</nowiki>'/run/user/1000/pulse/native'/><br />
</qemu:commandline><br />
</domain><br />
}}<br />
<br />
Change 1000 under the user directory to your user uid (which can be found by running the {{ic|id}} command. Remember to save the file and exit it without ending the process before continuing, otherwise the changes will not register. If you get the message {{ic|<nowiki>Domain [vmname] XML configuration edited.</nowiki>}} after exiting, it means that your changes have been applied.<br />
<br />
[[Restart]] {{ic|libvirtd.service}} and [[systemd/User|user service]] {{ic|pulseaudio.service}}.<br />
<br />
Virtual Machine audio will now be routed through the host as an application. The application {{Pkg|pavucontrol}} can be used to control the output device. Be aware that on Windows guests, this can cause audio crackling without [[#Slowed down audio pumped through HDMI on the video card|using Message-Signaled Interrupts.]]<br />
<br />
==== QEMU 3.0 audio changes ====<br />
As of QEMU 3.0 part of the audio patches have been merged ([https://www.reddit.com/r/VFIO/comments/97iuov/qemu_30_released/e49wmyd/ reddit link]). The {{AUR|qemu-patched}} package currently includes some additional audio patches, as some of the patches have not been officially up-streamed yet.<br />
<br />
You will need to change the chipset accordingly to how your VM is set up, i.e. {{ic|pc-q35-3.0}} or {{ic|pc-i440fx-3.0}} (after installing qemu 3.0) to use the new code paths:<br />
<br />
{{hc|$ virsh edit [vmname]|<br />
<domain type<nowiki>=</nowiki>'kvm'><br />
...<br />
<os><br />
<type arch<nowiki>=</nowiki>'x86_64' machine<nowiki>=</nowiki>'pc-q35-3.0'>hvm</type><br />
...<br />
</os><br />
}}<br />
<br />
{{hc|$ virsh edit [vmname]|<br />
<domain type<nowiki>=</nowiki>'kvm'><br />
...<br />
<os><br />
<type arch<nowiki>=</nowiki>'x86_64' machine<nowiki>=</nowiki>'pc-i440fx-3.0'>hvm</type><br />
...<br />
</os><br />
}}<br />
<br />
{{Note|<br />
* To speed up compilation time with {{AUR|qemu-patched}} use {{ic|1=--target-list=x86_64-softmmu}} to compile qemu with only x86_64 guest support.<br />
* Since Qemu 3.0 the XML arguments {{ic|qemu:env}} above are ''not'' needed if you run PulseAudio as your user and you have {{ic|1= nographics_allow_host_audio = 1}} enabled in {{ic|1=/etc/libvirt/qemu.conf}}. If you use a different user with QEMU/Libvirt, you will need to keep the {{ic|QEMU_PA_SERVER}} variable otherwise permission errors will occur.<br />
}}<br />
<br />
=== Physical disk/partition ===<br />
<br />
A whole disk or a partition may be used as a whole for improved I/O performance by adding an entry to the XML<br />
<br />
Virtio-BLK Example:<br />
{{hc|$ virsh edit [vmname]| <nowiki><br />
<devices><br />
...<br />
<disk type='block' device='disk'><br />
<driver name='qemu' type='raw' cache='none' io='native'/><br />
<source dev='/dev/disk/by-id/xxxxxxxx'/><br />
<target dev='vda' bus='virtio'/><br />
</disk><br />
...<br />
</devices><br />
</nowiki><br />
}}<br />
<br />
Virtio-SCSI Example:<br />
{{hc|$ virsh edit [vmname]| <nowiki><br />
<devices><br />
...<br />
<disk type='block' device='disk'><br />
<driver name='qemu' type='raw' cache='none' io='native'/><br />
<source dev='/dev/disk/by-id/xxxxxxxx'/><br />
<target dev='sda' bus='scsi'/><br />
</disk><br />
...<br />
</devices><br />
</nowiki><br />
}}<br />
<br />
To find out which disk/partition is associated with the one you would like to pass:<br />
<br />
{{hc|<br />
$ ls -l /dev/disk/by-id|<br />
ata-ST1000LM002-9VQ14L_Z0501SZ9 -> ../../sdd<br />
ata-ST1000LM002-9VQ14L_Z0501SZ9-part1 -> ../../sdd1<br />
}}<br />
<br />
You can also add the disk with Virt-Manager's '''Add Hardware''' menu and then type the disk you want in the '''Select or create custom storage''' box, e.g. '''/dev/disk/by-id/ata-ST1000LM002-9VQ14L_Z0501SZ9'''<br />
<br />
Depending on which bus you use, the above step will require either the VIOSTOR(bus=virtio) or VIOSCSI(bus=scsi) driver on Windows guests, refer to [[#Setting up the guest OS]] for the driver ISO.<br />
<br />
=== Gotchas ===<br />
<br />
==== Passing through a device that does not support resetting ====<br />
<br />
When the VM shuts down, all devices used by the guest are deinitialized by its OS in preparation for shutdown. In this state, those devices are no longer functional and must then be power-cycled before they can resume normal operation. Linux can handle this power-cycling on its own, but when a device has no known reset methods, it remains in this disabled state and becomes unavailable. Since Libvirt and Qemu both expect all host PCI devices to be ready to reattach to the host before completely stopping the VM, when encountering a device that will not reset, they will hang in a "Shutting down" state where they will not be able to be restarted until the host system has been rebooted. It is therefore reccomanded to only pass through PCI devices which the kernel is able to reset, as evidenced by the presence of a {{ic|reset}} file in the PCI device sysfs node, such as {{ic|/sys/bus/pci/devices/0000:00:1a.0/reset}}.<br />
<br />
The following bash command shows which devices can and cannot be reset.<br />
<br />
{{hc|<nowiki>for iommu_group in $(find /sys/kernel/iommu_groups/ -maxdepth 1 -mindepth 1 -type d);do echo "IOMMU group $(basename "$iommu_group")"; for device in $(\ls -1 "$iommu_group"/devices/); do if [[ -e "$iommu_group"/devices/"$device"/reset ]]; then echo -n "[RESET]"; fi; echo -n $'\t';lspci -nns "$device"; done; done</nowiki>|<br />
IOMMU group 0<br />
00:00.0 Host bridge [0600]: Intel Corporation Xeon E3-1200 v2/Ivy Bridge DRAM Controller [8086:0158] (rev 09)<br />
IOMMU group 1<br />
00:01.0 PCI bridge [0604]: Intel Corporation Xeon E3-1200 v2/3rd Gen Core processor PCI Express Root Port [8086:0151] (rev 09)<br />
01:00.0 VGA compatible controller [0300]: NVIDIA Corporation GK208 [GeForce GT 720] [10de:1288] (rev a1)<br />
01:00.1 Audio device [0403]: NVIDIA Corporation GK208 HDMI/DP Audio Controller [10de:0e0f] (rev a1)<br />
IOMMU group 2<br />
00:14.0 USB controller [0c03]: Intel Corporation 7 Series/C210 Series Chipset Family USB xHCI Host Controller [8086:1e31] (rev 04)<br />
IOMMU group 4<br />
[RESET] 00:1a.0 USB controller [0c03]: Intel Corporation 7 Series/C210 Series Chipset Family USB Enhanced Host Controller #2 [8086:1e2d] (rev 04)<br />
IOMMU group 5<br />
[RESET] 00:1b.0 Audio device [0403]: Intel Corporation 7 Series/C210 Series Chipset Family High Definition Audio Controller [8086:1e20] (rev 04)<br />
IOMMU group 10<br />
[RESET] 00:1d.0 USB controller [0c03]: Intel Corporation 7 Series/C210 Series Chipset Family USB Enhanced Host Controller #1 [8086:1e26] (rev 04)<br />
IOMMU group 13<br />
06:00.0 VGA compatible controller [0300]: NVIDIA Corporation GM204 [GeForce GTX 970] [10de:13c2] (rev a1)<br />
06:00.1 Audio device [0403]: NVIDIA Corporation GM204 High Definition Audio Controller [10de:0fbb] (rev a1)<br />
}}<br />
<br />
This signals that the xHCI USB controller in 00:14.0 cannot be reset and will therefore stop the VM from shutting down properly, while the integrated sound card in 00:1b.0 and the other two controllers in 00:1a.0 and 00:1d.0 do not share this problem and can be passed without issue.<br />
<br />
== Complete setups and examples ==<br />
<br />
If you have trouble configuring a certain mechanism in your setup, you might want to look up [[PCI_passthrough_via_OVMF/Examples|complete passthrough setup examples]]. A few users have described their setups and you might want to look up certain tricks from their configuration files.<br />
<br />
== Troubleshooting ==<br />
<br />
If your issue is not mentioned below, you may want to browse [[QEMU#Troubleshooting]].<br />
<br />
=== "Error 43: Driver failed to load" on Nvidia GPUs passed to Windows VMs ===<br />
<br />
{{Note|<br />
* This may also fix SYSTEM_THREAD_EXCEPTION_NOT_HANDLED boot crashes related to Nvidia drivers.<br />
* This may also fix problems under linux guests.<br />
}}<br />
<br />
Since version 337.88, Nvidia drivers on Windows check if an hypervisor is running and fail if it detects one, which results in an Error 43 in the Windows device manager. Starting with QEMU 2.5.0 and libvirt 1.3.3, the vendor_id for the hypervisor can be spoofed, which is enough to fool the Nvidia drivers into loading anyway. All one must do is add {{ic|<nowiki>hv_vendor_id=whatever</nowiki>}} to the cpu parameters in their QEMU command line, or by adding the following line to their libvirt domain configuration. It may help for the ID to be set to a 12-character alphanumeric (e.g. '1234567890ab') as opposed to longer or shorter strings.<br />
<br />
{{hc|$ virsh edit [vmname]|<nowiki><br />
...<br />
<features><br />
<hyperv><br />
...<br />
<vendor_id state='on' value='whatever'/><br />
...<br />
</hyperv><br />
...<br />
<kvm><br />
<hidden state='on'/><br />
</kvm><br />
</features><br />
...<br />
</nowiki>}}<br />
<br />
Users with older versions of QEMU and/or libvirt will instead have to disable a few hypervisor extensions, which can degrade performance substantially. If this is what you want to do, do the following replacement in your libvirt domain config file.<br />
<br />
{{hc|$ virsh edit [vmname]|<nowiki><br />
...<br />
<features><br />
<hyperv><br />
<relaxed state='on'/><br />
<vapic state='on'/><br />
<spinlocks state='on' retries='8191'/><br />
</hyperv><br />
...<br />
</features><br />
...<br />
<clock offset='localtime'><br />
<timer name='hypervclock' present='yes'/><br />
</clock><br />
...<br />
</nowiki>}}<br />
<br />
{{bc|<nowiki><br />
...<br />
<clock offset='localtime'><br />
<timer name='hypervclock' present='no'/><br />
</clock><br />
...<br />
<features><br />
<kvm><br />
<hidden state='on'/><br />
</kvm><br />
...<br />
<hyperv><br />
<relaxed state='off'/><br />
<vapic state='off'/><br />
<spinlocks state='off'/><br />
</hyperv><br />
...<br />
</features><br />
...<br />
</nowiki>}}<br />
<br />
=== "BAR 3: cannot reserve [mem]" error in dmesg after starting VM ===<br />
<br />
{{Expansion|This error is actually related to the boot_vgs issue and should be merged together with everything else concerning GPU ROMs.|section=UEFI (OVMF) Compatibility in VBIOS}}<br />
<br />
With respect to [https://www.linuxquestions.org/questions/linux-kernel-70/kernel-fails-to-assign-memory-to-pcie-device-4175487043/ this article]:<br />
<br />
If you still have code 43 check dmesg for memory reservation errors after starting up VM, if you have similar it could be the case:<br />
<br />
vfio-pci 0000:09:00.0: BAR 3: cannot reserve [mem 0xf0000000-0xf1ffffff 64bit pref]<br />
<br />
Find out a PCI Bridge your graphic card is connected to. This will give actual hierarchy of devices:<br />
<br />
$ lspci -t<br />
<br />
Before starting VM run following lines replacing IDs with actual from previous output.<br />
<br />
# echo 1 > /sys/bus/pci/devices/0000\:00\:03.1/remove<br />
# echo 1 > /sys/bus/pci/rescan<br />
<br />
{{Note|Probably setting [[kernel parameter]] {{ic|1=video=efifb:off}} is required as well. [https://pve.proxmox.com/wiki/Pci_passthrough#BAR_3:_can.27t_reserve_.5Bmem.5D_error Source]}}<br />
<br />
=== UEFI (OVMF) compatibility in VBIOS ===<br />
<br />
{{Remove|Flashing you guest GPU for the purpose of a GPU passthrough is '''never''' good advice. A full section should be dedicated to VBIOS compatibility.|section= UEFI (OVMF) Compatibility in VBIOS}}<br />
<br />
With respect to [https://pve.proxmox.com/wiki/Pci_passthrough#How_to_known_if_card_is_UEFI_.28ovmf.29_compatible this article]:<br />
<br />
Error 43 can be caused by the GPU's VBIOS without UEFI support. To check whenever your VBIOS supports it, you will have to use {{ic|rom-parser}}:<br />
<br />
$ git clone https://github.com/awilliam/rom-parser<br />
$ cd rom-parser && make<br />
<br />
Dump the GPU VBIOS:<br />
<br />
# echo 1 > /sys/bus/pci/devices/0000:01:00.0/rom<br />
# cat /sys/bus/pci/devices/0000:01:00.0/rom > /tmp/image.rom<br />
# echo 0 > /sys/bus/pci/devices/0000:01:00.0/rom<br />
<br />
And test it for compatibility:<br />
<br />
{{hc|$ ./rom-parser /tmp/image.rom|<br />
Valid ROM signature found @600h, PCIR offset 190h<br />
PCIR: type 0 (x86 PC-AT), vendor: 10de, device: 1184, class: 030000<br />
PCIR: revision 0, vendor revision: 1<br />
Valid ROM signature found @fa00h, PCIR offset 1ch<br />
PCIR: type 3 (EFI), vendor: 10de, device: 1184, class: 030000<br />
PCIR: revision 3, vendor revision: 0<br />
EFI: Signature Valid, Subsystem: Boot, Machine: X64<br />
Last image<br />
}}<br />
<br />
To be UEFI compatible, you need a "type 3 (EFI)" in the result. If it's not there, try updating your GPU VBIOS. GPU manufacturers often share VBIOS upgrades on their support pages. A large database of known compatible and working VBIOSes (along with their UEFI compatibility status!) is available on [https://www.techpowerup.com/vgabios/ TechPowerUp].<br />
<br />
Updated VBIOS can be used in the VM without flashing. To load it in QEMU:<br />
<br />
-device vfio-pci,host=07:00.0,......,romfile=/path/to/your/gpu/bios.bin \<br />
<br />
And in libvirt:<br />
<br />
{{bc|1=<br />
<hostdev><br />
...<br />
<rom file='/path/to/your/gpu/bios.bin'/><br />
...<br />
</hostdev><br />
}}<br />
<br />
One should compare VBIOS versions between host and guest systems using [https://www.techpowerup.com/download/nvidia-nvflash/ nvflash] (Linux versions under ''Show more versions'') or <br />
[https://www.techpowerup.com/download/techpowerup-gpu-z/ GPU-Z] (in Windows guest). To check the currently loaded VBIOS:<br />
<br />
{{hc|$ ./nvflash --version|<br />
...<br />
Version : 80.04.XX.00.97<br />
...<br />
UEFI Support : No<br />
UEFI Version : N/A<br />
UEFI Variant Id : N/A ( Unknown )<br />
UEFI Signer(s) : Unsigned<br />
...<br />
}}<br />
<br />
And to check a given VBIOS file:<br />
<br />
{{hc|$ ./nvflash --version NV299MH.rom|<br />
...<br />
Version : 80.04.XX.00.95<br />
...<br />
UEFI Support : Yes<br />
UEFI Version : 0x10022 (Jul 2 2013 @ 16377903 )<br />
UEFI Variant Id : 0x0000000000000004 ( GK1xx )<br />
UEFI Signer(s) : Microsoft Corporation UEFI CA 2011<br />
...<br />
}}<br />
<br />
If the external ROM did not work as it should in the guest, you will have to flash the newer VBIOS image to the GPU. In some cases it is possible to create your own VBIOS image with UEFI support using [https://www.win-raid.com/t892f16-AMD-and-Nvidia-GOP-update-No-requests-DIY.html GOPUpd] tool, however this is risky and may result in GPU brick.<br />
<br />
{{Warning|Failure during flashing may "brick" your GPU - recovery may be possible, but rarely easy and often requires additional hardware. '''DO NOT''' flash VBIOS images for other GPU models (different boards may use different VBIOSes, clocks, fan configuration). If it breaks, you get to keep all the pieces.}}<br />
<br />
In order to avoid the irreparable damage to your graphics adapter it is necessary to unload the NVIDIA kernel driver first:<br />
<br />
# modprobe -r nvidia_modeset nvidia <br />
<br />
Flashing the VBIOS can be done with:<br />
<br />
# ./nvflash romfile.bin<br />
<br />
{{Warning|'''DO NOT''' interrupt the flashing process, even if it looks like it's stuck. Flashing should take about a minute on most GPUs, but may take longer.}}<br />
<br />
=== Slowed down audio pumped through HDMI on the video card ===<br />
<br />
For some users VM's audio slows down/starts stuttering/becomes demonic after a while when it's pumped through HDMI on the video card. This usually also slows down graphics.<br />
A possible solution consists of enabling MSI (Message Signaled-Based Interrupts) instead of the default (Line-Based Interrupts).<br />
<br />
In order to check whether MSI is supported or enabled, run the following command as root:<br />
<br />
# lspci -vs $device | grep 'MSI:'<br />
<br />
where `$device` is the card's address (e.g. `01:00.0`).<br />
<br />
The output should be similar to:<br />
<br />
Capabilities: [60] MSI: Enable'''-''' Count=1/1 Maskable- 64bit+<br />
<br />
A {{ic|-}} after {{ic|Enable}} means MSI is supported, but not used by the VM, while a {{ic|+}} says that the VM is using it.<br />
<br />
The procedure to enable it is quite complex, instructions and an overview of the setting can be found [https://forums.guru3d.com/showthread.php?t=378044 here].<br />
<br />
Other hints can be found on the [https://lime-technology.com/wiki/index.php/UnRAID_6/VM_Guest_Support#Enable_MSI_for_Interrupts_to_Fix_HDMI_Audio_Support lime-technology's wiki], or on this article on [https://vfio.blogspot.it/2014/09/vfio-interrupts-and-how-to-coax-windows.html VFIO tips and tricks].<br />
<br />
A UI tool called [https://github.com/CHEF-KOCH/MSI-utility MSI Utility (FOSS Version 2)] works with Windows 10 64-bit and simplifies the process.<br />
<br />
In order to fix the issues enabling MSI on the 0 function of a nVidia card ({{ic|01:00.0 VGA compatible controller: NVIDIA Corporation GM206 [GeForce GTX 960] (rev a1) (prog-if 00 [VGA controller])}}) was not enough; it will also be required to enable it on the other function ({{ic|01:00.1 Audio device: NVIDIA Corporation Device 0fba (rev a1)}}) to fix the issue.<br />
<br />
=== No HDMI audio output on host when intel_iommu is enabled ===<br />
<br />
If after enabling {{ic|intel_iommu}} the HDMI output device of Intel GPU becomes unusable on the host then setting the option {{ic|igfx_off}} (i.e. {{ic|1=intel_iommu=on,igfx_off}}) might bring the audio back, please read {{ic|Graphics Problems?}} in [https://www.kernel.org/doc/Documentation/Intel-IOMMU.txt Intel-IOMMU.txt] for details about setting {{ic|igfx_off}}.<br />
<br />
=== X does not start after enabling vfio_pci ===<br />
<br />
This is related to the host GPU being detected as a secondary GPU, which causes X to fail/crash when it tries to load a driver for the guest GPU. To circumvent this, a Xorg configuration file specifying the BusID for the host GPU is required. The correct BusID can be acquired from lspci or the Xorg log. [https://www.redhat.com/archives/vfio-users/2016-August/msg00025.html Source →]<br />
<br />
{{hc|/etc/X11/xorg.conf.d/10-intel.conf|<nowiki><br />
Section "Device"<br />
Identifier "Intel GPU"<br />
Driver "modesetting"<br />
BusID "PCI:0:2:0"<br />
EndSection<br />
</nowiki>}}<br />
<br />
=== Chromium ignores integrated graphics for acceleration ===<br />
<br />
Chromium and friends will try to detect as many GPUs as they can in the system and pick which one is preferred (usually discrete NVIDIA/AMD graphics). It tries to pick a GPU by looking at PCI devices, not OpenGL renderers available in the system - the result is that Chromium may ignore the integrated GPU available for rendering and try to use the dedicated GPU bound to the {{ic|vfio-pci}} driver, and unusable on the host system, regardless of whenever a guest VM is running or not. This results in software rendering being used (leading to higher CPU load, which may also result in choppy video playback, scrolling and general un-smoothness).<br />
<br />
This can be fixed by [[Chromium/Tips and tricks#Forcing specific GPU|explicitly telling Chromium which GPU you want to use]].<br />
<br />
=== VM only uses one core ===<br />
<br />
For some users, even if IOMMU is enabled and the core count is set to more than 1, the VM still only uses one CPU core and thread. To solve this enable "Manually set CPU topology" in {{ic|virt-manager}} and set it to the desirable amount of CPU sockets, cores and threads. Keep in mind that "Threads" refers to the thread count per CPU, not the total count.<br />
<br />
=== Passthrough seems to work but no output is displayed ===<br />
<br />
Make sure if you are using virt-manager that UEFI firmware is selected for your virtual machine. Also, make sure you have passed the correct device to the VM.<br />
<br />
=== virt-manager has permission issues ===<br />
<br />
If you are getting a permission error with virt-manager add the following to your {{ic|/etc/libvirt/qemu.conf}}:<br />
<br />
group="kvm"<br />
user="''user''"<br />
<br />
If that does not work make sure your user account is added to the kvm and libvirt [[groups]].<br />
<br />
=== Host lockup after VM shutdown ===<br />
<br />
This issue seems to primarily affect users running a Windows 10 guest and usually after the VM has been run for a prolonged period of time: the host will experience multiple CPU core lockups (see [https://bbs.archlinux.org/viewtopic.php?id=206050&p=2]). To fix this try enabling Message Signal Interrupts on the GPU passed through to the guest. A good guide for how to do this can be found in [https://forums.guru3d.com/threads/windows-line-based-vs-message-signaled-based-interrupts.378044/].<br />
<br />
=== Host lockup if guest is left running during sleep ===<br />
<br />
VFIO-enabled virtual machines tend to become unstable if left running through a sleep/wakeup cycle and have been known to cause the host machine to lockup when an attempt is then made to shut them down. In order to avoid this, one can simply prevent the host from going into sleep while the guest is running using the following libvirt hook script and systemd unit. The hook file needs executable permissions to work.<br />
<br />
{{hc|/etc/libvirt/hooks/qemu|<nowiki><br />
#!/bin/bash<br />
<br />
OBJECT="$1"<br />
OPERATION="$2"<br />
SUBOPERATION="$3"<br />
EXTRA_ARG="$4"<br />
<br />
case "$OPERATION" in<br />
"prepare")<br />
systemctl start libvirt-nosleep@"$OBJECT"<br />
;;<br />
"release")<br />
systemctl stop libvirt-nosleep@"$OBJECT"<br />
;;<br />
esac<br />
</nowiki>}}<br />
<br />
{{hc|/etc/systemd/system/libvirt-nosleep@.service|<nowiki><br />
[Unit]<br />
Description=Preventing sleep while libvirt domain "%i" is running<br />
<br />
[Service]<br />
Type=simple<br />
ExecStart=/usr/bin/systemd-inhibit --what=sleep --why="Libvirt domain \"%i\" is running" --who=%U --mode=block sleep infinity<br />
</nowiki>}}<br />
<br />
== See also ==<br />
<br />
* [https://bbs.archlinux.org/viewtopic.php?id=162768 Discussion on Arch Linux forums] | [https://archive.is/kZYMt Archived link]<br />
* [https://docs.google.com/spreadsheet/ccc?key=0Aryg5nO-kBebdFozaW9tUWdVd2VHM0lvck95TUlpMlE User contributed hardware compatibility list]<br />
* [https://pastebin.com/rcnUZCv7 Example script from https://www.youtube.com/watch?v=37D2bRsthfI]<br />
* [https://vfio.blogspot.com/ Complete tutorial for PCI passthrough]<br />
* [https://www.redhat.com/archives/vfio-users/ VFIO users mailing list]<br />
* [https://webchat.freenode.net/?channels=vfio-users #vfio-users on freenode]<br />
* [https://www.youtube.com/watch?v=aLeWg11ZBn0 YouTube: Level1Linux - GPU Passthrough for Virtualization with Ryzen]<br />
* [https://www.reddit.com/r/VFIO A subreddit focused on vfio]</div>Nicman23https://wiki.archlinux.org/index.php?title=PCI_passthrough_via_OVMF&diff=538941PCI passthrough via OVMF2018-08-31T09:01:29Z<p>Nicman23: /* Qemu 3.0 Audio Changes */ added mention of removing overriding audio env variables</p>
<hr />
<div>[[Category:Virtualization]]<br />
[[ja:OVMF による PCI パススルー]]<br />
[[zh-hans:PCI passthrough via OVMF]]<br />
The Open Virtual Machine Firmware ([https://github.com/tianocore/tianocore.github.io/wiki/OVMF OVMF]) is a project to enable UEFI support for virtual machines. Starting with Linux 3.9 and recent versions of [[QEMU]], it is now possible to passthrough a graphics card, offering the VM native graphics performance which is useful for graphic-intensive tasks.<br />
<br />
Provided you have a desktop computer with a spare GPU you can dedicate to the host (be it an integrated GPU or an old OEM card, the brands do not even need to match) and that your hardware supports it (see [[#Prerequisites]]), it is possible to have a VM of any OS with its own dedicated GPU and near-native performance. For more information on techniques see the background [https://www.linux-kvm.org/images/b/b3/01x09b-VFIOandYou-small.pdf presentation (pdf)]. <br />
<br />
== Prerequisites ==<br />
<br />
A VGA Passthrough relies on a number of technologies that are not ubiquitous as of today and might not be available on your hardware. You will not be able to do this on your machine unless the following requirements are met :<br />
<br />
* Your CPU must support hardware virtualization (for kvm) and IOMMU (for the passthrough itself)<br />
** [https://ark.intel.com/Search/FeatureFilter?productType=processors&VTD=true List of compatible Intel CPUs (Intel VT-x and Intel VT-d)]<br />
** All AMD CPUs from the Bulldozer generation and up (including Zen) should be compatible.<br />
*** CPUs from the K10 generation (2007) do not have an IOMMU, so you '''need''' to have a motherboard with a [https://support.amd.com/TechDocs/43403.pdf#page=18 890FX] or [https://support.amd.com/TechDocs/48691.pdf#page=21 990FX] chipset to make it work, as those have their own IOMMU.<br />
* Your motherboard must also support IOMMU<br />
** Both the chipset and the BIOS must support it. It is not always easy to tell at a glance whether or not this is the case, but there is a fairly comprehensive list on the matter on the [https://wiki.xen.org/wiki/VTd_HowTo Xen wiki] as well as [[Wikipedia:List of IOMMU-supporting hardware]].<br />
* Your guest GPU ROM must support UEFI.<br />
** If you can find [https://www.techpowerup.com/vgabios/ any ROM in this list] that applies to your specific GPU and is said to support UEFI, you are generally in the clear. All GPUs from 2012 and later should support this, as Microsoft made UEFI a requirement for devices to be marketed as compatible with Windows 8.<br />
<br />
You will probably want to have a spare monitor or one with multiple input ports connected to different GPUs (the passthrough GPU will not display anything if there is no screen plugged in and using a VNC or Spice connection will not help your performance), as well as a mouse and a keyboard you can pass to your VM. If anything goes wrong, you will at least have a way to control your host machine this way.<br />
<br />
== Setting up IOMMU ==<br />
<br />
{{Note|<br />
* IOMMU is a generic name for Intel VT-d and AMD-Vi.<br />
* VT-d stands for ''Intel Virtualization Technology for Directed I/O'' and should not be confused with VT-x ''Intel Virtualization Technology''. VT-x allows one hardware platform to function as multiple “virtual” platforms while VT-d improves security and reliability of the systems and also improves performance of I/O devices in virtualized environments.}}<br />
<br />
Using IOMMU opens to features like PCI passthrough and memory protection from faulty or malicious devices, see [[wikipedia:Input-output memory management unit#Advantages]] and [https://www.quora.com/Memory-Management-computer-programming/Could-you-explain-IOMMU-in-plain-English Memory Management (computer programming): Could you explain IOMMU in plain English?].<br />
<br />
=== Enabling IOMMU ===<br />
<br />
Ensure that AMD-Vi/Intel VT-d is supported by the CPU and enabled in the BIOS settings. Both normally show up alongside other CPU features (meaning they could be in an overclocking-related menu) either with their actual names ("VT-d" or "AMD-Vi") or in more ambiguous terms such as "Virtualization technology", which may or may not be explained in the manual.<br />
<br />
Enable IOMMU support by setting the correct [[kernel parameter]] depending on the type of CPU in use:<br />
* For Intel CPUs (VT-d) set {{ic|1=intel_iommu=on}} <br />
* For AMD CPUs (AMD-Vi) set {{ic|1=amd_iommu=on}}<br />
<br />
You should also append the {{ic|1=iommu=pt}} parameter. This will prevent Linux from touching devices which cannot be passed through.<br />
<br />
After rebooting, check dmesg to confirm that IOMMU has been correctly enabled:<br />
<br />
{{hc|dmesg {{!}} grep -e DMAR -e IOMMU|<br />
[ 0.000000] ACPI: DMAR 0x00000000BDCB1CB0 0000B8 (v01 INTEL BDW 00000001 INTL 00000001)<br />
[ 0.000000] Intel-IOMMU: enabled<br />
[ 0.028879] dmar: IOMMU 0: reg_base_addr fed90000 ver 1:0 cap c0000020660462 ecap f0101a<br />
[ 0.028883] dmar: IOMMU 1: reg_base_addr fed91000 ver 1:0 cap d2008c20660462 ecap f010da<br />
[ 0.028950] IOAPIC id 8 under DRHD base 0xfed91000 IOMMU 1<br />
[ 0.536212] DMAR: No ATSR found<br />
[ 0.536229] IOMMU 0 0xfed90000: using Queued invalidation<br />
[ 0.536230] IOMMU 1 0xfed91000: using Queued invalidation<br />
[ 0.536231] IOMMU: Setting RMRR:<br />
[ 0.536241] IOMMU: Setting identity map for device 0000:00:02.0 [0xbf000000 - 0xcf1fffff]<br />
[ 0.537490] IOMMU: Setting identity map for device 0000:00:14.0 [0xbdea8000 - 0xbdeb6fff]<br />
[ 0.537512] IOMMU: Setting identity map for device 0000:00:1a.0 [0xbdea8000 - 0xbdeb6fff]<br />
[ 0.537530] IOMMU: Setting identity map for device 0000:00:1d.0 [0xbdea8000 - 0xbdeb6fff]<br />
[ 0.537543] IOMMU: Prepare 0-16MiB unity mapping for LPC<br />
[ 0.537549] IOMMU: Setting identity map for device 0000:00:1f.0 [0x0 - 0xffffff]<br />
[ 2.182790] [drm] DMAR active, disabling use of stolen memory<br />
}}<br />
<br />
=== Ensuring that the groups are valid ===<br />
<br />
The following script should allow you to see how your various PCI devices are mapped to IOMMU groups. If it does not return anything, you either have not enabled IOMMU support properly or your hardware does not support it.<br />
<br />
#!/bin/bash<br />
shopt -s nullglob<br />
for d in /sys/kernel/iommu_groups/*/devices/*; do <br />
n=${d#*/iommu_groups/*}; n=${n%%/*}<br />
printf 'IOMMU Group %s ' "$n"<br />
lspci -nns "${d##*/}"<br />
done;<br />
<br />
Example output:<br />
<br />
IOMMU Group 0 00:00.0 Host bridge [0600]: Intel Corporation 2nd Generation Core Processor Family DRAM Controller [8086:0104] (rev 09)<br />
IOMMU Group 1 00:16.0 Communication controller [0780]: Intel Corporation 6 Series/C200 Series Chipset Family MEI Controller #1 [8086:1c3a] (rev 04)<br />
IOMMU Group 2 00:19.0 Ethernet controller [0200]: Intel Corporation 82579LM Gigabit Network Connection [8086:1502] (rev 04)<br />
IOMMU Group 3 00:1a.0 USB controller [0c03]: Intel Corporation 6 Series/C200 Series Chipset Family USB Enhanced Host Controller #2 [8086:1c2d] (rev <br />
...<br />
<br />
An IOMMU group is the smallest set of physical devices that can be passed to a virtual machine. For instance, in the example above, both the GPU in 06:00.0 and its audio controller in 6:00.1 belong to IOMMU group 13 and can only be passed together. The frontal USB controller, however, has its own group (group 2) which is separate from both the USB expansion controller (group 10) and the rear USB controller (group 4), meaning that [[#USB controller|any of them could be passed to a VM without affecting the others]].<br />
<br />
=== Gotchas ===<br />
<br />
==== Plugging your guest GPU in an unisolated CPU-based PCIe slot ====<br />
<br />
Not all PCI-E slots are the same. Most motherboards have PCIe slots provided by both the CPU and the PCH. Depending on your CPU, it is possible that your processor-based PCIe slot does not support isolation properly, in which case the PCI slot itself will be appear to be grouped with the device that is connected to it.<br />
<br />
IOMMU Group 1 00:01.0 PCI bridge: Intel Corporation Xeon E3-1200 v2/3rd Gen Core processor PCI Express Root Port (rev 09)<br />
IOMMU Group 1 01:00.0 VGA compatible controller: NVIDIA Corporation GM107 [GeForce GTX 750] (rev a2)<br />
IOMMU Group 1 01:00.1 Audio device: NVIDIA Corporation Device 0fbc (rev a1)<br />
<br />
This is fine so long as only your guest GPU is included in here, such as above. Depending on what is plugged in your other PCIe slots and whether they are allocated to your CPU or your PCH, you may find yourself with additional devices within the same group, which would force you to pass those as well. If you are ok with passing everything that is in there to your VM, you are free to continue. Otherwise, you will either need to try and plug your GPU in your other PCIe slots (if you have any) and see if those provide isolation from the rest or to install the ACS override patch, which comes with its own drawbacks. See [[#Bypassing the IOMMU groups (ACS override patch)]] for more information.<br />
<br />
{{Note|If they are grouped with other devices in this manner, pci root ports and bridges should neither be bound to vfio at boot, nor be added to the VM.}}<br />
<br />
== Isolating the GPU ==<br />
<br />
In order to assign a device to a virtual machine, this device and all those sharing the same IOMMU group must have their driver replaced by a stub driver or a VFIO driver in order to prevent the host machine from interacting with them. In the case of most devices, this can be done on the fly right before the VM starts.<br />
<br />
However, due to their size and complexity, GPU drivers do not tend to support dynamic rebinding very well, so you cannot just have some GPU you use on the host be transparently passed to a VM without having both drivers conflict with each other. Because of this, it is generally advised to bind those placeholder drivers manually before starting the VM, in order to stop other drivers from attempting to claim it.<br />
<br />
The following section details how to configure a GPU so those placeholder drivers are bound early during the boot process, which makes said device inactive until a VM claims it or the driver is switched back. This is the preferred method, considering it has less caveats than switching drivers once the system is fully online.<br />
<br />
{{warning|Once you reboot after this procedure, whatever GPU you have configured will no longer be usable on the host until you reverse the manipulation. Make sure the GPU you intend to use on the host is properly configured before doing this - your motherboard should be set to display using the host GPU.}}<br />
<br />
Starting with Linux 4.1, the kernel includes vfio-pci. This is a VFIO driver, meaning it fulfills the same role as pci-stub did, but it can also control devices to an extent, such as by switching them into their D3 state when they are not in use.<br />
<br />
{{hc|$ modinfo vfio-pci|<br />
filename: /lib/modules/4.4.5-1-ARCH/kernel/drivers/vfio/pci/vfio-pci.ko.gz<br />
description: VFIO PCI - User Level meta-driver<br />
author: Alex Williamson <alex.williamson@redhat.com><br />
...<br />
}}<br />
<br />
Vfio-pci normally targets PCI devices by ID, meaning you only need to specify the IDs of the devices you intend to passthrough. For the following IOMMU group, you would want to bind vfio-pci with {{ic|10de:13c2}} and {{ic|10de:0fbb}}, which will be used as example values for the rest of this section.<br />
<br />
IOMMU Group 13 06:00.0 VGA compatible controller: NVIDIA Corporation GM204 [GeForce GTX 970] [10de:13c2] (rev a1)<br />
IOMMU Group 13 06:00.1 Audio device: NVIDIA Corporation GM204 High Definition Audio Controller [10de:0fbb] (rev a1)}}<br />
<br />
{{Note|You cannot specify which device to isolate using vendor-device ID pairs if the host GPU and the guest GPU share the same pair (i.e : if both are the same model). If this is your case, read [[#Using identical guest and host GPUs]] instead.}}<br />
<br />
You can then add those vendor-device ID pairs to the default parameters passed to vfio-pci whenever it is inserted into the kernel.<br />
<br />
{{hc|/etc/modprobe.d/vfio.conf|2=<br />
options vfio-pci ids=10de:13c2,10de:0fbb<br />
}}<br />
<br />
{{Note|If, as noted in [[#Plugging your guest GPU in an unisolated CPU-based PCIe slot]], your pci root port is part of your IOMMU group, you '''must not''' pass its ID to {{ic|vfio-pci}}, as it needs to remain attached to the host to function properly. Any other device within that group, however, should be left for {{ic|vfio-pci}} to bind with.}}<br />
<br />
This, however, does not guarantee that vfio-pci will be loaded before other graphics drivers. To ensure that, we need to statically bind it in the kernel image alongside with its dependencies. That means adding, in this order, {{ic|vfio_pci}}, {{ic|vfio}}, {{ic|vfio_iommu_type1}}, and {{ic|vfio_virqfd}} to [[mkinitcpio]]:<br />
<br />
{{hc|/etc/mkinitcpio.conf|2=<br />
MODULES=(... vfio_pci vfio vfio_iommu_type1 vfio_virqfd ...)<br />
}}<br />
<br />
{{Note|If you also have another driver loaded this way for [[Kernel mode setting#Early KMS start|early modesetting]] (such as {{ic|nouveau}}, {{ic|radeon}}, {{ic|amdgpu}}, {{ic|i915}}, etc.), all of the aforementioned VFIO modules must precede it.}}<br />
<br />
Also, ensure that the modconf hook is included in the HOOKS list of {{ic|mkinitcpio.conf}}:<br />
<br />
{{hc|/etc/mkinitcpio.conf|2=<br />
HOOKS=(... modconf ...)<br />
}}<br />
<br />
Since new modules have been added to the initramfs configuration, you must [[regenerate the initramfs]]. Should you change the IDs of the devices in {{ic|/etc/modprobe.d/vfio.conf}}, you will also have to regenerate it, as those parameters must be specified in the initramfs to be known during the early boot stages.<br />
<br />
Reboot and verify that vfio-pci has loaded properly and that it is now bound to the right devices.<br />
<br />
{{hc|$ dmesg {{!}} grep -i vfio|<br />
[ 0.329224] VFIO - User Level meta-driver version: 0.3<br />
[ 0.341372] vfio_pci: add [10de:13c2[ffff:ffff]] class 0x000000/00000000<br />
[ 0.354704] vfio_pci: add [10de:0fbb[ffff:ffff]] class 0x000000/00000000<br />
[ 2.061326] vfio-pci 0000:06:00.0: enabling device (0100 -> 0103)<br />
}}<br />
<br />
It is not necessary for all devices (or even expected device) from {{ic|vfio.conf}} to be in dmesg output. Sometimes a device does not appear in output at boot but actually is able to be visible and operatable in guest VM.<br />
<br />
{{hc|$ lspci -nnk -d 10de:13c2|<br />
06:00.0 VGA compatible controller: NVIDIA Corporation GM204 [GeForce GTX 970] [10de:13c2] (rev a1)<br />
Kernel driver in use: vfio-pci<br />
Kernel modules: nouveau nvidia<br />
}}<br />
<br />
{{hc|$ lspci -nnk -d 10de:0fbb|<br />
06:00.1 Audio device: NVIDIA Corporation GM204 High Definition Audio Controller [10de:0fbb] (rev a1)<br />
Kernel driver in use: vfio-pci<br />
Kernel modules: snd_hda_intel<br />
}}<br />
<br />
== Setting up an OVMF-based guest VM ==<br />
<br />
OVMF is an open-source UEFI firmware for QEMU virtual machines. While it's possible to use SeaBIOS to get similar results to an actual PCI passthough, the setup process is different and it is generally preferable to use the EFI method if your hardware supports it.<br />
<br />
=== Configuring libvirt ===<br />
<br />
[[Libvirt]] is a wrapper for a number of virtualization utilities that greatly simplifies the configuration and deployment process of virtual machines. In the case of KVM and QEMU, the frontend it provides allows us to avoid dealing with the permissions for QEMU and make it easier to add and remove various devices on a live VM. Its status as a wrapper, however, means that it might not always support all of the latest qemu features, which could end up requiring the use of a wrapper script to provide some extra arguments to QEMU.<br />
<br />
After installing {{Pkg|qemu}}, {{Pkg|libvirt}}, {{Pkg|ovmf}}, and {{Pkg|virt-manager}}, add the path to your OVMF firmware image and runtime variables template to your libvirt config so {{ic|virt-install}} or {{ic|virt-manager}} can find those later on.<br />
<br />
{{hc|/etc/libvirt/qemu.conf|2=<br />
nvram = [<br />
"/usr/share/ovmf/x64/OVMF_CODE.fd:/usr/share/ovmf/x64/OVMF_VARS.fd"<br />
]<br />
}}<br />
<br />
You can now [[enable]] and [[start]] {{ic|libvirtd.service}} and its logging component {{ic|virtlogd.socket}}.<br />
<br />
=== Setting up the guest OS ===<br />
<br />
The process of setting up a VM using {{ic|virt-manager}} is mostly self-explanatory, as most of the process comes with fairly comprehensive on-screen instructions.<br />
<br />
If using {{ic|virt-manager}}, you have to add your user to the libvirt [[group]] to ensure authentication.<br />
<br />
However, you should pay special attention to the following steps :<br />
<br />
* When the VM creation wizard asks you to name your VM (final step before clicking "Finish"), check the "Customize before install" checkbox.<br />
* In the "Overview" section, [https://i.imgur.com/73r2ctM.png set your firmware to "UEFI"]. If the option is grayed out, make sure that you have correctly specified the location of your firmware in {{ic|/etc/libvirt/qemu.conf}} and restart {{ic|libvirtd.service}}.<br />
* In the "CPUs" section, change your CPU model to "host-passthrough". If it is not in the list, you will have to type it by hand. This will ensure that your CPU is detected properly, since it causes libvirt to expose your CPU capabilities exactly as they are instead of only those it recognizes (which is the preferred default behavior to make CPU behavior easier to reproduce). Without it, some applications may complain about your CPU being of an unknown model.<br />
* If you want to minimize IO overhead, go into "Add Hardware" and add a Controller for SCSI drives of the "VirtIO SCSI" model. You can then change the default IDE disk for a SCSI disk, which will bind to said controller.<br />
** Windows VMs will not recognize those drives by default, so you need to download the ISO containing the drivers from [https://fedorapeople.org/groups/virt/virtio-win/direct-downloads/latest-virtio/ here] and add an IDE (or SATA for Windows 8.1 and newer) CD-ROM storage device linking to said ISO, otherwise you will not be able to get Windows to recognize it during the installation process. When prompted to select a disk to install windows on, load the drivers contained on the CD-ROM under ''vioscsi''.<br />
<br />
The rest of the installation process will take place as normal using a standard QXL video adapter running in a window. At this point, there is no need to install additional drivers for the rest of the virtual devices, since most of them will be removed later on. Once the guest OS is done installing, simply turn off the virtual machine. It is possible you will be dropped into the UEFI menu instead of starting the installation upon powering your VM for the first time. Sometimes the correct ISO file was not automatically detected and you will need to manually specify the drive to boot. By typing exit and navigating to "boot manager" you will enter a menu that allows you to choose between devices.<br />
<br />
=== Attaching the PCI devices ===<br />
<br />
With the installation done, it's now possible to edit the hardware details in libvirt and remove virtual integration devices, such as the spice channel and virtual display, the QXL video adapter, the emulated mouse and keyboard and the USB tablet device. Since that leaves you with no input devices, you may want to bind a few USB host devices to your VM as well, but remember to '''leave at least one mouse and/or keyboard assigned to your host''' in case something goes wrong with the guest. At this point, it also becomes possible to attach the PCI device that was isolated earlier; simply click on "Add Hardware" and select the PCI Host Devices you want to passthrough. If everything went well, the screen plugged into your GPU should show the OVMF splash screen and your VM should start up normally. From there, you can setup the drivers for the rest of your VM.<br />
<br />
=== Gotchas ===<br />
<br />
==== Using a non-EFI image on an OVMF-based VM ====<br />
<br />
The OVMF firmware does not support booting off non-EFI mediums. If the installation process drops you in a UEFI shell right after booting, you may have an invalid EFI boot media. Try using an alternate linux/windows image to determine if you have an invalid media.<br />
<br />
== Performance tuning ==<br />
<br />
Most use cases for PCI passthroughs relate to performance-intensive domains such as video games and GPU-accelerated tasks. While a PCI passthrough on its own is a step towards reaching native performance, there are still a few ajustments on the host and guest to get the most out of your VM.<br />
<br />
=== CPU Pinning ===<br />
<br />
The default behavior for KVM guests is to run operations coming from the guest as a number of threads representing virtual processors. Those threads are managed by the Linux scheduler like any other thread and are dispatched to any available CPU cores based on niceness and priority queues. Since switching between threads adds a bit of overhead (because context switching forces the core to change its cache between operations), this can noticeably harm performance on the guest. CPU pinning aims to resolve this as it overrides process scheduling and ensures that the VM threads will always run and only run on those specific cores.<br />
<br />
{{Note|For certain users enabling CPU pinning may introduce stuttering and short hangs, especially with the MuQSS scheduler (present in linux-ck and linux-zen kernels). You might want to try disabling pinning first if you experience similar issues, which effectively trades maximum performance for responsiveness at all times.}}<br />
<br />
==== CPU Topology ====<br />
<br />
Most modern CPU's support hardware multitasking, also known as hyper-threading on Intel CPU's or SMT on AMD CPU's. Hyper-threading/SMT is simply a very efficient way of running two threads on one CPU core at any given time. You will want to take into consideration that the CPU pinning you choose will greatly depend on what you do with your host while your VM is running.<br />
<br />
To find the topology for your CPU run {{ic|1=lscpu -e}}:<br />
<br />
{{Note|Pay special attention to the 4th column '''"CORE"''' as this shows the association of the Physical/Logical CPU cores}}<br />
<br />
{{ic|lscpu -e}} on a 6c/12t Ryzen 5 1600:<br />
<br />
CPU NODE SOCKET CORE L1d:L1i:L2:L3 ONLINE MAXMHZ MINMHZ<br />
0 0 0 0 0:0:0:0 yes 3800.0000 1550.0000<br />
1 0 0 0 0:0:0:0 yes 3800.0000 1550.0000<br />
2 0 0 1 1:1:1:0 yes 3800.0000 1550.0000<br />
3 0 0 1 1:1:1:0 yes 3800.0000 1550.0000<br />
4 0 0 2 2:2:2:0 yes 3800.0000 1550.0000<br />
5 0 0 2 2:2:2:0 yes 3800.0000 1550.0000<br />
6 0 0 3 3:3:3:1 yes 3800.0000 1550.0000<br />
7 0 0 3 3:3:3:1 yes 3800.0000 1550.0000<br />
8 0 0 4 4:4:4:1 yes 3800.0000 1550.0000<br />
9 0 0 4 4:4:4:1 yes 3800.0000 1550.0000<br />
10 0 0 5 5:5:5:1 yes 3800.0000 1550.0000<br />
11 0 0 5 5:5:5:1 yes 3800.0000 1550.0000<br />
<br />
{{ic|lscpu -e}} on a 6c/12t Intel 8700k:<br />
<br />
CPU NODE SOCKET CORE L1d:L1i:L2:L3 ONLINE MAXMHZ MINMHZ<br />
0 0 0 0 0:0:0:0 yes 4600.0000 800.0000<br />
1 0 0 1 1:1:1:0 yes 4600.0000 800.0000<br />
2 0 0 2 2:2:2:0 yes 4600.0000 800.0000<br />
3 0 0 3 3:3:3:0 yes 4600.0000 800.0000<br />
4 0 0 4 4:4:4:0 yes 4600.0000 800.0000<br />
5 0 0 5 5:5:5:0 yes 4600.0000 800.0000<br />
6 0 0 0 0:0:0:0 yes 4600.0000 800.0000<br />
7 0 0 1 1:1:1:0 yes 4600.0000 800.0000<br />
8 0 0 2 2:2:2:0 yes 4600.0000 800.0000<br />
9 0 0 3 3:3:3:0 yes 4600.0000 800.0000<br />
10 0 0 4 4:4:4:0 yes 4600.0000 800.0000<br />
11 0 0 5 5:5:5:0 yes 4600.0000 800.0000<br />
<br />
As we see above, with AMD '''Core 0''' is sequential with '''CPU 0 & 1''', whereas Intel places '''Core 0''' on '''CPU 0 & 6'''.<br />
<br />
If you prefer to have the host and guest running intensive tasks at the same time, it would then be preferable to leave at the very least, '''Core 0''' to the host. Next you will pin the amount of physical CPU's and their associated logical CPU's to the guest XML as shown below. It is also recommended to pin the emulator and iothread to '''Core 0''' as this helps eliminate latency within the VM.<br />
<br />
==== XML Examples ====<br />
{{Note|If you are not using Virtio storage drivers for your VM, ''do not'' use the '''iothread''' lines from the XML examples that are shown below.}}<br />
===== 4c/1t CPU w/o Hyperthreading Example =====<br />
{{hc|$ virsh edit [vmname]|<nowiki><br />
...<br />
<vcpu placement='static'>4</vcpu><br />
<cputune><br />
<vcpupin vcpu='0' cpuset='0'/><br />
<vcpupin vcpu='1' cpuset='1'/><br />
<vcpupin vcpu='2' cpuset='2'/><br />
<vcpupin vcpu='3' cpuset='3'/><br />
</cputune><br />
...<br />
</nowiki>}}<br />
<br />
===== 4c/2t Intel CPU Pinning Example =====<br />
{{hc|$ virsh edit [vmname]|<nowiki><br />
...<br />
<vcpu placement='static'>8</vcpu><br />
<iothreads>1</iothreads><br />
<cputune><br />
<vcpupin vcpu='0' cpuset='2'/><br />
<vcpupin vcpu='1' cpuset='8'/><br />
<vcpupin vcpu='2' cpuset='3'/><br />
<vcpupin vcpu='3' cpuset='9'/><br />
<vcpupin vcpu='4' cpuset='4'/><br />
<vcpupin vcpu='5' cpuset='10'/><br />
<vcpupin vcpu='6' cpuset='5'/><br />
<vcpupin vcpu='7' cpuset='11'/><br />
<emulatorpin cpuset='0,6'/><br />
<iothreadpin iothread='1' cpuset='0,6'/><br />
</cputune><br />
...<br />
<topology sockets='1' cores='4' threads='2'/><br />
...<br />
</nowiki>}}<br />
<br />
===== 4c/2t AMD CPU Example =====<br />
{{hc|$ virsh edit [vmname]|<nowiki><br />
...<br />
<vcpu placement='static'>8</vcpu><br />
<iothreads>1</iothreads><br />
<cputune><br />
<vcpupin vcpu='0' cpuset='2'/><br />
<vcpupin vcpu='1' cpuset='3'/><br />
<vcpupin vcpu='2' cpuset='4'/><br />
<vcpupin vcpu='3' cpuset='5'/><br />
<vcpupin vcpu='4' cpuset='6'/><br />
<vcpupin vcpu='5' cpuset='7'/><br />
<vcpupin vcpu='6' cpuset='8'/><br />
<vcpupin vcpu='7' cpuset='9'/><br />
<emulatorpin cpuset='0-1'/><br />
<iothreadpin iothread='1' cpuset='0-1'/><br />
</cputune><br />
...<br />
<topology sockets='1' cores='4' threads='2'/><br />
...<br />
</nowiki>}}<br />
<br />
{{Note|If further CPU isolation is needed, consider using the '''isolcpus''' kernel command-line parameter on the unused physical/logical cores.}}<br />
<br />
If you do not intend to be doing any computation-heavy work on the host (or even anything at all) at the same time as you would on the VM, you may want to pin your VM threads across all of your cores, so that the VM can fully take advantage of the spare CPU time the host has available. Be aware that pinning all physical and logical cores of your CPU could induce latency in the guest VM.<br />
<br />
=== Static huge pages ===<br />
<br />
When dealing with applications that require large amounts of memory, memory latency can become a problem since the more memory pages are being used, the more likely it is that this application will attempt to access information across multiple memory "pages", which is the base unit for memory allocation. Resolving the actual address of the memory page takes multiple steps, and so CPUs normally cache information on recently used memory pages to make subsequent uses on the same pages faster. Applications using large amounts of memory run into a problem where, for instance, a virtual machine uses 4GB of memory divided into 4kB pages (which is the default size for normal pages), meaning that such cache misses can become extremely frequent and greatly increase memory latency. Huge pages exist to mitigate this issue by giving larger individual pages to those applications, increasing the odds that multiple operations will target the same page in succession. This is normally handeled with transparent huge pages, which dynamically manages hugepages to keep up with the demand.<br />
<br />
On a VM with a PCI passthrough, however, it is '''not possible''' to benefit from transparent huge pages, as IOMMU requires that the guest's memory be allocated and pinned as soon as the VM starts. It is therefore required to allocate huge pages statically in order to benefit from them. <br />
<br />
{{Warning|Static huge pages lock down the allocated amount of memory, making it unavailable for applications that are not configured to use them. Allocating 4GBs worth of huge pages on a machine with 8GBs of memory will only leave you with 4GBs of available memory on the host '''even when the VM is not running'''.}}<br />
<br />
To allocate huge pages at boot, one must simply specify the desired amount on their kernel command line with {{ic|1=hugepages=''x''}}. For instance, reserving 1024 pages with {{ic|1=hugepages=1024}} and the default size of 2048kB per huge page creates 2GBs worth of memory for the virtual machine to use.<br />
<br />
If supported by CPU page size could be set manually. 1 GB huge page support could be verified by {{ic|<nowiki>grep pdpe1gb /proc/cpuinfo</nowiki>}}. Setting 1 GB huge page size via kernel parameters : {{ic|1=default_hugepagesz=1G hugepagesz=1G hugepages=X transparent_hugepage=never}}.<br />
<br />
Also, since static huge pages can only be used by applications that specifically request it, you must add this section in your libvirt domain configuration to allow kvm to benefit from them :<br />
<br />
{{hc|$ virsh edit [vmname]|<nowiki><br />
...<br />
<memoryBacking><br />
<hugepages/><br />
</memoryBacking><br />
...<br />
</nowiki>}}<br />
<br />
=== Dynamic huge pages ===<br />
<br />
{{Accuracy|Need futher testing if this variant as effective as static one}}<br />
<br />
Hugepages could be allocated dynamically via vm.nr_overcommit_hugepages parameter.<br />
<br />
{{hc|sysctl.d/10-kvm.conf|<nowiki><br />
vm.nr_hugepages = 0<br />
vm.nr_overcommit_hugepages = <num><br />
</nowiki>}}<br />
<br />
Where num - is the number of huge pages, which default size if 2Mb.<br />
Pages will be automatically allocated, and freed after VM stops.<br />
<br />
More manual way <br />
{{hc|manual dynamic huge pages|<nowiki><br />
echo <num> | sudo tee /sys/kernel/mm/hugepages/hugepages-2048kB/nr_hugepages<br />
echo <num> | sudo tee /sys/kernel/mm/hugepages/hugepages-1048576kB/nr_hugepages<br />
</nowiki>}}<br />
<br />
For 2Mb and 1Gb page size respectively.<br />
And they should be manually freed in the same way.<br />
<br />
It is hardly recommended to drop caches, compact memory and wait couple of seconds before starting VM, as there could be not enough free contiguous memory for required huge pages blocks. Especially after some uptime of the host system.<br />
{{hc|organize ram|<nowiki><br />
echo 3 | sudo tee /proc/sys/vm/drop_caches<br />
echo 1 | sudo tee /proc/sys/vm/compact_memory<br />
</nowiki>}}<br />
<br />
Theoretically, 1Gb pages works as 2Mb. But practically - no guaranteed way was found to get contiguous 1Gb memory blocks. Each consequent request of 1Gb blocks lead to lesser and lesser dynamically allocated count.<br />
<br />
=== CPU frequency governor ===<br />
<br />
Depending on the way your [[CPU frequency scaling|CPU governor]] is configured, the VM threads may not hit the CPU load thresholds for the frequency to ramp up. Indeed, KVM cannot actually change the CPU frequency on its own, which can be a problem if it does not scale up with vCPU usage as it would result in underwhelming performance. An easy way to see if it behaves correctly is to check if the frequency reported by {{ic|watch lscpu}} goes up when running a CPU-intensive task on the guest. If you are indeed experiencing stutter and the frequency does not go up to reach its reported maximum, it may be due to [https://lime-technology.com/forum/index.php?topic=46664.msg447678#msg447678 cpu scaling being controlled by the host OS]. In this case, try setting all cores to maximum frequency to see if this improves performance. Note that if you are using a modern intel chip with the default pstate driver, cpupower commands will be [[CPU frequency scaling#CPU frequency driver|ineffective]], so monitor {{ic|/proc/cpuinfo}} to make sure your cpu is actually at max frequency.<br />
<br />
=== CPU pinning with isolcpus ===<br />
<br />
Alternatively, make sure that you have isolated CPUs properly. In this example, let us assume you are using CPUs 4-7.<br />
Use the kernel parameters {{ic|isolcpus nohz_full rcu_nocbs}} to completely isolate the CPUs from the kernel. <br />
<br />
{{hc|sudo vim /etc/defaults/grub|<nowiki><br />
...<br />
GRUB_CMDLINE_LINUX="..your other params.. isolcpus=4-7 nohz_full=4-7 rcu_nocbs=4-7"<br />
...<br />
</nowiki>}}<br />
<br />
Then, run {{ic|qemu-system-x86_64}} with taskset and chrt:<br />
<br />
# chrt -r 1 taskset -c 4-7 qemu-system-x86_64 ...<br />
<br />
The {{ic|chrt}} command will ensure that the task scheduler will round-robin distribute work (otherwise it will all stay on the first cpu). For {{ic|taskset}}, the CPU numbers can be comma- and/or dash-separated, like "0,1,2,3" or "0-4" or "1,7-8,10" etc.<br />
<br />
See [https://www.reddit.com/r/VFIO/comments/6vgtpx/high_dpc_latency_and_audio_stuttering_on_windows/dm0sfto/ this reddit comment] for more info.<br />
<br />
=== Improving performance on AMD CPUs ===<br />
<br />
Previously, Nested Page Tables (NPT) had to be disabled on AMD systems running KVM to improve GPU performance because of a [https://sourceforge.net/p/kvm/bugs/230/ very old bug], but the trade off was decreased CPU performance, including stuttering.<br />
<br />
There is a [https://patchwork.kernel.org/patch/10027525/ kernel patch] that resolves this issue, which was accepted into kernel 4.14-stable and 4.9-stable. If you are running the official {{Pkg|linux}} or {{Pkg|linux-lts}} kernel the patch has already been applied (make sure you are on the latest). If you are running another kernel you might need to manually patch yourself.<br />
<br />
{{Note|Several Ryzen users (see [https://www.reddit.com/r/VFIO/comments/78i3jx/possible_fix_for_the_npt_issue_discussed_on_iommu/ this Reddit thread]) have tested the patch, and can confirm that it works, bringing GPU passthrough performance up to near native quality.}}<br />
<br />
<br />
Starting with QEMU 3.1 the TOPOEXT cpuid flag is disabled by default. In order to use hyperthreading(SMT) on AMD CPU's you need to manually enable it:<br />
<br />
<cpu mode='host-passthrough' check='none'><br />
<topology sockets='1' cores='4' threads='2'/><br />
<feature policy='require' name='topoext'/><br />
</cpu><br />
<br />
commit: https://git.qemu.org/?p=qemu.git;a=commit;h=7210a02c58572b2686a3a8d610c6628f87864aed<br />
<br />
=== Further Tuning ===<br />
<br />
More specialized VM tuning tips are available at [https://access.redhat.com/documentation/en-US/Red_Hat_Enterprise_Linux/7/html-single/Virtualization_Tuning_and_Optimization_Guide/index.html Red Hat's Virtualization Tuning and Optimization Guide]. This guide may be especially helpful if you are experiencing:<br />
<br />
* Moderate CPU load on the host during downloads/uploads from within the guest: See ''Bridge Zero Copy Transmit'' for a potential fix.<br />
* Guests capping out at certain network speeds during downloads/uploads despite virtio-net being used: See ''Multi-queue virtio-net'' for a potential fix.<br />
* Guests "stuttering" under high I/O, despite the same workload not affecting hosts to the same degree: See ''Multi-queue virtio-scsi'' for a potential fix.<br />
<br />
== Special procedures ==<br />
<br />
Certain setups require specific configuration tweaks in order to work properly. If you are having problems getting your host or your VM to work properly, see if your system matches one of the cases below and try adjusting your configuration accordingly.<br />
<br />
=== Using identical guest and host GPUs ===<br />
<br />
{{Expansion|A number of users have been having issues with this, it should probably be adressed by the article.|Talk:PCI passthrough via OVMF#Additionnal sections}}<br />
<br />
Due to how vfio-pci uses your vendor and device id pair to identify which device they need to bind to at boot, if you have two GPUs sharing such an ID pair you will not be able to get your passthough driver to bind with just one of them. This sort of setup makes it necessary to use a script, so that whichever driver you are using is instead assigned by pci bus address using the {{ic|driver_override}} mechanism.<br />
<br />
==== Script variants ====<br />
<br />
===== Passthrough all GPUs but the boot GPU =====<br />
<br />
Here, we will make a script to bind vfio-pci to all GPUs but the boot gpu. Create the script {{ic|/usr/bin/vfio-pci-override.sh}}:<br />
<br />
{{bc|<nowiki><br />
#!/bin/sh<br />
<br />
for i in /sys/devices/pci*/*/boot_vga; do<br />
if [ $(cat "$i") -eq 0 ]; then<br />
GPU="${i%/boot_vga}"<br />
AUDIO="$(echo "$GPU" | sed -e "s/0$/1/")"<br />
echo "vfio-pci" > "$GPU/driver_override"<br />
if [ -d "$AUDIO" ]; then<br />
echo "vfio-pci" > "$AUDIO/driver_override"<br />
fi<br />
fi<br />
done<br />
<br />
modprobe -i vfio-pci<br />
</nowiki>}}<br />
<br />
===== Passthrough selected GPU =====<br />
<br />
In this case we manually specify the GPU to bind.<br />
<br />
{{bc|<nowiki><br />
#!/bin/sh<br />
<br />
GROUP="0000:00:03.0"<br />
DEVS="0000:03:00.0 0000:03:00.1 ."<br />
<br />
if [ ! -z "$(ls -A /sys/class/iommu)" ]; then<br />
for DEV in $DEVS; do<br />
echo "vfio-pci" > /sys/bus/pci/devices/$GROUP/$DEV/driver_override<br />
done<br />
fi<br />
<br />
modprobe -i vfio-pci<br />
</nowiki>}}<br />
<br />
==== Script installation ====<br />
<br />
Create {{ic|/etc/modprobe.d/vfio.conf}} with the following:<br />
<br />
install vfio-pci /usr/bin/vfio-pci-override.sh<br />
<br />
Edit {{ic|/etc/mkinitcpio.conf}}<br />
<br />
Remove any video drivers from MODULES, and add {{ic|vfio-pci}}, and {{ic|vfio_iommu_type1}}<br />
<br />
MODULES=(ext4 vfat vfio-pci vfio_iommu_type1)<br />
<br />
Add {{ic|/etc/modprobe.d/vfio.conf}} and {{ic|/usr/bin/vfio-pci-override.sh}} to FILES:<br />
<br />
FILES=(/etc/modprobe.d/vfio.conf /usr/bin/vfio-pci-override.sh)<br />
<br />
[[Regenerate the initramfs]] and reboot:<br />
<br />
=== Passing the boot GPU to the guest ===<br />
<br />
{{Expansion|This is related to VBIOS issues and should be moved into a separate section regarding VBIOS compatibility.|section=UEFI (OVMF) Compatibility in VBIOS}}<br />
<br />
The GPU marked as {{ic|boot_vga}} is a special case when it comes to doing PCI passthroughs, since the BIOS needs to use it in order to display things like boot messages or the BIOS configuration menu. To do that, it makes [https://www.redhat.com/archives/vfio-users/2016-May/msg00224.html a copy of the VGA boot ROM which can then be freely modified]. This modified copy is the version the system gets to see, which the passthrough driver may reject as invalid. As such, it is generally recommended to change the boot GPU in the BIOS configuration so the host GPU is used instead or, if that's not possible, to swap the host and guest cards in the machine itself.<br />
<br />
=== Using Looking Glass to stream guest screen to the host ===<br />
<br />
It is possible to make VM share the monitor, and optionally a keyboard and a mouse with a help of [https://looking-glass.hostfission.com/ Looking Glass].<br />
<br />
==== Adding IVSHMEM Device to VM ====<br />
<br />
Looking glass works by creating a shared memory buffer between a host and a guest. This is a lot faster than streaming frames via localhost, but requires additional setup. <br />
<br />
With your VM turned off open the machine configuration<br />
<br />
{{hc|$ virsh edit [vmname]|<nowiki><br />
...<br />
<devices><br />
...<br />
<shmem name='looking-glass'><br />
<model type='ivshmem-plain'/><br />
<size unit='M'>32</size><br />
</shmem><br />
</devices><br />
...<br />
</nowiki>}}<br />
<br />
You should replace 32 with your own calculated value based on what resolution you are going to pass through. It can be calculated like this:<br />
<br />
width x height x 4 x 2 = total bytes<br />
total bytes / 1024 / 1024 = total megabytes + 2<br />
<br />
For example, in case of 1920x1080<br />
<br />
1920 x 1080 x 4 x 2 = 16,588,800 bytes<br />
16,588,800 / 1024 / 1024 = 15.82 MB + 2 = 17.82<br />
<br />
The result must be '''rounded up''' to the nearest power of two, and since 17.82 is bigger than 16 we should choose 32.<br />
<br />
Next create a script to create a shared memory file.<br />
<br />
{{hc|/usr/local/bin/looking-glass-init.sh|2=<br />
#!/bin/sh<br />
<br />
touch /dev/shm/looking-glass<br />
chown '''user''':kvm /dev/shm/looking-glass<br />
chmod 660 /dev/shm/looking-glass<br />
}}<br />
<br />
Replace user with your username.<br />
<br />
Remember to make the script [[executable]].<br />
<br />
Create a [[systemd]] unit to execute this script during boot<br />
<br />
{{hc|/etc/systemd/system/looking-glass-init.service|2=<br />
[Unit]<br />
Description=Create shared memory for looking glass<br />
<br />
[Service]<br />
Type=oneshot<br />
ExecStart=/usr/local/bin/looking-glass-init.sh<br />
<br />
[Install]<br />
WantedBy=multi-user.target<br />
}}<br />
<br />
[[Start]] and [[enable]] {{ic|looking-glass-init.service}}<br />
<br />
==== Installing the IVSHMEM Host to Windows guest ====<br />
<br />
Currently Windows would not notify users about a new IVSHMEM device, it would silently install a dummy driver. To actually enable the device you have to go into device manager and update the driver for the device under the "System Devices" node for '''"PCI standard RAM Controller"'''. Download a signed driver [https://github.com/virtio-win/kvm-guest-drivers-windows/issues/217 from issue 217], as it is not yet available elsewhere.<br />
<br />
Once the driver is installed you must download the latest [https://looking-glass.hostfission.com/downloads looking-glass-host] from the looking glass website and start it on your guest. In order to run it you would also need to install Microsoft Visual C++ Redistributable from [https://www.visualstudio.com/downloads/ Microsoft]<br />
It is also possible to make it start automatically on VM boot by editing the {{ic|HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Windows\CurrentVersion\Run}} registry and adding a path to the downloaded executable.<br />
<br />
==== Getting a client ====<br />
<br />
Looking glass client can be installed from AUR using {{AUR|looking-glass}} or {{AUR|looking-glass-git}} packages.<br />
<br />
You can start it once the VM is set up and running<br />
<br />
$ looking-glass-client<br />
<br />
If you do not want to use Spice to control the guest mouse and keyboard you can disable the Spice server.<br />
<br />
$ looking-glass-client -s<br />
<br />
Additionally you may want to start Looking Glass Client as full screen, otherwise the image may be scaled down resulting in poor image fidelity.<br />
<br />
$ looking-glass-client -F<br />
<br />
Launch with the {{ic|--help}} option for further information.<br />
<br />
=== Swap peripherals to and from the Host ===<br />
<br />
Looking Glass includes a Spice client in order to control mouse movement on the Windows guest. However this may have too much latency for certain applications, such as gaming. An alternative method is passing through specific USB devices for minimal latency. This allows for switching the devices between host and guest.<br />
<br />
First create a .xml file for the device(s) you wish to pass-through, which libvirt will use to identify the device.<br />
<br />
{{hc|/home/$USER/.VFIOinput/input_1.xml|2=<br />
<hostdev mode='subsystem' type='usb' managed='no'><br />
<source><br />
<vendor id='0x[Before Colon]'/><br />
<product id='0x[After Colon]'/><br />
</source><br />
</hostdev><br />
}}<br />
<br />
Replace [Before/After Colon] with the contents of the 'lsusb' command, specific to the device you want to pass-through.<br />
<br />
For instance my mouse is {{ic|Bus 005 Device 002: ID 1532:0037 Razer USA, Ltd}} so I would replace {{ic|vendor id}} with 1532, and {{ic|product id}} with 1037.<br />
<br />
Repeat this process for any additional USB devices you want to pass-through. If your mouse / keyboard has multiple entries in {{ic|lsusb}}, perhaps if it is wireless, then create additional xml files for each.<br />
<br />
{{Note|Don't forget to change the path & name of the script(s) above and below to match your user and specific system.}}<br />
<br />
Next a bash script file is needed to tell libvirt what to attach/detach the USB devices to the guest.<br />
<br />
{{hc|/home/$USER/.VFIOinput/input_attach.sh|2=<br />
#!/bin/bash<br />
<br />
virsh attach-device [VM-Name] [USBdevice]<br />
}}<br />
<br />
Replace [VM-Name] with the name of your virtual machine, which can be seen under virt-manager. Additionally replace [USBdevice] with the '''full''' path to the .xml file for the device you wish to pass-through. Add additional lines for more than 1 device. For example here is my script:<br />
<br />
{{hc|/home/ajmar/.VFIOinput/input_attach.sh|2=<br />
#!/bin/bash<br />
<br />
virsh attach-device win10 /home/ajmar/.VFIOinput/input_mouse.xml<br />
virsh attach-device win10 /home/ajmar/.VFIOinput/input_keyboard.xml<br />
}}<br />
<br />
Next duplicate the script file and replace {{ic|attach-device}} with {{ic|detach-device}}. Ensure both scripts are executable with {{ic|chmod +x $script.sh}}<br />
<br />
This 2 script files can now be executed to attach or detach your USB devices from the host to the guest VM. It is important to note that they may need to be executed as root. To run the script from the Windows VM, one possibility is using [[PuTTY]] to [[SSH]] into the host, and execute the script. On Windows PuTTY comes with plink.exe which can execute singular commands over SSH before then logging out, instead of opening a SSH terminal, all in the background.<br />
<br />
{{hc|detach_devices.bat|2=<br />
"C:\Program Files\PuTTY\plink.exe" root@$HOST_IP -pw $ROOTPASSWORD /home/$USER/.VFIOinput/input_detach.sh<br />
}}<br />
<br />
Replace {{ic|$HOST_IP}} with the Host [[Network configuration#IP addresses|IP Address]] and $ROOTPASSWORD with the root password.<br />
<br />
{{warning|This method is insecure if somebody has access to your VM, since they could open the file and read your password. It is advisable to use [[SSH keys]] instead!}}<br />
<br />
You may also want to execute the script files using key binds. On Windows one option is [https://autohotkey.com/ Autohotkey], and on the Host [[Xbindkeys]]. Because of the need to run the scripts as root, you may also need to use [[Polkit]] which can be used to authenticate specific executables as able to run as root without needing a password.<br />
<br />
=== Bypassing the IOMMU groups (ACS override patch) ===<br />
<br />
If you find your PCI devices grouped among others that you do not wish to pass through, you may be able to seperate them using Alex Williamson's ACS override patch. Make sure you understand [https://vfio.blogspot.com/2014/08/iommu-groups-inside-and-out.html the potential risk] of doing so. <br />
<br />
You will need a kernel with the patch applied. The easiest method to acquiring this is through the {{AUR|linux-vfio}} package. <br />
<br />
In addition, the ACS override patch needs to be enabled with kernel command line options. The patch file adds the following documentation:<br />
<br />
pcie_acs_override =<br />
[PCIE] Override missing PCIe ACS support for:<br />
downstream<br />
All downstream ports - full ACS capabilties<br />
multifunction<br />
All multifunction devices - multifunction ACS subset<br />
id:nnnn:nnnn<br />
Specfic device - full ACS capabilities<br />
Specified as vid:did (vendor/device ID) in hex<br />
<br />
The option {{ic|1=pcie_acs_override=downstream}} is typically sufficient.<br />
<br />
After installation and configuration, reconfigure your [[Kernel parameters|bootloader kernel parameters]] to load the new kernel with the {{ic|1=pcie_acs_override=}} option enabled.<br />
<br />
== Plain QEMU without libvirt ==<br />
<br />
Instead of setting up a virtual machine with the help of libvirt, plain QEMU commands with custom parameters can be used for running the VM intended to be used with PCI passthrough. This is desirable for some use cases like scripted setups, where the flexibility for usage with other scripts is needed.<br />
<br />
To achieve this after [[#Setting up IOMMU]] and [[#Isolating the GPU]], follow the [[QEMU]] article to setup the virtualized environment, [[QEMU#Enabling KVM|enable KVM]] on it and use the flag {{ic|1=-device vfio-pci,host=07:00.0}} replacing the identifier (07:00.0) with your actual device's ID that you used for the GPU isolation earlier.<br />
<br />
For utilizing the OVMF firmware, make sure the {{Pkg|ovmf}} package is installed, copy the UEFI variables from {{ic|/usr/share/ovmf/x64/OVMF_VARS.fd}} to temporary location like {{ic|/tmp/MY_VARS.fd}} and finally specify the OVMF paths by appending the following parameters to the QEMU command (order matters):<br />
<br />
* {{ic|1=-drive if=pflash,format=raw,readonly,file=/usr/share/ovmf/x64/OVMF_CODE.fd}} for the actual OVMF firmware binary, note the readonly option<br />
* {{ic|1=-drive if=pflash,format=raw,file=/tmp/MY_VARS.fd}} for the variables<br />
<br />
{{Note|QEMU's default SeaBIOS can be used instead of OVMF, but it's not recommended as it can cause issues with passthrough setups.}}<br />
<br />
It's recommended to study the QEMU article for ways to enhance the performance by using the [[QEMU#Installing virtio drivers|virtio drivers]] and other further configurations for the setup.<br />
<br />
You also might have to use the {{ic|1=-cpu host,kvm=off}} parameter to forward the host's CPU model info to the VM and fool the virtualization detection used by Nvidia's and possibly other manufacturers' device drivers trying to block the full hardware usage inside a virtualized system.<br />
<br />
== Passing though other devices ==<br />
<br />
=== USB controller ===<br />
<br />
If your motherboard has multiple USB controllers mapped to multiple groups, it is possible to pass those instead of USB devices. Passing an actual controller over an individual USB device provides the following advantages : <br />
<br />
* If a device disconnects or changes ID over the course of an given operation (such as a phone undergoing an update), the VM will not suddenly stop seeing it.<br />
* Any USB port managed by this controller is directly handled by the VM and can have its devices unplugged, replugged and changed without having to notify the hypervisor.<br />
* Libvirt will not complain if one of the USB devices you usually pass to the guest is missing when starting the VM.<br />
<br />
Unlike with GPUs, drivers for most USB controllers do not require any specific configuration to work on a VM and control can normally be passed back and forth between the host and guest systems with no side effects.<br />
<br />
{{Warning|Make sure your USB controller supports resetting: [[#Passing through a device that does not support resetting]]}}<br />
<br />
You can find out which PCI devices correspond to which controller and how various ports and devices are assigned to each one of them using this command :<br />
<br />
{{hc|$ <nowiki>for usb_ctrl in $(find /sys/bus/usb/devices/usb* -maxdepth 0 -type l); do pci_path="$(dirname "$(realpath "${usb_ctrl}")")"; echo "Bus $(cat "${usb_ctrl}/busnum") --> $(basename $pci_path) (IOMMU group $(basename $(realpath $pci_path/iommu_group)))"; lsusb -s "$(cat "${usb_ctrl}/busnum"):"; echo; done</nowiki>|<br />
Bus 1 --> 0000:00:1a.0 (IOMMU group 4)<br />
Bus 001 Device 004: ID 04f2:b217 Chicony Electronics Co., Ltd Lenovo Integrated Camera (0.3MP)<br />
Bus 001 Device 007: ID 0a5c:21e6 Broadcom Corp. BCM20702 Bluetooth 4.0 [ThinkPad]<br />
Bus 001 Device 008: ID 0781:5530 SanDisk Corp. Cruzer<br />
Bus 001 Device 002: ID 8087:0024 Intel Corp. Integrated Rate Matching Hub<br />
Bus 001 Device 001: ID 1d6b:0002 Linux Foundation 2.0 root hub<br />
<br />
Bus 2 --> 0000:00:1d.0 (IOMMU group 9)<br />
Bus 002 Device 006: ID 0451:e012 Texas Instruments, Inc. TI-Nspire Calculator<br />
Bus 002 Device 002: ID 8087:0024 Intel Corp. Integrated Rate Matching Hub<br />
Bus 002 Device 001: ID 1d6b:0002 Linux Foundation 2.0 root hub<br />
}}<br />
<br />
This laptop has 3 USB ports managed by 2 USB controllers, each with their own IOMMU group. In this example, Bus 001 manages a single USB port (with a SanDisk USB pendrive plugged into it so it appears on the list), but also a number of internal devices, such as the internal webcam and the bluetooth card. Bus 002, on the other hand, does not apprear to manage anything except for the calculator that is plugged into it. The third port is empty, which is why it does not show up on the list, but is actually managed by Bus 002.<br />
<br />
Once you have identified which controller manages which ports by plugging various devices into them and decided which one you want to passthrough, simply add it to the list of PCI host devices controlled by the VM in your guest configuration. No other configuration should be needed.<br />
<br />
{{Note|If your USB controller does not support resetting, is not in an isolated group, or is otherwise unable to be passed through then it may still be possible to accomplish similar results through [[udev]] rules. See [https://github.com/olavmrk/usb-libvirt-hotplug] which allows any device connected to specified USB ports to be automatically attached to a virtual machine.}}<br />
<br />
=== Passing VM audio to host via PulseAudio ===<br />
<br />
It is possible to route the virtual machine's audio to the host as an application using libvirt. This has the advantage of multiple audio streams being routable to one host output, and working with audio output devices that do not support passthrough. [[PulseAudio]] is required for this to work.<br />
<br />
First, remove the comment from the {{ic|1=#user = ""}} line. Then add your username in the quotations. This tells QEMU which user's pulseaudio stream to route through.<br />
<br />
{{hc|/etc/libvirt/qemu.conf|2=<br />
user = "example"<br />
}}<br />
<br />
Next, modify the libvirt configuration<br />
<br />
{{hc|$ virsh edit [vmname]|<br />
<domain type<nowiki>=</nowiki>'kvm'><br />
}}<br />
<br />
to<br />
<br />
{{hc|$ virsh edit [vmname]|<br />
<domain type<nowiki>=</nowiki>'kvm' xmlns:qemu<nowiki>='http://libvirt.org/schemas/domain/qemu/1.0'</nowiki>><br />
}}<br />
<br />
Then set the QEMU PulseAudio environment variables at the bottom of the libvirt xml file<br />
<br />
{{hc|$ virsh edit [vmname]|<br />
</devices><br />
</domain><br />
}}<br />
<br />
to<br />
<br />
{{hc|$ virsh edit [vmname]|<br />
</devices><br />
<qemu:commandline><br />
<qemu:env name<nowiki>=</nowiki>'QEMU_AUDIO_DRV' value<nowiki>=</nowiki>'pa'/><br />
<qemu:env name<nowiki>=</nowiki>'QEMU_PA_SERVER' value<nowiki>=</nowiki>'/run/user/1000/pulse/native'/><br />
</qemu:commandline><br />
</domain><br />
}}<br />
<br />
Change 1000 under the user directory to your user uid (which can be found by running the {{ic|id}} command. Remember to save the file and exit it without ending the process before continuing, otherwise the changes will not register. If you get the message {{ic|<nowiki>Domain [vmname] XML configuration edited.</nowiki>}} after exiting, it means that your changes have been applied.<br />
<br />
[[Restart]] {{ic|libvirtd.service}} and [[systemd/User|user service]] {{ic|pulseaudio.service}}.<br />
<br />
Virtual Machine audio will now be routed through the host as an application. The application {{Pkg|pavucontrol}} can be used to control the output device. Be aware that on Windows guests, this can cause audio crackling without [[#Slowed down audio pumped through HDMI on the video card|using Message-Signaled Interrupts.]]<br />
<br />
==== Qemu 3.0 Audio Changes ====<br />
As of QEMU 3.0 part of the audio patches have been merged ([https://www.reddit.com/r/VFIO/comments/97iuov/qemu_30_released/e49wmyd/ reddit link]). The {{AUR|qemu-patched}} package currently includes some additional audio patches, as some of the patches have not been officially up-streamed yet.<br />
<br />
You will need to change the chipset accordingly to how your VM is set up, i.e. {{ic|pc-q35-3.0}} or {{ic|pc-i440fx-3.0}} (after installing qemu 3.0) to use the new code paths:<br />
<br />
{{hc|$ virsh edit [vmname]|<br />
<domain type<nowiki>=</nowiki>'kvm'><br />
...<br />
<os><br />
<type arch<nowiki>=</nowiki>'x86_64' machine<nowiki>=</nowiki>'pc-q35-3.0'>hvm</type><br />
...<br />
</os><br />
}}<br />
<br />
{{hc|$ virsh edit [vmname]|<br />
<domain type<nowiki>=</nowiki>'kvm'><br />
...<br />
<os><br />
<type arch<nowiki>=</nowiki>'x86_64' machine<nowiki>=</nowiki>'pc-i440fx-3.0'>hvm</type><br />
...<br />
</os><br />
}}<br />
<br />
{{Note|To speed up compilation time with {{AUR|qemu-patched}} use {{ic|1=--target-list=x86_64-softmmu}} to compile qemu with only x86_64 guest support}}<br />
<br />
{{Note| You might need to remove the qemu:env lines that are mentioned in [[#Passing VM audio to host via PulseAudio]] }}<br />
<br />
=== Physical Disk/Partition ===<br />
<br />
A whole disk or a partition may be used as a whole for improved I/O performance by adding an entry to the XML<br />
<br />
Virtio-BLK Example:<br />
{{hc|$ virsh edit [vmname]| <nowiki><br />
<devices><br />
...<br />
<disk type='block' device='disk'><br />
<driver name='qemu' type='raw' cache='none' io='native'/><br />
<source dev='/dev/disk/by-id/xxxxxxxx'/><br />
<target dev='vda' bus='virtio'/><br />
</disk><br />
...<br />
</devices><br />
</nowiki><br />
}}<br />
<br />
Virtio-SCSI Example:<br />
{{hc|$ virsh edit [vmname]| <nowiki><br />
<devices><br />
...<br />
<disk type='block' device='disk'><br />
<driver name='qemu' type='raw' cache='none' io='native'/><br />
<source dev='/dev/disk/by-id/xxxxxxxx'/><br />
<target dev='sda' bus='scsi'/><br />
</disk><br />
...<br />
</devices><br />
</nowiki><br />
}}<br />
<br />
To find out which disk/partition is associated with the one you would like to pass:<br />
<br />
{{hc|<br />
$ ls -l /dev/disk/by-id|<br />
ata-ST1000LM002-9VQ14L_Z0501SZ9 -> ../../sdd<br />
ata-ST1000LM002-9VQ14L_Z0501SZ9-part1 -> ../../sdd1<br />
}}<br />
<br />
You can also add the disk with Virt-Manager's '''Add Hardware''' menu and then type the disk you want in the '''Select or create custom storage''' box, e.g. '''/dev/disk/by-id/ata-ST1000LM002-9VQ14L_Z0501SZ9'''<br />
<br />
Depending on which bus you use, the above step will require either the VIOSTOR(bus=virtio) or VIOSCSI(bus=scsi) driver on Windows guests, refer to [[#Setting up the guest OS]] for the driver ISO.<br />
<br />
=== Gotchas ===<br />
<br />
==== Passing through a device that does not support resetting ====<br />
<br />
When the VM shuts down, all devices used by the guest are deinitialized by its OS in preparation for shutdown. In this state, those devices are no longer functional and must then be power-cycled before they can resume normal operation. Linux can handle this power-cycling on its own, but when a device has no known reset methods, it remains in this disabled state and becomes unavailable. Since Libvirt and Qemu both expect all host PCI devices to be ready to reattach to the host before completely stopping the VM, when encountering a device that will not reset, they will hang in a "Shutting down" state where they will not be able to be restarted until the host system has been rebooted. It is therefore reccomanded to only pass through PCI devices which the kernel is able to reset, as evidenced by the presence of a {{ic|reset}} file in the PCI device sysfs node, such as {{ic|/sys/bus/pci/devices/0000:00:1a.0/reset}}.<br />
<br />
The following bash command shows which devices can and cannot be reset.<br />
<br />
{{hc|<nowiki>for iommu_group in $(find /sys/kernel/iommu_groups/ -maxdepth 1 -mindepth 1 -type d);do echo "IOMMU group $(basename "$iommu_group")"; for device in $(\ls -1 "$iommu_group"/devices/); do if [[ -e "$iommu_group"/devices/"$device"/reset ]]; then echo -n "[RESET]"; fi; echo -n $'\t';lspci -nns "$device"; done; done</nowiki>|<br />
IOMMU group 0<br />
00:00.0 Host bridge [0600]: Intel Corporation Xeon E3-1200 v2/Ivy Bridge DRAM Controller [8086:0158] (rev 09)<br />
IOMMU group 1<br />
00:01.0 PCI bridge [0604]: Intel Corporation Xeon E3-1200 v2/3rd Gen Core processor PCI Express Root Port [8086:0151] (rev 09)<br />
01:00.0 VGA compatible controller [0300]: NVIDIA Corporation GK208 [GeForce GT 720] [10de:1288] (rev a1)<br />
01:00.1 Audio device [0403]: NVIDIA Corporation GK208 HDMI/DP Audio Controller [10de:0e0f] (rev a1)<br />
IOMMU group 2<br />
00:14.0 USB controller [0c03]: Intel Corporation 7 Series/C210 Series Chipset Family USB xHCI Host Controller [8086:1e31] (rev 04)<br />
IOMMU group 4<br />
[RESET] 00:1a.0 USB controller [0c03]: Intel Corporation 7 Series/C210 Series Chipset Family USB Enhanced Host Controller #2 [8086:1e2d] (rev 04)<br />
IOMMU group 5<br />
[RESET] 00:1b.0 Audio device [0403]: Intel Corporation 7 Series/C210 Series Chipset Family High Definition Audio Controller [8086:1e20] (rev 04)<br />
IOMMU group 10<br />
[RESET] 00:1d.0 USB controller [0c03]: Intel Corporation 7 Series/C210 Series Chipset Family USB Enhanced Host Controller #1 [8086:1e26] (rev 04)<br />
IOMMU group 13<br />
06:00.0 VGA compatible controller [0300]: NVIDIA Corporation GM204 [GeForce GTX 970] [10de:13c2] (rev a1)<br />
06:00.1 Audio device [0403]: NVIDIA Corporation GM204 High Definition Audio Controller [10de:0fbb] (rev a1)<br />
}}<br />
<br />
This signals that the xHCI USB controller in 00:14.0 cannot be reset and will therefore stop the VM from shutting down properly, while the integrated sound card in 00:1b.0 and the other two controllers in 00:1a.0 and 00:1d.0 do not share this problem and can be passed without issue.<br />
<br />
== Complete setups and examples ==<br />
<br />
If you have trouble configuring a certain mechanism in your setup, you might want to look up [[PCI_passthrough_via_OVMF/Examples|complete passthrough setup examples]]. A few users have described their setups and you might want to look up certain tricks from their configuration files.<br />
<br />
== Troubleshooting ==<br />
<br />
If your issue is not mentioned below, you may want to browse [[QEMU#Troubleshooting]].<br />
<br />
=== "Error 43: Driver failed to load" on Nvidia GPUs passed to Windows VMs ===<br />
<br />
{{Note|<br />
* This may also fix SYSTEM_THREAD_EXCEPTION_NOT_HANDLED boot crashes related to Nvidia drivers.<br />
* This may also fix problems under linux guests.<br />
}}<br />
<br />
Since version 337.88, Nvidia drivers on Windows check if an hypervisor is running and fail if it detects one, which results in an Error 43 in the Windows device manager. Starting with QEMU 2.5.0 and libvirt 1.3.3, the vendor_id for the hypervisor can be spoofed, which is enough to fool the Nvidia drivers into loading anyway. All one must do is add {{ic|<nowiki>hv_vendor_id=whatever</nowiki>}} to the cpu parameters in their QEMU command line, or by adding the following line to their libvirt domain configuration. It may help for the ID to be set to a 12-character alphanumeric (e.g. '1234567890ab') as opposed to longer or shorter strings.<br />
<br />
{{hc|$ virsh edit [vmname]|<nowiki><br />
...<br />
<features><br />
<hyperv><br />
...<br />
<vendor_id state='on' value='whatever'/><br />
...<br />
</hyperv><br />
...<br />
<kvm><br />
<hidden state='on'/><br />
</kvm><br />
</features><br />
...<br />
</nowiki>}}<br />
<br />
Users with older versions of QEMU and/or libvirt will instead have to disable a few hypervisor extensions, which can degrade performance substantially. If this is what you want to do, do the following replacement in your libvirt domain config file.<br />
<br />
{{hc|$ virsh edit [vmname]|<nowiki><br />
...<br />
<features><br />
<hyperv><br />
<relaxed state='on'/><br />
<vapic state='on'/><br />
<spinlocks state='on' retries='8191'/><br />
</hyperv><br />
...<br />
</features><br />
...<br />
<clock offset='localtime'><br />
<timer name='hypervclock' present='yes'/><br />
</clock><br />
...<br />
</nowiki>}}<br />
<br />
{{bc|<nowiki><br />
...<br />
<clock offset='localtime'><br />
<timer name='hypervclock' present='no'/><br />
</clock><br />
...<br />
<features><br />
<kvm><br />
<hidden state='on'/><br />
</kvm><br />
...<br />
<hyperv><br />
<relaxed state='off'/><br />
<vapic state='off'/><br />
<spinlocks state='off'/><br />
</hyperv><br />
...<br />
</features><br />
...<br />
</nowiki>}}<br />
<br />
=== "BAR 3: cannot reserve [mem]" error in dmesg after starting VM ===<br />
<br />
{{Expansion|This error is actually related to the boot_vgs issue and should be merged together with everything else concerning GPU ROMs.|section=UEFI (OVMF) Compatibility in VBIOS}}<br />
<br />
With respect to [https://www.linuxquestions.org/questions/linux-kernel-70/kernel-fails-to-assign-memory-to-pcie-device-4175487043/ this article]:<br />
<br />
If you still have code 43 check dmesg for memory reservation errors after starting up VM, if you have similar it could be the case:<br />
<br />
vfio-pci 0000:09:00.0: BAR 3: cannot reserve [mem 0xf0000000-0xf1ffffff 64bit pref]<br />
<br />
Find out a PCI Bridge your graphic card is connected to. This will give actual hierarchy of devices:<br />
<br />
$ lspci -t<br />
<br />
Before starting VM run following lines replacing IDs with actual from previous output.<br />
<br />
# echo 1 > /sys/bus/pci/devices/0000\:00\:03.1/remove<br />
# echo 1 > /sys/bus/pci/rescan<br />
<br />
{{Note|Probably setting [[kernel parameter]] {{ic|1=video=efifb:off}} is required as well. [https://pve.proxmox.com/wiki/Pci_passthrough#BAR_3:_can.27t_reserve_.5Bmem.5D_error Source]}}<br />
<br />
=== UEFI (OVMF) Compatibility in VBIOS ===<br />
<br />
{{Remove|Flashing you guest GPU for the purpose of a GPU passthrough is '''never''' good advice. A full section should be dedicated to VBIOS compatibility.|section= UEFI (OVMF) Compatibility in VBIOS}}<br />
<br />
With respect to [https://pve.proxmox.com/wiki/Pci_passthrough#How_to_known_if_card_is_UEFI_.28ovmf.29_compatible this article]:<br />
<br />
Error 43 can be caused by the GPU's VBIOS without UEFI support. To check whenever your VBIOS supports it, you will have to use {{ic|rom-parser}}:<br />
<br />
$ git clone https://github.com/awilliam/rom-parser<br />
$ cd rom-parser && make<br />
<br />
Dump the GPU VBIOS:<br />
<br />
# echo 1 > /sys/bus/pci/devices/0000:01:00.0/rom<br />
# cat /sys/bus/pci/devices/0000:01:00.0/rom > /tmp/image.rom<br />
# echo 0 > /sys/bus/pci/devices/0000:01:00.0/rom<br />
<br />
And test it for compatibility:<br />
<br />
{{hc|$ ./rom-parser /tmp/image.rom|<br />
Valid ROM signature found @600h, PCIR offset 190h<br />
PCIR: type 0 (x86 PC-AT), vendor: 10de, device: 1184, class: 030000<br />
PCIR: revision 0, vendor revision: 1<br />
Valid ROM signature found @fa00h, PCIR offset 1ch<br />
PCIR: type 3 (EFI), vendor: 10de, device: 1184, class: 030000<br />
PCIR: revision 3, vendor revision: 0<br />
EFI: Signature Valid, Subsystem: Boot, Machine: X64<br />
Last image<br />
}}<br />
<br />
To be UEFI compatible, you need a "type 3 (EFI)" in the result. If it's not there, try updating your GPU VBIOS. GPU manufacturers often share VBIOS upgrades on their support pages. A large database of known compatible and working VBIOSes (along with their UEFI compatibility status!) is available on [https://www.techpowerup.com/vgabios/ TechPowerUp].<br />
<br />
Updated VBIOS can be used in the VM without flashing. To load it in QEMU:<br />
<br />
-device vfio-pci,host=07:00.0,......,romfile=/path/to/your/gpu/bios.bin \<br />
<br />
And in libvirt:<br />
<br />
{{bc|1=<br />
<hostdev><br />
...<br />
<rom file='/path/to/your/gpu/bios.bin'/><br />
...<br />
</hostdev><br />
}}<br />
<br />
One should compare VBIOS versions between host and guest systems using [https://www.techpowerup.com/download/nvidia-nvflash/ nvflash] (Linux versions under ''Show more versions'') or <br />
[https://www.techpowerup.com/download/techpowerup-gpu-z/ GPU-Z] (in Windows guest). To check the currently loaded VBIOS:<br />
<br />
{{hc|$ ./nvflash --version|<br />
...<br />
Version : 80.04.XX.00.97<br />
...<br />
UEFI Support : No<br />
UEFI Version : N/A<br />
UEFI Variant Id : N/A ( Unknown )<br />
UEFI Signer(s) : Unsigned<br />
...<br />
}}<br />
<br />
And to check a given VBIOS file:<br />
<br />
{{hc|$ ./nvflash --version NV299MH.rom|<br />
...<br />
Version : 80.04.XX.00.95<br />
...<br />
UEFI Support : Yes<br />
UEFI Version : 0x10022 (Jul 2 2013 @ 16377903 )<br />
UEFI Variant Id : 0x0000000000000004 ( GK1xx )<br />
UEFI Signer(s) : Microsoft Corporation UEFI CA 2011<br />
...<br />
}}<br />
<br />
If the external ROM did not work as it should in the guest, you will have to flash the newer VBIOS image to the GPU. In some cases it is possible to create your own VBIOS image with UEFI support using [https://www.win-raid.com/t892f16-AMD-and-Nvidia-GOP-update-No-requests-DIY.html GOPUpd] tool, however this is risky and may result in GPU brick.<br />
<br />
{{Warning|Failure during flashing may "brick" your GPU - recovery may be possible, but rarely easy and often requires additional hardware. '''DO NOT''' flash VBIOS images for other GPU models (different boards may use different VBIOSes, clocks, fan configuration). If it breaks, you get to keep all the pieces.}}<br />
<br />
In order to avoid the irreparable damage to your graphics adapter it is necessary to unload the NVIDIA kernel driver first:<br />
<br />
# modprobe -r nvidia_modeset nvidia <br />
<br />
Flashing the VBIOS can be done with:<br />
<br />
# ./nvflash romfile.bin<br />
<br />
{{Warning|'''DO NOT''' interrupt the flashing process, even if it looks like it's stuck. Flashing should take about a minute on most GPUs, but may take longer.}}<br />
<br />
=== Slowed down audio pumped through HDMI on the video card ===<br />
<br />
For some users VM's audio slows down/starts stuttering/becomes demonic after a while when it's pumped through HDMI on the video card. This usually also slows down graphics.<br />
A possible solution consists of enabling MSI (Message Signaled-Based Interrupts) instead of the default (Line-Based Interrupts).<br />
<br />
In order to check whether MSI is supported or enabled, run the following command as root:<br />
<br />
# lspci -vs $device | grep 'MSI:'<br />
<br />
where `$device` is the card's address (e.g. `01:00.0`).<br />
<br />
The output should be similar to:<br />
<br />
Capabilities: [60] MSI: Enable'''-''' Count=1/1 Maskable- 64bit+<br />
<br />
A {{ic|-}} after {{ic|Enable}} means MSI is supported, but not used by the VM, while a {{ic|+}} says that the VM is using it.<br />
<br />
The procedure to enable it is quite complex, instructions and an overview of the setting can be found [https://forums.guru3d.com/showthread.php?t=378044 here].<br />
<br />
Other hints can be found on the [https://lime-technology.com/wiki/index.php/UnRAID_6/VM_Guest_Support#Enable_MSI_for_Interrupts_to_Fix_HDMI_Audio_Support lime-technology's wiki], or on this article on [https://vfio.blogspot.it/2014/09/vfio-interrupts-and-how-to-coax-windows.html VFIO tips and tricks].<br />
<br />
A UI tool called [https://github.com/CHEF-KOCH/MSI-utility MSI Utility (FOSS Version 2)] works with Windows 10 64-bit and simplifies the process.<br />
<br />
In order to fix the issues enabling MSI on the 0 function of a nVidia card ({{ic|01:00.0 VGA compatible controller: NVIDIA Corporation GM206 [GeForce GTX 960] (rev a1) (prog-if 00 [VGA controller])}}) was not enough; it will also be required to enable it on the other function ({{ic|01:00.1 Audio device: NVIDIA Corporation Device 0fba (rev a1)}}) to fix the issue.<br />
<br />
=== No HDMI audio output on host when intel_iommu is enabled ===<br />
<br />
If after enabling {{ic|intel_iommu}} the HDMI output device of Intel GPU becomes unusable on the host then setting the option {{ic|igfx_off}} (i.e. {{ic|1=intel_iommu=on,igfx_off}}) might bring the audio back, please read {{ic|Graphics Problems?}} in [https://www.kernel.org/doc/Documentation/Intel-IOMMU.txt Intel-IOMMU.txt] for details about setting {{ic|igfx_off}}.<br />
<br />
=== X does not start after enabling vfio_pci ===<br />
<br />
This is related to the host GPU being detected as a secondary GPU, which causes X to fail/crash when it tries to load a driver for the guest GPU. To circumvent this, a Xorg configuration file specifying the BusID for the host GPU is required. The correct BusID can be acquired from lspci or the Xorg log. [https://www.redhat.com/archives/vfio-users/2016-August/msg00025.html Source →]<br />
<br />
{{hc|/etc/X11/xorg.conf.d/10-intel.conf|<nowiki><br />
Section "Device"<br />
Identifier "Intel GPU"<br />
Driver "modesetting"<br />
BusID "PCI:0:2:0"<br />
EndSection<br />
</nowiki>}}<br />
<br />
=== Chromium ignores integrated graphics for acceleration ===<br />
<br />
Chromium and friends will try to detect as many GPUs as they can in the system and pick which one is preferred (usually discrete NVIDIA/AMD graphics). It tries to pick a GPU by looking at PCI devices, not OpenGL renderers available in the system - the result is that Chromium may ignore the integrated GPU available for rendering and try to use the dedicated GPU bound to the {{ic|vfio-pci}} driver, and unusable on the host system, regardless of whenever a guest VM is running or not. This results in software rendering being used (leading to higher CPU load, which may also result in choppy video playback, scrolling and general un-smoothness).<br />
<br />
This can be fixed by [[Chromium/Tips and tricks#Forcing specific GPU|explicitly telling Chromium which GPU you want to use]].<br />
<br />
=== VM only uses one core ===<br />
<br />
For some users, even if IOMMU is enabled and the core count is set to more than 1, the VM still only uses one CPU core and thread. To solve this enable "Manually set CPU topology" in {{ic|virt-manager}} and set it to the desirable amount of CPU sockets, cores and threads. Keep in mind that "Threads" refers to the thread count per CPU, not the total count.<br />
<br />
=== Passthrough seems to work but no output is displayed ===<br />
<br />
Make sure if you are using virt-manager that UEFI firmware is selected for your virtual machine. Also, make sure you have passed the correct device to the VM.<br />
<br />
=== virt-manager has permission issues ===<br />
<br />
If you are getting a permission error with virt-manager add the following to your {{ic|/etc/libvirt/qemu.conf}}:<br />
<br />
group="kvm"<br />
user="''user''"<br />
<br />
If that does not work make sure your user account is added to the kvm and libvirt [[groups]].<br />
<br />
=== Host Lockup After VM Shutdown ===<br />
<br />
This issue seems to primarily affect users running a Windows 10 guest and usually after the VM has been run for a prolonged period of time: the host will experience multiple CPU core lockups (see [https://bbs.archlinux.org/viewtopic.php?id=206050&p=2]). To fix this try enabling Message Signal Interrupts on the GPU passed through to the guest. A good guide for how to do this can be found in [https://forums.guru3d.com/threads/windows-line-based-vs-message-signaled-based-interrupts.378044/].<br />
<br />
=== Host lockup if guest is left running during sleep ===<br />
<br />
VFIO-enabled virtual machines tend to become unstable if left running through a sleep/wakeup cycle and have been known to cause the host machine to lockup when an attempt is then made to shut them down. In order to avoid this, one can simply prevent the host from going into sleep while the guest is running using the following libvirt hook script and systemd unit. The hook file needs executable permissions to work.<br />
<br />
{{hc|/etc/libvirt/hooks/qemu|<nowiki><br />
#!/bin/bash<br />
<br />
OBJECT="$1"<br />
OPERATION="$2"<br />
SUBOPERATION="$3"<br />
EXTRA_ARG="$4"<br />
<br />
case "$OPERATION" in<br />
"prepare")<br />
systemctl start libvirt-nosleep@"$OBJECT"<br />
;;<br />
"release")<br />
systemctl stop libvirt-nosleep@"$OBJECT"<br />
;;<br />
esac<br />
</nowiki>}}<br />
<br />
{{hc|/etc/systemd/system/libvirt-nosleep@.service|<nowiki><br />
[Unit]<br />
Description=Preventing sleep while libvirt domain "%i" is running<br />
<br />
[Service]<br />
Type=simple<br />
ExecStart=/usr/bin/systemd-inhibit --what=sleep --why="Libvirt domain \"%i\" is running" --who=%U --mode=block sleep infinity<br />
</nowiki>}}<br />
<br />
== See also ==<br />
<br />
* [https://bbs.archlinux.org/viewtopic.php?id=162768 Discussion on Arch Linux forums] | [https://archive.is/kZYMt Archived link]<br />
* [https://docs.google.com/spreadsheet/ccc?key=0Aryg5nO-kBebdFozaW9tUWdVd2VHM0lvck95TUlpMlE User contributed hardware compatibility list]<br />
* [https://pastebin.com/rcnUZCv7 Example script from https://www.youtube.com/watch?v=37D2bRsthfI]<br />
* [https://vfio.blogspot.com/ Complete tutorial for PCI passthrough]<br />
* [https://www.redhat.com/archives/vfio-users/ VFIO users mailing list]<br />
* [https://webchat.freenode.net/?channels=vfio-users #vfio-users on freenode]<br />
* [https://www.youtube.com/watch?v=aLeWg11ZBn0 YouTube: Level1Linux - GPU Passthrough for Virtualization with Ryzen]</div>Nicman23https://wiki.archlinux.org/index.php?title=PCI_passthrough_via_OVMF&diff=538504PCI passthrough via OVMF2018-08-29T11:00:35Z<p>Nicman23: Better way - previous example did not work for my machine</p>
<hr />
<div>[[Category:Virtualization]]<br />
[[ja:OVMF による PCI パススルー]]<br />
[[zh-hans:PCI passthrough via OVMF]]<br />
The Open Virtual Machine Firmware ([https://github.com/tianocore/tianocore.github.io/wiki/OVMF OVMF]) is a project to enable UEFI support for virtual machines. Starting with Linux 3.9 and recent versions of [[QEMU]], it is now possible to passthrough a graphics card, offering the VM native graphics performance which is useful for graphic-intensive tasks.<br />
<br />
Provided you have a desktop computer with a spare GPU you can dedicate to the host (be it an integrated GPU or an old OEM card, the brands do not even need to match) and that your hardware supports it (see [[#Prerequisites]]), it is possible to have a VM of any OS with its own dedicated GPU and near-native performance. For more information on techniques see the background [https://www.linux-kvm.org/images/b/b3/01x09b-VFIOandYou-small.pdf presentation (pdf)]. <br />
<br />
== Prerequisites ==<br />
<br />
A VGA Passthrough relies on a number of technologies that are not ubiquitous as of today and might not be available on your hardware. You will not be able to do this on your machine unless the following requirements are met :<br />
<br />
* Your CPU must support hardware virtualization (for kvm) and IOMMU (for the passthrough itself)<br />
** [https://ark.intel.com/Search/FeatureFilter?productType=processors&VTD=true List of compatible Intel CPUs (Intel VT-x and Intel VT-d)]<br />
** All AMD CPUs from the Bulldozer generation and up (including Zen) should be compatible.<br />
*** CPUs from the K10 generation (2007) do not have an IOMMU, so you '''need''' to have a motherboard with a [https://support.amd.com/TechDocs/43403.pdf#page=18 890FX] or [https://support.amd.com/TechDocs/48691.pdf#page=21 990FX] chipset to make it work, as those have their own IOMMU.<br />
* Your motherboard must also support IOMMU<br />
** Both the chipset and the BIOS must support it. It is not always easy to tell at a glance whether or not this is the case, but there is a fairly comprehensive list on the matter on the [https://wiki.xen.org/wiki/VTd_HowTo Xen wiki] as well as [[Wikipedia:List of IOMMU-supporting hardware]].<br />
* Your guest GPU ROM must support UEFI.<br />
** If you can find [https://www.techpowerup.com/vgabios/ any ROM in this list] that applies to your specific GPU and is said to support UEFI, you are generally in the clear. All GPUs from 2012 and later should support this, as Microsoft made UEFI a requirement for devices to be marketed as compatible with Windows 8.<br />
<br />
You will probably want to have a spare monitor or one with multiple input ports connected to different GPUs (the passthrough GPU will not display anything if there is no screen plugged in and using a VNC or Spice connection will not help your performance), as well as a mouse and a keyboard you can pass to your VM. If anything goes wrong, you will at least have a way to control your host machine this way.<br />
<br />
== Setting up IOMMU ==<br />
<br />
{{Note|<br />
* IOMMU is a generic name for Intel VT-d and AMD-Vi.<br />
* VT-d stands for ''Intel Virtualization Technology for Directed I/O'' and should not be confused with VT-x ''Intel Virtualization Technology''. VT-x allows one hardware platform to function as multiple “virtual” platforms while VT-d improves security and reliability of the systems and also improves performance of I/O devices in virtualized environments.}}<br />
<br />
Using IOMMU opens to features like PCI passthrough and memory protection from faulty or malicious devices, see [[wikipedia:Input-output memory management unit#Advantages]] and [https://www.quora.com/Memory-Management-computer-programming/Could-you-explain-IOMMU-in-plain-English Memory Management (computer programming): Could you explain IOMMU in plain English?].<br />
<br />
=== Enabling IOMMU ===<br />
<br />
Ensure that AMD-Vi/Intel VT-d is supported by the CPU and enabled in the BIOS settings. Both normally show up alongside other CPU features (meaning they could be in an overclocking-related menu) either with their actual names ("VT-d" or "AMD-Vi") or in more ambiguous terms such as "Virtualization technology", which may or may not be explained in the manual.<br />
<br />
Enable IOMMU support by setting the correct [[kernel parameter]] depending on the type of CPU in use:<br />
* For Intel CPUs (VT-d) set {{ic|1=intel_iommu=on}} <br />
* For AMD CPUs (AMD-Vi) set {{ic|1=amd_iommu=on}}<br />
<br />
You should also append the {{ic|1=iommu=pt}} parameter. This will prevent Linux from touching devices which cannot be passed through.<br />
<br />
After rebooting, check dmesg to confirm that IOMMU has been correctly enabled:<br />
<br />
{{hc|dmesg {{!}} grep -e DMAR -e IOMMU|<br />
[ 0.000000] ACPI: DMAR 0x00000000BDCB1CB0 0000B8 (v01 INTEL BDW 00000001 INTL 00000001)<br />
[ 0.000000] Intel-IOMMU: enabled<br />
[ 0.028879] dmar: IOMMU 0: reg_base_addr fed90000 ver 1:0 cap c0000020660462 ecap f0101a<br />
[ 0.028883] dmar: IOMMU 1: reg_base_addr fed91000 ver 1:0 cap d2008c20660462 ecap f010da<br />
[ 0.028950] IOAPIC id 8 under DRHD base 0xfed91000 IOMMU 1<br />
[ 0.536212] DMAR: No ATSR found<br />
[ 0.536229] IOMMU 0 0xfed90000: using Queued invalidation<br />
[ 0.536230] IOMMU 1 0xfed91000: using Queued invalidation<br />
[ 0.536231] IOMMU: Setting RMRR:<br />
[ 0.536241] IOMMU: Setting identity map for device 0000:00:02.0 [0xbf000000 - 0xcf1fffff]<br />
[ 0.537490] IOMMU: Setting identity map for device 0000:00:14.0 [0xbdea8000 - 0xbdeb6fff]<br />
[ 0.537512] IOMMU: Setting identity map for device 0000:00:1a.0 [0xbdea8000 - 0xbdeb6fff]<br />
[ 0.537530] IOMMU: Setting identity map for device 0000:00:1d.0 [0xbdea8000 - 0xbdeb6fff]<br />
[ 0.537543] IOMMU: Prepare 0-16MiB unity mapping for LPC<br />
[ 0.537549] IOMMU: Setting identity map for device 0000:00:1f.0 [0x0 - 0xffffff]<br />
[ 2.182790] [drm] DMAR active, disabling use of stolen memory<br />
}}<br />
<br />
=== Ensuring that the groups are valid ===<br />
<br />
The following script should allow you to see how your various PCI devices are mapped to IOMMU groups. If it does not return anything, you either have not enabled IOMMU support properly or your hardware does not support it.<br />
<br />
#!/bin/bash<br />
shopt -s nullglob<br />
for d in /sys/kernel/iommu_groups/*/devices/*; do <br />
n=${d#*/iommu_groups/*}; n=${n%%/*}<br />
printf 'IOMMU Group %s ' "$n"<br />
lspci -nns "${d##*/}"<br />
done;<br />
<br />
Example output:<br />
<br />
IOMMU Group 0 00:00.0 Host bridge [0600]: Intel Corporation 2nd Generation Core Processor Family DRAM Controller [8086:0104] (rev 09)<br />
IOMMU Group 1 00:16.0 Communication controller [0780]: Intel Corporation 6 Series/C200 Series Chipset Family MEI Controller #1 [8086:1c3a] (rev 04)<br />
IOMMU Group 2 00:19.0 Ethernet controller [0200]: Intel Corporation 82579LM Gigabit Network Connection [8086:1502] (rev 04)<br />
IOMMU Group 3 00:1a.0 USB controller [0c03]: Intel Corporation 6 Series/C200 Series Chipset Family USB Enhanced Host Controller #2 [8086:1c2d] (rev <br />
...<br />
<br />
An IOMMU group is the smallest set of physical devices that can be passed to a virtual machine. For instance, in the example above, both the GPU in 06:00.0 and its audio controller in 6:00.1 belong to IOMMU group 13 and can only be passed together. The frontal USB controller, however, has its own group (group 2) which is separate from both the USB expansion controller (group 10) and the rear USB controller (group 4), meaning that [[#USB controller|any of them could be passed to a VM without affecting the others]].<br />
<br />
=== Gotchas ===<br />
<br />
==== Plugging your guest GPU in an unisolated CPU-based PCIe slot ====<br />
<br />
Not all PCI-E slots are the same. Most motherboards have PCIe slots provided by both the CPU and the PCH. Depending on your CPU, it is possible that your processor-based PCIe slot does not support isolation properly, in which case the PCI slot itself will be appear to be grouped with the device that is connected to it.<br />
<br />
IOMMU Group 1 00:01.0 PCI bridge: Intel Corporation Xeon E3-1200 v2/3rd Gen Core processor PCI Express Root Port (rev 09)<br />
IOMMU Group 1 01:00.0 VGA compatible controller: NVIDIA Corporation GM107 [GeForce GTX 750] (rev a2)<br />
IOMMU Group 1 01:00.1 Audio device: NVIDIA Corporation Device 0fbc (rev a1)<br />
<br />
This is fine so long as only your guest GPU is included in here, such as above. Depending on what is plugged in your other PCIe slots and whether they are allocated to your CPU or your PCH, you may find yourself with additional devices within the same group, which would force you to pass those as well. If you are ok with passing everything that is in there to your VM, you are free to continue. Otherwise, you will either need to try and plug your GPU in your other PCIe slots (if you have any) and see if those provide isolation from the rest or to install the ACS override patch, which comes with its own drawbacks. See [[#Bypassing the IOMMU groups (ACS override patch)]] for more information.<br />
<br />
{{Note|If they are grouped with other devices in this manner, pci root ports and bridges should neither be bound to vfio at boot, nor be added to the VM.}}<br />
<br />
== Isolating the GPU ==<br />
<br />
In order to assign a device to a virtual machine, this device and all those sharing the same IOMMU group must have their driver replaced by a stub driver or a VFIO driver in order to prevent the host machine from interacting with them. In the case of most devices, this can be done on the fly right before the VM starts.<br />
<br />
However, due to their size and complexity, GPU drivers do not tend to support dynamic rebinding very well, so you cannot just have some GPU you use on the host be transparently passed to a VM without having both drivers conflict with each other. Because of this, it is generally advised to bind those placeholder drivers manually before starting the VM, in order to stop other drivers from attempting to claim it.<br />
<br />
The following section details how to configure a GPU so those placeholder drivers are bound early during the boot process, which makes said device inactive until a VM claims it or the driver is switched back. This is the preferred method, considering it has less caveats than switching drivers once the system is fully online.<br />
<br />
{{warning|Once you reboot after this procedure, whatever GPU you have configured will no longer be usable on the host until you reverse the manipulation. Make sure the GPU you intend to use on the host is properly configured before doing this - your motherboard should be set to display using the host GPU.}}<br />
<br />
Starting with Linux 4.1, the kernel includes vfio-pci. This is a VFIO driver, meaning it fulfills the same role as pci-stub did, but it can also control devices to an extent, such as by switching them into their D3 state when they are not in use.<br />
<br />
{{hc|$ modinfo vfio-pci|<br />
filename: /lib/modules/4.4.5-1-ARCH/kernel/drivers/vfio/pci/vfio-pci.ko.gz<br />
description: VFIO PCI - User Level meta-driver<br />
author: Alex Williamson <alex.williamson@redhat.com><br />
...<br />
}}<br />
<br />
Vfio-pci normally targets PCI devices by ID, meaning you only need to specify the IDs of the devices you intend to passthrough. For the following IOMMU group, you would want to bind vfio-pci with {{ic|10de:13c2}} and {{ic|10de:0fbb}}, which will be used as example values for the rest of this section.<br />
<br />
IOMMU Group 13 06:00.0 VGA compatible controller: NVIDIA Corporation GM204 [GeForce GTX 970] [10de:13c2] (rev a1)<br />
IOMMU Group 13 06:00.1 Audio device: NVIDIA Corporation GM204 High Definition Audio Controller [10de:0fbb] (rev a1)}}<br />
<br />
{{Note|You cannot specify which device to isolate using vendor-device ID pairs if the host GPU and the guest GPU share the same pair (i.e : if both are the same model). If this is your case, read [[#Using identical guest and host GPUs]] instead.}}<br />
<br />
You can then add those vendor-device ID pairs to the default parameters passed to vfio-pci whenever it is inserted into the kernel.<br />
<br />
{{hc|/etc/modprobe.d/vfio.conf|2=<br />
options vfio-pci ids=10de:13c2,10de:0fbb<br />
}}<br />
<br />
{{Note|If, as noted in [[#Plugging your guest GPU in an unisolated CPU-based PCIe slot]], your pci root port is part of your IOMMU group, you '''must not''' pass its ID to {{ic|vfio-pci}}, as it needs to remain attached to the host to function properly. Any other device within that group, however, should be left for {{ic|vfio-pci}} to bind with.}}<br />
<br />
This, however, does not guarantee that vfio-pci will be loaded before other graphics drivers. To ensure that, we need to statically bind it in the kernel image alongside with its dependencies. That means adding, in this order, {{ic|vfio_pci}}, {{ic|vfio}}, {{ic|vfio_iommu_type1}}, and {{ic|vfio_virqfd}} to [[mkinitcpio]]:<br />
<br />
{{hc|/etc/mkinitcpio.conf|2=<br />
MODULES=(... vfio_pci vfio vfio_iommu_type1 vfio_virqfd ...)<br />
}}<br />
<br />
{{Note|If you also have another driver loaded this way for [[Kernel mode setting#Early KMS start|early modesetting]] (such as {{ic|nouveau}}, {{ic|radeon}}, {{ic|amdgpu}}, {{ic|i915}}, etc.), all of the aforementioned VFIO modules must precede it.}}<br />
<br />
Also, ensure that the modconf hook is included in the HOOKS list of {{ic|mkinitcpio.conf}}:<br />
<br />
{{hc|/etc/mkinitcpio.conf|2=<br />
HOOKS=(... modconf ...)<br />
}}<br />
<br />
Since new modules have been added to the initramfs configuration, you must [[regenerate the initramfs]]. Should you change the IDs of the devices in {{ic|/etc/modprobe.d/vfio.conf}}, you will also have to regenerate it, as those parameters must be specified in the initramfs to be known during the early boot stages.<br />
<br />
Reboot and verify that vfio-pci has loaded properly and that it is now bound to the right devices.<br />
<br />
{{hc|$ dmesg {{!}} grep -i vfio|<br />
[ 0.329224] VFIO - User Level meta-driver version: 0.3<br />
[ 0.341372] vfio_pci: add [10de:13c2[ffff:ffff]] class 0x000000/00000000<br />
[ 0.354704] vfio_pci: add [10de:0fbb[ffff:ffff]] class 0x000000/00000000<br />
[ 2.061326] vfio-pci 0000:06:00.0: enabling device (0100 -> 0103)<br />
}}<br />
<br />
It is not necessary for all devices (or even expected device) from {{ic|vfio.conf}} to be in dmesg output. Sometimes a device does not appear in output at boot but actually is able to be visible and operatable in guest VM.<br />
<br />
{{hc|$ lspci -nnk -d 10de:13c2|<br />
06:00.0 VGA compatible controller: NVIDIA Corporation GM204 [GeForce GTX 970] [10de:13c2] (rev a1)<br />
Kernel driver in use: vfio-pci<br />
Kernel modules: nouveau nvidia<br />
}}<br />
<br />
{{hc|$ lspci -nnk -d 10de:0fbb|<br />
06:00.1 Audio device: NVIDIA Corporation GM204 High Definition Audio Controller [10de:0fbb] (rev a1)<br />
Kernel driver in use: vfio-pci<br />
Kernel modules: snd_hda_intel<br />
}}<br />
<br />
== Setting up an OVMF-based guest VM ==<br />
<br />
OVMF is an open-source UEFI firmware for QEMU virtual machines. While it's possible to use SeaBIOS to get similar results to an actual PCI passthough, the setup process is different and it is generally preferable to use the EFI method if your hardware supports it.<br />
<br />
=== Configuring libvirt ===<br />
<br />
[[Libvirt]] is a wrapper for a number of virtualization utilities that greatly simplifies the configuration and deployment process of virtual machines. In the case of KVM and QEMU, the frontend it provides allows us to avoid dealing with the permissions for QEMU and make it easier to add and remove various devices on a live VM. Its status as a wrapper, however, means that it might not always support all of the latest qemu features, which could end up requiring the use of a wrapper script to provide some extra arguments to QEMU.<br />
<br />
After installing {{Pkg|qemu}}, {{Pkg|libvirt}}, {{Pkg|ovmf}}, and {{Pkg|virt-manager}}, add the path to your OVMF firmware image and runtime variables template to your libvirt config so {{ic|virt-install}} or {{ic|virt-manager}} can find those later on.<br />
<br />
{{hc|/etc/libvirt/qemu.conf|2=<br />
nvram = [<br />
"/usr/share/ovmf/x64/OVMF_CODE.fd:/usr/share/ovmf/x64/OVMF_VARS.fd"<br />
]<br />
}}<br />
<br />
You can now [[enable]] and [[start]] {{ic|libvirtd.service}} and its logging component {{ic|virtlogd.socket}}.<br />
<br />
=== Setting up the guest OS ===<br />
<br />
The process of setting up a VM using {{ic|virt-manager}} is mostly self-explanatory, as most of the process comes with fairly comprehensive on-screen instructions.<br />
<br />
If using {{ic|virt-manager}}, you have to add your user to the libvirt [[group]] to ensure authentication.<br />
<br />
However, you should pay special attention to the following steps :<br />
<br />
* When the VM creation wizard asks you to name your VM (final step before clicking "Finish"), check the "Customize before install" checkbox.<br />
* In the "Overview" section, [https://i.imgur.com/73r2ctM.png set your firmware to "UEFI"]. If the option is grayed out, make sure that you have correctly specified the location of your firmware in {{ic|/etc/libvirt/qemu.conf}} and restart {{ic|libvirtd.service}}.<br />
* In the "CPUs" section, change your CPU model to "host-passthrough". If it is not in the list, you will have to type it by hand. This will ensure that your CPU is detected properly, since it causes libvirt to expose your CPU capabilities exactly as they are instead of only those it recognizes (which is the preferred default behavior to make CPU behavior easier to reproduce). Without it, some applications may complain about your CPU being of an unknown model.<br />
* If you want to minimize IO overhead, go into "Add Hardware" and add a Controller for SCSI drives of the "VirtIO SCSI" model. You can then change the default IDE disk for a SCSI disk, which will bind to said controller.<br />
** Windows VMs will not recognize those drives by default, so you need to download the ISO containing the drivers from [https://fedorapeople.org/groups/virt/virtio-win/direct-downloads/latest-virtio/ here] and add an IDE (or SATA for Windows 8.1 and newer) CD-ROM storage device linking to said ISO, otherwise you will not be able to get Windows to recognize it during the installation process. When prompted to select a disk to install windows on, load the drivers contained on the CD-ROM under ''vioscsi''.<br />
<br />
The rest of the installation process will take place as normal using a standard QXL video adapter running in a window. At this point, there is no need to install additional drivers for the rest of the virtual devices, since most of them will be removed later on. Once the guest OS is done installing, simply turn off the virtual machine. It is possible you will be dropped into the UEFI menu instead of starting the installation upon powering your VM for the first time. Sometimes the correct ISO file was not automatically detected and you will need to manually specify the drive to boot. By typing exit and navigating to "boot manager" you will enter a menu that allows you to choose between devices.<br />
<br />
=== Attaching the PCI devices ===<br />
<br />
With the installation done, it's now possible to edit the hardware details in libvirt and remove virtual integration devices, such as the spice channel and virtual display, the QXL video adapter, the emulated mouse and keyboard and the USB tablet device. Since that leaves you with no input devices, you may want to bind a few USB host devices to your VM as well, but remember to '''leave at least one mouse and/or keyboard assigned to your host''' in case something goes wrong with the guest. At this point, it also becomes possible to attach the PCI device that was isolated earlier; simply click on "Add Hardware" and select the PCI Host Devices you want to passthrough. If everything went well, the screen plugged into your GPU should show the OVMF splash screen and your VM should start up normally. From there, you can setup the drivers for the rest of your VM.<br />
<br />
=== Gotchas ===<br />
<br />
==== Using a non-EFI image on an OVMF-based VM ====<br />
<br />
The OVMF firmware does not support booting off non-EFI mediums. If the installation process drops you in a UEFI shell right after booting, you may have an invalid EFI boot media. Try using an alternate linux/windows image to determine if you have an invalid media.<br />
<br />
== Performance tuning ==<br />
<br />
Most use cases for PCI passthroughs relate to performance-intensive domains such as video games and GPU-accelerated tasks. While a PCI passthrough on its own is a step towards reaching native performance, there are still a few ajustments on the host and guest to get the most out of your VM.<br />
<br />
=== CPU pinning ===<br />
<br />
The default behavior for KVM guests is to run operations coming from the guest as a number of threads representing virtual processors. Those threads are managed by the Linux scheduler like any other thread and are dispatched to any available CPU cores based on niceness and priority queues. Since switching between threads adds a bit of overhead (because context switching forces the core to change its cache between operations), this can noticeably harm performance on the guest. CPU pinning aims to resolve this as it overrides process scheduling and ensures that the VM threads will always run and only run on those specific cores. Here, for instance, the guest cores 0, 1, 2 and 3 are mapped to the host cores 4, 5, 6 and 7 respectively.<br />
<br />
{{Note|For certain users enabling CPU pinning may introduce stuttering and short hangs, especially with the MuQSS scheduler (present in linux-ck and linux-zen kernels). You might want to try disabling pinning first if you experience similar issues, which effectively trades maximum performance for responsiveness at all times.}}<br />
<br />
{{hc|$ virsh edit [vmname]|<nowiki><br />
...<br />
<vcpu placement='static'>4</vcpu><br />
<cputune><br />
<vcpupin vcpu='0' cpuset='4'/><br />
<vcpupin vcpu='1' cpuset='5'/><br />
<vcpupin vcpu='2' cpuset='6'/><br />
<vcpupin vcpu='3' cpuset='7'/><br />
</cputune><br />
...<br />
</nowiki>}}<br />
<br />
==== The case of Hyper-threading ====<br />
<br />
If your CPU supports hardware multitasking, also known as Hyper-threading on Intel chips, there are two ways you can go with your CPU pinning. That is, Hyper-threading is simply a very efficient way of running two threads on one CPU at any given time, so while it may give you 8 logical cores on what would otherwise be a quad-core CPU, if the physical core is overloaded, the logical core will not be of any use. One could pin their VM threads on 2 physical cores and their 2 respective threads, but any task overloading those two cores will not be helped by the extra two logical cores, since in the end you are only passing through two cores out of four, not four out of eight. What you should do knowing this depends on what you intend to do with your host while your VM is running.<br />
<br />
This is the abridged content of {{ic|/proc/cpuinfo}} on a quad-core machine with hyper-threading.<br />
<br />
{{hc|$ grep -e "processor" -e "core id" -e "^$" /proc/cpuinfo|<br />
processor : 0<br />
core id : 0<br />
<br />
processor : 1<br />
core id : 1<br />
<br />
processor : 2<br />
core id : 2<br />
<br />
processor : 3<br />
core id : 3<br />
<br />
processor : 4<br />
core id : 0<br />
<br />
processor : 5<br />
core id : 1<br />
<br />
processor : 6<br />
core id : 2<br />
<br />
processor : 7<br />
core id : 3<br />
}}<br />
<br />
If you do not intend to be doing any computation-heavy work on the host (or even anything at all) at the same time as you would on the VM, it would probably be better to pin your VM threads across all of your logical cores, so that the VM can fully take advantage of the spare CPU time on all your cores.<br />
<br />
On the quad-core machine mentioned above, it would look like this :<br />
<br />
{{hc|$ virsh edit [vmname]|<nowiki><br />
...<br />
<vcpu placement='static'>4</vcpu><br />
<cputune><br />
<vcpupin vcpu='0' cpuset='4'/><br />
<vcpupin vcpu='1' cpuset='5'/><br />
<vcpupin vcpu='2' cpuset='6'/><br />
<vcpupin vcpu='3' cpuset='7'/><br />
</cputune><br />
...<br />
<cpu mode='custom' match='exact'><br />
...<br />
<topology sockets='1' cores='4' threads='1'/><br />
...<br />
</cpu><br />
...<br />
</nowiki>}}<br />
<br />
If you would instead prefer to have the host and guest running intensive tasks at the same time, it would then be preferable to pin a limited amount of physical cores and their respective threads on the guest and leave the rest to the host to avoid the two competing for CPU time.<br />
<br />
On the quad-core machine mentioned above, it would look like this :<br />
<br />
{{hc|$ virsh edit [vmname]|<nowiki><br />
...<br />
<vcpu placement='static'>4</vcpu><br />
<cputune><br />
<vcpupin vcpu='0' cpuset='2'/><br />
<vcpupin vcpu='1' cpuset='6'/><br />
<vcpupin vcpu='2' cpuset='3'/><br />
<vcpupin vcpu='3' cpuset='7'/><br />
</cputune><br />
...<br />
<cpu mode='custom' match='exact'><br />
...<br />
<topology sockets='1' cores='2' threads='2'/><br />
...<br />
</cpu><br />
...<br />
</nowiki>}}<br />
<br />
=== Static huge pages ===<br />
<br />
When dealing with applications that require large amounts of memory, memory latency can become a problem since the more memory pages are being used, the more likely it is that this application will attempt to access information across multiple memory "pages", which is the base unit for memory allocation. Resolving the actual address of the memory page takes multiple steps, and so CPUs normally cache information on recently used memory pages to make subsequent uses on the same pages faster. Applications using large amounts of memory run into a problem where, for instance, a virtual machine uses 4GB of memory divided into 4kB pages (which is the default size for normal pages), meaning that such cache misses can become extremely frequent and greatly increase memory latency. Huge pages exist to mitigate this issue by giving larger individual pages to those applications, increasing the odds that multiple operations will target the same page in succession. This is normally handeled with transparent huge pages, which dynamically manages hugepages to keep up with the demand.<br />
<br />
On a VM with a PCI passthrough, however, it is '''not possible''' to benefit from transparent huge pages, as IOMMU requires that the guest's memory be allocated and pinned as soon as the VM starts. It is therefore required to allocate huge pages statically in order to benefit from them. <br />
<br />
{{Warning|Static huge pages lock down the allocated amount of memory, making it unavailable for applications that are not configured to use them. Allocating 4GBs worth of huge pages on a machine with 8GBs of memory will only leave you with 4GBs of available memory on the host '''even when the VM is not running'''.}}<br />
<br />
To allocate huge pages at boot, one must simply specify the desired amount on their kernel command line with {{ic|1=hugepages=''x''}}. For instance, reserving 1024 pages with {{ic|1=hugepages=1024}} and the default size of 2048kB per huge page creates 2GBs worth of memory for the virtual machine to use.<br />
<br />
If supported by CPU page size could be set manually. 1 GB huge page support could be verified by {{ic|<nowiki>grep pdpe1gb /proc/cpuinfo</nowiki>}}. Setting 1 GB huge page size via kernel parameters : {{ic|1=default_hugepagesz=1G hugepagesz=1G hugepages=X transparent_hugepage=never}}.<br />
<br />
Also, since static huge pages can only be used by applications that specifically request it, you must add this section in your libvirt domain configuration to allow kvm to benefit from them :<br />
<br />
{{hc|$ virsh edit [vmname]|<nowiki><br />
...<br />
<memoryBacking><br />
<hugepages/><br />
</memoryBacking><br />
...<br />
</nowiki>}}<br />
<br />
=== Dynamic huge pages ===<br />
<br />
{{Accuracy|Need futher testing if this variant as effective as static one}}<br />
<br />
Hugepages could be allocated dynamically via vm.nr_overcommit_hugepages parameter.<br />
<br />
{{hc|sysctl.d/10-kvm.conf|<nowiki><br />
vm.nr_hugepages = 0<br />
vm.nr_overcommit_hugepages = <num><br />
</nowiki>}}<br />
<br />
Where num - is the number of huge pages, which default size if 2Mb.<br />
Pages will be automatically allocated, and freed after VM stops.<br />
<br />
More manual way <br />
{{hc|manual dynamic huge pages|<nowiki><br />
echo <num> | sudo tee /sys/kernel/mm/hugepages/hugepages-2048kB/nr_hugepages<br />
echo <num> | sudo tee /sys/kernel/mm/hugepages/hugepages-1048576kB/nr_hugepages<br />
</nowiki>}}<br />
<br />
For 2Mb and 1Gb page size respectively.<br />
And they should be manually freed in the same way.<br />
<br />
It is hardly recommended to drop caches, compact memory and wait couple of seconds before starting VM, as there could be not enough free contiguous memory for required huge pages blocks. Especially after some uptime of the host system.<br />
{{hc|organize ram|<nowiki><br />
echo 3 | sudo tee /proc/sys/vm/drop_caches<br />
echo 1 | sudo tee /proc/sys/vm/compact_memory<br />
</nowiki>}}<br />
<br />
Theoretically, 1Gb pages works as 2Mb. But practically - no guaranteed way was found to get contiguous 1Gb memory blocks. Each consequent request of 1Gb blocks lead to lesser and lesser dynamically allocated count.<br />
<br />
=== CPU frequency governor ===<br />
<br />
Depending on the way your [[CPU frequency scaling|CPU governor]] is configured, the VM threads may not hit the CPU load thresholds for the frequency to ramp up. Indeed, KVM cannot actually change the CPU frequency on its own, which can be a problem if it does not scale up with vCPU usage as it would result in underwhelming performance. An easy way to see if it behaves correctly is to check if the frequency reported by {{ic|watch lscpu}} goes up when running a CPU-intensive task on the guest. If you are indeed experiencing stutter and the frequency does not go up to reach its reported maximum, it may be due to [https://lime-technology.com/forum/index.php?topic=46664.msg447678#msg447678 cpu scaling being controlled by the host OS]. In this case, try setting all cores to maximum frequency to see if this improves performance. Note that if you are using a modern intel chip with the default pstate driver, cpupower commands will be [[CPU frequency scaling#CPU frequency driver|ineffective]], so monitor {{ic|/proc/cpuinfo}} to make sure your cpu is actually at max frequency.<br />
<br />
=== High DPC Latency ===<br />
<br />
{{Accuracy|As far as I can tell all virtio modules listed here are for virtual devices used when Linux runs as a guest. Loading them on the host serves no purpose.}}<br />
<br />
If you are experiencing high DPC and/or interrupt latency in your Guest VM, ensure you have [[Kernel modules#Manual module handling|loaded the needed virtio kernel modules]] on the host kernel. Loadable virtio kernel modules include: {{ic|virtio-pci}}, {{ic|virtio-net}}, {{ic|virtio-blk}}, {{ic|virtio-balloon}}, {{ic|virtio-ring}} and {{ic|virtio}}. <br />
<br />
After loading one or more of these modules, {{ic|lsmod {{!}} grep virtio}} executed on the host should not return empty.<br />
<br />
==== CPU pinning with isolcpus ====<br />
<br />
Alternatively, make sure that you have isolated CPUs properly. In this example, let us assume you are using CPUs 4-7.<br />
Use the kernel parameters {{ic|isolcpus nohz_full rcu_nocbs}} to completely isolate the CPUs from the kernel. <br />
<br />
{{hc|sudo vim /etc/defaults/grub|<nowiki><br />
...<br />
GRUB_CMDLINE_LINUX="..your other params.. isolcpus=4-7 nohz_full=4-7 rcu_nocbs=4-7"<br />
...<br />
</nowiki>}}<br />
<br />
Then, run {{ic|qemu-system-x86_64}} with taskset and chrt:<br />
<br />
# chrt -r 1 taskset -c 4-7 qemu-system-x86_64 ...<br />
<br />
The {{ic|chrt}} command will ensure that the task scheduler will round-robin distribute work (otherwise it will all stay on the first cpu). For {{ic|taskset}}, the CPU numbers can be comma- and/or dash-separated, like "0,1,2,3" or "0-4" or "1,7-8,10" etc.<br />
<br />
See [https://www.reddit.com/r/VFIO/comments/6vgtpx/high_dpc_latency_and_audio_stuttering_on_windows/dm0sfto/ this reddit comment] for more info.<br />
<br />
=== Improving performance on AMD CPUs ===<br />
<br />
Previously, Nested Page Tables (NPT) had to be disabled on AMD systems running KVM to improve GPU performance because of a [https://sourceforge.net/p/kvm/bugs/230/ very old bug], but the trade off was decreased CPU performance, including stuttering.<br />
<br />
There is a [https://patchwork.kernel.org/patch/10027525/ kernel patch] that resolves this issue, which was accepted into kernel 4.14-stable and 4.9-stable. If you are running the official {{Pkg|linux}} or {{Pkg|linux-lts}} kernel the patch has already been applied (make sure you are on the latest). If you are running another kernel you might need to manually patch yourself.<br />
<br />
{{Note|Several Ryzen users (see [https://www.reddit.com/r/VFIO/comments/78i3jx/possible_fix_for_the_npt_issue_discussed_on_iommu/ this Reddit thread]) have tested the patch, and can confirm that it works, bringing GPU passthrough performance up to near native quality.}}<br />
<br />
=== Further Tuning ===<br />
<br />
More specialized VM tuning tips are available at [https://access.redhat.com/documentation/en-US/Red_Hat_Enterprise_Linux/7/html-single/Virtualization_Tuning_and_Optimization_Guide/index.html Red Hat's Virtualization Tuning and Optimization Guide]. This guide may be especially helpful if you are experiencing:<br />
<br />
* Moderate CPU load on the host during downloads/uploads from within the guest: See ''Bridge Zero Copy Transmit'' for a potential fix.<br />
* Guests capping out at certain network speeds during downloads/uploads despite virtio-net being used: See ''Multi-queue virtio-net'' for a potential fix.<br />
* Guests "stuttering" under high I/O, despite the same workload not affecting hosts to the same degree: See ''Multi-queue virtio-scsi'' for a potential fix.<br />
<br />
== Special procedures ==<br />
<br />
Certain setups require specific configuration tweaks in order to work properly. If you are having problems getting your host or your VM to work properly, see if your system matches one of the cases below and try adjusting your configuration accordingly.<br />
<br />
=== Using identical guest and host GPUs ===<br />
<br />
{{Expansion|A number of users have been having issues with this, it should probably be adressed by the article.|Talk:PCI passthrough via OVMF#Additionnal sections}}<br />
<br />
Due to how vfio-pci uses your vendor and device id pair to identify which device they need to bind to at boot, if you have two GPUs sharing such an ID pair you will not be able to get your passthough driver to bind with just one of them. This sort of setup makes it necessary to use a script, so that whichever driver you are using is instead assigned by pci bus address using the {{ic|driver_override}} mechanism.<br />
<br />
==== Script variants ====<br />
<br />
===== Passthrough all GPUs but the boot GPU =====<br />
<br />
Here, we will make a script to bind vfio-pci to all GPUs but the boot gpu. Create the script {{ic|/usr/bin/vfio-pci-override.sh}}:<br />
<br />
{{bc|<nowiki><br />
#!/bin/sh<br />
<br />
for i in /sys/devices/pci*/*/boot_vga; do<br />
if [ $(cat "$i") -eq 0 ]; then<br />
GPU="${i%/boot_vga}"<br />
AUDIO="$(echo "$GPU" | sed -e "s/0$/1/")"<br />
echo "vfio-pci" > "$GPU/driver_override"<br />
if [ -d "$AUDIO" ]; then<br />
echo "vfio-pci" > "$AUDIO/driver_override"<br />
fi<br />
fi<br />
done<br />
<br />
modprobe -i vfio-pci<br />
</nowiki>}}<br />
<br />
===== Passthrough selected GPU =====<br />
<br />
In this case we manually specify the GPU to bind.<br />
<br />
{{bc|<nowiki><br />
#!/bin/sh<br />
<br />
GROUP="0000:00:03.0"<br />
DEVS="0000:03:00.0 0000:03:00.1 ."<br />
<br />
if [ ! -z "$(ls -A /sys/class/iommu)" ]; then<br />
for DEV in $DEVS; do<br />
echo "vfio-pci" > /sys/bus/pci/devices/$GROUP/$DEV/driver_override<br />
done<br />
fi<br />
<br />
modprobe -i vfio-pci<br />
</nowiki>}}<br />
<br />
==== Script installation ====<br />
<br />
Create {{ic|/etc/modprobe.d/vfio.conf}} with the following:<br />
<br />
install vfio-pci /usr/bin/vfio-pci-override.sh<br />
<br />
Edit {{ic|/etc/mkinitcpio.conf}}<br />
<br />
Remove any video drivers from MODULES, and add {{ic|vfio-pci}}, and {{ic|vfio_iommu_type1}}<br />
<br />
MODULES=(ext4 vfat vfio-pci vfio_iommu_type1)<br />
<br />
Add {{ic|/etc/modprobe.d/vfio.conf}} and {{ic|/usr/bin/vfio-pci-override.sh}} to FILES:<br />
<br />
FILES=(/etc/modprobe.d/vfio.conf /usr/bin/vfio-pci-override.sh)<br />
<br />
[[Regenerate the initramfs]] and reboot:<br />
<br />
=== Passing the boot GPU to the guest ===<br />
<br />
{{Expansion|This is related to VBIOS issues and should be moved into a separate section regarding VBIOS compatibility.|section=UEFI (OVMF) Compatibility in VBIOS}}<br />
<br />
The GPU marked as {{ic|boot_vga}} is a special case when it comes to doing PCI passthroughs, since the BIOS needs to use it in order to display things like boot messages or the BIOS configuration menu. To do that, it makes [https://www.redhat.com/archives/vfio-users/2016-May/msg00224.html a copy of the VGA boot ROM which can then be freely modified]. This modified copy is the version the system gets to see, which the passthrough driver may reject as invalid. As such, it is generally recommended to change the boot GPU in the BIOS configuration so the host GPU is used instead or, if that's not possible, to swap the host and guest cards in the machine itself.<br />
<br />
=== Using Looking Glass to stream guest screen to the host ===<br />
<br />
It is possible to make VM share the monitor, and optionally a keyboard and a mouse with a help of [https://looking-glass.hostfission.com/ Looking Glass].<br />
<br />
==== Adding IVSHMEM Device to VM ====<br />
<br />
Looking glass works by creating a shared memory buffer between a host and a guest. This is a lot faster than streaming frames via localhost, but requires additional setup. <br />
<br />
With your VM turned off open the machine configuration<br />
<br />
{{hc|$ virsh edit [vmname]|<nowiki><br />
...<br />
<devices><br />
...<br />
<shmem name='looking-glass'><br />
<model type='ivshmem-plain'/><br />
<size unit='M'>32</size><br />
</shmem><br />
</devices><br />
...<br />
</nowiki>}}<br />
<br />
You should replace 32 with your own calculated value based on what resolution you are going to pass through. It can be calculated like this:<br />
<br />
width x height x 4 x 2 = total bytes<br />
total bytes / 1024 / 1024 = total megabytes + 2<br />
<br />
For example, in case of 1920x1080<br />
<br />
1920 x 1080 x 4 x 2 = 16,588,800 bytes<br />
16,588,800 / 1024 / 1024 = 15.82 MB + 2 = 17.82<br />
<br />
The result must be '''rounded up''' to the nearest power of two, and since 17.82 is bigger than 16 we should choose 32.<br />
<br />
Next create a script to create a shared memory file.<br />
<br />
{{hc|/usr/local/bin/looking-glass-init.sh|2=<br />
#!/bin/sh<br />
<br />
touch /dev/shm/looking-glass<br />
chown '''user''':kvm /dev/shm/looking-glass<br />
chmod 660 /dev/shm/looking-glass<br />
}}<br />
<br />
Replace user with your username.<br />
<br />
Remember to make the script [[executable]].<br />
<br />
Create a [[systemd]] unit to execute this script during boot<br />
<br />
{{hc|/etc/systemd/system/looking-glass-init.service|2=<br />
[Unit]<br />
Description=Create shared memory for looking glass<br />
<br />
[Service]<br />
Type=oneshot<br />
ExecStart=/usr/local/bin/looking-glass-init.sh<br />
<br />
[Install]<br />
WantedBy=multi-user.target<br />
}}<br />
<br />
[[Start]] and [[enable]] {{ic|looking-glass-init.service}}<br />
<br />
==== Installing the IVSHMEM Host to Windows guest ====<br />
<br />
Currently Windows would not notify users about a new IVSHMEM device, it would silently install a dummy driver. To actually enable the device you have to go into device manager and update the driver for the device under the "System Devices" node for '''"PCI standard RAM Controller"'''. Download a signed driver [https://github.com/virtio-win/kvm-guest-drivers-windows/issues/217 from issue 217], as it is not yet available elsewhere.<br />
<br />
Once the driver is installed you must download the latest [https://looking-glass.hostfission.com/downloads looking-glass-host] from the looking glass website and start it on your guest. In order to run it you would also need to install Microsoft Visual C++ Redistributable from [https://www.visualstudio.com/downloads/ Microsoft]<br />
It is also possible to make it start automatically on VM boot by editing the {{ic|HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Windows\CurrentVersion\Run}} registry and adding a path to the downloaded executable.<br />
<br />
==== Getting a client ====<br />
<br />
Looking glass client can be installed from AUR using {{AUR|looking-glass}} or {{AUR|looking-glass-git}} packages.<br />
<br />
You can start it once the VM is set up and running<br />
<br />
$ looking-glass-client<br />
<br />
If you do not want to use Spice to control the guest mouse and keyboard you can disable the Spice server.<br />
<br />
$ looking-glass-client -s<br />
<br />
Additionally you may want to start Looking Glass Client as full screen, otherwise the image may be scaled down resulting in poor image fidelity.<br />
<br />
$ looking-glass-client -F<br />
<br />
Launch with the {{ic|--help}} option for further information.<br />
<br />
=== Swap peripherals to and from the Host ===<br />
<br />
Looking Glass includes a Spice client in order to control mouse movement on the Windows guest. However this may have too much latency for certain applications, such as gaming. An alternative method is passing through specific USB devices for minimal latency. This allows for switching the devices between host and guest.<br />
<br />
First create a .xml file for the device(s) you wish to pass-through, which libvirt will use to identify the device.<br />
<br />
{{hc|/home/$USER/.VFIOinput/input_1.xml|2=<br />
<hostdev mode='subsystem' type='usb' managed='no'><br />
<source><br />
<vendor id='0x[Before Colon]'/><br />
<product id='0x[After Colon]'/><br />
</source><br />
</hostdev><br />
}}<br />
<br />
Replace [Before/After Colon] with the contents of the 'lsusb' command, specific to the device you want to pass-through.<br />
<br />
For instance my mouse is {{ic|Bus 005 Device 002: ID 1532:0037 Razer USA, Ltd}} so I would replace {{ic|vendor id}} with 1532, and {{ic|product id}} with 1037.<br />
<br />
Repeat this process for any additional USB devices you want to pass-through. If your mouse / keyboard has multiple entries in {{ic|lsusb}}, perhaps if it is wireless, then create additional xml files for each.<br />
<br />
{{Note|Don't forget to change the path & name of the script(s) above and below to match your user and specific system.}}<br />
<br />
Next a bash script file is needed to tell libvirt what to attach/detach the USB devices to the guest.<br />
<br />
{{hc|/home/$USER/.VFIOinput/input_attach.sh|2=<br />
#!/bin/bash<br />
<br />
virsh attach-device [VM-Name] [USBdevice]<br />
}}<br />
<br />
Replace [VM-Name] with the name of your virtual machine, which can be seen under virt-manager. Additionally replace [USBdevice] with the '''full''' path to the .xml file for the device you wish to pass-through. Add additional lines for more than 1 device. For example here is my script:<br />
<br />
{{hc|/home/ajmar/.VFIOinput/input_attach.sh|2=<br />
#!/bin/bash<br />
<br />
virsh attach-device win10 /home/ajmar/.VFIOinput/input_mouse.xml<br />
virsh attach-device win10 /home/ajmar/.VFIOinput/input_keyboard.xml<br />
}}<br />
<br />
Next duplicate the script file and replace {{ic|attach-device}} with {{ic|detach-device}}. Ensure both scripts are executable with {{ic|chmod +x $script.sh}}<br />
<br />
This 2 script files can now be executed to attach or detach your USB devices from the host to the guest VM. It is important to note that they may need to be executed as root. To run the script from the Windows VM, one possibility is using [[PuTTY]] to [[SSH]] into the host, and execute the script. On Windows PuTTY comes with plink.exe which can execute singular commands over SSH before then logging out, instead of opening a SSH terminal, all in the background.<br />
<br />
{{hc|detach_devices.bat|2=<br />
"C:\Program Files\PuTTY\plink.exe" root@$HOST_IP -pw $ROOTPASSWORD /home/$USER/.VFIOinput/input_detach.sh<br />
}}<br />
<br />
Replace {{ic|$HOST_IP}} with the Host [[Network configuration#IP addresses|IP Address]] and $ROOTPASSWORD with the root password.<br />
<br />
{{warning|This method is insecure if somebody has access to your VM, since they could open the file and read your password. It is advisable to use [[SSH keys]] instead!}}<br />
<br />
You may also want to execute the script files using key binds. On Windows one option is [https://autohotkey.com/ Autohotkey], and on the Host [[Xbindkeys]]. Because of the need to run the scripts as root, you may also need to use [[Polkit]] which can be used to authenticate specific executables as able to run as root without needing a password.<br />
<br />
=== Bypassing the IOMMU groups (ACS override patch) ===<br />
<br />
If you find your PCI devices grouped among others that you do not wish to pass through, you may be able to seperate them using Alex Williamson's ACS override patch. Make sure you understand [https://vfio.blogspot.com/2014/08/iommu-groups-inside-and-out.html the potential risk] of doing so. <br />
<br />
You will need a kernel with the patch applied. The easiest method to acquiring this is through the {{AUR|linux-vfio}} package. <br />
<br />
In addition, the ACS override patch needs to be enabled with kernel command line options. The patch file adds the following documentation:<br />
<br />
pcie_acs_override =<br />
[PCIE] Override missing PCIe ACS support for:<br />
downstream<br />
All downstream ports - full ACS capabilties<br />
multifunction<br />
All multifunction devices - multifunction ACS subset<br />
id:nnnn:nnnn<br />
Specfic device - full ACS capabilities<br />
Specified as vid:did (vendor/device ID) in hex<br />
<br />
The option {{ic|1=pcie_acs_override=downstream}} is typically sufficient.<br />
<br />
After installation and configuration, reconfigure your [[Kernel parameters|bootloader kernel parameters]] to load the new kernel with the {{ic|1=pcie_acs_override=}} option enabled.<br />
<br />
== Plain QEMU without libvirt ==<br />
<br />
Instead of setting up a virtual machine with the help of libvirt, plain QEMU commands with custom parameters can be used for running the VM intended to be used with PCI passthrough. This is desirable for some use cases like scripted setups, where the flexibility for usage with other scripts is needed.<br />
<br />
To achieve this after [[#Setting up IOMMU]] and [[#Isolating the GPU]], follow the [[QEMU]] article to setup the virtualized environment, [[QEMU#Enabling KVM|enable KVM]] on it and use the flag {{ic|1=-device vfio-pci,host=07:00.0}} replacing the identifier (07:00.0) with your actual device's ID that you used for the GPU isolation earlier.<br />
<br />
For utilizing the OVMF firmware, make sure the {{Pkg|ovmf}} package is installed, copy the UEFI variables from {{ic|/usr/share/ovmf/x64/OVMF_VARS.fd}} to temporary location like {{ic|/tmp/MY_VARS.fd}} and finally specify the OVMF paths by appending the following parameters to the QEMU command (order matters):<br />
<br />
* {{ic|1=-drive if=pflash,format=raw,readonly,file=/usr/share/ovmf/x64/OVMF_CODE.fd}} for the actual OVMF firmware binary, note the readonly option<br />
* {{ic|1=-drive if=pflash,format=raw,file=/tmp/MY_VARS.fd}} for the variables<br />
<br />
{{Note|QEMU's default SeaBIOS can be used instead of OVMF, but it's not recommended as it can cause issues with passthrough setups.}}<br />
<br />
It's recommended to study the QEMU article for ways to enhance the performance by using the [[QEMU#Installing virtio drivers|virtio drivers]] and other further configurations for the setup.<br />
<br />
You also might have to use the {{ic|1=-cpu host,kvm=off}} parameter to forward the host's CPU model info to the VM and fool the virtualization detection used by Nvidia's and possibly other manufacturers' device drivers trying to block the full hardware usage inside a virtualized system.<br />
<br />
== Passing though other devices ==<br />
<br />
=== USB controller ===<br />
<br />
If your motherboard has multiple USB controllers mapped to multiple groups, it is possible to pass those instead of USB devices. Passing an actual controller over an individual USB device provides the following advantages : <br />
<br />
* If a device disconnects or changes ID over the course of an given operation (such as a phone undergoing an update), the VM will not suddenly stop seeing it.<br />
* Any USB port managed by this controller is directly handled by the VM and can have its devices unplugged, replugged and changed without having to notify the hypervisor.<br />
* Libvirt will not complain if one of the USB devices you usually pass to the guest is missing when starting the VM.<br />
<br />
Unlike with GPUs, drivers for most USB controllers do not require any specific configuration to work on a VM and control can normally be passed back and forth between the host and guest systems with no side effects.<br />
<br />
{{Warning|Make sure your USB controller supports resetting: [[#Passing through a device that does not support resetting]]}}<br />
<br />
You can find out which PCI devices correspond to which controller and how various ports and devices are assigned to each one of them using this command :<br />
<br />
{{hc|$ <nowiki>for usb_ctrl in $(find /sys/bus/usb/devices/usb* -maxdepth 0 -type l); do pci_path="$(dirname "$(realpath "${usb_ctrl}")")"; echo "Bus $(cat "${usb_ctrl}/busnum") --> $(basename $pci_path) (IOMMU group $(basename $(realpath $pci_path/iommu_group)))"; lsusb -s "$(cat "${usb_ctrl}/busnum"):"; echo; done</nowiki>|<br />
Bus 1 --> 0000:00:1a.0 (IOMMU group 4)<br />
Bus 001 Device 004: ID 04f2:b217 Chicony Electronics Co., Ltd Lenovo Integrated Camera (0.3MP)<br />
Bus 001 Device 007: ID 0a5c:21e6 Broadcom Corp. BCM20702 Bluetooth 4.0 [ThinkPad]<br />
Bus 001 Device 008: ID 0781:5530 SanDisk Corp. Cruzer<br />
Bus 001 Device 002: ID 8087:0024 Intel Corp. Integrated Rate Matching Hub<br />
Bus 001 Device 001: ID 1d6b:0002 Linux Foundation 2.0 root hub<br />
<br />
Bus 2 --> 0000:00:1d.0 (IOMMU group 9)<br />
Bus 002 Device 006: ID 0451:e012 Texas Instruments, Inc. TI-Nspire Calculator<br />
Bus 002 Device 002: ID 8087:0024 Intel Corp. Integrated Rate Matching Hub<br />
Bus 002 Device 001: ID 1d6b:0002 Linux Foundation 2.0 root hub<br />
}}<br />
<br />
This laptop has 3 USB ports managed by 2 USB controllers, each with their own IOMMU group. In this example, Bus 001 manages a single USB port (with a SanDisk USB pendrive plugged into it so it appears on the list), but also a number of internal devices, such as the internal webcam and the bluetooth card. Bus 002, on the other hand, does not apprear to manage anything except for the calculator that is plugged into it. The third port is empty, which is why it does not show up on the list, but is actually managed by Bus 002.<br />
<br />
Once you have identified which controller manages which ports by plugging various devices into them and decided which one you want to passthrough, simply add it to the list of PCI host devices controlled by the VM in your guest configuration. No other configuration should be needed.<br />
<br />
{{Note|If your USB controller does not support resetting, is not in an isolated group, or is otherwise unable to be passed through then it may still be possible to accomplish similar results through [[udev]] rules. See [https://github.com/olavmrk/usb-libvirt-hotplug] which allows any device connected to specified USB ports to be automatically attached to a virtual machine.}}<br />
<br />
=== Passing VM audio to host via PulseAudio ===<br />
<br />
<br />
==== With official repository qemu: ====<br />
<br />
It is possible to route the virtual machine's audio to the host as an application using libvirt. This has the advantage of multiple audio streams being routable to one host output, and working with audio output devices that do not support passthrough. [[PulseAudio]] is required for this to work.<br />
<br />
First, remove the comment from the {{ic|1=#user = ""}} line. Then add your username in the quotations. This tells QEMU which user's pulseaudio stream to route through.<br />
<br />
{{hc|/etc/libvirt/qemu.conf|2=<br />
user = "example"<br />
}}<br />
<br />
Next, modify the libvirt configuration<br />
<br />
{{hc|$ virsh edit [vmname]|<br />
<domain type<nowiki>=</nowiki>'kvm'><br />
}}<br />
<br />
to<br />
<br />
{{hc|$ virsh edit [vmname]|<br />
<domain type<nowiki>=</nowiki>'kvm' xmlns:qemu<nowiki>='http://libvirt.org/schemas/domain/qemu/1.0'</nowiki>><br />
}}<br />
<br />
Then set the QEMU PulseAudio environment variables at the bottom of the libvirt xml file<br />
<br />
{{hc|$ virsh edit [vmname]|<br />
</devices><br />
</domain><br />
}}<br />
<br />
to<br />
<br />
{{hc|$ virsh edit [vmname]|<br />
</devices><br />
<qemu:commandline><br />
<qemu:env name<nowiki>=</nowiki>'QEMU_AUDIO_DRV' value<nowiki>=</nowiki>'pa'/><br />
<qemu:env name<nowiki>=</nowiki>'QEMU_PA_SERVER' value<nowiki>=</nowiki>'/run/user/1000/pulse/native'/><br />
</qemu:commandline><br />
</domain><br />
}}<br />
<br />
Change 1000 under the user directory to your user uid (which can be found by running the {{ic|id}} command. Remember to save the file and exit it without ending the process before continuing, otherwise the changes will not register. If you get the message {{ic|<nowiki>Domain [vmname] XML configuration edited.</nowiki>}} after exiting, it means that your changes have been applied.<br />
<br />
[[Restart]] {{ic|libvirtd.service}} and [[systemd/User|user service]] {{ic|pulseaudio.service}}.<br />
<br />
Virtual Machine audio will now be routed through the host as an application. The application {{Pkg|pavucontrol}} can be used to control the output device. Be aware that on Windows guests, this can cause audio crackling without [[#Slowed down audio pumped through HDMI on the video card|using Message-Signaled Interrupts.]]<br />
<br />
====With qemu-patched====<br />
<br />
This package {{AUR|qemu-patched}} includes some audio patches which only require to change the chipset to improve audio with ich6, while for some cases the previous method did not result in improvement.<br />
<br />
Change the chipset to pc-q35-3.0 (after installing qemu 3.0 that the above package is based on):<br />
<br />
{{hc|$ virsh edit [vmname]|<br />
<domain type<nowiki>=</nowiki>'kvm'><br />
...<br />
<os><br />
<type arch<nowiki>=</nowiki>'x86_64' machine<nowiki>=</nowiki>'pc-q35-3.0'>hvm</type><br />
...<br />
</os><br />
}}<br />
<br />
To speed up compilation use --target-list=x86_64-softmmu to compile qemu with only x86_64 guest support.<br />
<br />
=== Physical Disk/Partition ===<br />
<br />
A whole disk or a partition may be used as a whole for improved I/O performance by adding an entry to XML<br />
<br />
{{hc|$ virsh edit [vmname]| <nowiki><br />
<devices><br />
...<br />
<disk type='block' device='disk'><br />
<driver name='qemu' type='raw' cache='none' io='native'/><br />
<source dev='/dev/sdXX'/><br />
<target dev='vda' bus='virtio'/><br />
<address type='pci' domain='0x0000' bus='0x02' slot='0x0a' function='0x0'/><br />
</disk><br />
...<br />
</devices><br />
</nowiki><br />
}}<br />
<br />
This would require a driver on Windows guests, refer to [[#Setting up the guest OS]].<br />
<br />
=== Gotchas ===<br />
<br />
==== Passing through a device that does not support resetting ====<br />
<br />
When the VM shuts down, all devices used by the guest are deinitialized by its OS in preparation for shutdown. In this state, those devices are no longer functional and must then be power-cycled before they can resume normal operation. Linux can handle this power-cycling on its own, but when a device has no known reset methods, it remains in this disabled state and becomes unavailable. Since Libvirt and Qemu both expect all host PCI devices to be ready to reattach to the host before completely stopping the VM, when encountering a device that will not reset, they will hang in a "Shutting down" state where they will not be able to be restarted until the host system has been rebooted. It is therefore reccomanded to only pass through PCI devices which the kernel is able to reset, as evidenced by the presence of a {{ic|reset}} file in the PCI device sysfs node, such as {{ic|/sys/bus/pci/devices/0000:00:1a.0/reset}}.<br />
<br />
The following bash command shows which devices can and cannot be reset.<br />
<br />
{{hc|<nowiki>for iommu_group in $(find /sys/kernel/iommu_groups/ -maxdepth 1 -mindepth 1 -type d);do echo "IOMMU group $(basename "$iommu_group")"; for device in $(\ls -1 "$iommu_group"/devices/); do if [[ -e "$iommu_group"/devices/"$device"/reset ]]; then echo -n "[RESET]"; fi; echo -n $'\t';lspci -nns "$device"; done; done</nowiki>|<br />
IOMMU group 0<br />
00:00.0 Host bridge [0600]: Intel Corporation Xeon E3-1200 v2/Ivy Bridge DRAM Controller [8086:0158] (rev 09)<br />
IOMMU group 1<br />
00:01.0 PCI bridge [0604]: Intel Corporation Xeon E3-1200 v2/3rd Gen Core processor PCI Express Root Port [8086:0151] (rev 09)<br />
01:00.0 VGA compatible controller [0300]: NVIDIA Corporation GK208 [GeForce GT 720] [10de:1288] (rev a1)<br />
01:00.1 Audio device [0403]: NVIDIA Corporation GK208 HDMI/DP Audio Controller [10de:0e0f] (rev a1)<br />
IOMMU group 2<br />
00:14.0 USB controller [0c03]: Intel Corporation 7 Series/C210 Series Chipset Family USB xHCI Host Controller [8086:1e31] (rev 04)<br />
IOMMU group 4<br />
[RESET] 00:1a.0 USB controller [0c03]: Intel Corporation 7 Series/C210 Series Chipset Family USB Enhanced Host Controller #2 [8086:1e2d] (rev 04)<br />
IOMMU group 5<br />
[RESET] 00:1b.0 Audio device [0403]: Intel Corporation 7 Series/C210 Series Chipset Family High Definition Audio Controller [8086:1e20] (rev 04)<br />
IOMMU group 10<br />
[RESET] 00:1d.0 USB controller [0c03]: Intel Corporation 7 Series/C210 Series Chipset Family USB Enhanced Host Controller #1 [8086:1e26] (rev 04)<br />
IOMMU group 13<br />
06:00.0 VGA compatible controller [0300]: NVIDIA Corporation GM204 [GeForce GTX 970] [10de:13c2] (rev a1)<br />
06:00.1 Audio device [0403]: NVIDIA Corporation GM204 High Definition Audio Controller [10de:0fbb] (rev a1)<br />
}}<br />
<br />
This signals that the xHCI USB controller in 00:14.0 cannot be reset and will therefore stop the VM from shutting down properly, while the integrated sound card in 00:1b.0 and the other two controllers in 00:1a.0 and 00:1d.0 do not share this problem and can be passed without issue.<br />
<br />
== Complete setups and examples ==<br />
<br />
If you have trouble configuring a certain mechanism in your setup, you might want to look up [[PCI_passthrough_via_OVMF/Examples|complete passthrough setup examples]]. A few users have described their setups and you might want to look up certain tricks from their configuration files.<br />
<br />
== Troubleshooting ==<br />
<br />
If your issue is not mentioned below, you may want to browse [[QEMU#Troubleshooting]].<br />
<br />
=== "Error 43: Driver failed to load" on Nvidia GPUs passed to Windows VMs ===<br />
<br />
{{Note|<br />
* This may also fix SYSTEM_THREAD_EXCEPTION_NOT_HANDLED boot crashes related to Nvidia drivers.<br />
* This may also fix problems under linux guests.<br />
}}<br />
<br />
Since version 337.88, Nvidia drivers on Windows check if an hypervisor is running and fail if it detects one, which results in an Error 43 in the Windows device manager. Starting with QEMU 2.5.0 and libvirt 1.3.3, the vendor_id for the hypervisor can be spoofed, which is enough to fool the Nvidia drivers into loading anyway. All one must do is add {{ic|<nowiki>hv_vendor_id=whatever</nowiki>}} to the cpu parameters in their QEMU command line, or by adding the following line to their libvirt domain configuration. It may help for the ID to be set to a 12-character alphanumeric (e.g. '1234567890ab') as opposed to longer or shorter strings.<br />
<br />
{{hc|$ virsh edit [vmname]|<nowiki><br />
...<br />
<features><br />
<hyperv><br />
...<br />
<vendor_id state='on' value='whatever'/><br />
...<br />
</hyperv><br />
...<br />
<kvm><br />
<hidden state='on'/><br />
</kvm><br />
</features><br />
...<br />
</nowiki>}}<br />
<br />
Users with older versions of QEMU and/or libvirt will instead have to disable a few hypervisor extensions, which can degrade performance substantially. If this is what you want to do, do the following replacement in your libvirt domain config file.<br />
<br />
{{hc|$ virsh edit [vmname]|<nowiki><br />
...<br />
<features><br />
<hyperv><br />
<relaxed state='on'/><br />
<vapic state='on'/><br />
<spinlocks state='on' retries='8191'/><br />
</hyperv><br />
...<br />
</features><br />
...<br />
<clock offset='localtime'><br />
<timer name='hypervclock' present='yes'/><br />
</clock><br />
...<br />
</nowiki>}}<br />
<br />
{{bc|<nowiki><br />
...<br />
<clock offset='localtime'><br />
<timer name='hypervclock' present='no'/><br />
</clock><br />
...<br />
<features><br />
<kvm><br />
<hidden state='on'/><br />
</kvm><br />
...<br />
<hyperv><br />
<relaxed state='off'/><br />
<vapic state='off'/><br />
<spinlocks state='off'/><br />
</hyperv><br />
...<br />
</features><br />
...<br />
</nowiki>}}<br />
<br />
=== "BAR 3: cannot reserve [mem]" error in dmesg after starting VM ===<br />
<br />
{{Expansion|This error is actually related to the boot_vgs issue and should be merged together with everything else concerning GPU ROMs.|section=UEFI (OVMF) Compatibility in VBIOS}}<br />
<br />
With respect to [https://www.linuxquestions.org/questions/linux-kernel-70/kernel-fails-to-assign-memory-to-pcie-device-4175487043/ this article]:<br />
<br />
If you still have code 43 check dmesg for memory reservation errors after starting up VM, if you have similar it could be the case:<br />
<br />
vfio-pci 0000:09:00.0: BAR 3: cannot reserve [mem 0xf0000000-0xf1ffffff 64bit pref]<br />
<br />
Find out a PCI Bridge your graphic card is connected to. This will give actual hierarchy of devices:<br />
<br />
$ lspci -t<br />
<br />
Before starting VM run following lines replacing IDs with actual from previous output.<br />
<br />
# echo 1 > /sys/bus/pci/devices/0000\:00\:03.1/remove<br />
# echo 1 > /sys/bus/pci/rescan<br />
<br />
{{Note|Probably setting [[kernel parameter]] {{ic|1=video=efifb:off}} is required as well. [https://pve.proxmox.com/wiki/Pci_passthrough#BAR_3:_can.27t_reserve_.5Bmem.5D_error Source]}}<br />
<br />
=== UEFI (OVMF) Compatibility in VBIOS ===<br />
<br />
{{Remove|Flashing you guest GPU for the purpose of a GPU passthrough is '''never''' good advice. A full section should be dedicated to VBIOS compatibility.|section= UEFI (OVMF) Compatibility in VBIOS}}<br />
<br />
With respect to [https://pve.proxmox.com/wiki/Pci_passthrough#How_to_known_if_card_is_UEFI_.28ovmf.29_compatible this article]:<br />
<br />
Error 43 can be caused by the GPU's VBIOS without UEFI support. To check whenever your VBIOS supports it, you will have to use {{ic|rom-parser}}:<br />
<br />
$ git clone https://github.com/awilliam/rom-parser<br />
$ cd rom-parser && make<br />
<br />
Dump the GPU VBIOS:<br />
<br />
# echo 1 > /sys/bus/pci/devices/0000:01:00.0/rom<br />
# cat /sys/bus/pci/devices/0000:01:00.0/rom > /tmp/image.rom<br />
# echo 0 > /sys/bus/pci/devices/0000:01:00.0/rom<br />
<br />
And test it for compatibility:<br />
<br />
{{hc|$ ./rom-parser /tmp/image.rom|<br />
Valid ROM signature found @600h, PCIR offset 190h<br />
PCIR: type 0 (x86 PC-AT), vendor: 10de, device: 1184, class: 030000<br />
PCIR: revision 0, vendor revision: 1<br />
Valid ROM signature found @fa00h, PCIR offset 1ch<br />
PCIR: type 3 (EFI), vendor: 10de, device: 1184, class: 030000<br />
PCIR: revision 3, vendor revision: 0<br />
EFI: Signature Valid, Subsystem: Boot, Machine: X64<br />
Last image<br />
}}<br />
<br />
To be UEFI compatible, you need a "type 3 (EFI)" in the result. If it's not there, try updating your GPU VBIOS. GPU manufacturers often share VBIOS upgrades on their support pages. A large database of known compatible and working VBIOSes (along with their UEFI compatibility status!) is available on [https://www.techpowerup.com/vgabios/ TechPowerUp].<br />
<br />
Updated VBIOS can be used in the VM without flashing. To load it in QEMU:<br />
<br />
-device vfio-pci,host=07:00.0,......,romfile=/path/to/your/gpu/bios.bin \<br />
<br />
And in libvirt:<br />
<br />
{{bc|1=<br />
<hostdev><br />
...<br />
<rom file='/path/to/your/gpu/bios.bin'/><br />
...<br />
</hostdev><br />
}}<br />
<br />
One should compare VBIOS versions between host and guest systems using [https://www.techpowerup.com/download/nvidia-nvflash/ nvflash] (Linux versions under ''Show more versions'') or <br />
[https://www.techpowerup.com/download/techpowerup-gpu-z/ GPU-Z] (in Windows guest). To check the currently loaded VBIOS:<br />
<br />
{{hc|$ ./nvflash --version|<br />
...<br />
Version : 80.04.XX.00.97<br />
...<br />
UEFI Support : No<br />
UEFI Version : N/A<br />
UEFI Variant Id : N/A ( Unknown )<br />
UEFI Signer(s) : Unsigned<br />
...<br />
}}<br />
<br />
And to check a given VBIOS file:<br />
<br />
{{hc|$ ./nvflash --version NV299MH.rom|<br />
...<br />
Version : 80.04.XX.00.95<br />
...<br />
UEFI Support : Yes<br />
UEFI Version : 0x10022 (Jul 2 2013 @ 16377903 )<br />
UEFI Variant Id : 0x0000000000000004 ( GK1xx )<br />
UEFI Signer(s) : Microsoft Corporation UEFI CA 2011<br />
...<br />
}}<br />
<br />
If the external ROM did not work as it should in the guest, you will have to flash the newer VBIOS image to the GPU. In some cases it is possible to create your own VBIOS image with UEFI support using [https://www.win-raid.com/t892f16-AMD-and-Nvidia-GOP-update-No-requests-DIY.html GOPUpd] tool, however this is risky and may result in GPU brick.<br />
<br />
{{Warning|Failure during flashing may "brick" your GPU - recovery may be possible, but rarely easy and often requires additional hardware. '''DO NOT''' flash VBIOS images for other GPU models (different boards may use different VBIOSes, clocks, fan configuration). If it breaks, you get to keep all the pieces.}}<br />
<br />
In order to avoid the irreparable damage to your graphics adapter it is necessary to unload the NVIDIA kernel driver first:<br />
<br />
# modprobe -r nvidia_modeset nvidia <br />
<br />
Flashing the VBIOS can be done with:<br />
<br />
# ./nvflash romfile.bin<br />
<br />
{{Warning|'''DO NOT''' interrupt the flashing process, even if it looks like it's stuck. Flashing should take about a minute on most GPUs, but may take longer.}}<br />
<br />
=== Slowed down audio pumped through HDMI on the video card ===<br />
<br />
For some users VM's audio slows down/starts stuttering/becomes demonic after a while when it's pumped through HDMI on the video card. This usually also slows down graphics.<br />
A possible solution consists of enabling MSI (Message Signaled-Based Interrupts) instead of the default (Line-Based Interrupts).<br />
<br />
In order to check whether MSI is supported or enabled, run the following command as root:<br />
<br />
# lspci -vs $device | grep 'MSI:'<br />
<br />
where `$device` is the card's address (e.g. `01:00.0`).<br />
<br />
The output should be similar to:<br />
<br />
Capabilities: [60] MSI: Enable'''-''' Count=1/1 Maskable- 64bit+<br />
<br />
A {{ic|-}} after {{ic|Enable}} means MSI is supported, but not used by the VM, while a {{ic|+}} says that the VM is using it.<br />
<br />
The procedure to enable it is quite complex, instructions and an overview of the setting can be found [https://forums.guru3d.com/showthread.php?t=378044 here].<br />
<br />
Other hints can be found on the [https://lime-technology.com/wiki/index.php/UnRAID_6/VM_Guest_Support#Enable_MSI_for_Interrupts_to_Fix_HDMI_Audio_Support lime-technology's wiki], or on this article on [https://vfio.blogspot.it/2014/09/vfio-interrupts-and-how-to-coax-windows.html VFIO tips and tricks].<br />
<br />
A UI tool called [https://github.com/CHEF-KOCH/MSI-utility MSI Utility (FOSS Version 2)] works with Windows 10 64-bit and simplifies the process.<br />
<br />
In order to fix the issues enabling MSI on the 0 function of a nVidia card ({{ic|01:00.0 VGA compatible controller: NVIDIA Corporation GM206 [GeForce GTX 960] (rev a1) (prog-if 00 [VGA controller])}}) was not enough; it will also be required to enable it on the other function ({{ic|01:00.1 Audio device: NVIDIA Corporation Device 0fba (rev a1)}}) to fix the issue.<br />
<br />
=== No HDMI audio output on host when intel_iommu is enabled ===<br />
<br />
If after enabling {{ic|intel_iommu}} the HDMI output device of Intel GPU becomes unusable on the host then setting the option {{ic|igfx_off}} (i.e. {{ic|1=intel_iommu=on,igfx_off}}) might bring the audio back, please read {{ic|Graphics Problems?}} in [https://www.kernel.org/doc/Documentation/Intel-IOMMU.txt Intel-IOMMU.txt] for details about setting {{ic|igfx_off}}.<br />
<br />
=== X does not start after enabling vfio_pci ===<br />
<br />
This is related to the host GPU being detected as a secondary GPU, which causes X to fail/crash when it tries to load a driver for the guest GPU. To circumvent this, a Xorg configuration file specifying the BusID for the host GPU is required. The correct BusID can be acquired from lspci or the Xorg log. [https://www.redhat.com/archives/vfio-users/2016-August/msg00025.html Source →]<br />
<br />
{{hc|/etc/X11/xorg.conf.d/10-intel.conf|<nowiki><br />
Section "Device"<br />
Identifier "Intel GPU"<br />
Driver "modesetting"<br />
BusID "PCI:0:2:0"<br />
EndSection<br />
</nowiki>}}<br />
<br />
=== Chromium ignores integrated graphics for acceleration ===<br />
<br />
Chromium and friends will try to detect as many GPUs as they can in the system and pick which one is preferred (usually discrete NVIDIA/AMD graphics). It tries to pick a GPU by looking at PCI devices, not OpenGL renderers available in the system - the result is that Chromium may ignore the integrated GPU available for rendering and try to use the dedicated GPU bound to the {{ic|vfio-pci}} driver, and unusable on the host system, regardless of whenever a guest VM is running or not. This results in software rendering being used (leading to higher CPU load, which may also result in choppy video playback, scrolling and general un-smoothness).<br />
<br />
This can be fixed by [[Chromium/Tips and tricks#Forcing specific GPU|explicitly telling Chromium which GPU you want to use]].<br />
<br />
=== VM only uses one core ===<br />
<br />
For some users, even if IOMMU is enabled and the core count is set to more than 1, the VM still only uses one CPU core and thread. To solve this enable "Manually set CPU topology" in {{ic|virt-manager}} and set it to the desirable amount of CPU sockets, cores and threads. Keep in mind that "Threads" refers to the thread count per CPU, not the total count.<br />
<br />
=== Passthrough seems to work but no output is displayed ===<br />
<br />
Make sure if you are using virt-manager that UEFI firmware is selected for your virtual machine. Also, make sure you have passed the correct device to the VM.<br />
<br />
=== virt-manager has permission issues ===<br />
<br />
If you are getting a permission error with virt-manager add the following to your {{ic|/etc/libvirt/qemu.conf}}:<br />
<br />
group="kvm"<br />
user="''user''"<br />
<br />
If that does not work make sure your user account is added to the kvm and libvirt [[groups]].<br />
<br />
=== Host Lockup After VM Shutdown ===<br />
<br />
This issue seems to primarily affect users running a Windows 10 guest and usually after the VM has been run for a prolonged period of time: the host will experience multiple CPU core lockups (see [https://bbs.archlinux.org/viewtopic.php?id=206050&p=2]). To fix this try enabling Message Signal Interrupts on the GPU passed through to the guest. A good guide for how to do this can be found in [https://forums.guru3d.com/threads/windows-line-based-vs-message-signaled-based-interrupts.378044/].<br />
<br />
=== Host lockup if guest is left running during sleep ===<br />
<br />
VFIO-enabled virtual machines tend to become unstable if left running through a sleep/wakeup cycle and have been known to cause the host machine to lockup when an attempt is then made to shut them down. In order to avoid this, one can simply prevent the host from going into sleep while the guest is running using the following libvirt hook script and systemd unit. The hook file needs executable permissions to work.<br />
<br />
{{hc|/etc/libvirt/hooks/qemu|<nowiki><br />
#!/bin/bash<br />
<br />
OBJECT="$1"<br />
OPERATION="$2"<br />
SUBOPERATION="$3"<br />
EXTRA_ARG="$4"<br />
<br />
case "$OPERATION" in<br />
"prepare")<br />
systemctl start libvirt-nosleep@"$OBJECT"<br />
;;<br />
"release")<br />
systemctl stop libvirt-nosleep@"$OBJECT"<br />
;;<br />
esac<br />
</nowiki>}}<br />
<br />
{{hc|/etc/systemd/system/libvirt-nosleep@.service|<nowiki><br />
[Unit]<br />
Description=Preventing sleep while libvirt domain "%i" is running<br />
<br />
[Service]<br />
Type=simple<br />
ExecStart=/usr/bin/systemd-inhibit --what=sleep --why="Libvirt domain \"%i\" is running" --who=%U --mode=block sleep infinity<br />
</nowiki>}}<br />
<br />
== See also ==<br />
<br />
* [https://bbs.archlinux.org/viewtopic.php?id=162768 Discussion on Arch Linux forums] | [https://archive.is/kZYMt Archived link]<br />
* [https://docs.google.com/spreadsheet/ccc?key=0Aryg5nO-kBebdFozaW9tUWdVd2VHM0lvck95TUlpMlE User contributed hardware compatibility list]<br />
* [https://pastebin.com/rcnUZCv7 Example script from https://www.youtube.com/watch?v=37D2bRsthfI]<br />
* [https://vfio.blogspot.com/ Complete tutorial for PCI passthrough]<br />
* [https://www.redhat.com/archives/vfio-users/ VFIO users mailing list]<br />
* [https://webchat.freenode.net/?channels=vfio-users #vfio-users on freenode]<br />
* [https://www.youtube.com/watch?v=aLeWg11ZBn0 YouTube: Level1Linux - GPU Passthrough for Virtualization with Ryzen]</div>Nicman23https://wiki.archlinux.org/index.php?title=Sway&diff=428163Sway2016-03-27T19:39:01Z<p>Nicman23: Added some information regarding man sway-input and the output of sway-msg -t get_inputs</p>
<hr />
<div>[[Category:Tiling WMs]]<br />
[[Category:Dynamic WMs]]<br />
[[ja:Sway]]<br />
''sway'' (SirCmpwn's Wayland window manager) is an attempt to create a [[Wayland]] version of [[i3]].<br />
<br />
{{Note|This is still a work in progress so caution is advised. However, it is deemed ready for regular use by the creator.}}<br />
<br />
== Installation ==<br />
<br />
''sway'' can be [[installed]] with the {{AUR|sway-git}} package. If you already use i3, then copy your i3 configuration to {{ic|~/.config/sway/config}} and it will work out of the box. Otherwise, copy the sample configuration file to {{ic|~/.config/sway/config}}. It is located at {{ic|/etc/sway/config}}, unless the {{ic|DFALLBACK_CONFIG_DIR}} flag has been set. See the sway(5) [[man page]] for information on the configuration.<br />
<br />
== Starting sway ==<br />
=== From TTY ===<br />
You can start sway by simply typing {{ic|sway}} in the TTY.<br />
<br />
=== Using a display manager ===<br />
The sway session is located at {{ic|/usr/share/wayland-sessions/sway.desktop}}. It is automatically recognized by GDM.<br />
<br />
=== From X ===<br />
If you want to start ''sway'' in an X session for testing purposes it is possible to start it as a regular program.<br />
<br />
== Configuration ==<br />
=== Keymap ===<br />
By default, sway starts with the US QWERTY keymap. You can override this behaviour by starting sway with<br />
$ XKB_DEFAULT_LAYOUT=gb XKB_DEFAULT_VARIANT=colemak XKB_DEFAULT_MODEL=pc101 sway<br />
This will launch sway with the keyboard set to the Colemak variant of the British keymap with the 101-key keyboard model.<br />
<br />
If you are using a display manager, you can not simply prepend the above line to the {{ic|sway.desktop}} file. As root, create the following file:<br />
{{hc|/usr/bin/sway-gb-ck|2=<br />
#!/bin/sh<br />
XKB_DEFAULT_LAYOUT=gb XKB_DEFAULT_VARIANT=colemak XKB_DEFAULT_MODEL=pc101 sway}}<br />
Then, create a {{ic|sway-gb-ck.desktop}} file that starts the above script:<br />
{{hc|/usr/share/wayland-sessions/sway.desktop|2=<br />
[Desktop Entry]<br />
Name=Sway British(Colemak)<br />
Comment=SirCmpwn's Wayland window manager with the British Colemak keyboard layout<br />
Exec=sway-gb-ck<br />
Type=Application<br />
}}<br />
<br />
=== Statusbar ===<br />
Installing the program {{Pkg|i3status}} is an easy way to get a practical, default statusbar. All one has to do is add following snippet at the end of your sway config:<br />
{{hc|~/.config/sway/config|<nowiki><br />
bar {<br />
status_command i3status<br />
}<br />
</nowiki>}}<br />
If you want to achieve colored output of i3status, you can adjust following part in the i3status configuration:<br />
{{hc|~/.config/i3status/config|<nowiki><br />
general {<br />
colors = true<br />
interval = 5<br />
}<br />
</nowiki>}}<br />
In both examples, the system-wide installed configuration files has been copied over to the user directory and then modified.<br />
<br />
=== Wallpaper ===<br />
This line, which can be appended at the end of your sway configuration, sets a background image on all displays (output matches all with name {{ic|"*"}}):<br />
{{hc|~/.config/sway/config|<nowiki><br />
output "*" background /home/onny/pictures/fredwang_norway.jpg fill<br />
</nowiki>}}<br />
Of course you have to replace the file name and path according to your wallpaper.<br />
<br />
=== Input devices ===<br />
Its possible to tweak specific input device configurations. For example to enable tap-to-click for a touchpad, add an input block:<br />
{{hc|~/.config/sway/config|<nowiki><br />
input "2:14:ETPS/2_Elantech_Touchpad" {<br />
tap enabled<br />
}<br />
</nowiki>}}<br />
Where as the device identifier can be queried with:<br />
swaymsg -t get_inputs<br />
Output from the command, sometimes has "\" to escape some symbols like "/" (ie {{ic|"2:14:ETPS\/2_Elantech_Touchpad"}}) and need to be removed<br />
<br />
More documentation and options like acceleration profiles can be found with <br />
man sway-input<br />
<br />
=== Custom keybindings ===<br />
[[Extra_keyboard_keys|Special keys]] on your keyboard can be used to execute commands, for example to control your volume or your monitor brightness:<br />
{{hc|~/.config/sway/config|<nowiki><br />
bindsym XF86AudioRaiseVolume exec pactl set-sink-volume 0 +15%<br />
bindsym XF86AudioLowerVolume exec pactl set-sink-volume 0 -15%<br />
bindsym XF86AudioToggle exec pactl set-sink-mute 0 toggle<br />
bindsym XF86MonBrightnessDown exec dsplight down 5<br />
bindsym XF86MonBrightnessUp exec dsplight up 5<br />
</nowiki>}}<br />
<br />
== See also ==<br />
<br />
* [https://github.com/SirCmpwn/sway Github project]<br />
* [http://swaywm.org Website]</div>Nicman23https://wiki.archlinux.org/index.php?title=LXDE&diff=416186LXDE2016-01-19T10:42:09Z<p>Nicman23: Add that lxpanel and specifically its menus can be configured - stub</p>
<hr />
<div>[[Category:Desktop environments]]<br />
[[cs:LXDE]]<br />
[[es:LXDE]]<br />
[[fr:LXDE]]<br />
[[it:LXDE]]<br />
[[ja:LXDE]]<br />
[[pl:LXDE]]<br />
[[ru:LXDE]]<br />
[[sr:LXDE]]<br />
[[tr:LXDE Masaüstü Ortamı]]<br />
[[uk:LXDE]]<br />
[[zh-CN:LXDE]]<br />
[[zh-TW:LXDE]]<br />
{{Related articles start}}<br />
{{Related|Desktop environment}}<br />
{{Related|Display manager}}<br />
{{Related|Window manager}}<br />
{{Related|Openbox}}<br />
{{Related|PCManFM}}<br />
{{Related|LXDM}}<br />
{{Related|LXQt}}<br />
{{Related|File manager functionality#Mounting}}<br />
{{Related articles end}}<br />
<br />
From project [http://lxde.org/ home page]:<br />
<br />
: ''The "Lightweight X11 Desktop Environment" is an extremely fast-performing and energy-saving desktop environment. Maintained by an international community of developers, it comes with a beautiful interface, multi-language support, standard keyboard short cuts and additional features like tabbed file browsing. LXDE uses less CPU and less RAM than other environments. It is especially designed for cloud computers with low hardware specifications, such as, netbooks, mobile devices (e.g. MIDs) or older computers.''<br />
<br />
== Installation ==<br />
<br />
LXDE requires at least {{Pkg|lxde-common}} and {{Pkg|openbox}} (or another window manager) to be [[install]]ed. The {{Grp|lxde}} group contains the full desktop.<br />
<br />
=== GTK+ 3 version ===<br />
<br />
An experimental GTK+ 3 build of LXDE can be installed with the ''lxde-gtk3'' group from the unofficial [[Unofficial user repositories#city|city]] repository. The packages are available also from the [[AUR]]: [https://aur.archlinux.org/packages.php?K=lxde-gtk3 search for packages tagged with lxde-gtk3].<br />
<br />
While it works mostly, there are some known issues with [https://sourceforge.net/p/lxde/bugs/769/ gpicview], [https://sourceforge.net/p/lxde/bugs/768/ lxappearance-obconf], [https://sourceforge.net/p/lxde/patches/513/ lxlauncher] and [https://sourceforge.net/p/lxde/bugs/773/ lxpanel].<br />
<br />
== Starting the desktop ==<br />
<br />
=== Graphical log-in ===<br />
<br />
[[LXDM]] is the default display manager for LXDE and is installed as part of the {{Grp|lxde}} group. See also [[Display manager]].<br />
<br />
=== Console ===<br />
<br />
To use '''startx''', you will need to define LXDE in [[xinitrc]]:<br />
<br />
{{hc|~/.xinitrc|exec startlxde}}<br />
<br />
See also [[Start X at login]].<br />
<br />
== Tips and tricks ==<br />
<br />
=== Application menu editing ===<br />
<br />
{{Merge|Desktop entries}}<br />
<br />
The application menu works by resolving the {{ic|.desktop}} files located in {{ic|/usr/share/applications}}. Many desktop environments run programs that supersede these settings to allow customization of the menu. LXDE has yet to create an application menu editor but you can manually build them yourself if you are so inclined. Third party menu editor can be found in [[AUR]] - {{AUR|lxmed}}<br />
<br />
To add or edit a menu item, create or link to the {{ic|.desktop}} file in {{ic|/usr/share/applications}}, {{ic|/usr/local/share/applications}}, or {{ic|~/.local/share/applications}}. (The latter two have the advantage of putting your application outside of directories governed by {{ic|pacman}}.) Consult [http://standards.freedesktop.org/desktop-entry-spec/latest/ the desktop entry specification] on freedesktop.org for structures of {{ic|.desktop}} files.<br />
<br />
To remove items from the menu, instead of deleting the {{ic|.desktop}} files, you can edit the file and add the following line in the file:<br />
<br />
NoDisplay=true<br />
<br />
To expedite the process for a good number of files you can put it in a loop. For example:<br />
<br />
$ cd /usr/share/applications<br />
$ for i in program1.desktop program2.desktop ...; do cp /usr/share/applications/$i \<br />
/home/user/.local/share/applications/; echo "NoDisplay=true" >> \<br />
/home/user/.local/share/applications/$i; done<br />
<br />
This will work for all applications except KDE applications. For these, the only way to remove them from the menu is to log into KDE itself and use its menu editor. For every item that you do not want displayed, check the 'Show only in KDE' option. If adding NoDisplay=True will not work, you can add ShowOnlyIn=XFCE.<br />
<br />
=== Autostart ===<br />
<br />
Applications can be automatically started in several ways.<br />
<br />
==== Desktop files ====<br />
<br />
{{Tip|{{ic|.desktop}} files can be manipulated with the {{AUR|lxsession-edit}}{{Broken package link|{{aur-mirror|lxsession-edit}}}} package.}}<br />
<br />
See [[Desktop entries#Autostart]].<br />
<br />
==== Lxsession ====<br />
<br />
Each line in {{ic|~/.config/lxsession/LXDE/autostart}} represents a command to be executed. If a line starts with {{ic|@}}, and the command following it crashes, the command is automatically re-executed. For example:<br />
<br />
{{hc|~/.config/lxsession/LXDE/autostart|<br />
@lxterminal<br />
@leafpad<br />
}}<br />
<br />
{{Note|Unlike Openbox, these commands do ''not'' end with a {{ic|&}} symbol.}}<br />
<br />
There is also a global autostart file at {{ic|/etc/xdg/lxsession/LXDE/autostart}}.<br />
<br />
{{Note|If both files are present, lxsession only executes the local file as of v0.4.9}}<br />
<br />
=== Bindings ===<br />
<br />
Mouse and key bindings (i.e. keyboard shortcuts) are implemented with Openbox. LXDE users should follow the [http://openbox.org/wiki/Help:Bindings Openbox wiki] to edit {{ic|~/.config/openbox/lxde-rc.xml}}.<br />
<br />
An optional GUI for editing the key bindings is provided by the {{AUR|obkey}} package. Whle it edits {{ic|rc.xml}} by default, you can direct it to the LXDE configuration as follows:<br />
<br />
$ obkey ~/.config/openbox/lxde-rc.xml<br />
<br />
See [http://code.google.com/p/obkey/] for more information.<br />
<br />
=== Cursors ===<br />
<br />
LXAppearance, provided by the {{pkg|lxappearance}} package, is a graphical tool that can determine a number of aspects of the user interface including the cursor theme. Settings configured using LXAppearance are written to {{ic|~/.gtkrc-2.0}}, {{ic|~/.config/gtk-3.0/settings.ini}}, and {{ic|~/.icons/default/index.theme}}. See also [[Cursor themes]].<br />
<br />
=== Digital clock applet time ===<br />
<br />
You can right click on the digital clock applet on the panel and set how it displays the current time using the strftime format - see [http://linux.die.net/man/3/strftime man strftime] for details.<br />
<br />
=== Font settings ===<br />
<br />
See [[Font configuration]]. {{Pkg|lxappearance-obconf}} configures LXDE-specific settings.<br />
<br />
=== Keyboard layout ===<br />
<br />
See [[Keyboard configuration in Xorg]] for generic instructions. A keyboard layout applet is included with ''lxpanel''.<br />
<br />
See [[#Autostart programs]] for a way to automatically start ''setxkbmap'' in LXDE.<br />
<br />
=== Screen locking ===<br />
<br />
LXDE does not come with a screen locker of its own; see [[List of applications/Security#Screen lockers]] for alternatives.<br />
<br />
{{ic|/etc/xdg/lxsession/LXDE/autostart}} from {{Pkg|lxde-common}} lists [[XScreenSaver]], which will be launched automatically. See [[#Autostart]] when using a different locker. See [[DPMS]] on how to control the screen saver without external programs.<br />
<br />
=== lxpanel icons ===<br />
<br />
{{Accuracy|{{ic|.local/share/}} preferred}}<br />
<br />
Default icons used by lxpanel are stored in {{ic|/usr/share/pixmaps}} and any custom icons you want lxpanel to use need to be saved there as well.<br />
<br />
You can change default icons for applications by taking the following steps:<br />
<br />
# Save the new icon to /usr/share/pixmaps<br />
# Use a text editor to open the {{ic|.desktop}} file of the program whose icon you want to change in {{ic|/usr/share/applications}}.<br />
# Change<br />
<br />
Icon=/default/icon/.png<br />
<br />
to:<br />
<br />
Icon=/name/of/new/icon/added/to/pixmaps/.png<br />
<br />
=== lxpanel menus ===<br />
<br />
The panel's menus can be configured in {{ic|/etc/xdg/menus/lxde-applications.menu}} as per the [[xdg-menu]] format to work with applications from other sessions (notably [[mate]]) to add some of the function-ability that lxde lacks.<br />
<br />
<br />
=== Replace Openbox ===<br />
<br />
''lxsession'' uses the [[window manager]] defined in {{ic|~/.config/lxsession/LXDE/desktop.conf}}. If this file does not exist, it searches in {{ic|/etc/xdg/lxsession/LXDE/desktop.conf}} instead.<br />
<br />
Replace {{ic|openbox-lxde}} in either file with a window manager of choice:<br />
<br />
[Session]<br />
window_manager=openbox-lxde<br />
<br />
For metacity:<br />
<br />
window_manager=metacity<br />
<br />
For compiz:<br />
<br />
window_manager=compiz<br />
<br />
Alternatively, you can autostart {{ic|wm --replace}} using the method defined in [[#Lxsession]] where ''wm'' is the name of the window manager executable being started. This method does mean that Openbox will be started first on each login and will then immediately be replaced by the autostarted window manager.<br />
<br />
Note that since openbox dispatches the desktop-wide keyboard shortcuts in LXDE, users who want to replace it and still use these shortcuts will need to reimplement this functionality themselves. A good option is [[xbindkeys]].<br />
<br />
=== Shutdown, reboot, suspend and hibernate options (LXSession-logout) ===<br />
<br />
This requires installation of {{Pkg|upower}}.<br />
<br />
== Troubleshooting ==<br />
<br />
=== NTFS with Chinese characters ===<br />
<br />
{{Merge|NTFS-3G}}<br />
<br />
For a storage device with an NTFS filesystem, you will need to install the [[NTFS-3G]] package. Generally, PCManFM works well with NTFS filesystems, however there is one bug affecting NTFS users that if you have files or directories on an NTFS filesystem, the names of which contain non-latin characters (e.g. Chinese characters) may disappear when opening (or auto-mounting) the NTFS volume. This happens because the lxsession mount-helper is not correctly parsing the policies and locale options. There is a workaround for this:<br />
<br />
Create a new {{ic|/usr/local/bin/mount.ntfs-3g}} with a new Bash script containing:<br />
<br />
#!/bin/bash<br />
/usr/bin/ntfs-3g $1 $2 -o locale=en_US.UTF-8<br />
<br />
And then make it executable:<br />
<br />
# chmod +x /usr/local/bin/mount.ntfs-3g<br />
<br />
=== lxpanel crashes with some themes or browsing particular web pages ===<br />
With some gtk themes ([[AUR]] - {{AUR|gtk-theme-greybird}}{{Broken package link|{{aur-mirror|gtk-theme-greybird}}}}), launch lxpanel will get errors below.<br />
<br />
lxpanel: cairo-scaled-font.c:459: _cairo_scaled_glyph_page_destroy: Assertion `!scaled_font->cache_frozen' failed.<br />
<br />
Try install {{Pkg|ttf-dejavu}} in this case.<br />
<br />
If lxpanel crashes when browsing particular unicode web pages, try install {{Pkg|ttf-droid}}.<br />
<br />
== See also ==<br />
<br />
* [http://lxlinux.com/ Linux LXDE Guide]<br />
* [http://lxde.sourceforge.net LXDE (Sourceforge)]<br />
* [http://forum.lxde.org LXDE forum]</div>Nicman23