User:CodingKoopa/Direct Rendering Infrastructure

From ArchWiki

gpu go brrr

this article is a WIP

  • Should the graphics APIs be ordered from the kernel out or from the applications in?
  • Maybe have a wikitable / Template:Text art showing a visual representation of the different layers? I'm thinking something structured like Wikipedia:File:The Linux Graphics Stack and glamor.svg. Maybe have the sections linked inside it?
  • Maybe explain succinctly application toolkits like GTK/Qt?
  • /usr/lib/pkgconfig has some info on libraries such as GL

TODO: integrate Wayland information from https://doc.qt.io/qt-6/wayland-and-qt.html - this is a good primer on why graphics buffers are necessary TODO: consider using definition syntax - see mkinitcpio TODO: see https://github.com/dolphin-emu/dolphin/pull/9015 as reference

Graphics APIs

It's useful to be aware of the different APIs that Linux programs use to render and display graphics.

  • Xlib (also known as libX11) is the original X11 client-side library for interacting with the X server. As of its conception, this API was the only way for programs to construct and display a user interface.
  • XCB (also known as libXCB) is the (relatively) newer X11 client-side library, providing an asynchronous interface, and more direct access to the display server. Xlib and XCB together comprise X11's 2D APIs, sometimes called "X Rendering" for its exclusivity to the X Server.
  • OpenGL (sometimes "OGL", rarely "Desktop GL") is a ubiquitous and long-standing API for rendering 2D and 3D graphics on desktop platforms, designed with hardware acceleration in mind. The OpenGL API does not provide a means for creating and managing a context; for this, a window-system API is required.
  • OpenGL ES (often "GLES") is a subset of OpenGL designed for embedded systems. Since modern desktop Linux OpenGL implementations provide at least GLES 2.0 [1] [2], sufficiently low versions of GLES are now the least common denominator of hardware accelerated graphics APIs [3].
  • GLX is an OpenGL window-system API for X11.
  • EGL is a platform-agnostic OpenGL/GLES window-system API. Though "window-system" is the best term to describe the role that both GLX and EGL play, EGL's functionality is more broad than servicing programs that target a windowing system. EGL can be also used to target lower level "headless rendering" platforms such as the DRM itself, which is necessary for Wayland compositors to function.
    Initially, up to and including EGL 1.4, the only method to get a display was eglGetDisplay(), which internally selects the specific platform as an implementation detail [4]. To provide control over the underlying EGL platform is used, the EGL_EXT_platform_base extension for EGL 1.4 was developed, which the other EGL_EXT_platform_platform platform extensions are based on. With EGL 1.5, the EGL_EXT_platform_base extension's functionality was merged into the main API, and the most of the existing platforms were promoted to the Khronos Group namespace as EGL_KHR_platform_platform.
    To reiterate, EGL_EXT_platform_platform extensions have a transitive dependency on EGL 1.4, and EGL_KHR_platform_platform have a dependency on EGL 1.5. On desktop Linux, the following platforms exist:
  • Vulkan is a 3D API intended to be the successor of OpenGL, providing less overhead and more control over how the GPU is utilized, at the cost of a higher code complexity. The Vulkan API does provide everything needed to get started with rendering, unlike OpenGL's requirement for a window-system API.
  • Gallium3D is an architecture for creating graphics drivers comprised of interfaces and supporting libraries, used by Mesa and several other open-source drivers.
  • DXVK, VKD3D, wined3d, and ANGLE are translation layers for using 3D API other than that which the code directly uses.
  • VA-API, VDPAU, and NVENC/NVDEC are auxiliary APIs for using the GPU to decode video. See Hardware video acceleration for more details.

The following shifts and trends are worth noting:

  • Programs written directly using the X11 2D APIs have become uncommon, with OpenGL ES generally being the preference for new applications that wish to be as portable as possible. However, the X11 APIs still may be targeted by UI frameworks with platform abstraction layers. For instance, Qt unconditionally defaults to using XCB [7].
  • OpenGL programs are switching from GLX to EGL, as a more portable and potentially faster window-system API [8] [9].
    • Understanding that EGL hasn't always been a viable (or existing) option for OpenGL programs on Xorg reveals why there may seem to be a dichotomy between Xorg programs using GLX, and Wayland using EGL. For instance, mpv's wayland option correctly implies EGL (the only option), but its x11 option implies GLX, even though EGL is now an option [10].
  • Games and other performance-critical applications are steadily adopting Vulkan, though usually without dropping OpenGL support if present. [11] [12].
  • EGLStream adoption has been met with opposition for its exclusivity to Nvidia drivers [13] [14] [15] [16] [17]. Some still supported the platform, as it was the only way for a Wayland compositor to support Nvidia GPUs. In 2021, though, Nvidia drivers gained support for the GBM platform [18], further de-incentivizing EGLStream usage [19] [20].

Background

Historically, most X clients drew their graphics using indirect rendering, issuing drawing commands to the Xorg display server. The Device Dependent X (DDX) driver is the component of the X server that interacts directly with the GPU to render the X clients, as well as perform mode setting. Any other programs, such as a full-screen video players, would assume exclusive access to the GPU to render its graphics. This poses the following issues:

  • In order for the DDX to directly issue commands to the GPU, Xorg must be running as root. This may be undesirable; see Xorg#Rootless Xorg.
  • Without a kernel-space intermediary, a program like a video player that requires direct rendering access could cause a conflict with an active X server.
  • The kernel must perform mode setting to initialize the console. The DDX still needs to perform mode setting too, though. As a result, there is more mode setting than needed, and duplicated code. See Wikipedia:Direct Rendering Manager#Kernel Mode Setting for other drawbacks of User Mode Setting.

This issue was solved by introducing the kernel-space Direct Rendering Infrastructure (DRI) for managing displays and rendering graphics from user-space without requiring elevated permissions or causing conflicts. The components of DRI of interest are:

The consumers of the DRI now are:

  • OpenGL/GLES clients direct rendering using a DRI client that bypasses the display server [21] [22].
  • Wayland clients using the Wayland EGL platform to render directly to a surface in a generic shared buffer. The EGL platform implementation then displays the surfaces using the DRM, while accounting for damage requests [23].
  • X clients using the Xlib or XCB EGL platform to render directly to an X11 surface. The DDX then displays the X11 surfaces using the DRM, accounting for damage requests and working with the compositor if present [24].
  • X clients using the X11 2D APIs, which the DDX will render and display.
  • X clients using the the Xorg DRI extension to direct render by proxy of the DDX.
  • X clients using the Xorg GLX extension to issue OpenGL commands to the server for software rendering by an internal OpenGL implementation, as a form of indirect rendering.
    • GLX predates the DRI, but has since been updated to support passing through to the system's hardware accelerated OpenGL implementation if available, for direct rendering.

For more information, see Wikipedia:Free and open-source graphics device driver#History.

Graphics Drivers

A modern Linux graphics stack is comprised of the following:

  • A kernel module providing the DRM driver, providing low-level hardware support. These are all present within the kernel tree, with the exception of the proprietary Nvidia kernel module [25].
  • Included with Xorg: Implementations of the X11 client-side libraries:
    • Xlib, implemented in libX11.so from libx11.
    • XCB, implemented in libxcb.so from libxcb.
  • Required for Xorg: A DDX driver, which provides:
    • Mode setting via KMS, while supporting the Xorg RandR extension.
    • Support for "X rendering" (programs using the Xlib and XCB 2D APIs).
    • Optionally, hardware acceleration for X rendering.
    • Optionally, hardware acceleration for video decoding by deferring to what was guessed to be the correct vendor-specific implementation [26]. VDPAU is a special case where the DDX can delegate the displaying of the video to VDPAU, in addition to decoding [27].
Note: Vendor-specific DDX drivers are being phased out in favor of the generic modesetting(4) driver. Unlike the other generic drivers, the modesetting driver does offer hardware acceleration, via Glamor. If available, Glamor implements X rendering by translating the render primitives into OpenGL commands, which are passed through the system's OpenGL stack. Glamor is viable for day-to-day usage, and is even recommended over the specialized drivers in some cases; see Intel graphics#Installation.
  • Optional for Xorg: A compositor, which receives an off-screen buffer of the Xorg scene for visual manipulation.
  • Generally required for Wayland but optional for Xorg: An OpenGL driver. As suggested by the New Linux OpenGL ABI Proposal, a complete OpenGL implementation is comprised of:
    • Vendor-neutral entrypoints that dispatch to vendor libraries. These are now provided by libglvnd (the GL Vendor-Neutral Dispatch library) in libglvnd:
      • libGL.so, meant to provide entrypoints for OpenGL 1.2, GLX 1.3, and extensions of the two. This was the original de facto API for using OpenGL on Linux (which, then, would be equivalent to "OpenGL on Xorg") [28] [29]. Before Andy Ritger's OpenGL ABI proposal, each vendor shipped their own version of this shared object, and in doing so often took liberties in introducing symbols from newer OpenGL and GLX versions [30].
      • libOpenGL.so, providing entrypoints for OpenGL 4.4 and backwards-compatibility extensions. The behavior of this shared object is more predictable than libGL thanks to less historical baggage. It does not include entrypoints for any window-system libraries.
      • libGLESv1_CM.so, providing entrypoints for the OpenGL ES 1 common profile.
      • libGLESv2.so, providing entrypoints for the OpenGL ES 2.0 through 3.2 [31].
      • libGLX.so, providing entrypoints for GLX.
      • libEGL.so, providing entrypoints for EGL.
    • Vendor-specific shared objects that issue commands to the GPU using the DRM. Mesa provides an implementation for software rendering, as well as implementations for most hardware vendors. There are only two libraries in this category:
      • libGLX_vendor.so, the GLX implementation.
      • libEGL_vendor.so, the EGL implementation.
        • libglvnd's EGL dispatching requires the implementation to also provide an EGL Installable Client Driver (ICD), described in /usr/share/glvnd/egl_vendor.d [32].
        • Since Nvidia's EGL implementation uses the EGL Platform Interface, external platform implementations are also needed. The external platform for GBM is provided by nvidia-utils, and the external platform for Wayland with EGLStreams is provided by egl-wayland. Mesa, on the other hand, does not have the same infrastructure for defining platforms; it internally provides libgbm for any headless rendering, which on Arch is already included with mesa [33].
  • Optional: A Vulkan driver. Mesa provides an implementation for software rendering.


TODO: more library vulkan notes

TODO: crocus, iris, i915

Retrieved from "https://wiki.archlinux.org/index.php?title=User:CodingKoopa/Direct_Rendering_Infrastructure&oldid=754530"