Problem

When ARM hardware moves towards ACPI instead of device tree, the first level OS needs to be able to extract information stored in ACPI and invoke ACPI functions to enumerate and manage hardware.

ACPI functions are stored as AML (ACPI Machine Language) bytecode and are executed within the context of the first level OS kernel via an AML interpreter, which is a large and complicated piece of software.

For KVM this is not a significant challenge, as Linux will have to solve this problem and KVM just leverages Linux support for the hardware it runs on. However, for Xen, this is challenging, because one of the key strengths of the Xen hypervisor is its small size and small TCB, and adding a complete AML interpreter to the Xen hypervisor is unacceptable.

Further, both the Xen hypervisor and Dom0 needs access to ACPI, because they both manage separate aspects of the hardware (the hypervisor core resources and Dom0 most peripherals).

LCA 14 Meeting

The minutes from the meeting discussing this page at LCA14 can be found here

ACPI Lingo Glossary

Term

Name

Type

Description

AML

ACPI Machine Language

bytecode to be interpreted
by an AML interpreter

Table

ACPI Tables

Tables are data structures, they can
be either fixed-format or AML-based

RSDP

Root System Description Pointer

The ACPI root pointer" (**)

RSDT

Root System Description Table

static

collection of pointers to other tables

XSDT

Extended System Description Table

static

collection of pointers to other tables

DSDT

Differentiated System Description Table

AML

SSDT

Secondary System Description Table

AML

STAO

Status Override Table

AML

Background

At LCU13 we had a session (http://lcu-13.zerista.com/event/member/85124) about ACPI on Xen. This led to a series of questions, which was discussed in a lengthy e-mail thread between the Xen developers and the ACPI experts (*).

The full minutes of the discussion are available here: https://docs.google.com/document/d/1D20tOOQxtNDZc-WsI34Ke_1E9NhGocFMSOlCzV_mF-0/edit?pli=1

In the following I try to capture the main question points and the answers about how to to deal with the unique ACPI on Xen challenges.

Static Requirements

To boot Xen on ACPI ARM, the following things are required to be statically available (no use of AML):

  • CPUs
  • Memory
  • GICs, including their MMIO regions and their interrupts
  • arch_timers, including their MMIO regions and their interrupts
  • UARTs, including their MMIO regions and interrupts
  • IOMMUs, including their MMIO regions and device/IOMMU topology
  • PSCI must be adopted for CPU power management (device power management is OK in AML)
  • Xen must be able to add a "blacklist table" to prevent Dom0 from using at least the UART and the IOMMUs

CPUs

As of the 5.0A specification, CPUs are not described in a C-like struct; they are described as Devices in the ACPI namespace which requires execution of the ACPI interpreter.

However, LEG has been working with the ASWG (ACPI Spec Working Group, if the acronym is unfamiliar) and there are several from Linaro and elsewhere trying to make sure that at least the initial CPUs available are described in a C-like struct (aka, a static ACPI table).

Memory

ACPI does not describe memory, UEFI describes memory, which does not require ACPI.

GICs

See the MADT table. Section 5.2.12. The GIC is described in section 5.2.12.14, page 145. The mapping of GIC interrupts to the global interrupt number space is defined in the GIC distributor structure. Section 5.2.12.14

GICV, GICH are not supported in ACPI 5.0 spec, the ACPI proposal for GICv3 may cover it.

arch_timers

Timers can be initialized by GTDT table. Section 5.2.24. RFC patch for upstream: https://lkml.org/lkml/2013/12/3/889

UARTs

For early prints from Xen, SPCR table is used for describing UART information.

IOMMUs

Not described properly in ACPI yet, but something for SMMUs is definitely needed. We need to check on the CSRT, in the meantime; it can possibly be bent to do what is needed and would be statically available.

Stefano asked an open question with respect to device/IOMMU topology:

The one thing that concerns me the most, whether we describe SMMUs in the CSRT or in another table, is how do we list the devices and the corresponding stream IDs that can be programmed into a specific SMMU. For PCI devices things are simple because we could simply list the BDFs. For other devices, I don't know. Given that the full list of devices is only available as part of the ACPI namespace AFAICT, resorting to ACPI namespace references would work but would defeat the purpose of having the SMMU available early, before any AML has been interpreted.

Is there another way to reference a non-PCI device that doesn't involve the ACPI namespace? Otherwise, can we simply dismiss non-PCI devices?

PSCI

Presence of PSCI is described in FADT table.Only v0.2 is supported for which calls can be inferred from PSCI specs.

Blacklist table

(This approach is no longer applicable see Status override table section below).

The blacklist table is the idea of Xen adding an ACPI table which "hides" certain devices or system properties from Dom0. For example, even though the UART is described in the DSDT, and the DSDT is passed unmodified to Dom0, Dom0 should not attempt to use the UART because Xen is using it. Attempts by Dom0 to actually access the UART anyway is not a security concern, because Xen will not map the UART MMIO regions in the Stage 2 mappings. The GIC's virtualization extensions is another example.

Minimum requirements:

  • Change number of cores (DT?)
  • Change amount of memory (DT?)
  • Change memory map (?)
  • Hide GIC virt. support
  • Hide IOMMU (or certain features of stage-2 IOMMUs)
  • Hide the UART

Should be possible to implement now, but someone must actually look at implementation before proposing something to the UEFI Forum / ACPI Working Group.

It may not be essential that this get standardized through the working group; to avoid collisions with other tables, we may want to reserve the table name (the four-char tag like "MADT" or "CSRT"). Just the same, we could work with this table as a Xen only table. We will need to have some priorities shift to spend time on this table, but the work needed looks something like this:

  1. define the table structure (essentially, a list of names), but in ACPI form.
  2. modify iasl (the ASL compiler) so it knows how to build such a table -- this may have to be carried as a patch in the ACPICA tools for Linux until the table gets into the standard.
  3. make sure the kernel loads and exposes the table properly.
  4. modify the ACPI namespace lookup to ignore items in the blacklist.

Status Override Table

In this approach, a status method "_STA" is added to the devices that need to be hidden from Dom0 in DSDT, and the added method returns 0 to disable ACPI probing of devices used by Xen and hidden from Dom0.

Each device in a DSDT has a _STA method which is created behind the scenes by the core if it is not explicitly defined by the DSDT itself. If not defined the default is to return 1 always. The purpose of the _STA is that DSDT can do things like probe for IDs in memory to detect if a device is actually there. If a 0 is returned ACPI will not attempt to probe device or make any use of the device.

A separate Status override table(STAO) is defined which contains the path of device and special one byte "uart" field to indicate the presence/absence of SPCR table. In case uart field is present DOM0 reads SPCR table and adds _STA method to the DSDT device describing UART, thereby disabling the UART. For other devices path needs to be described in STAO table. DOM0 then sets _STA to 0 to disable corresponding devices from DSDT. Over the blacklist proposal this has the advantage of not creating "holes" in the DSDT tree which may lead to strange side effects in ACPICA core. Work needed to accomplish this would be something like this.

  1. Define the table structure (essentially, a list of names), but in ACPI form. b. Modify XSDT to add entry corresponding to the table. c. Make sure the kernel loads and exposes the table properly.

Dom0 and ACPI

The central question is whether ACPI is passed on to Dom0 or if Xen simply generates an FDT for Dom0. The conclusion is that Dom0 will need access to the hardware ACPI, because:

  • not pasing ACPI requires parsing everything from ACPI in Xen
  • the translation is not well-defined
  • no way to handle AML operations with DT
  • device power management needs to be done by Dom0, and that is going to be through ACPI
  • not all the Linux (Dom0) drivers might be tested with device tree going forward.

Dom0 needs to see a modified view of the original hardware, but that is addressed with blacklisting, see above.

Boot protocol for Dom0

Some questions in this area were raised:

  • How does the kernel get to the ACPI data:
    • The UEFI system table is passed to the kernel at boot time. The system table is already mapped into memory. The kernel obtains the system table pointer by reading a property from the DT. (A device tree structure is always passed to the kernel, even if it is mostly empty)
    • More info: UEFI builds a "System Table", which is essentially the same for all UEFI implementations. In that table, there is a pointer to a struct that is an ACPI table called the RSDP. That pointer in the System Table is all ACPI needs; the RSDP contains enough info to find all of the other tables. It's all just pointers and structs at this point. The tables themselves are mapped into memory by UEFI and that memory map is passed on to the kernel.
  • Could Xen execute AML code (potentially in an AML interpreter VM) and then pass on ACPI to Dom0:
    • Absolutely not, AML may modify itself, there's no locking mechanisms etc. It's either pass it all to Dom0 or pass static DSDT/SSDT to Dom0, separate from the real hardware ones.
  • Windows as Dom0
    • While Windows as DomU's is something we may care about in the short term, we are not considering Windows for Dom0 in the short term, and must deal with that later.

Xen node

For Xen node a seperate table is defined ("XENV") aka Xen Environment table, this table will contain the address and size of grant table for sharing memory between Xen and DOM0 and event channel interrupt.DOM0 will configure grant table information and event interrupt based on information provided in this table.

Notes

(*) Specifically everyone on the mail thread were: Christoffer Dall, Al Stone, Grant Likely, Leif Lindholm, Stefano Stabellini, Ian Campbell, Julien Grall, Mark Rutland, and Andrea Gallo).

(**) RDSP discovery in UEFI, from ACPI 5.0 spec: "In Unified Extensible Firmware Interface (UEFI) enabled systems, a pointer to the RSDP structure exists within the EFI System Table"

References

LEG/Engineering/Virtualization/ACPI_on_Xen (last modified 2014-12-12 14:06:27)