701ab935 - vfio/nvgrace-gpu: Add vfio pci variant module for grace hopper

Tue, 20 Feb 2024 11:50:55 +0000

vfio/nvgrace-gpu: Add vfio pci variant module for grace hopperNVIDIA's upcoming Grace Hopper Superchip provides a PCI-like devicefor the on-chip GPU that is the logical OS representation of theinternal proprietary chip-to-chip cache coherent interconnect.The device is peculiar compared to a real PCI device in that whilstthere is a real 64b PCI BAR1 (comprising region 2 & region 3) on thedevice, it is not used to access device memory once the fasterchip-to-chip interconnect is initialized (occurs at the time of hostsystem boot). The device memory is accessed instead using the chip-to-chipinterconnect that is exposed as a contiguous physically addressableregion on the host. This device memory aperture can be obtained from hostACPI table using device_property_read_u64(), according to the FWspecification. Since the device memory is cache coherent with the CPU,it can be mmap into the user VMA with a cacheable mapping usingremap_pfn_range() and used like a regular RAM. The device memoryis not added to the host kernel, but mapped directly as this reducesmemory wastage due to struct pages.There is also a requirement of a minimum reserved 1G uncached region(termed as resmem) to support the Multi-Instance GPU (MIG) feature [1].This is to work around a HW defect. Based on [2], the requisite properties(uncached, unaligned access) can be achieved through a VM mapping (S1)of NORMAL_NC and host (S2) mapping with MemAttr[2:0]=0b101. To providea different non-cached property to the reserved 1G region, it needs tobe carved out from the device memory and mapped as a separate regionin Qemu VMA with pgprot_writecombine(). pgprot_writecombine() sets theQemu VMA page properties (pgprot) as NORMAL_NC.Provide a VFIO PCI variant driver that adapts the unique device memoryrepresentation into a more standard PCI representation facing userspace.The variant driver exposes these two regions - the non-cached reserved(resmem) and the cached rest of the device memory (termed as usemem) asseparate VFIO 64b BAR regions. This is divergent from the baremetalapproach, where the device memory is exposed as a device memory region.The decision for a different approach was taken in view of the fact thatit would necessiate additional code in Qemu to discover and insert thoseregions in the VM IPA, along with the additional VM ACPI DSDT changes tocommunicate the device memory region IPA to the VM workloads. Moreover,this behavior would have to be added to a variety of emulators (beyondtop of tree Qemu) out there desiring grace hopper support.Since the device implements 64-bit BAR0, the VFIO PCI variant drivermaps the uncached carved out region to the next available PCI BAR (i.e.comprising of region 2 and 3). The cached device memory aperture isassigned BAR region 4 and 5. Qemu will then naturally generate a PCIdevice in the VM with the uncached aperture reported as BAR2 region,the cacheable as BAR4. The variant driver provides emulation for thesefake BARs' PCI config space offset registers.The hardware ensures that the system does not crash when the memoryis accessed with the memory enable turned off. It synthesis ~0 readsand dropped writes on such access. So there is no need to support thedisablement/enablement of BAR through PCI_COMMAND config space register.The memory layout on the host looks like the following: devmem (memlength)|--------------------------------------------------||-------------cached------------------------|--NC--|| |usemem.memphys resmem.memphysPCI BARs need to be aligned to the power-of-2, but the actual memory on thedevice may not. A read or write access to the physical address from thelast device PFN up to the next power-of-2 aligned physical addressresults in reading ~0 and dropped writes. Note that the GPU devicedriver [6] is capable of knowing the exact device memory size throughseparate means. The device memory size is primarily kept in the systemACPI tables for use by the VFIO PCI variant module.Note that the usemem memory is added by the VM Nvidia device driver [5]to the VM kernel as memblocks. Hence make the usable memory size memblock(MEMBLK_SIZE) aligned. This is a hardwired ABI value between the GPU FW andVFIO driver. The VM device driver make use of the same value for itscalculation to determine USEMEM size.Currently there is no provision in KVM for a S2 mapping withMemAttr[2:0]=0b101, but there is an ongoing effort to provide the same [3].As previously mentioned, resmem is mapped pgprot_writecombine(), thatsets the Qemu VMA page properties (pgprot) as NORMAL_NC. Using theproposed changes in [3] and [4], KVM marks the region withMemAttr[2:0]=0b101 in S2.If the device memory properties are not present, the driver registers thevfio-pci-core function pointers. Since there are no ACPI memory propertiesgenerated for the VM, the variant driver inside the VM will only usethe vfio-pci-core ops and hence try to map the BARs as non cached. Thisis not a problem as the CPUs have FWB enabled which blocks the VMmapping's ability to override the cacheability set by the host mapping.This goes along with a qemu series [6] to provides the necessaryimplementation of the Grace Hopper Superchip firmware specification sothat the guest operating system can see the correct ACPI modeling forthe coherent GPU device. Verified with the CUDA workload in the VM.[1] https://www.nvidia.com/en-in/technologies/multi-instance-gpu/[2] section D8.5.5 of https://developer.arm.com/documentation/ddi0487/latest/[3] https://lore.kernel.org/all/20240211174705.31992-1-ankita@nvidia.com/[4] https://lore.kernel.org/all/20230907181459.18145-2-ankita@nvidia.com/[5] https://github.com/NVIDIA/open-gpu-kernel-modules[6] https://lore.kernel.org/all/20231203060245.31593-1-ankita@nvidia.com/Reviewed-by: Kevin Tian Reviewed-by: Yishai Hadas Reviewed-by: Zhi Wang Signed-off-by: Aniket Agashe Signed-off-by: Ankit Agrawal Link: https://lore.kernel.org/r/20240220115055.23546-4-ankita@nvidia.comSigned-off-by: Alex Williamson List of files: /linux-6.15/drivers/vfio/pci/nvgrace-gpu/Makefile

Changes in Makefile

701ab935 - vfio/nvgrace-gpu: Add vfio pci variant module for grace hopper