vmd(4): Major driver refactoring

- Re-implement pcib interface to use standard pci bus driver on top of
vmd(4) instead of custom one.
- Re-implement memory/bus resource allocation to properly hand

vmd(4): Major driver refactoring

- Re-implement pcib interface to use standard pci bus driver on top of
vmd(4) instead of custom one.
- Re-implement memory/bus resource allocation to properly handle even
complicated configurations.
- Re-implement interrupt handling to evenly distribute children's MSI/
MSI-X interrupts between available vmd(4) MSI-X vectors and setup them
to be handled by standard OS mechanisms with minimal overhead, except
sharing when unavoidable.

Successfully tested on Dell XPS 13 laptop with Core i7-1185G7 CPU (VMD
device ID 0x9a0b) and single NVMe SSD, dual-booting with Windows 10.

Successfully tested on Supermicro X11DPI-NT motherboard with Xeon(R)
Gold 6242R CPUs (VMD device ID 0x201d), simultaneously handling NVMe
SSD on one PCIe port and PLX bridge with 3 NVMe and 1 AHCI SSDs on
another. Handles SSD hot-plug (except Optane 905p for some reason,
which are not detected until manual bus rescan) and enabled IOMMU
(directly connected SSDs work, but ones connected to the PLX fail
without errors from IOMMU).

MFC after: 2 weeks
Sponsored by: iXsystems, Inc.
Differential revision: https://reviews.freebsd.org/D31762

(cherry picked from commit 7af4475a6e31202a865b1dd3727018659b44470f)

show more ...