Changes in Kconfig

858ce2f5 - cxl: Add FWCTL support to CXL

Fri, 07 Mar 2025 20:55:31 +0000

cxl: Add FWCTL support to CXLAdd fwctl support code to allow sending of CXL feature commands fromuserspace through as ioctls via FWCTL. Provide initial setup bits. TheCXL PCI probe function will call devm_cxl_setup_fwctl() after thecxl_memdev has been enumerated in order to setup FWCTL char device underthe cxl_memdev like the existing memdev char device for issuing CXL rawmailbox commands from userspace via ioctls.Link: https://patch.msgid.link/r/20250307205648.1021626-2-dave.jiang@intel.comSigned-off-by: Dave Jiang Reviewed-by: Li Ming Reviewed-by: Jonathan Cameron Signed-off-by: Jason Gunthorpe List of files: /linux-6.15/drivers/cxl/Kconfig

516e5bd0 - cxl: Add mce notifier to emit aliased address for extended linear cache

Wed, 26 Feb 2025 16:21:21 +0000

cxl: Add mce notifier to emit aliased address for extended linear cacheBelow is a setup with extended linear cache configuration with an examplelayout of memory region shown below presented as a single memory regionconsists of 256G memory where there's 128G of DRAM and 128G of CXL memory.The kernel sees a region of total 256G of system memory. 128G DRAM 128G CXL memory|-----------------------------------|-------------------------------------|Data resides in either DRAM or far memory (FM) with no replication. Hotdata is swapped into DRAM by the hardware behind the scenes. When error isdetected in one location, it is possible that error also resides in thealiased location. Therefore when a memory location that is flagged by MCEis part of the special region, the aliased memory location needs to beofflined as well.Add an mce notify callback to identify if the MCE address location is partof an extended linear cache region and handle accordingly.Added symbol export to set_mce_nospec() in x86 code in order to callset_mce_nospec() from the CXL MCE notify callback.Link: https://lore.kernel.org/linux-cxl/668333b17e4b2_5639294fd@dwillia2-xfh.jf.intel.com.notmuch/Reviewed-by: Jonathan Cameron Reviewed-by: Li Ming Reviewed-by: Alison Schofield Link: https://patch.msgid.link/20250226162224.3633792-5-dave.jiang@intel.comSigned-off-by: Dave Jiang List of files: /linux-6.15/drivers/cxl/Kconfig

f0e6a232 - cxl: Add Get Supported Features command for kernel usage

Thu, 20 Feb 2025 19:42:40 +0000

cxl: Add Get Supported Features command for kernel usageCXL spec r3.2 8.2.9.6.1 Get Supported Features (Opcode 0500h)The command retrieve the list of supported device-specific features(identified by UUID) and general information about each Feature.The driver will retrieve the Feature entries in order to make checks andprovide information for the Get Feature and Set Feature command. One ofthe main piece of information retrieved are the effects a Set Featurecommand would have for a particular feature. The retrieved Featureentries are stored in the cxl_mailbox context.The setup of Features is initiated via devm_cxl_setup_features() during thepci probe function before the cxl_memdev is enumerated.Reviewed-by: Jonathan Cameron Reviewed-by: Dan Williams Reviewed-by: Li Ming Reviewed-by: Davidlohr Bueso Tested-by: Shiju Jose Link: https://patch.msgid.link/20250220194438.2281088-3-dave.jiang@intel.comSigned-off-by: Dave Jiang List of files: /linux-6.15/drivers/cxl/Kconfig

6575b268 - cxl/port: Fix CXL port initialization order when the subsystem is built-in

Fri, 25 Oct 2024 19:32:55 +0000

cxl/port: Fix CXL port initialization order when the subsystem is built-inWhen the CXL subsystem is built-in the module init order is determinedby Makefile order. That order violates expectations. The expectation isthat cxl_acpi and cxl_mem can race to attach. If cxl_acpi wins the race,cxl_mem will find the enabled CXL root ports it needs. If cxl_acpi losesthe race it will retrigger cxl_mem to attach via cxl_bus_rescan(). Thatflow only works if cxl_acpi can assume ports are enabled immediatelyupon cxl_acpi_probe() return. That in turn can only happen in theCONFIG_CXL_ACPI=y case if the cxl_port driver is registered beforecxl_acpi_probe() runs.Fix up the order to prevent initialization failures. Ensure thatcxl_port is built-in when cxl_acpi is also built-in, arrange forMakefile order to resolve the subsys_initcall() order of cxl_port andcxl_acpi, and arrange for Makefile order to resolve thedevice_initcall() (module_init()) order of the remaining objects.As for what contributed to this not being found earlier, the CXLregression environment, cxl_test, builds all CXL functionality as amodule to allow to symbol mocking and other dynamic reload tests. As aresult there is no regression coverage for the built-in case.Reported-by: Gregory Price Closes: http://lore.kernel.org/20241004212504.1246-1-gourry@gourry.netTested-by: Gregory Price Fixes: 8dd2bc0f8e02 ("cxl/mem: Add the cxl_mem driver")Cc: stable@vger.kernel.orgCc: Davidlohr Bueso Cc: Jonathan Cameron Cc: Dave Jiang Cc: Alison Schofield Cc: Vishal Verma Cc: Ira Weiny Reviewed-by: Jonathan Cameron Reviewed-by: Ira Weiny Tested-by: Alejandro Lucero Reviewed-by: Alejandro Lucero Signed-off-by: Dan Williams Link: https://patch.msgid.link/172988474904.476062.7961350937442459266.stgit@dwillia2-xfh.jf.intel.comSigned-off-by: Ira Weiny List of files: /linux-6.15/drivers/cxl/Kconfig

1b5695b0 - mm: make range-to-target_node lookup facility a part of numa_memblks

Wed, 07 Aug 2024 06:41:09 +0000

mm: make range-to-target_node lookup facility a part of numa_memblksThe x86 implementation of range-to-target_node lookup (i.e. phys_to_target_node() and memory_add_physaddr_to_nid()) relies onnuma_memblks.Since numa_memblks are now part of the generic code, move these functionsfrom x86 to mm/numa_memblks.c and select CONFIG_NUMA_KEEP_MEMINFO whenCONFIG_NUMA_MEMBLKS=y for dax and cxl.[rppt@kernel.org: fix build] Link: https://lkml.kernel.org/r/ZtVfSt_zloPdDqVB@kernel.orgLink: https://lkml.kernel.org/r/20240807064110.1003856-26-rppt@kernel.orgSigned-off-by: Mike Rapoport (Microsoft) Reviewed-by: Jonathan Cameron Tested-by: Zi Yan # for x86_64 and arm64Tested-by: Jonathan Cameron [arm64 + CXL via QEMU]Reviewed-by: Dan Williams Acked-by: David Hildenbrand Cc: Alexander Gordeev Cc: Andreas Larsson Cc: Arnd Bergmann Cc: Borislav Petkov Cc: Catalin Marinas Cc: Christophe Leroy Cc: Dave Hansen Cc: Davidlohr Bueso Cc: David S. Miller Cc: Greg Kroah-Hartman Cc: Heiko Carstens Cc: Huacai Chen Cc: Ingo Molnar Cc: Jiaxun Yang Cc: John Paul Adrian Glaubitz Cc: Jonathan Corbet Cc: Michael Ellerman Cc: Palmer Dabbelt Cc: Rafael J. Wysocki Cc: Rob Herring (Arm) Cc: Samuel Holland Cc: Thomas Bogendoerfer Cc: Thomas Gleixner Cc: Vasily Gorbik Cc: Will Deacon Signed-off-by: Andrew Morton List of files: /linux-6.15/drivers/cxl/Kconfig

4cce9c6d - cxl: Fix use of phys_to_target_node() for x86

Wed, 24 Apr 2024 15:47:56 +0000

cxl: Fix use of phys_to_target_node() for x86The CXL driver uses both functions phys_to_target_node() andmemory_add_physaddr_to_nid(). The x86 architecture relies on theNUMA_KEEP_MEMINFO kernel option enabled for both functions to workcorrect. Update Kconfig to make sure the option is always enabled forthe driver.Suggested-by: Dan Williams Link: http://lore.kernel.org/r/65f8b191c0422_aa222941b@dwillia2-mobl3.amr.corp.intel.com.notmuchReviewed-by: Ira Weiny Reviewed-by: Dan Williams Signed-off-by: Robert Richter Link: https://lore.kernel.org/r/20240424154756.2152614-1-rrichter@amd.comSigned-off-by: Dave Jiang List of files: /linux-6.15/drivers/cxl/Kconfig

a46aba14 - cxl: remove CONFIG_CXL_PMU entry in drivers/cxl/Kconfig

Sun, 04 Feb 2024 09:46:13 +0000

cxl: remove CONFIG_CXL_PMU entry in drivers/cxl/KconfigCommit 5d7107c72796 ("perf: CXL Performance Monitoring Unit driver")added the config entries for CXL_PMU in drivers/cxl/Kconfig anddrivers/perf/Kconfig, so it can be toggled from multiple locations:[1] Device Drivers -> PCI support -> CXL (Compute Expres Link) Devices -> CXL Performance Monitoring Unit[2] Device Drivers -> Performance monitor support -> CXL Performance Monitoring UnitThis complicates things, and nobody else does this.I kept the one in drivers/perf/Kconfig because CONFIG_CXL_PMU controlsthe compilation of drivers/perf/cxl_pmu.c.Acked-by: Davidlohr Bueso Reviewed-by: Dave Jiang Reviewed-by: Jonathan Cameron Signed-off-by: Masahiro Yamada List of files: /linux-6.15/drivers/cxl/Kconfig

ad6f04c0 - cxl: Add callback to parse the DSMAS subtables from CDAT

Thu, 21 Dec 2023 22:03:13 +0000

cxl: Add callback to parse the DSMAS subtables from CDATProvide a callback function to the CDAT parser in order to parse theDevice Scoped Memory Affinity Structure (DSMAS). Each DSMAS structurecontains the DPA range and its associated attributes in each entry. Seethe CDAT specification for details. The device handle and the DPA rangeis saved and to be associated with the DSLBIS locality data when theDSLBIS entries are parsed. The xarray is a local variable. When thetotal path performance data is calculated and storred this xarray can bediscarded.Coherent Device Attribute Table 1.03 2.1 Device Scoped memory AffinityStructure (DSMAS)Reviewed-by: Jonathan Cameron Signed-off-by: Dave Jiang Link: https://lore.kernel.org/r/170319619355.2212653.2675953129671561293.stgit@djiang5-mobl3Signed-off-by: Dan Williams List of files: /linux-6.15/drivers/cxl/Kconfig

9171dfcd - cxl: fix CONFIG_FW_LOADER dependency

Mon, 03 Jul 2023 11:29:13 +0000

cxl: fix CONFIG_FW_LOADER dependencyWhen FW_LOADER is disabled, cxl fails to link:arm-linux-gnueabi-ld: drivers/cxl/core/memdev.o: in function `cxl_memdev_setup_fw_upload':memdev.c:(.text+0x90e): undefined reference to `firmware_upload_register'memdev.c:(.text+0x93c): undefined reference to `firmware_upload_unregister'In order to use the firmware_upload_register() function, both FW_LOADERand FW_UPLOAD have to be enabled, which is a bit confusing. In addition,the dependency is on the wrong symbol, as the caller is part of thecxl_core.ko module, not the cxl_mem.ko module.Fixes: 9521875bbe005 ("cxl: add a firmware update mechanism using the sysfs firmware loader")Signed-off-by: Arnd Bergmann Link: https://lore.kernel.org/r/20230703112928.332321-1-arnd@kernel.orgReviewed-by: Xiao Yang Reviewed-by: Dave Jiang Signed-off-by: Vishal Verma List of files: /linux-6.15/drivers/cxl/Kconfig

5d7107c7 - perf: CXL Performance Monitoring Unit driver

Fri, 26 May 2023 09:58:23 +0000

perf: CXL Performance Monitoring Unit driverCXL rev 3.0 introduces a standard performance monitoring hardwareblock to CXL. Instances are discovered using CXL Register Locator DVSECentries. Each CXL component may have multiple PMUs.This initial driver supports a subset of types of counter.It supports counters that are either fixed or configurable, but requiresthat they support the ability to freeze and write value whilst frozen.Development done with QEMU model which will be posted shortly.Example:$ perf stat -a -e cxl_pmu_mem0.0/h2d_req_snpcur/ -e cxl_pmu_mem0.0/h2d_req_snpdata/ -e cxl_pmu_mem0.0/clock_ticks/ sleep 1Performance counter stats for 'system wide':96,757,023,244,321 cxl_pmu_mem0.0/h2d_req_snpcur/96,757,023,244,365 cxl_pmu_mem0.0/h2d_req_snpdata/193,514,046,488,653 cxl_pmu_mem0.0/clock_ticks/ 1.090539600 seconds time elapsedReviewed-by: Dave Jiang Reviewed-by: Kan Liang Signed-off-by: Jonathan Cameron Link: https://lore.kernel.org/r/20230526095824.16336-5-Jonathan.Cameron@huawei.comSigned-off-by: Dan Williams List of files: /linux-6.15/drivers/cxl/Kconfig

9521875b - cxl: add a firmware update mechanism using the sysfs firmware loader

Wed, 14 Jun 2023 17:17:40 +0000

cxl: add a firmware update mechanism using the sysfs firmware loaderThe sysfs based firmware loader mechanism was created to easily allowuserspace to upload firmware images to FPGA cards. This also happens tobe pretty suitable to create a user-initiated but kernel-controlledfirmware update mechanism for CXL devices, using the CXL specifiedmailbox commands.Since firmware update commands can be long-running, and can be processedin the background by the endpoint device, it is desirable to have theability to chunk the firmware transfer down to smaller pieces, so thatone operation does not monopolize the mailbox, locking out any otherlong running background commands entirely - e.g. security commands like'sanitize' or poison scanning operations.The firmware loader mechanism allows a natural way to perform thischunking, as after each mailbox command, that is restricted to themaximum mailbox payload size, the cxl memdev driver relinquishes controlback to the fw_loader system and awaits the next chunk of data totransfer. This opens opportunities for other background commands toaccess the mailbox and send their own slices of background commands.Add the necessary helpers and state tracking to be able to perform the'Get FW Info', 'Transfer FW', and 'Activate FW' mailbox commands asdescribed in the CXL spec. Wire these up to the firmware loadercallbacks, and register with that system to create the memX/firmware/sysfs ABI.Cc: Davidlohr Bueso Cc: Jonathan Cameron Cc: Russ Weight Cc: Alison Schofield Cc: Ira Weiny Cc: Dave Jiang Cc: Ben Widawsky Cc: Dan Williams Reviewed-by: Jonathan Cameron Reviewed-by: Dave Jiang Signed-off-by: Vishal Verma Link: https://lore.kernel.org/r/20230602-vv-fw_update-v4-1-c6265bd7343b@intel.comSigned-off-by: Dan Williams List of files: /linux-6.15/drivers/cxl/Kconfig

45d235c5 - cxl/region: Enable CONFIG_CXL_REGION to be toggled

Fri, 10 Feb 2023 09:06:27 +0000

cxl/region: Enable CONFIG_CXL_REGION to be toggledAdd help text and a label so the CXL_REGION config option can betoggled. This is mainly to enable compile testing without regionsupport.Reviewed-by: Vishal Verma Reviewed-by: Jonathan Cameron Reviewed-by: Dave Jiang Reviewed-by: Gregory Price Tested-by: Fan Ni Link: https://lore.kernel.org/r/167601998765.1924368.258370414771847699.stgit@dwillia2-xfh.jf.intel.comSigned-off-by: Dan Williams List of files: /linux-6.15/drivers/cxl/Kconfig

cbbd05d0 - cxl: fix spelling mistakes

Wed, 25 Jan 2023 03:22:21 +0000

cxl: fix spelling mistakesCorrect spelling mistakes (reported by codespell).Signed-off-by: Randy Dunlap Cc: Alison Schofield Cc: Vishal Verma Cc: Ira Weiny Cc: Ben Widawsky Cc: Dan Williams Cc: linux-cxl@vger.kernel.orgReviewed-by: Vishal Verma Reviewed-by: Alison Schofield Link: https://lore.kernel.org/r/20230125032221.21277-1-rdunlap@infradead.orgSigned-off-by: Dan Williams List of files: /linux-6.15/drivers/cxl/Kconfig

d18bc74a - cxl/region: Manage CPU caches relative to DPA invalidation events

Thu, 01 Dec 2022 22:03:41 +0000

cxl/region: Manage CPU caches relative to DPA invalidation eventsA "DPA invalidation event" is any scenario where the contents of a DPA(Device Physical Address) is modified in a way that is incoherent withCPU caches, or if the HPA (Host Physical Address) to DPA associationchanges due to a remapping event.PMEM security events like Unlock and Passphrase Secure Erase alreadymanage caches through LIBNVDIMM, so that leaves HPA to DPA remap eventsthat need cache management by the CXL core. Those only happen when theboot time CXL configuration has changed. That event occurs whenuserspace attaches an endpoint decoder to a region configuration, andthat region is subsequently activated.The implications of not invalidating caches between remap events is thatreads from the region at different points in time may return differentresults due to stale cached data from the previous HPA to DPA mapping.Without a guarantee that the region contents after cxl_region_probe()are written before being read (a layering-violation assumption thatcxl_region_probe() can not make) the CXL subsystem needs to ensure thatreads that precede writes see consistent results.A CONFIG_CXL_REGION_INVALIDATION_TEST option is added to support debugand unit testing of the CXL implementation in QEMU or other environmentswhere cpu_cache_has_invalidate_memregion() returns false. This may provetoo restrictive for QEMU where the HDM decoders are emulated, but inthat case the CXL subsystem needs some new mechanism / indication thatthe HDM decoder is emulated and not a passthrough of real hardware.Reviewed-by: Dave Jiang Link: https://lore.kernel.org/r/166993222098.1995348.16604163596374520890.stgit@dwillia2-xfh.jf.intel.comSigned-off-by: Dan Williams List of files: /linux-6.15/drivers/cxl/Kconfig

23a22cd1 - cxl/region: Allocate HPA capacity to regions

Mon, 25 Apr 2022 18:43:44 +0000

cxl/region: Allocate HPA capacity to regionsAfter a region's interleave parameters (ways and granularity) are set,add a way for regions to allocate HPA (host physical address space) fromthe free capacity in their parent root-decoder. The allocator for thiscapacity reuses the 'struct resource' based allocator used forCONFIG_DEVICE_PRIVATE.Once the tuple of "ways, granularity, [uuid], and size" is set theregion configuration transitions to the CXL_CONFIG_INTERLEAVE_ACTIVEstate which is a precursor to allowing endpoint decoders to be added toa region.Co-developed-by: Ben Widawsky Signed-off-by: Ben Widawsky Reviewed-by: Jonathan Cameron Link: https://lore.kernel.org/r/165784335630.1758207.420216490941955417.stgit@dwillia2-xfh.jf.intel.comSigned-off-by: Dan Williams List of files: /linux-6.15/drivers/cxl/Kconfig

779dd20c - cxl/region: Add region creation support

Tue, 08 Jun 2021 17:28:34 +0000

cxl/region: Add region creation supportCXL 2.0 allows for dynamic provisioning of new memory regions (systemphysical address resources like "System RAM" and "Persistent Memory").Whereas DDR and PMEM resources are conveyed statically at boot, CXLallows for assembling and instantiating new regions from the availablecapacity of CXL memory expanders in the system.Sysfs with an "echo $region_name > $create_region_attribute" interfaceis chosen as the mechanism to initiate the provisioning process. Thiswas chosen over ioctl() and netlink() to keep the configurationinterface entirely in a pseudo-fs interface, and it was chosen overconfigfs since, aside from this one creation event, the interface isread-mostly. I.e. configfs supports cases where an object is designed tobe provisioned each boot, like an iSCSI storage target, and CXL regioncreation is mostly for PMEM regions which are created usually onceper-lifetime of a server instance. This is an improvement over nvdimmthat pre-created "seed" devices that tended to confuse users looking todetermine which devices are active and which are idle.Recall that the major change that CXL brings over previous persistentmemory architectures is the ability to dynamically define new regions.Compare that to drivers like 'nfit' where the region configuration isstatically defined by platform firmware.Regions are created as a child of a root decoder that encompasses anaddress space with constraints. When created through sysfs, the rootdecoder is explicit. When created from an LSA's region structure a rootdecoder will possibly need to be inferred by the driver.Upon region creation through sysfs, a vacant region is created with aunique name. Regions have a number of attributes that must be configuredbefore the region can be bound to the driver where HDM decoder programis completed.An example of creating a new region:- Allocate a new region name:region=$(cat /sys/bus/cxl/devices/decoder0.0/create_pmem_region)- Create a new region by name:whileregion=$(cat /sys/bus/cxl/devices/decoder0.0/create_pmem_region)! echo $region > /sys/bus/cxl/devices/decoder0.0/create_pmem_regiondo true; done- Region now exists in sysfs:stat -t /sys/bus/cxl/devices/decoder0.0/$region- Delete the region, and name:echo $region > /sys/bus/cxl/devices/decoder0.0/delete_regionSigned-off-by: Ben Widawsky Reviewed-by: Jonathan Cameron Link: https://lore.kernel.org/r/165784333909.1758207.794374602146306032.stgit@dwillia2-xfh.jf.intel.com[djbw: simplify locking, reword changelog]Signed-off-by: Dan Williams List of files: /linux-6.15/drivers/cxl/Kconfig

3eddcc93 - cxl/pci: Create PCI DOE mailbox's for memory devices

Tue, 19 Jul 2022 20:52:47 +0000

cxl/pci: Create PCI DOE mailbox's for memory devicesDOE mailbox objects will be needed for various mailbox communicationswith each memory device.Iterate each DOE mailbox capability and create PCI DOE mailbox objectsas found.It is not anticipated that this is the final resting place for theiteration of the DOE devices. The support of switch ports will drivethis code into the PCIe side. In this imagined architecture the CXLport driver would then query into the PCI device for the DOE mailboxarray.For now creating the mailboxes in the CXL port is good enough for theendpoints. Later PCIe ports will need to support this to support switchports more generically.Cc: Dan Williams Cc: Davidlohr Bueso Cc: Lukas Wunner Reviewed-by: Jonathan Cameron Signed-off-by: Ira Weiny Link: https://lore.kernel.org/r/20220719205249.566684-5-ira.weiny@intel.comSigned-off-by: Dan Williams List of files: /linux-6.15/drivers/cxl/Kconfig

9ea4dcf4 - PM: CXL: Disable suspend

Fri, 22 Apr 2022 22:58:11 +0000

PM: CXL: Disable suspendThe CXL specification claims S3 support at a hardware level, but at asystem software level there are some missing pieces. Section 9.4 (CXL2.0) rightly claims that "CXL mem adapters may need aux power to retainmemory context across S3", but there is no enumeration mechanism for theOS to determine if a given adapter has that support. Moreover the savestate and resume image for the system may inadvertantly end up in a CXLdevice that needs to be restored before the save state is recoverable.I.e. a circular dependency that is not resolvable without a third partysave-area.Arrange for the cxl_mem driver to fail S3 attempts. This still nominalyallows for suspend, but requires unbinding all CXL memory devices beforethe suspend to ensure the typical DRAM flow is taken. The cxl_mem unbindflow is intended to also tear down all CXL memory regions associatedwith a given cxl_memdev.It is reasonable to assume that any device participating in a System RAMrange published in the EFI memory map is covered by aux power andsave-area outside the device itself. So this restriction can beminimized in the future once pre-existing region enumeration supportarrives, and perhaps a spec update to clarify if the EFI memory map issufficent for determining the range of devices managed byplatform-firmware for S3 support.Per Rafael, if the CXL configuration prevents suspend then it shouldfail early before tasks are frozen, and mem_sleep should stop showing'mem' as an option [1]. Effectively CXL augments the platform suspend->valid() op since, for example, the ACPI ops are not aware of the CXL /PCI dependencies. Given the split role of platform firmware vs OSprovisioned CXL memory it is up to the cxl_mem driver to determine ifthe CXL configuration has elements that platform firmware may not beprepared to restore.Link: https://lore.kernel.org/r/CAJZ5v0hGVN_=3iU8OLpHY3Ak35T5+JcBM-qs8SbojKrpd0VXsA@mail.gmail.com [1]Cc: "Rafael J. Wysocki" Cc: Pavel Machek Cc: Len Brown Reviewed-by: Rafael J. Wysocki Link: https://lore.kernel.org/r/165066828317.3907920.5690432272182042556.stgit@dwillia2-desk3.amr.corp.intel.comSigned-off-by: Dan Williams List of files: /linux-6.15/drivers/cxl/Kconfig

8dd2bc0f - cxl/mem: Add the cxl_mem driver

Fri, 04 Feb 2022 15:18:31 +0000

cxl/mem: Add the cxl_mem driverAt this point the subsystem can enumerate all CXL ports (CXL.mem decoderesources in upstream switch ports and host bridges) in a system. Thelast mile is connecting those ports to endpoints.The cxl_mem driver connects an endpoint device to the platform CXL.memprotoctol decode-topology. At ->probe() time it walks itsdevice-topology-ancestry and adds a CXL Port object at every UpstreamPort hop until it gets to CXL root. The CXL root object is only presentafter a platform firmware driver registers platform CXL resources. ForACPI based platform this is managed by the ACPI0017 device and thecxl_acpi driver.The ports are registered such that disabling a given port automaticallyunregisters all descendant ports, and the chain can only be registeredafter the root is established.Given ACPI device scanning may run asynchronously compared to PCI devicescanning the root driver is tasked with rescanning the bus after theroot successfully probes.Conversely if any ports in a chain between the root and an endpointbecomes disconnected it subsequently triggers the endpoint tounregister. Given lock depenedencies the endpoint unregistration happensin a workqueue asynchronously. If userspace cares about synchronizingdelayed work after port events the /sys/bus/cxl/flush attribute isavailable for that purpose.Reported-by: Randy Dunlap Signed-off-by: Ben Widawsky Reviewed-by: Jonathan Cameron [djbw: clarify changelog, rework hotplug support]Link: https://lore.kernel.org/r/164398782997.903003.9725273241627693186.stgit@dwillia2-desk3.amr.corp.intel.comSigned-off-by: Dan Williams List of files: /linux-6.15/drivers/cxl/Kconfig

54cdbf84 - cxl/port: Add a driver for 'struct cxl_port' objects

Tue, 01 Feb 2022 21:07:51 +0000

cxl/port: Add a driver for 'struct cxl_port' objectsThe need for a CXL port driver and a dedicated cxl_bus_type is driven bya need to simultaneously support 2 independent physical memory decodedomains (cache coherent CXL.mem and uncached PCI.mmio) that alsointersect at a single PCIe device node. A CXL Port is a device thatadvertises a CXL Component Register block with an "HDM DecoderCapability Structure".>From Documentation/driver-api/cxl/memory-devices.rst: Similar to how a RAID driver takes disk objects and assembles them into a new logical device, the CXL subsystem is tasked to take PCIe and ACPI objects and assemble them into a CXL.mem decode topology. The need for runtime configuration of the CXL.mem topology is also similar to RAID in that different environments with the same hardware configuration may decide to assemble the topology in contrasting ways. One may choose performance (RAID0) striping memory across multiple Host Bridges and endpoints while another may opt for fault tolerance and disable any striping in the CXL.mem topology.The port driver identifies whether an endpoint Memory Expander isconnected to a CXL topology. If an active (bound to the 'cxl_port'driver) CXL Port is not found at every PCIe Switch Upstream port and anactive "root" CXL Port then the device is just a plain PCIe endpointonly capable of participating in PCI.mmio and DMA cycles, not CXL.memcoherent interleave sets.The 'cxl_port' driver lets the CXL subsystem leverage driver-coreinfrastructure for setup and teardown of register resources andcommunicating device activation status to userspace. The cxl_bus_typecan rendezvous the async arrival of platform level CXL resources (viathe 'cxl_acpi' driver) with the asynchronous enumeration of MemoryExpander endpoints, while also implementing a hierarchical locking modelindependent of the associated 'struct pci_dev' locking model. Thelocking for dport and decoder enumeration is now handled in the corerather than callers.For now the port driver only enumerates and registers CXL resources(downstream port metadata and decoder resources) later it will be usedto take action on its decoders in response to CXL.mem regionprovisioning requests.Note1: cxlpci.h has long depended on pci.h, but port.c was the first tonot include pci.h. Carry that dependency in cxlpci.h.Note2: cxl port enumeration and probing complicates CXL subsystem initto the point that it helps to have centralized debug logging of probeevents in cxl_bus_probe().Reported-by: kernel test robot Signed-off-by: Ben Widawsky Reviewed-by: Jonathan Cameron Co-developed-by: Dan Williams Link: https://lore.kernel.org/r/164374948116.464348.1772618057599155408.stgit@dwillia2-desk3.amr.corp.intel.comSigned-off-by: Dan Williams List of files: /linux-6.15/drivers/cxl/Kconfig