cxl/Documentation: Remove 'mixed' from sysfs mode doc

Commit 188e9529a606 ("cxl: Remove the CXL_DECODER_MIXED mistake")
removed the mixed mode.

Remove it from the sysfs documentation.

Cc: Dan Will

cxl/Documentation: Remove 'mixed' from sysfs mode doc

Commit 188e9529a606 ("cxl: Remove the CXL_DECODER_MIXED mistake")
removed the mixed mode.

Remove it from the sysfs documentation.

Cc: Dan Williams <[email protected]>
Signed-off-by: Ira Weiny <[email protected]>
Reviewed-by: Jonathan Cameron <[email protected]>
Reviewed-by: Dave Jiang <[email protected]>
Link: https://patch.msgid.link/[email protected]
Signed-off-by: Dave Jiang <[email protected]>

show more ...

cxl/pmem: Export dirty shutdown count via sysfs

Similar to how the acpi_nfit driver exports Optane dirty shutdown count,
introduce:

/sys/bus/cxl/devices/nvdimm-bridge0/ndbusX/nmemY/cxl/dirty_shut

cxl/pmem: Export dirty shutdown count via sysfs

Similar to how the acpi_nfit driver exports Optane dirty shutdown count,
introduce:

/sys/bus/cxl/devices/nvdimm-bridge0/ndbusX/nmemY/cxl/dirty_shutdown

Under the conditions that 1) dirty shutdown can be set, 2) Device GPF
DVSEC exists, and 3) the count itself can be retrieved.

Suggested-by: Dan Williams <[email protected]>
Signed-off-by: Davidlohr Bueso <[email protected]>
Reviewed-by: Dave Jiang <[email protected]>
Reviewed-by: Ira Weiny <[email protected]>
Reviewed-by: Jonathan Cameron <[email protected]>
Link: https://patch.msgid.link/[email protected]
Signed-off-by: Dave Jiang <[email protected]>

show more ...

cxl: Document missing sysfs files

Add to the ABI documentation the payload_max and label_storage_size
read-only files, which have been there since the early days.

Signed-off-by: Davidlohr Bueso <da

cxl: Document missing sysfs files

Add to the ABI documentation the payload_max and label_storage_size
read-only files, which have been there since the early days.

Signed-off-by: Davidlohr Bueso <[email protected]>
Reviewed-by: Li Ming <[email protected]>
Reviewed-by: Dave Jiang <[email protected]>
Reviewed-by: Ira Weiny <[email protected]>
Link: https://patch.msgid.link/[email protected]
Signed-off-by: Dave Jiang <[email protected]>

show more ...

cxl: Plug typos in ABI doc

Trivially update where necessary.

Reviewed-by: Jonathan Cameron <[email protected]>
Signed-off-by: Davidlohr Bueso <[email protected]>
Reviewed-by: Li Ming <min

cxl: Plug typos in ABI doc

Trivially update where necessary.

Reviewed-by: Jonathan Cameron <[email protected]>
Signed-off-by: Davidlohr Bueso <[email protected]>
Reviewed-by: Li Ming <[email protected]>
Reviewed-by: Dave Jiang <[email protected]>
Reviewed-by: Ira Weiny <[email protected]>
Link: https://patch.msgid.link/[email protected]
Signed-off-by: Dave Jiang <[email protected]>

show more ...

cxl/region: Add sysfs attribute for locality attributes of CXL regions

Add read/write latencies and bandwidth sysfs attributes for the enabled CXL
region. The bandwidth is the aggregated bandwidth o

cxl/region: Add sysfs attribute for locality attributes of CXL regions

Add read/write latencies and bandwidth sysfs attributes for the enabled CXL
region. The bandwidth is the aggregated bandwidth of all devices that
contribute to the CXL region. The latency is the worst latency of the
device amongst all the devices that contribute to the CXL region.

Reviewed-by: Jonathan Cameron <[email protected]>
Tested-by: Jonathan Cameron <[email protected]>
Signed-off-by: Dave Jiang <[email protected]>
Link: https://lore.kernel.org/r/[email protected]
Signed-off-by: Dan Williams <[email protected]>

show more ...

cxl: Export sysfs attributes for memory device QoS class

Export qos_class sysfs attributes for the CXL memory device. The QoS clas
should show up as /sys/bus/cxl/devices/memX/ram/qos_class for the v

cxl: Export sysfs attributes for memory device QoS class

Export qos_class sysfs attributes for the CXL memory device. The QoS clas
should show up as /sys/bus/cxl/devices/memX/ram/qos_class for the volatile
partition and /sys/bus/cxl/devices/memX/pmem/qos_class for the persistent
partition. The QTG ID is retrieved via _DSM after supplying the
calculated bandwidth and latency for the entire CXL path from device to
the CPU. This ID is used to match up to the root decoder QoS class to
determine which CFMWS the memory range of a hotplugged CXL mem device
should be assigned under.

While there may be multiple DSMAS exported by the device CDAT, the driver
will only expose the first QTG ID per partition in sysfs for now. In the
future when multiple QTG IDs are necessary, they can be exposed. [1]

[1]: https://lore.kernel.org/linux-cxl/167571650007.587790.10040913293130712882.stgit@djiang5-mobl3.local/T/#md2a47b1ead3e1ba08f50eab29a4af1aed1d215ab

Suggested-by: Dan Williams <[email protected]>
Reviewed-by: Jonathan Cameron <[email protected]>
Signed-off-by: Dave Jiang <[email protected]>
Link: https://lore.kernel.org/r/170319625698.2212653.17544381274847420961.stgit@djiang5-mobl3
Signed-off-by: Dan Williams <[email protected]>

show more ...

cxl: Export QTG ids from CFMWS to sysfs as qos_class attribute

Export the QoS Throttling Group ID from the CXL Fixed Memory Window
Structure (CFMWS) under the root decoder sysfs attributes as qos_cl

cxl: Export QTG ids from CFMWS to sysfs as qos_class attribute

Export the QoS Throttling Group ID from the CXL Fixed Memory Window
Structure (CFMWS) under the root decoder sysfs attributes as qos_class.

CXL rev3.0 9.17.1.3 CXL Fixed Memory Window Structure (CFMWS)

cxl cli will use this id to match with the _DSM retrieved id for a
hot-plugged CXL memory device DPA memory range to make sure that the
DPA range is under the right CFMWS window.

Reviewed-by: Davidlohr Bueso <[email protected]>
Reviewed-by: Ira Weiny <[email protected]>
Reviewed-by: Jonathan Cameron <[email protected]>
Signed-off-by: Dave Jiang <[email protected]>
Link: https://lore.kernel.org/r/169713681699.2205276.14475306324720093079.stgit@djiang5-mobl3
Signed-off-by: Dan Williams <[email protected]>

show more ...

cxl: Add decoders_committed sysfs attribute to cxl_port

This attribute allows cxl-cli to determine whether there are decoders
committed to a memdev. This is only a snapshot of the state, and
doesn'

cxl: Add decoders_committed sysfs attribute to cxl_port

This attribute allows cxl-cli to determine whether there are decoders
committed to a memdev. This is only a snapshot of the state, and
doesn't offer any protection or serialization against a concurrent
disable-region operation.

Reviewed-by: Jim Harris <[email protected]>
Suggested-by: Dan Williams <[email protected]>
Signed-off-by: Dave Jiang <[email protected]>
Link: https://lore.kernel.org/r/169747907439.272156.10261062080830155662.stgit@djiang5-mobl3
Signed-off-by: Dan Williams <[email protected]>

show more ...

cxl/memdev: Only show sanitize sysfs files when supported

If the device does not support Sanitize or Secure Erase commands,
hide the respective sysfs interfaces such that the operation can
never be

cxl/memdev: Only show sanitize sysfs files when supported

If the device does not support Sanitize or Secure Erase commands,
hide the respective sysfs interfaces such that the operation can
never be attempted.

In order to be generic, keep track of the enabled security commands
found in the CEL - the driver does not support Security Passthrough.

Signed-off-by: Davidlohr Bueso <[email protected]>
Link: https://lore.kernel.org/r/[email protected]
Reviewed-by: Dave Jiang <[email protected]>
Signed-off-by: Vishal Verma <[email protected]>

show more ...

cxl/memdev: Improve sanitize ABI descriptions

Be more detailed about the CPU cache management situation. The same
goes for both sanitize and secure erase.

Signed-off-by: Davidlohr Bueso <dave@stgol

cxl/memdev: Improve sanitize ABI descriptions

Be more detailed about the CPU cache management situation. The same
goes for both sanitize and secure erase.

Signed-off-by: Davidlohr Bueso <[email protected]>
Link: https://lore.kernel.org/r/[email protected]
Reviewed-by: Dave Jiang <[email protected]>
Signed-off-by: Vishal Verma <[email protected]>

show more ...

cxl: add a firmware update mechanism using the sysfs firmware loader

The sysfs based firmware loader mechanism was created to easily allow
userspace to upload firmware images to FPGA cards. This als

cxl: add a firmware update mechanism using the sysfs firmware loader

The sysfs based firmware loader mechanism was created to easily allow
userspace to upload firmware images to FPGA cards. This also happens to
be pretty suitable to create a user-initiated but kernel-controlled
firmware update mechanism for CXL devices, using the CXL specified
mailbox commands.

Since firmware update commands can be long-running, and can be processed
in the background by the endpoint device, it is desirable to have the
ability to chunk the firmware transfer down to smaller pieces, so that
one operation does not monopolize the mailbox, locking out any other
long running background commands entirely - e.g. security commands like
'sanitize' or poison scanning operations.

The firmware loader mechanism allows a natural way to perform this
chunking, as after each mailbox command, that is restricted to the
maximum mailbox payload size, the cxl memdev driver relinquishes control
back to the fw_loader system and awaits the next chunk of data to
transfer. This opens opportunities for other background commands to
access the mailbox and send their own slices of background commands.

Add the necessary helpers and state tracking to be able to perform the
'Get FW Info', 'Transfer FW', and 'Activate FW' mailbox commands as
described in the CXL spec. Wire these up to the firmware loader
callbacks, and register with that system to create the memX/firmware/
sysfs ABI.

Cc: Davidlohr Bueso <[email protected]>
Cc: Jonathan Cameron <[email protected]>
Cc: Russ Weight <[email protected]>
Cc: Alison Schofield <[email protected]>
Cc: Ira Weiny <[email protected]>
Cc: Dave Jiang <[email protected]>
Cc: Ben Widawsky <[email protected]>
Cc: Dan Williams <[email protected]>
Reviewed-by: Jonathan Cameron <[email protected]>
Reviewed-by: Dave Jiang <[email protected]>
Signed-off-by: Vishal Verma <[email protected]>
Link: https://lore.kernel.org/r/[email protected]
Signed-off-by: Dan Williams <[email protected]>

show more ...

cxl/mem: Support Secure Erase

Implement support for the non-pmem exclusive secure erase, per
CXL specs. Create a write-only 'security/erase' sysfs file to
perform the requested operation.

As with t

cxl/mem: Support Secure Erase

Implement support for the non-pmem exclusive secure erase, per
CXL specs. Create a write-only 'security/erase' sysfs file to
perform the requested operation.

As with the sanitation this requires the device being offline
and thus no active HPA-DPA decoding.

The expectation is that userspace can use it such as:

cxl disable-memdev memX
echo 1 > /sys/bus/cxl/devices/memX/security/erase
cxl enable-memdev memX

Reviewed-by: Jonathan Cameron <[email protected]>
Reviewed-by: Dave Jiang <[email protected]>
Reviewed-by: Fan Ni <[email protected]>
Signed-off-by: Davidlohr Bueso <[email protected]>
Link: https://lore.kernel.org/r/[email protected]
Signed-off-by: Dan Williams <[email protected]>

show more ...

cxl/mem: Wire up Sanitization support

Implement support for CXL 3.0 8.2.9.8.5.1 Sanitize. This is done by
adding a security/sanitize' memdev sysfs file to trigger the operation
and extend the status

cxl/mem: Wire up Sanitization support

Implement support for CXL 3.0 8.2.9.8.5.1 Sanitize. This is done by
adding a security/sanitize' memdev sysfs file to trigger the operation
and extend the status file to make it poll(2)-capable for completion.
Unlike all other background commands, this is the only operation that
is special and monopolizes the device for long periods of time.

In addition to the traditional pmem security requirements, all regions
must also be offline in order to perform the operation. This permits
avoiding explicit global CPU cache management, relying instead on the
implict cache management when a region transitions between
CXL_CONFIG_ACTIVE and CXL_CONFIG_COMMIT.

The expectation is that userspace can use it such as:

cxl disable-memdev memX
echo 1 > /sys/bus/cxl/devices/memX/security/sanitize
cxl wait-sanitize memX
cxl enable-memdev memX

Reviewed-by: Dave Jiang <[email protected]>
Reviewed-by: Jonathan Cameron <[email protected]>
Signed-off-by: Davidlohr Bueso <[email protected]>
Link: https://lore.kernel.org/r/[email protected]
Signed-off-by: Dan Williams <[email protected]>

show more ...

cxl/mem: Introduce security state sysfs file

Add a read-only sysfs file to display the security state
of a device (currently only pmem):

/sys/bus/cxl/devices/memX/security/state

This introduce

cxl/mem: Introduce security state sysfs file

Add a read-only sysfs file to display the security state
of a device (currently only pmem):

/sys/bus/cxl/devices/memX/security/state

This introduces a cxl_security_state structure that is
to be the placeholder for common CXL security features.

Reviewed-by: Jonathan Cameron <[email protected]>
Reviewed-by: Fan Ni <[email protected]>
Signed-off-by: Davidlohr Bueso <[email protected]>
Reviewed-by: Dave Jiang <[email protected]>
Link: https://lore.kernel.org/r/[email protected]
Signed-off-by: Dan Williams <[email protected]>

show more ...

cxl/memdev: Add trigger_poison_list sysfs attribute

When a boolean 'true' is written to this attribute the memdev driver
retrieves the poison list from the device. The list consists of
addresses tha

cxl/memdev: Add trigger_poison_list sysfs attribute

When a boolean 'true' is written to this attribute the memdev driver
retrieves the poison list from the device. The list consists of
addresses that are poisoned, or would result in poison if accessed,
and the source of the poison. This attribute is only visible for
devices supporting the capability. The retrieved errors are logged
as kernel events when cxl_poison event tracing is enabled.

Signed-off-by: Alison Schofield <[email protected]>
Reviewed-by: Jonathan Cameron <[email protected]>
Reviewed-by: Ira Weiny <[email protected]>
Link: https://lore.kernel.org/r/1081cfdc8a349dc754779642d584707e56db26ba.1681838291.git.alison.schofield@intel.com
Tested-by: Jonathan Cameron <[email protected]>
Signed-off-by: Dan Williams <[email protected]>

show more ...

cxl/region: Add volatile region creation support

Expand the region creation infrastructure to enable 'ram'
(volatile-memory) regions. The internals of create_pmem_region_store()
and create_pmem_regi

cxl/region: Add volatile region creation support

Expand the region creation infrastructure to enable 'ram'
(volatile-memory) regions. The internals of create_pmem_region_store()
and create_pmem_region_show() are factored out into helpers
__create_region() and __create_region_show() for the 'ram' case to
reuse.

Reviewed-by: Vishal Verma <[email protected]>
Reviewed-by: Gregory Price <[email protected]>
Reviewed-by: Dave Jiang <[email protected]>
Reviewed-by: Ira Weiny <[email protected]>
Reviewed-by: Jonathan Cameron <[email protected]>
Tested-by: Fan Ni <[email protected]>
Link: https://lore.kernel.org/r/167601995775.1924368.352616146815830591.stgit@dwillia2-xfh.jf.intel.com
Signed-off-by: Dan Williams <[email protected]>

show more ...

cxl/region: Support empty uuids for non-pmem regions

Shipping versions of the cxl-cli utility expect all regions to have a
'uuid' attribute. In preparation for 'ram' regions, update the 'uuid'
attri

cxl/region: Support empty uuids for non-pmem regions

Shipping versions of the cxl-cli utility expect all regions to have a
'uuid' attribute. In preparation for 'ram' regions, update the 'uuid'
attribute to return an empty string which satisfies the current
expectations of 'cxl list -R'. Otherwise, 'cxl list -R' fails in the
presence of regions with the 'uuid' attribute missing. Force the
attribute to be read-only as there is no facility or expectation for a
'ram' region to recall its uuid from one boot to the next.

Reviewed-by: Vishal Verma <[email protected]>
Tested-by: Fan Ni <[email protected]>
Reviewed-by: Ira Weiny <[email protected]>
Reviewed-by: Jonathan Cameron <[email protected]>
Link: https://lore.kernel.org/r/167601994558.1924368.12612811533724694444.stgit@dwillia2-xfh.jf.intel.com
Signed-off-by: Dan Williams <[email protected]>

show more ...

cxl/region: Add a mode attribute for regions

In preparation for a new region type, "ram" regions, add a mode
attribute to clarify the mode of the decoders that can be added to a
region. Share the in

cxl/region: Add a mode attribute for regions

In preparation for a new region type, "ram" regions, add a mode
attribute to clarify the mode of the decoders that can be added to a
region. Share the internals of mode_show() (for decoders) with the
region case.

Reviewed-by: Vishal Verma <[email protected]>
Reviewed-by: Dave Jiang <[email protected]>
Reviewed-by: Gregory Price <[email protected]>
Reviewed-by: Ira Weiny <[email protected]>
Reviewed-by: Jonathan Cameron <[email protected]>
Tested-by: Fan Ni <[email protected]>
Link: https://lore.kernel.org/r/167601993930.1924368.4305018565539515665.stgit@dwillia2-xfh.jf.intel.com
Signed-off-by: Dan Williams <[email protected]>

show more ...

cxl/Documentation: Update references to attributes added in v6.0

Prior to Linus deciding that the kernel that following v5.19 would be
v6.0, the CXL ABI documentation already referenced v5.20. In pr

cxl/Documentation: Update references to attributes added in v6.0

Prior to Linus deciding that the kernel that following v5.19 would be
v6.0, the CXL ABI documentation already referenced v5.20. In preparation
for updating these entries update the kernel version to v6.0.

Reviewed-by: Vishal Verma <[email protected]>
Reviewed-by: Dave Jiang <[email protected]>
Reviewed-by: Gregory Price <[email protected]>
Reviewed-by: Ira Weiny <[email protected]>
Reviewed-by: Davidlohr Bueso <[email protected]>
Reviewed-by: Jonathan Cameron <[email protected]>
Tested-by: Fan Ni <[email protected]>
Link: https://lore.kernel.org/r/167601993360.1924368.14122892663883462813.stgit@dwillia2-xfh.jf.intel.com
Signed-off-by: Dan Williams <[email protected]>

show more ...

cxl/port: Link the 'parent_dport' in portX/ and endpointX/ sysfs

Similar to the justification in:

1b58b4cac6fc ("cxl/port: Record parent dport when adding ports")

...userspace wants to know the ro

cxl/port: Link the 'parent_dport' in portX/ and endpointX/ sysfs

Similar to the justification in:

1b58b4cac6fc ("cxl/port: Record parent dport when adding ports")

...userspace wants to know the routing information for ports for
calculating the memdev order for region creation among other things.
Cache the information the kernel discovers at enumeration time in a
'parent_dport' attribute to save userspace the time of trawling sysfs
to recover the same information.

Reviewed-by: Jonathan Cameron <[email protected]>
Link: https://lore.kernel.org/r/167124082375.1626103.6047000000121298560.stgit@dwillia2-xfh.jf.intel.com
Signed-off-by: Dan Williams <[email protected]>

show more ...

cxl/hdm: Commit decoder state to hardware

After all the soft validation of the region has completed, convey the
region configuration to hardware while being careful to commit decoders
in specificati

cxl/hdm: Commit decoder state to hardware

After all the soft validation of the region has completed, convey the
region configuration to hardware while being careful to commit decoders
in specification mandated order. In addition to programming the endpoint
decoder base-address, interleave ways and granularity, the switch
decoder target lists are also established.

While the kernel can enforce spec-mandated commit order, it can not
enforce spec-mandated reset order. For example, the kernel can't stop
someone from removing an endpoint device that is occupying decoderN in a
switch decoder where decoderN+1 is also committed. To reset decoderN,
decoderN+1 must be torn down first. That "tear down the world"
implementation is saved for a follow-on patch.

Callback operations are provided for the 'commit' and 'reset'
operations. While those callbacks may prove useful for CXL accelerators
(Type-2 devices with memory) the primary motivation is to enable a
simple way for cxl_test to intercept those operations.

Reviewed-by: Jonathan Cameron <[email protected]>
Link: https://lore.kernel.org/r/165784338418.1758207.14659830845389904356.stgit@dwillia2-xfh.jf.intel.com
Signed-off-by: Dan Williams <[email protected]>

show more ...

cxl/region: Enable the assignment of endpoint decoders to regions

The region provisioning process involves allocating DPA to a set of
endpoint decoders, and HPA plus the region geometry to a region

cxl/region: Enable the assignment of endpoint decoders to regions

The region provisioning process involves allocating DPA to a set of
endpoint decoders, and HPA plus the region geometry to a region device.
Then the decoder is assigned to the region. At this point several
validation steps can be performed to validate that the decoder is
suitable to participate in the region.

Co-developed-by: Ben Widawsky <[email protected]>
Signed-off-by: Ben Widawsky <[email protected]>
Reviewed-by: Jonathan Cameron <[email protected]>
Reported-by: kernel test robot <[email protected]>
Link: https://lore.kernel.org/r/165784336184.1758207.16403282029203949622.stgit@dwillia2-xfh.jf.intel.com
Signed-off-by: Dan Williams <[email protected]>

show more ...

cxl/region: Allocate HPA capacity to regions

After a region's interleave parameters (ways and granularity) are set,
add a way for regions to allocate HPA (host physical address space) from
the free

cxl/region: Allocate HPA capacity to regions

After a region's interleave parameters (ways and granularity) are set,
add a way for regions to allocate HPA (host physical address space) from
the free capacity in their parent root-decoder. The allocator for this
capacity reuses the 'struct resource' based allocator used for
CONFIG_DEVICE_PRIVATE.

Once the tuple of "ways, granularity, [uuid], and size" is set the
region configuration transitions to the CXL_CONFIG_INTERLEAVE_ACTIVE
state which is a precursor to allowing endpoint decoders to be added to
a region.

Co-developed-by: Ben Widawsky <[email protected]>
Signed-off-by: Ben Widawsky <[email protected]>
Reviewed-by: Jonathan Cameron <[email protected]>
Link: https://lore.kernel.org/r/165784335630.1758207.420216490941955417.stgit@dwillia2-xfh.jf.intel.com
Signed-off-by: Dan Williams <[email protected]>

show more ...

cxl/region: Add interleave geometry attributes

Add ABI to allow the number of devices that comprise a region to be
set as well as the interleave granularity for the region.

Signed-off-by: Ben Widaw

cxl/region: Add interleave geometry attributes

Add ABI to allow the number of devices that comprise a region to be
set as well as the interleave granularity for the region.

Signed-off-by: Ben Widawsky <[email protected]>
[djbw: reword changelog]
Reviewed-by: Jonathan Cameron <[email protected]>
Link: https://lore.kernel.org/r/[email protected]
Signed-off-by: Dan Williams <[email protected]>

show more ...

cxl/region: Add a 'uuid' attribute

The process of provisioning a region involves triggering the creation of
a new region object, pouring in the configuration, and then binding that
configured object

cxl/region: Add a 'uuid' attribute

The process of provisioning a region involves triggering the creation of
a new region object, pouring in the configuration, and then binding that
configured object to the region driver to start its operation. For
persistent memory regions the CXL specification mandates that it
identified by a uuid. Add an ABI for userspace to specify a region's
uuid.

Signed-off-by: Ben Widawsky <[email protected]>
[djbw: simplify locking]
Reviewed-by: Jonathan Cameron <[email protected]>
Link: https://lore.kernel.org/r/165784334465.1758207.8224025435884752570.stgit@dwillia2-xfh.jf.intel.com
Signed-off-by: Dan Williams <[email protected]>

show more ...