powercap: intel_rapl: Introduce APIs for PMU support

Introduce two new APIs rapl_package_add_pmu()/rapl_package_remove_pmu().

RAPL driver can invoke these APIs to expose its supported energy
counte

powercap: intel_rapl: Introduce APIs for PMU support

Introduce two new APIs rapl_package_add_pmu()/rapl_package_remove_pmu().

RAPL driver can invoke these APIs to expose its supported energy
counters via perf PMU. The new RAPL PMU is fully compatible with current
MSR RAPL PMU, including using the same PMU name and events
name/id/unit/scale, etc.

For example, use below command
perf stat -e power/energy-pkg/ -e power/energy-ram/ FOO
to get the energy consumption if power/energy-pkg/ and power/energy-ram/
events are available in the "perf list" output.

This does not introduce any conflict because TPMI RAPL is the only user
of these APIs currently, and it never co-exists with MSR RAPL.

Note that RAPL Packages can be probed/removed dynamically, and the
events supported by each TPMI RAPL device can be different. Thus the
RAPL PMU support is done on demand, which means
1. PMU is registered only if it is needed by a RAPL Package. PMU events
for unsupported counters are not exposed.
2. PMU is unregistered and registered when a new RAPL Package is probed
and supports new counters that are not supported by current PMU.
For example, on a dual-package system using TPMI RAPL, it is possible
that Package 1 behaves as TPMI domain root and supports Psys domain.
In this case, register PMU without Psys event when probing Package 0,
and re-register the PMU with Psys event when probing Package 1.
3. PMU is unregistered when all registered RAPL Packages don't need PMU.

Signed-off-by: Zhang Rui <[email protected]>
Signed-off-by: Rafael J. Wysocki <[email protected]>

show more ...

powercap: intel_rapl: Fix locking in TPMI RAPL

The RAPL framework uses CPU hotplug locking to protect the rapl_packages
list and rp->lead_cpu to guarantee that

1. the RAPL package device is not un

powercap: intel_rapl: Fix locking in TPMI RAPL

The RAPL framework uses CPU hotplug locking to protect the rapl_packages
list and rp->lead_cpu to guarantee that

1. the RAPL package device is not unprobed and freed
2. the cached rp->lead_cpu is always valid

for operations like powercap sysfs accesses.

Current RAPL APIs assume being called from CPU hotplug callbacks which
hold the CPU hotplug lock, but TPMI RAPL driver invokes the APIs in the
driver's .probe() function without acquiring the CPU hotplug lock.

Fix the problem by providing both locked and lockless versions of RAPL
APIs.

Fixes: 9eef7f9da928 ("powercap: intel_rapl: Introduce RAPL TPMI interface driver")
Signed-off-by: Zhang Rui <[email protected]>
Cc: 6.5+ <[email protected]> # 6.5+
Signed-off-by: Rafael J. Wysocki <[email protected]>

show more ...

powercap: intel_rapl: Fix a sparse warning in TPMI interface

Depends on the interface used, the RAPL registers can be either MSR
indexes or memory mapped IO addresses. Current RAPL common code uses

powercap: intel_rapl: Fix a sparse warning in TPMI interface

Depends on the interface used, the RAPL registers can be either MSR
indexes or memory mapped IO addresses. Current RAPL common code uses u64
to save both MSR and memory mapped IO registers. With this, when
handling register address with an __iomem annotation, it triggers a
sparse warning like below:

sparse warnings: (new ones prefixed by >>)
>> drivers/powercap/intel_rapl_tpmi.c:141:41: sparse: sparse: incorrect type in initializer (different address spaces) @@ expected unsigned long long [usertype] *tpmi_rapl_regs @@ got void [noderef] __iomem * @@
drivers/powercap/intel_rapl_tpmi.c:141:41: sparse: expected unsigned long long [usertype] *tpmi_rapl_regs
drivers/powercap/intel_rapl_tpmi.c:141:41: sparse: got void [noderef] __iomem *

Fix the problem by using a union to save the registers instead.

Suggested-by: David Laight <[email protected]>
Reported-by: kernel test robot <[email protected]>
Closes: https://lore.kernel.org/oe-kbuild-all/[email protected]/
Tested-by: Wang Wendy <[email protected]>
Signed-off-by: Zhang Rui <[email protected]>
[ rjw: Subject and changelog edits ]
Signed-off-by: Rafael J. Wysocki <[email protected]>

show more ...

powercap: intel_rapl: Introduce core support for TPMI interface

Compared with existing RAPL MSR/MMIO Interface, the RAPL TPMI Interface
1. has per Power Limit register, thus has per Power Limit Lock

powercap: intel_rapl: Introduce core support for TPMI interface

Compared with existing RAPL MSR/MMIO Interface, the RAPL TPMI Interface
1. has per Power Limit register, thus has per Power Limit Lock and
Enable bit.
2. doesn't have Power Limit Clamp bit.
3. the Power Limit Lock and Enable bits have different bit offsets.
These mean RAPL TPMI Interface needs its own primitive information.

RAPL TPMI Interface also has per domain unit register but with a
different register layout. This requires a TPMI specific rapl_defaults
call to decode the unit register.

Introduce the RAPL core support for TPMI Interface.

Signed-off-by: Zhang Rui <[email protected]>
Tested-by: Wang Wendy <[email protected]>
Signed-off-by: Rafael J. Wysocki <[email protected]>

show more ...

powercap: intel_rapl: Introduce RAPL I/F type

Different RAPL Interfaces may have different primitive information and
rapl_defaults calls.

To better distinguish this difference in the RAPL framework

powercap: intel_rapl: Introduce RAPL I/F type

Different RAPL Interfaces may have different primitive information and
rapl_defaults calls.

To better distinguish this difference in the RAPL framework code,
introduce a new enum to represent different types of RAPL Interfaces.

No functional change.

Signed-off-by: Zhang Rui <[email protected]>
Tested-by: Wang Wendy <[email protected]>
Signed-off-by: Rafael J. Wysocki <[email protected]>

show more ...

powercap: intel_rapl: Make cpu optional for rapl_package

MSR RAPL Interface always removes a rapl_package when all the CPUs in
that rapl_package are offlined. This is because it relies on an online

powercap: intel_rapl: Make cpu optional for rapl_package

MSR RAPL Interface always removes a rapl_package when all the CPUs in
that rapl_package are offlined. This is because it relies on an online
CPU to access the MSR.

But for RAPL Interface using MMIO registers, when all the cpus within
the rapl_package are offlined,
1. the register can still be accessed
2. monitoring and setting the Power Pimits for the rapl_package is still
meaningful because of uncore power.

This means that, a valid rapl_package doesn't rely on one or more cpus
being onlined.

For this sense, make cpu optional for rapl_package. A rapl_package can
be registered either using a CPU id to represent the physical
package/die, or using the physical package id directly.

Note that, the thermal throttling interrupt is not disabled via
MSR_IA32_PACKAGE_THERM_INTERRUPT for such rapl_package at the moment.
If it is still needed in the future, this can be achieved by selecting
an onlined CPU using the physical package id.

Note that, processor_thermal_rapl, the current MMIO RAPL Interface
driver, can also be converted to register using a package id instead.
But this is not done right now because processor_thermal_rapl driver
works on single-package systems only, and offlining the only package
will not happen. So keep the previous logic.

Signed-off-by: Zhang Rui <[email protected]>
Tested-by: Wang Wendy <[email protected]>
Signed-off-by: Rafael J. Wysocki <[email protected]>

show more ...

powercap: intel_rapl: Add support for lock bit per Power Limit

With RAPL MSR/MMIO Interface, each RAPL domain has one Power Limit
register. Each Power Limit register has one lock bit which tells the

powercap: intel_rapl: Add support for lock bit per Power Limit

With RAPL MSR/MMIO Interface, each RAPL domain has one Power Limit
register. Each Power Limit register has one lock bit which tells the OS
if the power limit register can be used or not.
Depending on the number of power limits supported by the power limit
register, the lock bit may apply to one or more power limits.

With RAPL TPMI Interface, each RAPL domain has multiple Power Limits,
and each Power Limit has its own register, with a lock bit.

To handle this, introduce support for lock bit per Power Limit.

For existing RAPL MSR/MMIO Interfaces, the lock bit in the Power Limit
register applies to all the Power Limits controlled by this register.

Remove the per domain DOMAIN_STATE_BIOS_LOCKED flag at the same time
because it can be replaced by the per Power Limit lock.

No functional change intended.

Signed-off-by: Zhang Rui <[email protected]>
Tested-by: Wang Wendy <[email protected]>
Signed-off-by: Rafael J. Wysocki <[email protected]>

show more ...

powercap: intel_rapl: Cleanup Power Limits support

The same set of operations are shared by different Powert Limits,
including Power Limit get/set, Power Limit enable/disable, clamping
enable/disabl

powercap: intel_rapl: Cleanup Power Limits support

The same set of operations are shared by different Powert Limits,
including Power Limit get/set, Power Limit enable/disable, clamping
enable/disable, time window get/set, and max power get/set, etc.

But the same operation for different Power Limit has different
primitives because they use different registers/register bits.

A lot of dirty/duplicate code was introduced to handle this difference.

Introduce a universal way to issue Power Limit operations.
Instead of using hardcoded primitive name directly, use Power Limit id
+ operation type, and hide all the Power Limit difference details in a
central place, get_pl_prim(). Two helpers, rapl_read_pl_data() and
rapl_write_pl_data(), are introduced at the same time to simplify the
code for issuing Power Limit operations.

Signed-off-by: Zhang Rui <[email protected]>
Tested-by: Wang Wendy <[email protected]>
Signed-off-by: Rafael J. Wysocki <[email protected]>

show more ...

powercap: intel_rapl: Change primitive order

The same set of operations are shared by different Powert Limits,
including Power Limit get/set, Power Limit enable/disable, clamping
enable/disable, tim

powercap: intel_rapl: Change primitive order

The same set of operations are shared by different Powert Limits,
including Power Limit get/set, Power Limit enable/disable, clamping
enable/disable, time window get/set, and max power get/set, etc.

But the same operation for different Power Limit has different
primitives because they use different registers/register bits.

A lot of dirty/duplicate code was introduced to handle this difference.

Instead of using hardcoded primitive name directly, using Power Limit id
+ operation type is much cleaner.

For this sense, move POWER_LIMIT1/POWER_LIMIT2/POWER_LIMIT4 to the
beginning of enum rapl_primitives so that they can be reused as
Power Limit ids.

No functional change.

Signed-off-by: Zhang Rui <[email protected]>
Tested-by: Wang Wendy <[email protected]>
Signed-off-by: Rafael J. Wysocki <[email protected]>

show more ...

powercap: intel_rapl: Support per domain energy/power/time unit

RAPL MSR/MMIO Interface has package scope unit register but some RAPL
domains like Dram/Psys may use a fixed energy unit value instead

powercap: intel_rapl: Support per domain energy/power/time unit

RAPL MSR/MMIO Interface has package scope unit register but some RAPL
domains like Dram/Psys may use a fixed energy unit value instead of the
default unit value on certain platforms.
RAPL TPMI Interface supports per domain unit register.

For the above reasons, add support for per domain unit register and per
domain energy/power/time unit.

When per domain unit register is not available, use the package scope
unit register as the per domain unit register for each RAPL domain so
that this change is transparent to MSR/MMIO Interface.

No functional change intended.

Signed-off-by: Zhang Rui <[email protected]>
Tested-by: Wang Wendy <[email protected]>
Signed-off-by: Rafael J. Wysocki <[email protected]>

show more ...

powercap: intel_rapl: Support per Interface primitive information

RAPL primitive information is Interface specific.

Although current MSR and MMIO Interface share the same RAPL primitives,
new Inter

powercap: intel_rapl: Support per Interface primitive information

RAPL primitive information is Interface specific.

Although current MSR and MMIO Interface share the same RAPL primitives,
new Interface like TPMI has its own RAPL primitive information.

Save the primitive information in the Interface private structure.

Plus, using variant name "rp" for struct rapl_primitive_info is
confusing because "rp" is also used for struct rapl_package.
Use "rpi" as the variant name for struct rapl_primitive_info, and rename
the previous rpi[] array to avoid conflict.

No functional change.

Signed-off-by: Zhang Rui <[email protected]>
Tested-by: Wang Wendy <[email protected]>
Signed-off-by: Rafael J. Wysocki <[email protected]>

show more ...

powercap: intel_rapl: Support per Interface rapl_defaults

rapl_defaults is Interface specific.

Although current MSR and MMIO Interface share the same rapl_defaults,
new Interface like TPMI need its

powercap: intel_rapl: Support per Interface rapl_defaults

rapl_defaults is Interface specific.

Although current MSR and MMIO Interface share the same rapl_defaults,
new Interface like TPMI need its own rapl_defaults callbacks.

Save the rapl_defaults information in the Interface private structure.

No functional change.

Signed-off-by: Zhang Rui <[email protected]>
Tested-by: Wang Wendy <[email protected]>
Signed-off-by: Rafael J. Wysocki <[email protected]>

show more ...

powercap: intel_rapl: Remove unused field in struct rapl_if_priv

After commit f1e8d7560d30 ("powercap/intel_rapl: enumerate Psys RAPL
domain together with package RAPL domain"), the platform_rapl_do

powercap: intel_rapl: Remove unused field in struct rapl_if_priv

After commit f1e8d7560d30 ("powercap/intel_rapl: enumerate Psys RAPL
domain together with package RAPL domain"), the platform_rapl_domain field
is not used anymore. Remove it from rapl_if_priv structure.

Fixes: f1e8d7560d30 ("powercap/intel_rapl: enumerate Psys RAPL domain together with package RAPL domain")
Signed-off-by: Zhang Rui <[email protected]>
Signed-off-by: Rafael J. Wysocki <[email protected]>

show more ...

powercap: intel_rapl: support new layout of Psys PowerLimit Register on SPR

On Sapphire Rapids, the layout of the Psys domain Power Limit Register
is different from from what it was before.

Enhance

powercap: intel_rapl: support new layout of Psys PowerLimit Register on SPR

On Sapphire Rapids, the layout of the Psys domain Power Limit Register
is different from from what it was before.

Enhance the code to support the new Psys PL register layout.

Signed-off-by: Zhang Rui <[email protected]>
Reported-and-tested-by: Alkattan Dana <[email protected]>
[ rjw: Subject and changelog edits ]
Signed-off-by: Rafael J. Wysocki <[email protected]>

show more ...

powercap: RAPL: Fix struct declaration in header file

struct rapl_package is declared twice in intel_rapl.h, once at line 80
and once earlier.

Code inspection suggests that the first instance shoul

powercap: RAPL: Fix struct declaration in header file

struct rapl_package is declared twice in intel_rapl.h, once at line 80
and once earlier.

Code inspection suggests that the first instance should be struct
rapl_domain rather than rapl_package, so change it.

Signed-off-by: Wan Jiabing <[email protected]>
[ rjw: Subject and changelog edits ]
Signed-off-by: Rafael J. Wysocki <[email protected]>

show more ...

powercap/intel_rapl: enumerate Psys RAPL domain together with package RAPL domain

On multi-package systems, the Psys MSR is only valid for CPUs on
specific package (master package). The current code

powercap/intel_rapl: enumerate Psys RAPL domain together with package RAPL domain

On multi-package systems, the Psys MSR is only valid for CPUs on
specific package (master package). The current code makes the
assumption that package 0 is the master package, but this is not
true on new platforms like SPR.

Fix the problem by emuerating the Psys RAPL domain for every
package, so CPUs in slave packages will read 0 for the Psys energy
counter and only CPUs in master packages can get a valid reading
and register the Psys RAPL domain.

The sysfs I/F for the Psys RAPL domain is not changed.

Signed-off-by: Zhang Rui <[email protected]>
[ rjw: Subject and changelog edits ]
Signed-off-by: Rafael J. Wysocki <[email protected]>

show more ...

powercap: Add Power Limit4 support

Modern Intel Mobile platforms support power limit4 (PL4), which is
the SoC package level maximum power limit (in Watts). It can be used
to preemptively limits pote

powercap: Add Power Limit4 support

Modern Intel Mobile platforms support power limit4 (PL4), which is
the SoC package level maximum power limit (in Watts). It can be used
to preemptively limits potential SoC power to prevent power spikes
from tripping the power adapter and battery over-current protection.
This patch enables this feature by exposing package level peak power
capping control to userspace via RAPL sysfs interface. With this,
application like DTPF can modify PL4 power limit, the similar way
of other package power limit (PL1).
As this feature is not tested on previous generations, here it is
enabled only for the platform that has been verified to work,
for safety concerns.

Signed-off-by: Sumeet Pawnikar <[email protected]>
Co-developed-by: Zhang Rui <[email protected]>
Signed-off-by: Zhang Rui <[email protected]>
Reviewed-by: Srinivas Pandruvada <[email protected]>
Tested-by: Srinivas Pandruvada <[email protected]>
Signed-off-by: Rafael J. Wysocki <[email protected]>

show more ...

intel_rapl: need linux/cpuhotplug.h for enum cpuhp_state

Fixes: 7ebf8eff63b4 ("intel_rapl: introduce struct rapl_if_private")
Signed-off-by: Stephen Rothwell <[email protected]>
Signed-off-by: Ra

intel_rapl: need linux/cpuhotplug.h for enum cpuhp_state

Fixes: 7ebf8eff63b4 ("intel_rapl: introduce struct rapl_if_private")
Signed-off-by: Stephen Rothwell <[email protected]>
Signed-off-by: Rafael J. Wysocki <[email protected]>

show more ...

intel_rapl: support two power limits for every RAPL domain

RAPL MSR interface supports 2 power limits for package domain, and 1 power
limit for other domains, while RAPL MMIO interface supports 2 po

intel_rapl: support two power limits for every RAPL domain

RAPL MSR interface supports 2 power limits for package domain, and 1 power
limit for other domains, while RAPL MMIO interface supports 2 power limits
for both package and dram domains.
And when 2 power limits are supported, the FW_LOCK bit is in bit 63 of the
register, instead of bit 31.

Remove the assumption that only pakcage domain supports 2 power limits.
And allow the RAPL interface driver to specify the number of power limits
supported, for every single RAPL domain it owns..

Reviewed-by: Pandruvada, Srinivas <[email protected]>
Tested-by: Pandruvada, Srinivas <[email protected]>
Signed-off-by: Zhang Rui <[email protected]>
Signed-off-by: Rafael J. Wysocki <[email protected]>

show more ...

intel_rapl: support 64 bit register

RAPL MMIO interface uses 64 bit registers, thus force use 64 bit register
for all the RAPL code.

Reviewed-by: Pandruvada, Srinivas <[email protected]

intel_rapl: support 64 bit register

RAPL MMIO interface uses 64 bit registers, thus force use 64 bit register
for all the RAPL code.

Reviewed-by: Pandruvada, Srinivas <[email protected]>
Tested-by: Pandruvada, Srinivas <[email protected]>
Signed-off-by: Zhang Rui <[email protected]>
Signed-off-by: Rafael J. Wysocki <[email protected]>

show more ...

intel_rapl: abstract RAPL common code

Split intel_rapl.c to intel_rapl_common.c and intel_rapl_msr.c, where
intel_rapl_common.c contains the common code that can be used by both MSR
and MMIO interfa

intel_rapl: abstract RAPL common code

Split intel_rapl.c to intel_rapl_common.c and intel_rapl_msr.c, where
intel_rapl_common.c contains the common code that can be used by both MSR
and MMIO interface.
intel_rapl_msr.c contains the implementation of RAPL MSR interface.

Reviewed-by: Pandruvada, Srinivas <[email protected]>
Tested-by: Pandruvada, Srinivas <[email protected]>
Signed-off-by: Zhang Rui <[email protected]>
Signed-off-by: Rafael J. Wysocki <[email protected]>

show more ...

intel_rapl: abstract register access operations

MSR and MMIO RAPL interfaces have different ways to access the registers,
thus in order to abstract the register access operations, two callbacks,
.re

intel_rapl: abstract register access operations

MSR and MMIO RAPL interfaces have different ways to access the registers,
thus in order to abstract the register access operations, two callbacks,
.read_raw()/.write_raw() are introduced, and they should be implemented by
MSR RAPL and MMIO RAPL interface driver respectly.

This patch implements them for the MSR I/F only.

Reviewed-by: Pandruvada, Srinivas <[email protected]>
Tested-by: Pandruvada, Srinivas <[email protected]>
Signed-off-by: Zhang Rui <[email protected]>
Signed-off-by: Rafael J. Wysocki <[email protected]>

show more ...

intel_rapl: abstract register address

MSR and MMIO RAPL interface have different sets of registers, thus the
RAPL register address should be obtained from interface specific
structure, i.e. struct r

intel_rapl: abstract register address

MSR and MMIO RAPL interface have different sets of registers, thus the
RAPL register address should be obtained from interface specific
structure, i.e. struct rapl_if_private, instead.

Reviewed-by: Pandruvada, Srinivas <[email protected]>
Tested-by: Pandruvada, Srinivas <[email protected]>
Signed-off-by: Zhang Rui <[email protected]>
Signed-off-by: Rafael J. Wysocki <[email protected]>

show more ...

intel_rapl: introduce struct rapl_if_private

Introduce a new structure, rapl_if_private, to save the private data
for different RAPL Interface.

Reviewed-by: Pandruvada, Srinivas <srinivas.pandruvad

intel_rapl: introduce struct rapl_if_private

Introduce a new structure, rapl_if_private, to save the private data
for different RAPL Interface.

Reviewed-by: Pandruvada, Srinivas <[email protected]>
Tested-by: Pandruvada, Srinivas <[email protected]>
Signed-off-by: Zhang Rui <[email protected]>
Signed-off-by: Rafael J. Wysocki <[email protected]>

show more ...

intel_rapl: introduce intel_rapl.h

Create a new header file for the common definitions that might be used
by different RAPL Interface.

Reviewed-by: Pandruvada, Srinivas <[email protected]

intel_rapl: introduce intel_rapl.h

Create a new header file for the common definitions that might be used
by different RAPL Interface.

Reviewed-by: Pandruvada, Srinivas <[email protected]>
Tested-by: Pandruvada, Srinivas <[email protected]>
Signed-off-by: Zhang Rui <[email protected]>
Signed-off-by: Rafael J. Wysocki <[email protected]>

show more ...