Changes in interrupt.h

d65d411c - treewide: context_tracking: Rename CONTEXT_* into CT_STATE_*

Tue, 25 Jul 2023 11:08:50 +0000

treewide: context_tracking: Rename CONTEXT_* into CT_STATE_*Context tracking state related symbols currently use a mix of theCONTEXT_ (e.g. CONTEXT_KERNEL) and CT_SATE_ (e.g. CT_STATE_MASK) prefixes.Clean up the naming and make the ctx_state enum use the CT_STATE_ prefix.Suggested-by: Frederic Weisbecker Signed-off-by: Valentin Schneider Acked-by: Frederic Weisbecker Acked-by: Thomas Gleixner Signed-off-by: Neeraj Upadhyay List of files: /linux-6.15/arch/powerpc/include/asm/interrupt.h

0db880fc - powerpc: Avoid nmi_enter/nmi_exit in real mode interrupt.

Wed, 10 Apr 2024 04:30:06 +0000

powerpc: Avoid nmi_enter/nmi_exit in real mode interrupt.nmi_enter()/nmi_exit() touches per cpu variables which can lead to kernelcrash when invoked during real mode interrupt handling (e.g. early HMI/MCEinterrupt handler) if percpu allocation comes from vmalloc area.Early HMI/MCE handlers are called through DEFINE_INTERRUPT_HANDLER_NMI()wrapper which invokes nmi_enter/nmi_exit calls. We don't see any issue whenpercpu allocation is from the embedded first chunk. However withCONFIG_NEED_PER_CPU_PAGE_FIRST_CHUNK enabled there are chances where percpuallocation can come from the vmalloc area.With kernel command line "percpu_alloc=page" we can force percpu allocationto come from vmalloc area and can see kernel crash in machine_check_early:[ 1.215714] NIP [c000000000e49eb4] rcu_nmi_enter+0x24/0x110[ 1.215717] LR [c0000000000461a0] machine_check_early+0xf0/0x2c0[ 1.215719] --- interrupt: 200[ 1.215720] [c000000fffd73180] [0000000000000000] 0x0 (unreliable)[ 1.215722] [c000000fffd731b0] [0000000000000000] 0x0[ 1.215724] [c000000fffd73210] [c000000000008364] machine_check_early_common+0x134/0x1f8Fix this by avoiding use of nmi_enter()/nmi_exit() in real mode if percpufirst chunk is not embedded.Reviewed-by: Christophe Leroy Tested-by: Shirisha Ganta Signed-off-by: Mahesh Salgaonkar Signed-off-by: Michael Ellerman Link: https://msgid.link/20240410043006.81577-1-mahesh@linux.ibm.com List of files: /linux-6.15/arch/powerpc/include/asm/interrupt.h

d5835fb6 - powerpc: Use user_mode() macro when possible

Fri, 16 Feb 2024 10:10:36 +0000

powerpc: Use user_mode() macro when possibleThere is a nice macro to check user mode.Use it instead of open coding anding with MSR_PR to increasereadability and avoid having to comment what that anding is for.Signed-off-by: Christophe Leroy Signed-off-by: Michael Ellerman Link: https://msgid.link/fbf74887dcf1f1ba9e1680fc3247cbb581b00662.1708078228.git.christophe.leroy@csgroup.eu List of files: /linux-6.15/arch/powerpc/include/asm/interrupt.h

fb3b72a3 - powerpc: Consolidate 32-bit and 64-bit interrupt_enter_prepare

Sat, 21 Jan 2023 09:58:03 +0000

powerpc: Consolidate 32-bit and 64-bit interrupt_enter_prepareThere are two separeate implementations for 32-bit and 64-bit whichmostly do the same thing. Consolidating on one implementation endsup being smaller and simpler, there is just irq soft-mask reconcilethat is specific to 64-bit.There should be no real functional change with this patch, but itdoes make the context tracking calls necessary for 32-bit to supportcontext tracking.Signed-off-by: Nicholas Piggin Signed-off-by: Michael Ellerman Link: https://lore.kernel.org/r/20230121095805.2823731-2-npiggin@gmail.com List of files: /linux-6.15/arch/powerpc/include/asm/interrupt.h

2e7ec190 - powerpc/64s: Add missing declaration for machine_check_early_boot()

Fri, 25 Nov 2022 11:34:42 +0000

powerpc/64s: Add missing declaration for machine_check_early_boot()There's no declaration for machine_check_early_boot(), which leads to abuild failure with W=1. Add one.Fixes: 2f5182cffa43 ("powerpc/64s: early boot machine check handler")Signed-off-by: Michael Ellerman Link: https://lore.kernel.org/r/20221125132521.2167039-1-mpe@ellerman.id.au List of files: /linux-6.15/arch/powerpc/include/asm/interrupt.h

f7bff6e7 - powerpc/64/interrupt: avoid BUG/WARN recursion in interrupt entry

Mon, 26 Sep 2022 05:43:04 +0000

powerpc/64/interrupt: avoid BUG/WARN recursion in interrupt entryBUG/WARN are handled with a program interrupt which can turn into aninfinite recursion when there are bugs in interrupt handler entry(which can be irritated by bugs in other parts of the code).There is one feeble attempt to avoid this recursion, but it missesseveral cases. Make a tidier macro for this and switch most bugs inthe interrupt entry wrapper over to use it.Signed-off-by: Nicholas Piggin Signed-off-by: Michael Ellerman Link: https://lore.kernel.org/r/20220926054305.2671436-7-npiggin@gmail.com List of files: /linux-6.15/arch/powerpc/include/asm/interrupt.h

56adbb7a - powerpc/64/interrupt: Fix false warning in context tracking due to idle state

Mon, 26 Sep 2022 05:42:59 +0000

powerpc/64/interrupt: Fix false warning in context tracking due to idle stateCommit 171476775d32 ("context_tracking: Convert state to atomic_t")added a CONTEXT_IDLE state which can be encountered by interrupts fromkernel mode in the idle thread, causing a false positive warning.Fixes: 171476775d32 ("context_tracking: Convert state to atomic_t")Signed-off-by: Nicholas Piggin Signed-off-by: Michael Ellerman Link: https://lore.kernel.org/r/20220926054305.2671436-2-npiggin@gmail.com List of files: /linux-6.15/arch/powerpc/include/asm/interrupt.h

f8971c62 - powerpc: Change system_call_exception calling convention

Wed, 21 Sep 2022 06:56:00 +0000

powerpc: Change system_call_exception calling conventionChange system_call_exception arguments to pass a pointer to a stackframe container caller state, as well as the original r0, whichdetermines the number of the syscall. This has been observed to yieldimproved performance to passing them by registers, circumventing theneed to allocate a stack frame.Signed-off-by: Rohan McLure Reviewed-by: Nicholas Piggin [mpe: Retain clearing of high bits of args for compat tasks]Signed-off-by: Michael Ellerman Link: https://lore.kernel.org/r/20220921065605.1051927-21-rmclure@linux.ibm.com List of files: /linux-6.15/arch/powerpc/include/asm/interrupt.h

e0d68273 - powerpc: Remove CONFIG_PPC_BOOK3E

Mon, 19 Sep 2022 17:01:33 +0000

powerpc: Remove CONFIG_PPC_BOOK3ECONFIG_PPC_BOOK3E is redundant with CONFIG_PPC_BOOK3E_64.The later is more explicit about the fact that it's a 64 bits target.Remove CONFIG_PPC_BOOK3E.Signed-off-by: Christophe Leroy Signed-off-by: Michael Ellerman Link: https://lore.kernel.org/r/5d0891490813c19cdcfc04678f512ea68cba3e64.1663606876.git.christophe.leroy@csgroup.eu List of files: /linux-6.15/arch/powerpc/include/asm/interrupt.h

46d60bdb - powerpc: Include asm/firmware.h in all users of firmware_has_feature()

Fri, 06 May 2022 09:14:24 +0000

powerpc: Include asm/firmware.h in all users of firmware_has_feature()Trying to remove asm/ppc_asm.h from all places that don't need itleads to several failures linked to firmware_has_feature().To fix it, include asm/firmware.h in all files usingfirmware_has_feature()All users found with: git grep -L "firmware\.h" ` git grep -l "firmware_has_feature("`Signed-off-by: Christophe Leroy Signed-off-by: Michael Ellerman Link: https://lore.kernel.org/r/11956ec181a034b51a881ac9c059eea72c679a73.1651828453.git.christophe.leroy@csgroup.eu List of files: /linux-6.15/arch/powerpc/include/asm/interrupt.h

5352090a - powerpc/kasan: Don't instrument non-maskable or raw interrupts

Wed, 18 May 2022 10:06:17 +0000

powerpc/kasan: Don't instrument non-maskable or raw interruptsDisable address sanitization for raw and non-maskable interrupthandlers, because they can run in real mode, where we cannot accessthe shadow memory. (Note that kasan_arch_is_ready() doesn't test forreal mode, since it is a static branch for speed, and in any case notall the entry points to the generic KASAN code are protected bykasan_arch_is_ready guards.)The changes to interrupt_nmi_enter/exit_prepare() look larger thanthey actually are. The changes are equivalent to adding!IS_ENABLED(CONFIG_KASAN) to the conditions for calling nmi_enter() ornmi_exit() in real mode. That is, the code is equivalent to using thefollowing condition for calling nmi_enter/exit: if (((!IS_ENABLED(CONFIG_PPC_BOOK3S_64) || !firmware_has_feature(FW_FEATURE_LPAR) || radix_enabled()) && !IS_ENABLED(CONFIG_KASAN) || (mfmsr() & MSR_DR))That unwieldy condition has been split into several statements withcomments, for easier reading.The nmi_ipi_lock functions that call atomic functions (i.e.,nmi_ipi_lock_start(), nmi_ipi_lock() and nmi_ipi_unlock()), besidesbeing marked noinstr, now call arch_atomic_* functions instead ofatomic_* functions because with KASAN enabled, the atomic_* functionsare wrappers which explicitly do address sanitization on theirarguments. Since we are trying to avoid address sanitization, we haveto use the lower-level arch_atomic_* versions.In hv_nmi_check_nonrecoverable(), the regs_set_unrecoverable() callhas been open-coded so as to avoid having to either trust the inliningor mark regs_set_unrecoverable() as noinstr.[paulus@ozlabs.org: combined a few work-in-progress commits of Daniel's and wrote the commit message.]Signed-off-by: Daniel Axtens Signed-off-by: Paul Mackerras Signed-off-by: Michael Ellerman Link: https://lore.kernel.org/r/YoTFGaKM8Pd46PIK@cleo List of files: /linux-6.15/arch/powerpc/include/asm/interrupt.h

76222808 - powerpc: Move C prototypes out of asm-prototypes.h

Fri, 04 Mar 2022 17:04:05 +0000

powerpc: Move C prototypes out of asm-prototypes.hWe originally added asm-prototypes.h in commit 42f5b4cacd78 ("powerpc:Introduce asm-prototypes.h"). It's purpose was for prototypes of Cfunctions that are only called from asm, in order to fix sparsewarnings about missing prototypes.A few months later Nick added a different use case incommit 4efca4ed05cb ("kbuild: modversions for EXPORT_SYMBOL() for asm")for C prototypes for exported asm functions. This is basically theinverse of our original usage.Since then we've added various prototypes to asm-prototypes.h for bothreasons, meaning we now need to unstitch it all.Dispatch prototypes of C functions into relevant headers and keeponly the prototypes for functions defined in assembly.For the time being, leave prom_init() there because moving itinto asm/prom.h or asm/setup.h conflicts withdrivers/gpu/drm/nouveau/nvkm/subdev/bios/shadowrom.oThis will be fixed later by untaggling asm/pci.h and asm/prom.hor by renaming the function in shadowrom.cSigned-off-by: Christophe Leroy Signed-off-by: Michael Ellerman Link: https://lore.kernel.org/r/62d46904eca74042097acf4cb12c175e3067f3d1.1646413435.git.christophe.leroy@csgroup.eu List of files: /linux-6.15/arch/powerpc/include/asm/interrupt.h

973e2e64 - powerpc/interrupt: Remove struct interrupt_state

Fri, 25 Feb 2022 16:36:22 +0000

powerpc/interrupt: Remove struct interrupt_stateSince commit ceff77efa4f8 ("powerpc/64e/interrupt: Use new interruptcontext tracking scheme") struct interrupt_state has been empty andunused.Remove it.Signed-off-by: Christophe Leroy Signed-off-by: Michael Ellerman Link: https://lore.kernel.org/r/1d862ce3eab3da6ca7ac47d4a78a18f154462511.1645806970.git.christophe.leroy@csgroup.eu List of files: /linux-6.15/arch/powerpc/include/asm/interrupt.h

8b91cee5 - powerpc/64s/hash: Make hash faults work in NMI context

Fri, 04 Feb 2022 03:53:48 +0000

powerpc/64s/hash: Make hash faults work in NMI contextHash faults are not resoved in NMI context, instead causing the accessto fail. This is done because perf interrupts can get backtracesincluding walking the user stack, and taking a hash fault on those coulddeadlock on the HPTE lock if the perf interrupt hits while the same HPTElock is being held by the hash fault code. The user-access for the stackwalking will notice the access failed and deal with that in the perfcode.The reason to allow perf interrupts in is to better profile hash faults.The problem with this is any hash fault on a kernel access that happensin NMI context will crash, because kernel accesses must not fail.Hard lockups, system reset, machine checks that access vmalloc spaceincluding modules and including stack backtracing and symbol lookup inmodules, per-cpu data, etc could all run into this problem.Fix this by disallowing perf interrupts in the hash fault code (thedirect hash fault is covered by MSR[EE]=0 so the PMI disable just needsto extend to the preload case). This simplifies the tricky logic in hashfaults and perf, at the cost of reduced profiling of hash faults.perf can still latch addresses when interrupts are disabled, it justwon't get the stack trace at that point, so it would still find hotspots, just sometimes with confusing stack chains.An alternative could be to allow perf interrupts here but always do theslowpath stack walk if we are in nmi context, but that slows down allperf interrupt stack walking on hash though and it does not remove asmuch tricky code.Reported-by: Laurent Dufour Signed-off-by: Nicholas Piggin Tested-by: Laurent Dufour Reviewed-by: Aneesh Kumar K.V Signed-off-by: Michael Ellerman Link: https://lore.kernel.org/r/20220204035348.545435-1-npiggin@gmail.com List of files: /linux-6.15/arch/powerpc/include/asm/interrupt.h

ecb1057c - powerpc/64/interrupt: reduce expensive debug tests

Wed, 22 Sep 2021 14:54:51 +0000

powerpc/64/interrupt: reduce expensive debug testsMove the assertions requiring restart table searches underCONFIG_PPC_IRQ_SOFT_MASK_DEBUG.Signed-off-by: Nicholas Piggin Signed-off-by: Michael Ellerman Link: https://lore.kernel.org/r/20210922145452.352571-6-npiggin@gmail.com List of files: /linux-6.15/arch/powerpc/include/asm/interrupt.h

ff0b0d6e - powerpc/64s/interrupt: handle MSR EE and RI in interrupt entry wrapper

Wed, 22 Sep 2021 14:54:48 +0000

powerpc/64s/interrupt: handle MSR EE and RI in interrupt entry wrapperThe mtmsrd to enable MSR[RI] can be combined with the mtmsrd to enableMSR[EE] in interrupt entry code, for those interrupts which enable EE.This helps performance of important synchronous interrupts (e.g., pagefaults).This is similar to what commit dd152f70bdc1 ("powerpc/64s: system callavoid setting MSR[RI] until we set MSR[EE]") does for system calls.Do this by enabling EE and RI together at the beginning of the entrywrapper if PACA_IRQ_HARD_DIS is clear, and only enabling RI if it isset.Asynchronous interrupts set PACA_IRQ_HARD_DIS, but synchronous onesleave it unchanged, so by default they always get EE=1 unless they haveinterrupted a caller that is hard disabled. When the sync interruptlater calls interrupt_cond_local_irq_enable(), it will not requireanother mtmsrd because MSR[EE] was already enabled here.This avoids one mtmsrd L=1 for synchronous interrupts on 64s, whichsaves about 20 cycles on POWER9. And for kernel-mode interrupts, bothsynchronous and asynchronous, this saves an additional 40 cycles due tothe mtmsrd being moved ahead of mfspr SPRN_AMR, which prevents a SPRscoreboard stall.Signed-off-by: Nicholas Piggin Signed-off-by: Michael Ellerman Link: https://lore.kernel.org/r/20210922145452.352571-3-npiggin@gmail.com List of files: /linux-6.15/arch/powerpc/include/asm/interrupt.h

4423eb5a - powerpc/64/interrupt: make normal synchronous interrupts enable MSR[EE] if possible

Wed, 22 Sep 2021 14:54:47 +0000

powerpc/64/interrupt: make normal synchronous interrupts enable MSR[EE] if possibleMake synchronous interrupt handler entry wrappers enable MSR[EE] ifMSR[EE] was enabled in the interrupted context. IRQs are soft-disabledat this point so there is no change to high level code, but it's amasked interrupt could fire.This is a performance disadvantage for interrupts which do not latercall interrupt_cond_local_irq_enable(), because an an additional mtmsrdor wrtee instruction is executed. However the important synchronousinterrupts (e.g., page fault) do enable interrupts, so the performancedisadvantage is mostly avoided.In the next patch, MSR[RI] enabling can be combined with MSR[EE]enabling, which mitigates the performance drop for the former and givesa performance advanage for the latter interrupts, on 64s machines. 64eis coming along for the ride for now to avoid divergences with 64s inthis tricky code.Signed-off-by: Nicholas Piggin Signed-off-by: Michael Ellerman Link: https://lore.kernel.org/r/20210922145452.352571-2-npiggin@gmail.com List of files: /linux-6.15/arch/powerpc/include/asm/interrupt.h

42e03bc5 - powerpc/kuap: Prepare for supporting KUAP on BOOK3E/64

Tue, 19 Oct 2021 07:29:26 +0000

powerpc/kuap: Prepare for supporting KUAP on BOOK3E/64Also call kuap_lock() and kuap_save_and_lock() frominterrupt functions with CONFIG_PPC64.For book3s/64 we keep them empty as it is done in assembly.Also do the locked assert when switching task unless it isbook3s/64.Signed-off-by: Christophe Leroy Signed-off-by: Michael Ellerman Link: https://lore.kernel.org/r/1cbf94e26e6d6e2e028fd687588a7e6622d454a6.1634627931.git.christophe.leroy@csgroup.eu List of files: /linux-6.15/arch/powerpc/include/asm/interrupt.h

937fb700 - powerpc/kuap: Add kuap_lock()

Tue, 19 Oct 2021 07:29:23 +0000

powerpc/kuap: Add kuap_lock()Add kuap_lock() and call it when entering interrupts from user.It is called kuap_lock() as it is similar to kuap_save_and_lock()without the save.However book3s/32 already have a kuap_lock(). Rename itkuap_lock_addr().Signed-off-by: Christophe Leroy Signed-off-by: Michael Ellerman Link: https://lore.kernel.org/r/4437e2deb9f6f549f7089d45e9c6f96a7e77905a.1634627931.git.christophe.leroy@csgroup.eu List of files: /linux-6.15/arch/powerpc/include/asm/interrupt.h

526d4a4c - powerpc/32s: Do kuep_lock() and kuep_unlock() in assembly

Tue, 19 Oct 2021 07:29:17 +0000

powerpc/32s: Do kuep_lock() and kuep_unlock() in assemblyWhen interrupt and syscall entries where converted to C, KUEP lockingand unlocking was also converted. It improved performance by unrollingthe loop, and allowed easily implementing boot time deactivation ofKUEP.However, null_syscall selftest shows that KUEP is still heavy(361 cycles with KUEP, 212 cycles without).A way to improve more is to group 'mtsr's together, instead ofrepeating 'addi' + 'mtsr' several times.In order to do that, more registers need to be available. In C, GCCwill always be able to provide the requested number of registers, butat the cost of saving some data on the stack, which is counterperformant here.So let's do it in assembly, when we have full control of whichregister can be used. It also has the advantage of locking earlierand unlocking later and it helps GCC generating less tricky code.The only drawback is to make boot time deactivation less straightforward and require 'hand' instruction patching.Group 'mtsr's by 4.With this change, null_syscall selftest reports 336 cycles. Withoutthe change it was 361 cycles, that's a 7% reduction.Signed-off-by: Christophe Leroy Signed-off-by: Michael Ellerman Link: https://lore.kernel.org/r/115cb279e9b9948dfd93a065e047081c59e3a2a6.1634627931.git.christophe.leroy@csgroup.eu List of files: /linux-6.15/arch/powerpc/include/asm/interrupt.h