arm64: Return early when break handler is found on linked-list

The search for breakpoint handlers iterate through the entire
linked list. Given that all registered hook has a valid fn field, and no

arm64: Return early when break handler is found on linked-list

The search for breakpoint handlers iterate through the entire
linked list. Given that all registered hook has a valid fn field, and no
registered hooks share the same mask and imm. This commit optimize the
efficiency slightly by returning early as a matching handler is found.

Signed-off-by: Liao Chang <[email protected]>
Acked-by: Will Deacon <[email protected]>
Link: https://lore.kernel.org/r/[email protected]
Signed-off-by: Catalin Marinas <[email protected]>

show more ...

arm64: mops: Handle MOPS exceptions from EL1

We will soon be using MOPS instructions in the kernel, so wire up the
exception handler to handle exceptions from EL1 caused by the copy/set
operation be

arm64: mops: Handle MOPS exceptions from EL1

We will soon be using MOPS instructions in the kernel, so wire up the
exception handler to handle exceptions from EL1 caused by the copy/set
operation being stopped and resumed on a different type of CPU.

Add a helper for advancing the single step state machine, similarly to
what the EL0 exception handler does.

Signed-off-by: Kristina Martsenko <[email protected]>
Link: https://lore.kernel.org/r/[email protected]
Signed-off-by: Catalin Marinas <[email protected]>

show more ...

arm64: Introduce esr_brk_comment, esr_is_cfi_brk

As it is already used in two places, move esr_comment() to a header for
re-use, with a clearer name.

Introduce esr_is_cfi_brk() to detect kCFI BRK s

arm64: Introduce esr_brk_comment, esr_is_cfi_brk

As it is already used in two places, move esr_comment() to a header for
re-use, with a clearer name.

Introduce esr_is_cfi_brk() to detect kCFI BRK syndromes, currently used
by early_brk64() but soon to also be used by hypervisor code.

Signed-off-by: Pierre-Clément Tosi <[email protected]>
Acked-by: Will Deacon <[email protected]>
Link: https://lore.kernel.org/r/[email protected]
Signed-off-by: Oliver Upton <[email protected]>

show more ...

arm64: kgdb: Set PSTATE.SS to 1 to re-enable single-step

Currently only the first attempt to single-step has any effect. After
that all further stepping remains "stuck" at the same program counter
v

arm64: kgdb: Set PSTATE.SS to 1 to re-enable single-step

Currently only the first attempt to single-step has any effect. After
that all further stepping remains "stuck" at the same program counter
value.

Refer to the ARM Architecture Reference Manual (ARM DDI 0487E.a) D2.12,
PSTATE.SS=1 should be set at each step before transferring the PE to the
'Active-not-pending' state. The problem here is PSTATE.SS=1 is not set
since the second single-step.

After the first single-step, the PE transferes to the 'Inactive' state,
with PSTATE.SS=0 and MDSCR.SS=1, thus PSTATE.SS won't be set to 1 due to
kernel_active_single_step()=true. Then the PE transferes to the
'Active-pending' state when ERET and returns to the debugger by step
exception.

Before this patch:
==================
Entering kdb (current=0xffff3376039f0000, pid 1) on processor 0 due to Keyboard Entry
[0]kdb>

[0]kdb>
[0]kdb> bp write_sysrq_trigger
Instruction(i) BP #0 at 0xffffa45c13d09290 (write_sysrq_trigger)
is enabled addr at ffffa45c13d09290, hardtype=0 installed=0

[0]kdb> go
$ echo h > /proc/sysrq-trigger

Entering kdb (current=0xffff4f7e453f8000, pid 175) on processor 1 due to Breakpoint @ 0xffffad651a309290
[1]kdb> ss

Entering kdb (current=0xffff4f7e453f8000, pid 175) on processor 1 due to SS trap @ 0xffffad651a309294
[1]kdb> ss

Entering kdb (current=0xffff4f7e453f8000, pid 175) on processor 1 due to SS trap @ 0xffffad651a309294
[1]kdb>

After this patch:
=================
Entering kdb (current=0xffff6851c39f0000, pid 1) on processor 0 due to Keyboard Entry
[0]kdb> bp write_sysrq_trigger
Instruction(i) BP #0 at 0xffffc02d2dd09290 (write_sysrq_trigger)
is enabled addr at ffffc02d2dd09290, hardtype=0 installed=0

[0]kdb> go
$ echo h > /proc/sysrq-trigger

Entering kdb (current=0xffff6851c53c1840, pid 174) on processor 1 due to Breakpoint @ 0xffffc02d2dd09290
[1]kdb> ss

Entering kdb (current=0xffff6851c53c1840, pid 174) on processor 1 due to SS trap @ 0xffffc02d2dd09294
[1]kdb> ss

Entering kdb (current=0xffff6851c53c1840, pid 174) on processor 1 due to SS trap @ 0xffffc02d2dd09298
[1]kdb> ss

Entering kdb (current=0xffff6851c53c1840, pid 174) on processor 1 due to SS trap @ 0xffffc02d2dd0929c
[1]kdb>

Fixes: 44679a4f142b ("arm64: KGDB: Add step debugging support")
Co-developed-by: Wei Li <[email protected]>
Signed-off-by: Wei Li <[email protected]>
Signed-off-by: Sumit Garg <[email protected]>
Tested-by: Douglas Anderson <[email protected]>
Acked-by: Daniel Thompson <[email protected]>
Tested-by: Daniel Thompson <[email protected]>
Link: https://lore.kernel.org/r/[email protected]
Signed-off-by: Will Deacon <[email protected]>

show more ...

arm64/sysreg: Add _EL1 into ID_AA64DFR0_EL1 definition names

Normally we include the full register name in the defines for fields within
registers but this has not been followed for ID registers. In

arm64/sysreg: Add _EL1 into ID_AA64DFR0_EL1 definition names

Normally we include the full register name in the defines for fields within
registers but this has not been followed for ID registers. In preparation
for automatic generation of defines add the _EL1s into the defines for
ID_AA64DFR0_EL1 to follow the convention. No functional changes.

Signed-off-by: Mark Brown <[email protected]>
Link: https://lore.kernel.org/r/[email protected]
Signed-off-by: Catalin Marinas <[email protected]>

show more ...

arm64/sysreg: Align field names in ID_AA64DFR0_EL1 with architecture

The naming scheme the architecture uses for the fields in ID_AA64DFR0_EL1
does not align well with kernel conventions, using as i

arm64/sysreg: Align field names in ID_AA64DFR0_EL1 with architecture

The naming scheme the architecture uses for the fields in ID_AA64DFR0_EL1
does not align well with kernel conventions, using as it does a lot of
MixedCase in various arrangements. In preparation for automatically
generating the defines for this register rename the defines used to match
what is in the architecture.

Signed-off-by: Mark Brown <[email protected]>
Link: https://lore.kernel.org/r/[email protected]
Signed-off-by: Catalin Marinas <[email protected]>

show more ...

arm64: Treat ESR_ELx as a 64-bit register

In the initial release of the ARM Architecture Reference Manual for
ARMv8-A, the ESR_ELx registers were defined as 32-bit registers. This
changed in 2018 wi

arm64: Treat ESR_ELx as a 64-bit register

In the initial release of the ARM Architecture Reference Manual for
ARMv8-A, the ESR_ELx registers were defined as 32-bit registers. This
changed in 2018 with version D.a (ARM DDI 0487D.a) of the architecture,
when they became 64-bit registers, with bits [63:32] defined as RES0. In
version G.a, a new field was added to ESR_ELx, ISS2, which covers bits
[36:32]. This field is used when the Armv8.7 extension FEAT_LS64 is
implemented.

As a result of the evolution of the register width, Linux stores it as
both a 64-bit value and a 32-bit value, which hasn't affected correctness
so far as Linux only uses the lower 32 bits of the register.

Make the register type consistent and always treat it as 64-bit wide. The
register is redefined as an "unsigned long", which is an unsigned
double-word (64-bit quantity) for the LP64 machine (aapcs64 [1], Table 1,
page 14). The type was chosen because "unsigned int" is the most frequent
type for ESR_ELx and because FAR_ELx, which is used together with ESR_ELx
in exception handling, is also declared as "unsigned long". The 64-bit type
also makes adding support for architectural features that use fields above
bit 31 easier in the future.

The KVM hypervisor will receive a similar update in a subsequent patch.

[1] https://github.com/ARM-software/abi-aa/releases/download/2021Q3/aapcs64.pdf

Signed-off-by: Alexandru Elisei <[email protected]>
Reviewed-by: Marc Zyngier <[email protected]>
Link: https://lore.kernel.org/r/[email protected]
Signed-off-by: Catalin Marinas <[email protected]>

show more ...

arm64: expose FAR_EL1 tag bits in siginfo

The kernel currently clears the tag bits (i.e. bits 56-63) in the fault
address exposed via siginfo.si_addr and sigcontext.fault_address. However,
the tag b

arm64: expose FAR_EL1 tag bits in siginfo

The kernel currently clears the tag bits (i.e. bits 56-63) in the fault
address exposed via siginfo.si_addr and sigcontext.fault_address. However,
the tag bits may be needed by tools in order to accurately diagnose
memory errors, such as HWASan [1] or future tools based on the Memory
Tagging Extension (MTE).

Expose these bits via the arch_untagged_si_addr mechanism, so that
they are only exposed to signal handlers with the SA_EXPOSE_TAGBITS
flag set.

[1] http://clang.llvm.org/docs/HardwareAssistedAddressSanitizerDesign.html

Signed-off-by: Peter Collingbourne <[email protected]>
Reviewed-by: Catalin Marinas <[email protected]>
Link: https://linux-review.googlesource.com/id/Ia8876bad8c798e0a32df7c2ce1256c4771c81446
Link: https://lore.kernel.org/r/0010296597784267472fa13b39f8238d87a72cf8.1605904350.git.pcc@google.com
Signed-off-by: Catalin Marinas <[email protected]>

show more ...

arm64: Improve diagnostics when trapping BRK with FAULT_BRK_IMM

When generating instructions at runtime, for example due to kernel text
patching or the BPF JIT, we can emit a trapping BRK instructio

arm64: Improve diagnostics when trapping BRK with FAULT_BRK_IMM

When generating instructions at runtime, for example due to kernel text
patching or the BPF JIT, we can emit a trapping BRK instruction if we
are asked to encode an invalid instruction such as an out-of-range]
branch. This is indicative of a bug in the caller, and will result in a
crash on executing the generated code. Unfortunately, the message from
the crash is really unhelpful, and mumbles something about ptrace:

| Unexpected kernel BRK exception at EL1
| Internal error: ptrace BRK handler: f2000100 [#1] SMP

We can do better than this. Install a break handler for FAULT_BRK_IMM,
which is the immediate used to encode the "I've been asked to generate
an invalid instruction" error, and triage the faulting PC to determine
whether or not the failure occurred in the BPF JIT.

Link: https://lore.kernel.org/r/20200915141707.GB26439@willie-the-truck
Reported-by: Ilias Apalodimas <[email protected]>
Signed-off-by: Will Deacon <[email protected]>

show more ...

arm64: Use test_tsk_thread_flag() for checking TIF_SINGLESTEP

Rather than open-code test_tsk_thread_flag() at each callsite, simply
replace the couple of offenders with calls to test_tsk_thread_flag

arm64: Use test_tsk_thread_flag() for checking TIF_SINGLESTEP

Rather than open-code test_tsk_thread_flag() at each callsite, simply
replace the couple of offenders with calls to test_tsk_thread_flag()
directly.

Signed-off-by: Will Deacon <[email protected]>

show more ...

arm64: ptrace: Override SPSR.SS when single-stepping is enabled

Luis reports that, when reverse debugging with GDB, single-step does not
function as expected on arm64:

| I've noticed, under very

arm64: ptrace: Override SPSR.SS when single-stepping is enabled

Luis reports that, when reverse debugging with GDB, single-step does not
function as expected on arm64:

| I've noticed, under very specific conditions, that a PTRACE_SINGLESTEP
| request by GDB won't execute the underlying instruction. As a consequence,
| the PC doesn't move, but we return a SIGTRAP just like we would for a
| regular successful PTRACE_SINGLESTEP request.

The underlying problem is that when the CPU register state is restored
as part of a reverse step, the SPSR.SS bit is cleared and so the hardware
single-step state can transition to the "active-pending" state, causing
an unexpected step exception to be taken immediately if a step operation
is attempted.

In hindsight, we probably shouldn't have exposed SPSR.SS in the pstate
accessible by the GPR regset, but it's a bit late for that now. Instead,
simply prevent userspace from configuring the bit to a value which is
inconsistent with the TIF_SINGLESTEP state for the task being traced.

Cc: <[email protected]>
Cc: Mark Rutland <[email protected]>
Cc: Keno Fischer <[email protected]>
Link: https://lore.kernel.org/r/[email protected]
Reported-by: Luis Machado <[email protected]>
Tested-by: Luis Machado <[email protected]>
Signed-off-by: Will Deacon <[email protected]>

show more ...

arm64: debug: mark a function as __init to save some memory

'debug_monitors_init()' is only called via 'postcore_initcall'.
It can be marked as __init to save a few bytes of memory.

Signed-off-by:

arm64: debug: mark a function as __init to save some memory

'debug_monitors_init()' is only called via 'postcore_initcall'.
It can be marked as __init to save a few bytes of memory.

Signed-off-by: Christophe JAILLET <[email protected]>
Reviewed-by: Douglas Anderson <[email protected]>
Link: https://lore.kernel.org/r/[email protected]
Signed-off-by: Will Deacon <[email protected]>

show more ...

arm64: Call debug_traps_init() from trap_init() to help early kgdb

A new kgdb feature will soon land (kgdb_earlycon) that lets us run
kgdb much earlier. In order for everything to work properly it'

arm64: Call debug_traps_init() from trap_init() to help early kgdb

A new kgdb feature will soon land (kgdb_earlycon) that lets us run
kgdb much earlier. In order for everything to work properly it's
important that the break hook is setup by the time we process
"kgdbwait".

Right now the break hook is setup in debug_traps_init() and that's
called from arch_initcall(). That's a bit too late since
kgdb_earlycon really needs things to be setup by the time the system
calls dbg_late_init().

We could fix this by adding call_break_hook() into early_brk64() and
that works fine. However, it's a little ugly. Instead, let's just
add a call to debug_traps_init() straight from trap_init(). There's
already a documented dependency between trap_init() and
debug_traps_init() and this makes the dependency more obvious rather
than just relying on a comment.

NOTE: this solution isn't early enough to let us select the
"ARCH_HAS_EARLY_DEBUG" KConfig option that is introduced by the
kgdb_earlycon patch series. That would only be set if we could do
breakpoints when early params are parsed. This patch only enables
"late early" breakpoints, AKA breakpoints when dbg_late_init() is
called. It's expected that this should be fine for most people.

It should also be noted that if you crash you can still end up in kgdb
earlier than debug_traps_init(). Since you don't need breakpoints to
debug a crash that's fine.

Suggested-by: Will Deacon <[email protected]>
Signed-off-by: Douglas Anderson <[email protected]>
Acked-by: Will Deacon <[email protected]>
Cc: Catalin Marinas <[email protected]>
Cc: Will Deacon <[email protected]>
Link: https://lore.kernel.org/r/20200513160501.1.I0b5edf030cc6ebef6ab4829f8867cdaea42485d8@changeid
Signed-off-by: Will Deacon <[email protected]>

show more ...

arm64: Remove unneeded rcu_read_lock from debug handlers

Remove rcu_read_lock()/rcu_read_unlock() from debug exception
handlers since we are sure those are not preemptible and
interrupts are off.

A

arm64: Remove unneeded rcu_read_lock from debug handlers

Remove rcu_read_lock()/rcu_read_unlock() from debug exception
handlers since we are sure those are not preemptible and
interrupts are off.

Acked-by: Paul E. McKenney <[email protected]>
Signed-off-by: Masami Hiramatsu <[email protected]>
Signed-off-by: Will Deacon <[email protected]>

show more ...

treewide: Replace GPLv2 boilerplate/reference with SPDX - rule 234

Based on 1 normalized pattern(s):

this program is free software you can redistribute it and or modify
it under the terms of th

treewide: Replace GPLv2 boilerplate/reference with SPDX - rule 234

Based on 1 normalized pattern(s):

this program is free software you can redistribute it and or modify
it under the terms of the gnu general public license version 2 as
published by the free software foundation this program is
distributed in the hope that it will be useful but without any
warranty without even the implied warranty of merchantability or
fitness for a particular purpose see the gnu general public license
for more details you should have received a copy of the gnu general
public license along with this program if not see http www gnu org
licenses

extracted by the scancode license scanner the SPDX license identifier

GPL-2.0-only

has been chosen to replace the boilerplate/reference in 503 file(s).

Signed-off-by: Thomas Gleixner <[email protected]>
Reviewed-by: Alexios Zavras <[email protected]>
Reviewed-by: Allison Randal <[email protected]>
Reviewed-by: Enrico Weigelt <[email protected]>
Cc: [email protected]
Link: https://lkml.kernel.org/r/[email protected]
Signed-off-by: Greg Kroah-Hartman <[email protected]>

show more ...

arm64: Clear OSDLR_EL1 on CPU boot

Some firmwares may reboot CPUs with OS Double Lock set. Make sure that
it is unlocked, in order to use debug exceptions.

Cc: <[email protected]>
Signed-off-b

arm64: Clear OSDLR_EL1 on CPU boot

Some firmwares may reboot CPUs with OS Double Lock set. Make sure that
it is unlocked, in order to use debug exceptions.

Cc: <[email protected]>
Signed-off-by: Jean-Philippe Brucker <[email protected]>
Signed-off-by: Will Deacon <[email protected]>

show more ...

arm64: debug: Clean up brk_handler()

brk_handler() now looks pretty strange and can be refactored to drop its
funny 'handler_found' local variable altogether.

Reviewed-by: Mark Rutland <mark.rutlan

arm64: debug: Clean up brk_handler()

brk_handler() now looks pretty strange and can be refactored to drop its
funny 'handler_found' local variable altogether.

Reviewed-by: Mark Rutland <[email protected]>
Signed-off-by: Will Deacon <[email protected]>

show more ...

arm64: probes: Move magic BRK values into brk-imm.h

kprobes and uprobes reserve some BRK immediates for installing their
probes. Define these along with the other reservations in brk-imm.h
and renam

arm64: probes: Move magic BRK values into brk-imm.h

kprobes and uprobes reserve some BRK immediates for installing their
probes. Define these along with the other reservations in brk-imm.h
and rename the ESR definitions to be consistent with the others that we
already have.

Reviewed-by: Mark Rutland <[email protected]>
Signed-off-by: Will Deacon <[email protected]>

show more ...

arm64: kprobes: Avoid calling kprobes debug handlers explicitly

Kprobes bypasses our debug hook registration code so that it doesn't
get tangled up with recursive debug exceptions from things like l

arm64: kprobes: Avoid calling kprobes debug handlers explicitly

Kprobes bypasses our debug hook registration code so that it doesn't
get tangled up with recursive debug exceptions from things like lockdep:

http://lists.infradead.org/pipermail/linux-arm-kernel/2015-February/324385.html

However, since then, (a) the hook list has become RCU protected and (b)
the kprobes hooks were found not to filter out exceptions from userspace
correctly. On top of that, the step handler is invoked directly from
single_step_handler(), which *does* use the debug hook list, so it's
clearly not the end of the world.

For now, have kprobes use the debug hook registration API like everybody
else. We can revisit this in the future if this is found to limit
coverage significantly.

Reviewed-by: Mark Rutland <[email protected]>
Signed-off-by: Will Deacon <[email protected]>

show more ...

arm64: debug: Separate debug hooks based on target exception level

Mixing kernel and user debug hooks together is highly error-prone as it
relies on all of the hooks to figure out whether the except

arm64: debug: Separate debug hooks based on target exception level

Mixing kernel and user debug hooks together is highly error-prone as it
relies on all of the hooks to figure out whether the exception came from
kernel or user, and then to act accordingly.

Make our debug hook code a little more robust by maintaining separate
hook lists for user and kernel, with separate registration functions
to force callers to be explicit about the exception levels that they
care about.

Reviewed-by: Mark Rutland <[email protected]>
Signed-off-by: Will Deacon <[email protected]>

show more ...

arm64: debug: Remove meaningless comment

The comment next to the definition of our 'break_hook' list head is
at best wrong but mainly just meaningless. Rip it out.

Reviewed-by: Mark Rutland <mark.r

arm64: debug: Remove meaningless comment

The comment next to the definition of our 'break_hook' list head is
at best wrong but mainly just meaningless. Rip it out.

Reviewed-by: Mark Rutland <[email protected]>
Signed-off-by: Will Deacon <[email protected]>

show more ...

arm64: debug: Rename addr parameter for non-watchpoint exception hooks

Since the 'addr' parameter contains an UNKNOWN value for non-watchpoint
debug exceptions, rename it to 'unused' for those hooks

arm64: debug: Rename addr parameter for non-watchpoint exception hooks

Since the 'addr' parameter contains an UNKNOWN value for non-watchpoint
debug exceptions, rename it to 'unused' for those hooks so we don't get
tempted to use it in the future.

Reviewed-by: Mark Rutland <[email protected]>
Signed-off-by: Will Deacon <[email protected]>

show more ...

signal/arm64: Add and use arm64_force_sig_fault where appropriate

Wrap force_sig_fault with a helper that calls arm64_show_signal
and call arm64_force_sig_fault where appropraite.

Reviewed-by: Cata

signal/arm64: Add and use arm64_force_sig_fault where appropriate

Wrap force_sig_fault with a helper that calls arm64_show_signal
and call arm64_force_sig_fault where appropraite.

Reviewed-by: Catalin Marinas <[email protected]>
Tested-by: Catalin Marinas <[email protected]>
Signed-off-by: Eric W. Biederman <[email protected]>

show more ...

signal/arm64: Remove unneeded tsk parameter from arm64_force_sig_info

Every caller passes in current for tsk so there is no need to pass
tsk. Instead make tsk a local variable initialized to curren

signal/arm64: Remove unneeded tsk parameter from arm64_force_sig_info

Every caller passes in current for tsk so there is no need to pass
tsk. Instead make tsk a local variable initialized to current.

Reviewed-by: Catalin Marinas <[email protected]>
Tested-by: Catalin Marinas <[email protected]>
Signed-off-by: "Eric W. Biederman" <[email protected]>

show more ...

arm64: Use arm64_force_sig_info instead of force_sig_info

Using arm64_force_sig_info means that printing messages about unhandled
signals is dealt with for us, so use that in preference to force_sig

arm64: Use arm64_force_sig_info instead of force_sig_info

Using arm64_force_sig_info means that printing messages about unhandled
signals is dealt with for us, so use that in preference to force_sig_info
and remove any homebrew printing code.

Signed-off-by: Will Deacon <[email protected]>

show more ...