ARM: remove Thumb2 __builtin_thread_pointer workaround for Clang

Now that the minimum supported version of LLVM for building the kernel has
been bumped to 13.0.1, the conditional expression added to

ARM: remove Thumb2 __builtin_thread_pointer workaround for Clang

Now that the minimum supported version of LLVM for building the kernel has
been bumped to 13.0.1, the conditional expression added to get_current()
by commit c1e42efacb9b ("ARM: 9151/1: Thumb2: avoid
__builtin_thread_pointer() on Clang") is always true, as the build will
fail during the configuration stage for older LLVM versions. Remove it,
effectively reverting the aforementioned change.

Link: https://lkml.kernel.org/r/20240125-bump-min-llvm-ver-to-13-0-1-v1-4-f5ff9bda41c5@kernel.org
Signed-off-by: Nathan Chancellor <[email protected]>
Reviewed-by: Kees Cook <[email protected]>
Cc: Albert Ou <[email protected]>
Cc: "Aneesh Kumar K.V (IBM)" <[email protected]>
Cc: Ard Biesheuvel <[email protected]>
Cc: Borislav Petkov (AMD) <[email protected]>
Cc: Catalin Marinas <[email protected]>
Cc: Conor Dooley <[email protected]>
Cc: Dave Hansen <[email protected]>
Cc: Ingo Molnar <[email protected]>
Cc: Mark Rutland <[email protected]>
Cc: Masahiro Yamada <[email protected]>
Cc: Michael Ellerman <[email protected]>
Cc: "Naveen N. Rao" <[email protected]>
Cc: Nicholas Piggin <[email protected]>
Cc: Nicolas Schier <[email protected]>
Cc: Palmer Dabbelt <[email protected]>
Cc: Paul Walmsley <[email protected]>
Cc: Russell King <[email protected]>
Cc: Thomas Gleixner <[email protected]>
Cc: Will Deacon <[email protected]>
Signed-off-by: Andrew Morton <[email protected]>

show more ...

ARM: make get_current() and __my_cpu_offset() __always_inline

The get_current() and __my_cpu_offset() accessors evaluate to only a
single instruction emitted inline, but due to the size of the asm s

ARM: make get_current() and __my_cpu_offset() __always_inline

The get_current() and __my_cpu_offset() accessors evaluate to only a
single instruction emitted inline, but due to the size of the asm string
that is created for SMP+v6 configurations, the compiler assumes
otherwise, and may emit the functions out of line instead.

So use __always_inline to avoid this.

Signed-off-by: Ard Biesheuvel <[email protected]>
Reviewed-by: Nick Desaulniers <[email protected]>

show more ...

ARM: assembler: define a Kconfig symbol for group relocation support

Nathan reports the group relocations go out of range in pathological
cases such as allyesconfig kernels, which have little chance

ARM: assembler: define a Kconfig symbol for group relocation support

Nathan reports the group relocations go out of range in pathological
cases such as allyesconfig kernels, which have little chance of actually
booting but are still used in validation.

So add a Kconfig symbol for this feature, and make it depend on
!COMPILE_TEST.

Signed-off-by: Ard Biesheuvel <[email protected]>

show more ...

ARM: 9176/1: avoid literal references in inline assembly

Nathan reports that the new get_current() and per-CPU offset accessors
may cause problems at build time due to the use of a literal to hold t

ARM: 9176/1: avoid literal references in inline assembly

Nathan reports that the new get_current() and per-CPU offset accessors
may cause problems at build time due to the use of a literal to hold the
address of the respective variables. This is due to the fact that LLD
before v14 does not support the PC-relative group relocations that are
normally used for this, and the fallback relies on literals but does not
emit the literal pools explictly using the .ltorg directive.

./arch/arm/include/asm/current.h:53:6: error: out of range pc-relative fixup value
asm(LOAD_SYM_ARMV6(%0, __current) : "=r"(cur));
^
./arch/arm/include/asm/insn.h:25:2: note: expanded from macro 'LOAD_SYM_ARMV6'
" ldr " #reg ", =" #sym " nt"
^
<inline asm>:1:3: note: instantiated into assembly here
ldr r0, =__current
^

Since emitting a literal pool in this particular case is not possible,
let's avoid the LOAD_SYM_ARMV6() entirely, and use the ordinary C
assigment instead.

As it turns out, there are other such cases, and here, using .ltorg to
emit the literal pool within range of the LDR instruction would be
possible due to the presence of an unconditional branch right after it.
Unfortunately, putting .ltorg directives in subsections appears to
confuse the Clang inline assembler, resulting in similar errors even
though the .ltorg is most definitely within range.

So let's fix this by emitting the literal explicitly, and not rely on
the assembler to figure this out. This means we have move the fallback
out of the LOAD_SYM_ARMV6() macro and into the callers.

Link: https://github.com/ClangBuiltLinux/linux/issues/1551

Fixes: 9c46929e7989 ("ARM: implement THREAD_INFO_IN_TASK for uniprocessor systems")
Reported-by: Nathan Chancellor <[email protected]>
Tested-by: Nathan Chancellor <[email protected]>
Signed-off-by: Ard Biesheuvel <[email protected]>
Signed-off-by: Russell King (Oracle) <[email protected]>

show more ...

ARM: implement THREAD_INFO_IN_TASK for uniprocessor systems

On UP systems, only a single task can be 'current' at the same time,
which means we can use a global variable to track it. This means we c

ARM: implement THREAD_INFO_IN_TASK for uniprocessor systems

On UP systems, only a single task can be 'current' at the same time,
which means we can use a global variable to track it. This means we can
also enable THREAD_INFO_IN_TASK for those systems, as in that case,
thread_info is accessed via current rather than the other way around,
removing the need to store thread_info at the base of the task stack.
This, in turn, permits us to enable IRQ stacks and vmap'ed stacks on UP
systems as well.

To partially mitigate the performance overhead of this arrangement, use
a ADD/ADD/LDR sequence with the appropriate PC-relative group
relocations to load the value of current when needed. This means that
accessing current will still only require a single load as before,
avoiding the need for a literal to carry the address of the global
variable in each function. However, accessing thread_info will now
require this load as well.

Acked-by: Linus Walleij <[email protected]>
Acked-by: Nicolas Pitre <[email protected]>
Signed-off-by: Ard Biesheuvel <[email protected]>
Tested-by: Marc Zyngier <[email protected]>
Tested-by: Vladimir Murzin <[email protected]> # ARMv7M

show more ...

ARM: 9151/1: Thumb2: avoid __builtin_thread_pointer() on Clang

If available, we use the __builtin_thread_pointer() helper to get the
value of the TLS register, to help the compiler understand that i

ARM: 9151/1: Thumb2: avoid __builtin_thread_pointer() on Clang

If available, we use the __builtin_thread_pointer() helper to get the
value of the TLS register, to help the compiler understand that it
doesn't need to reload it every time we access 'current'.

Unfortunately, Clang fails to emit the MRC system register read
directly when building for Thumb2, and instead, it issues a call to the
__aeabi_read_tp helper, which the kernel does not provide, and so this
result in link failures at build time.

So create a special case for this, and emit the MRC directly using an
asm() block, just like we do when the helper is not available to begin
with.

Link: https://github.com/ClangBuiltLinux/linux/issues/1485

Reviewed-by: Nick Desaulniers <[email protected]>
Reviewed-by: Nathan Chancellor <[email protected]>
Tested-by: Nathan Chancellor <[email protected]>
Reviewed-by: Kees Cook <[email protected]>
Signed-off-by: Ard Biesheuvel <[email protected]>
Signed-off-by: Russell King (Oracle) <[email protected]>

show more ...

ARM: smp: Store current pointer in TPIDRURO register if available

Now that the user space TLS register is assigned on every return to user
space, we can use it to keep the 'current' pointer while ru

ARM: smp: Store current pointer in TPIDRURO register if available

Now that the user space TLS register is assigned on every return to user
space, we can use it to keep the 'current' pointer while running in the
kernel. This removes the need to access it via thread_info, which is
located at the base of the stack, but will be moved out of there in a
subsequent patch.

Use the __builtin_thread_pointer() helper when available - this will
help GCC understand that reloading the value within the same function is
not necessary, even when using the per-task stack protector (which also
generates accesses via the TLS register). For example, the generated
code below loads TPIDRURO only once, and uses it to access both the
stack canary and the preempt_count fields.

<do_one_initcall>:
e92d 41f0 stmdb sp!, {r4, r5, r6, r7, r8, lr}
ee1d 4f70 mrc 15, 0, r4, cr13, cr0, {3}
4606 mov r6, r0
b094 sub sp, #80 ; 0x50
f8d4 34e8 ldr.w r3, [r4, #1256] ; 0x4e8 <- stack canary
9313 str r3, [sp, #76] ; 0x4c
f8d4 8004 ldr.w r8, [r4, #4] <- preempt count

Co-developed-by: Keith Packard <[email protected]>
Signed-off-by: Keith Packard <[email protected]>
Signed-off-by: Ard Biesheuvel <[email protected]>
Reviewed-by: Linus Walleij <[email protected]>
Tested-by: Amit Daniel Kachhap <[email protected]>

show more ...

[ARM] move include/asm-arm to arch/arm/include/asm

Move platform independent header files to arch/arm/include/asm, leaving
those in asm/arch* and asm/plat* alone.

Signed-off-by: Russell King <rmk+k

[ARM] move include/asm-arm to arch/arm/include/asm

Move platform independent header files to arch/arm/include/asm, leaving
those in asm/arch* and asm/plat* alone.

Signed-off-by: Russell King <[email protected]>

show more ...