s390/entry: Pass the asce as parameter to sie64a()

Pass the guest ASCE explicitly as parameter, instead of having sie64a()
take it from lowcore.

This removes hidden state from lowcore, and makes th

s390/entry: Pass the asce as parameter to sie64a()

Pass the guest ASCE explicitly as parameter, instead of having sie64a()
take it from lowcore.

This removes hidden state from lowcore, and makes things look cleaner.

Signed-off-by: Claudio Imbrenda <[email protected]>
Reviewed-by: Nico Boehr <[email protected]>
Link: https://lore.kernel.org/r/[email protected]
Signed-off-by: Heiko Carstens <[email protected]>
Signed-off-by: Vasily Gorbik <[email protected]>

show more ...

s390/vdso: Introduce and use struct stack_frame_vdso_wrapper

Introduce and use struct stack_frame_vdso_wrapper within vdso user wrapper
code. With this structure it is possible to automatically gen

s390/vdso: Introduce and use struct stack_frame_vdso_wrapper

Introduce and use struct stack_frame_vdso_wrapper within vdso user wrapper
code. With this structure it is possible to automatically generate an
asm-offset define which can be used to save and restore the return address
of the calling function.

Also use STACK_FRAME_USER_OVERHEAD instead of STACK_FRAME_OVERHEAD to
document that the code works with user space stack frames with the standard
stack frame layout.

Fixes: aa44433ac4ee ("s390: add USER_STACKTRACE support")
Reviewed-by: Jens Remus <[email protected]>
Signed-off-by: Heiko Carstens <[email protected]>
Signed-off-by: Alexander Gordeev <[email protected]>

show more ...

s390/stacktrace: Merge perf_callchain_user() and arch_stack_walk_user()

The two functions perf_callchain_user() and arch_stack_walk_user() are
nearly identical. Reduce code duplication and add a com

s390/stacktrace: Merge perf_callchain_user() and arch_stack_walk_user()

The two functions perf_callchain_user() and arch_stack_walk_user() are
nearly identical. Reduce code duplication and add a common helper which can
be called by both functions.

Fixes: aa44433ac4ee ("s390: add USER_STACKTRACE support")
Reviewed-by: Jens Remus <[email protected]>
Signed-off-by: Heiko Carstens <[email protected]>
Signed-off-by: Alexander Gordeev <[email protected]>

show more ...

s390/switch_to: use generic header file

Move the switch_to() implementation to process.c and use the generic
switch_to.h header file instead, like some other architectures.

This addresses also the

s390/switch_to: use generic header file

Move the switch_to() implementation to process.c and use the generic
switch_to.h header file instead, like some other architectures.

This addresses also the oddity that the old switch_to() implementation
assigns the return value of __switch_to() to 'prev' instead of 'last',
like it should.

Remove also all includes of switch_to.h from C files, except process.c.

Acked-by: Alexander Gordeev <[email protected]>
Signed-off-by: Heiko Carstens <[email protected]>

show more ...

s390/perf: implement perf_callchain_user()

Daan De Meyer and Neal Gompa reported that s390 does not support perf user
stack unwinding.

This was never implemented since this requires user space to b

s390/perf: implement perf_callchain_user()

Daan De Meyer and Neal Gompa reported that s390 does not support perf user
stack unwinding.

This was never implemented since this requires user space to be compiled
with the -mbackchain compile option, which until now no distribution
did. However this is going to change with Fedora. Therefore provide a
perf_callchain_user() implementation.

Note that due to the way s390 sets up stack frames the provided call chains
can contain invalid values. This is especially true for the first stack
frame, where it is not possible to tell if the return address has been
written to the stack already or not.

Reported-by: Daan De Meyer <[email protected]>
Reported-by: Neal Gompa <[email protected]>
Closes: https://lore.kernel.org/all/CAO8sHcn3+_qrnvp0580aK7jN0Wion5F7KYeBAa4MnCY4mqABPA@mail.gmail.com/
Link: https://lore.kernel.org/all/[email protected]
Reviewed-by: Neal Gompa <[email protected]>
Acked-by: Ilya Leoshkevich <[email protected]>
Signed-off-by: Heiko Carstens <[email protected]>
Signed-off-by: Vasily Gorbik <[email protected]>

show more ...

s390/kdump: remove nodat stack restriction for calling nodat functions

To allow calling of DAT-off code from kernel the stack needs
to be switched to nodat_stack (or other stack mapped as 1:1).

Bef

s390/kdump: remove nodat stack restriction for calling nodat functions

To allow calling of DAT-off code from kernel the stack needs
to be switched to nodat_stack (or other stack mapped as 1:1).

Before call_nodat() macro was introduced that was necessary
to provide the very same memory address for STNSM and STOSM
instructions. If the kernel would stay on a random stack
(e.g. a virtually mapped one) then a virtual address provided
for STNSM instruction could differ from the physical address
needed for the corresponding STOSM instruction.

After call_nodat() macro is introduced the kernel stack does
not need to be mapped 1:1 anymore, since the macro stores the
physical memory address of return PSW in a register before
entering DAT-off mode. This way the return LPSWE instruction
is able to pick the correct memory location and restore the
DAT-on mode. That however might fail in case the 16-byte return
PSW happened to cross page boundary: PSW mask and PSW address
could end up in two separate non-contiguous physical pages.

Align the return PSW on 16-byte boundary so it always fits
into a single physical page. As result any stack (including
the virtually mapped one) could be used for calling DAT-off
code and prior switching to nodat_stack becomes unnecessary.

Signed-off-by: Alexander Gordeev <[email protected]>
Reviewed-by: Heiko Carstens <[email protected]>
Signed-off-by: Heiko Carstens <[email protected]>
Signed-off-by: Vasily Gorbik <[email protected]>

show more ...

s390/kdump: rework invocation of DAT-off code

Calling kdump kernel is a two-step process that involves
invocation of the purgatory code: first time - to verify
the new kernel checksum and second tim

s390/kdump: rework invocation of DAT-off code

Calling kdump kernel is a two-step process that involves
invocation of the purgatory code: first time - to verify
the new kernel checksum and second time - to call the new
kernel itself.

The purgatory code operates on real addresses and does not
expect any memory protection. Therefore, before the purgatory
code is entered the DAT mode is always turned off. However,
it is only restored upon return from the new kernel checksum
verification. In case the purgatory was called to start the
new kernel and failed the control is returned to the old
kernel, but the DAT mode continues staying off.

The new kernel start failure is unlikely and leads to the
disabled wait state anyway. Still that poses a risk, since
the kernel code in general is not DAT-off safe and even
calling the disabled_wait() function might crash.

Introduce call_nodat() macro that allows entering DAT-off
mode, calling an arbitrary function and restoring DAT mode
back on. Switch all invocations of DAT-off code to that
macro and avoid the above described scenario altogether.

Name the call_nodat() macro in small letters after the
already existing call_on_stack() and put it to the same
header file.

Signed-off-by: Alexander Gordeev <[email protected]>
Reviewed-by: Heiko Carstens <[email protected]>
[[email protected]: some small modifications to call_nodat() macro]
Signed-off-by: Heiko Carstens <[email protected]>
Signed-off-by: Vasily Gorbik <[email protected]>

show more ...

s390/stacktrace: remove call_on_stack_noreturn()

There is no user left of call_on_stack_noreturn() - remove it.

Reviewed-by: Vasily Gorbik <[email protected]>
Signed-off-by: Heiko Carstens <hca@lin

s390/stacktrace: remove call_on_stack_noreturn()

There is no user left of call_on_stack_noreturn() - remove it.

Reviewed-by: Vasily Gorbik <[email protected]>
Signed-off-by: Heiko Carstens <[email protected]>
Signed-off-by: Vasily Gorbik <[email protected]>

show more ...

s390/entry: sort out physical vs virtual pointers usage in sie64a

Fix virtual vs physical address confusion (which currently are the
same).

sie_block is accessed in entry.S and passed it to hardwar

s390/entry: sort out physical vs virtual pointers usage in sie64a

Fix virtual vs physical address confusion (which currently are the
same).

sie_block is accessed in entry.S and passed it to hardware, which is why
both its physical and virtual address are needed. To avoid every caller
having to do the virtual-physical conversion, add a new function sie64a()
which converts the virtual address to physical.

Signed-off-by: Nico Boehr <[email protected]>
Reviewed-by: Alexander Gordeev <[email protected]>
Reviewed-by: Claudio Imbrenda <[email protected]>
Link: https://lore.kernel.org/r/[email protected]
Message-Id: <[email protected]>
Signed-off-by: Janosch Frank <[email protected]>

show more ...

s390/stack: add union to reflect kvm stack slot usages

Add a union which describes how the empty stack slots are being used
by kvm and perf. This should help to avoid another bug like the one
which

s390/stack: add union to reflect kvm stack slot usages

Add a union which describes how the empty stack slots are being used
by kvm and perf. This should help to avoid another bug like the one
which was fixed with commit c9bfb460c3e4 ("s390/perf: obtain sie_block
from the right address").

Reviewed-by: Nico Boehr <[email protected]>
Tested-by: Nico Boehr <[email protected]>
Signed-off-by: Heiko Carstens <[email protected]>

show more ...

s390/stack: merge empty stack frame slots

Merge empty1 and empty2 arrays within the stack frame to one single
array. This is possible since with commit 42b01a553a56 ("s390: always
use the packed sta

s390/stack: merge empty stack frame slots

Merge empty1 and empty2 arrays within the stack frame to one single
array. This is possible since with commit 42b01a553a56 ("s390: always
use the packed stack layout") the alternative stack frame layout is
gone.

Reviewed-by: Nico Boehr <[email protected]>
Reviewed-by: Alexander Gordeev <[email protected]>
Signed-off-by: Heiko Carstens <[email protected]>

show more ...

s390: current_stack_pointer shouldn't be a function

s390 defines current_stack_pointer as function while all other
architectures use 'register unsigned long asm("<stackptr reg>").

This make codes l

s390: current_stack_pointer shouldn't be a function

s390 defines current_stack_pointer as function while all other
architectures use 'register unsigned long asm("<stackptr reg>").

This make codes like the following from check_stack_object() fail:

if (IS_ENABLED(CONFIG_STACK_GROWSUP)) {
if ((void *)current_stack_pointer < obj + len)
return BAD_STACK;
} else {
if (obj < (void *)current_stack_pointer)
return BAD_STACK;
}

because this would compare the address of current_stack_pointer() and
not the stackpointer value.

Reported-by: Karsten Graul <[email protected]>
Fixes: 2792d84e6da5 ("usercopy: Check valid lifetime via stack depth")
Cc: Kees Cook <[email protected]>
Cc: Vasily Gorbik <[email protected]>
Cc: Alexander Gordeev <[email protected]>
Signed-off-by: Sven Schnelle <[email protected]>
Reviewed-by: Heiko Carstens <[email protected]>
Signed-off-by: Heiko Carstens <[email protected]>

show more ...

s390: always use the packed stack layout

-mpacked-stack option has been supported by both minimum
gcc and clang versions for a while. With commit e2bc3e91d91e
("scripts/min-tool-version.sh: Raise mi

s390: always use the packed stack layout

-mpacked-stack option has been supported by both minimum
gcc and clang versions for a while. With commit e2bc3e91d91e
("scripts/min-tool-version.sh: Raise minimum clang version to 13.0.0
for s390") minimum clang version now also supports a combination
of flags -mpacked-stack -mbackchain -pg -mfentry and fulfills
all requirements to always enable the packed stack layout.

Reviewed-by: Heiko Carstens <[email protected]>
Signed-off-by: Vasily Gorbik <[email protected]>

show more ...

s390/unwind: use current_frame_address() to unwind current task

current_stack_pointer() simply returns current value of %r15. If
current_stack_pointer() caller allocates stack (which is the case in

s390/unwind: use current_frame_address() to unwind current task

current_stack_pointer() simply returns current value of %r15. If
current_stack_pointer() caller allocates stack (which is the case in
unwind code) %r15 points to a stack frame allocated for callees, meaning
current_stack_pointer() caller (e.g. stack_trace_save) will end up in
the stacktrace. This is not expected by stack_trace_save*() callers and
causes problems.

current_frame_address() on the other hand returns function stack frame
address, which matches %r15 upon function invocation. Using it in
get_stack_pointer() makes it more aligned with x86 implementation
(according to BACKTRACE_SELF_TEST output) and meets stack_trace_save*()
caller's expectations, notably KCSAN.

Also make sure unwind_start is always inlined.

Reported-by: Nathan Chancellor <[email protected]>
Suggested-by: Marco Elver <[email protected]>
Signed-off-by: Vasily Gorbik <[email protected]>
Tested-by: Marco Elver <[email protected]>
Tested-by: Nathan Chancellor <[email protected]>
Link: https://lore.kernel.org/r/patch.git-04dd26be3043.your-ad-here.call-01630504868-ext-6188@work.hours
Signed-off-by: Heiko Carstens <[email protected]>

show more ...

s390: rename CALL_ON_STACK_NORETURN() to call_on_stack_noreturn()

Lower case matches the call_on_stack() macro and is easier to read.

Reviewed-by: Sven Schnelle <[email protected]>
Signed-off-by:

s390: rename CALL_ON_STACK_NORETURN() to call_on_stack_noreturn()

Lower case matches the call_on_stack() macro and is easier to read.

Reviewed-by: Sven Schnelle <[email protected]>
Signed-off-by: Heiko Carstens <[email protected]>
Signed-off-by: Vasily Gorbik <[email protected]>

show more ...

s390: add type checking to CALL_ON_STACK_NORETURN() macro

Make sure the to be called function takes no arguments (and returns void).
Otherwise usage of CALL_ON_STACK_NORETURN() would generate broken

s390: add type checking to CALL_ON_STACK_NORETURN() macro

Make sure the to be called function takes no arguments (and returns void).
Otherwise usage of CALL_ON_STACK_NORETURN() would generate broken code.

Reviewed-by: Sven Schnelle <[email protected]>
Signed-off-by: Heiko Carstens <[email protected]>
Signed-off-by: Vasily Gorbik <[email protected]>

show more ...

s390: remove old CALL_ON_STACK() macro

Reviewed-by: Sven Schnelle <[email protected]>
Signed-off-by: Heiko Carstens <[email protected]>
Signed-off-by: Vasily Gorbik <[email protected]>

s390: introduce proper type handling call_on_stack() macro

The existing CALL_ON_STACK() macro allows for subtle bugs:

- There is no type checking of the function that is being called. That
is: mi

s390: introduce proper type handling call_on_stack() macro

The existing CALL_ON_STACK() macro allows for subtle bugs:

- There is no type checking of the function that is being called. That
is: missing or too many arguments do not cause any compile error or
warning. The same is true if the return type of the called function
changes. This can lead to quite random bugs.

- Sign and zero extension of arguments is missing. Given that the s390
C ABI requires that the caller of a function performs proper sign
and zero extension this can also lead to subtle bugs.

- If arguments to the CALL_ON_STACK() macros contain functions calls
register corruption can happen due to register asm constructs being
used.

Therefore introduce a new call_on_stack() macro which is supposed to
fix all these problems.

Reviewed-by: Sven Schnelle <[email protected]>
Signed-off-by: Heiko Carstens <[email protected]>
Signed-off-by: Vasily Gorbik <[email protected]>

show more ...

s390/stack: fix possible register corruption with stack switch helper

The CALL_ON_STACK macro is used to call a C function from inline
assembly, and therefore must consider the C ABI, which says tha

s390/stack: fix possible register corruption with stack switch helper

The CALL_ON_STACK macro is used to call a C function from inline
assembly, and therefore must consider the C ABI, which says that only
registers 6-13, and 15 are non-volatile (restored by the called
function).

The inline assembly incorrectly marks all registers used to pass
parameters to the called function as read-only input operands, instead
of operands that are read and written to. This might result in
register corruption depending on usage, compiler, and compile options.

Fix this by marking all operands used to pass parameters as read/write
operands. To keep the code simple even register 6, if used, is marked
as read-write operand.

Fixes: ff340d2472ec ("s390: add stack switch helper")
Cc: <[email protected]> # 4.20
Reviewed-by: Vasily Gorbik <[email protected]>
Signed-off-by: Heiko Carstens <[email protected]>
Signed-off-by: Vasily Gorbik <[email protected]>

show more ...

s390/unwind: add machine check handler stack

Fixes: b61b1595124a ("s390: add stack for machine check handler")
Signed-off-by: Vasily Gorbik <[email protected]>
Signed-off-by: Heiko Carstens <hca@lin

s390/unwind: add machine check handler stack

Fixes: b61b1595124a ("s390: add stack for machine check handler")
Signed-off-by: Vasily Gorbik <[email protected]>
Signed-off-by: Heiko Carstens <[email protected]>

show more ...

s390: fix register clobbering in CALL_ON_STACK

CALL_ON_STACK defines and initializes register variables. Inline
assembly which follows might trigger compiler to generate memory access
for "stack" ar

s390: fix register clobbering in CALL_ON_STACK

CALL_ON_STACK defines and initializes register variables. Inline
assembly which follows might trigger compiler to generate memory access
for "stack" argument (e.g. in case of S390_lowcore.nodat_stack). This
memory access produces a function call under kasan with outline
instrumentation which clobbers registers.

Switch "stack" argument in CALL_ON_STACK helper to use memory reference
constraint and perform load instead.

Reviewed-by: Heiko Carstens <[email protected]>
Signed-off-by: Vasily Gorbik <[email protected]>

show more ...

s390/unwind: always inline get_stack_pointer

Always inline get_stack_pointer() to avoid potential problems
due to compiler inlining decisions, i.e. getting stack pointer of
get_stack_pointer() itsel

s390/unwind: always inline get_stack_pointer

Always inline get_stack_pointer() to avoid potential problems
due to compiler inlining decisions, i.e. getting stack pointer of
get_stack_pointer() itself which is later reused.

Reviewed-by: Heiko Carstens <[email protected]>
Signed-off-by: Vasily Gorbik <[email protected]>

show more ...

s390: avoid misusing CALL_ON_STACK for task stack setup

CALL_ON_STACK is intended to be used for temporary stack switching with
potential return to the caller.

When CALL_ON_STACK is misused to swit

s390: avoid misusing CALL_ON_STACK for task stack setup

CALL_ON_STACK is intended to be used for temporary stack switching with
potential return to the caller.

When CALL_ON_STACK is misused to switch from nodat stack to task stack
back_chain information would later lead stack unwinder from task stack into
(per cpu) nodat stack which is reused for other purposes. This would
yield confusing unwinding result or errors.

To avoid that introduce CALL_ON_STACK_NORETURN to be used instead. It
makes sure that back_chain is zeroed and unwinder finishes gracefully
ending up at task pt_regs.

Reviewed-by: Heiko Carstens <[email protected]>
Signed-off-by: Vasily Gorbik <[email protected]>

show more ...

s390: correct CALL_ON_STACK back_chain saving

Currently CALL_ON_STACK saves r15 as back_chain in the first stack frame of
the stack we about to switch to. But if a function which uses CALL_ON_STACK

s390: correct CALL_ON_STACK back_chain saving

Currently CALL_ON_STACK saves r15 as back_chain in the first stack frame of
the stack we about to switch to. But if a function which uses CALL_ON_STACK
calls other function it allocates a stack frame for a callee. In this
case r15 is pointing to a callee stack frame and not a stack frame of
function itself. This results in dummy unwinding entry with random
sp and ip values.

Introduce and utilize current_frame_address macro to get an address of
actual function stack frame.

Reviewed-by: Heiko Carstens <[email protected]>
Signed-off-by: Vasily Gorbik <[email protected]>

show more ...

s390/unwind: unify task is current checks

Avoid mixture of task == NULL and task == current meaning the same
thing and simply always initialize task with current in unwind_start.

Reviewed-by: Heiko

s390/unwind: unify task is current checks

Avoid mixture of task == NULL and task == current meaning the same
thing and simply always initialize task with current in unwind_start.

Reviewed-by: Heiko Carstens <[email protected]>
Signed-off-by: Vasily Gorbik <[email protected]>

show more ...