mm: make arch_get_unmapped_area() take vm_flags by default

Patch series "mm: Care about shadow stack guard gap when getting an
unmapped area", v2.

As covered in the commit log for c44357c2e76b ("x8

mm: make arch_get_unmapped_area() take vm_flags by default

Patch series "mm: Care about shadow stack guard gap when getting an
unmapped area", v2.

As covered in the commit log for c44357c2e76b ("x86/mm: care about shadow
stack guard gap during placement") our current mmap() implementation does
not take care to ensure that a new mapping isn't placed with existing
mappings inside it's own guard gaps. This is particularly important for
shadow stacks since if two shadow stacks end up getting placed adjacent to
each other then they can overflow into each other which weakens the
protection offered by the feature.

On x86 there is a custom arch_get_unmapped_area() which was updated by the
above commit to cover this case by specifying a start_gap for allocations
with VM_SHADOW_STACK. Both arm64 and RISC-V have equivalent features and
use the generic implementation of arch_get_unmapped_area() so let's make
the equivalent change there so they also don't get shadow stack pages
placed without guard pages. The arm64 and RISC-V shadow stack
implementations are currently on the list:

https://lore.kernel.org/r/20240829-arm64-gcs-v12-0-42fec94743
https://lore.kernel.org/lkml/[email protected]/

Given the addition of the use of vm_flags in the generic implementation we
also simplify the set of possibilities that have to be dealt with in the
core code by making arch_get_unmapped_area() take vm_flags as standard.
This is a bit invasive since the prototype change touches quite a few
architectures but since the parameter is ignored the change is
straightforward, the simplification for the generic code seems worth it.


This patch (of 3):

When we introduced arch_get_unmapped_area_vmflags() in 961148704acd ("mm:
introduce arch_get_unmapped_area_vmflags()") we did so as part of properly
supporting guard pages for shadow stacks on x86_64, which uses a custom
arch_get_unmapped_area(). Equivalent features are also present on both
arm64 and RISC-V, both of which use the generic implementation of
arch_get_unmapped_area() and will require equivalent modification there.
Rather than continue to deal with having two versions of the functions
let's bite the bullet and have all implementations of
arch_get_unmapped_area() take vm_flags as a parameter.

The new parameter is currently ignored by all implementations other than
x86. The only caller that doesn't have a vm_flags available is
mm_get_unmapped_area(), as for the x86 implementation and the wrapper used
on other architectures this is modified to supply no flags.

No functional changes.

Link: https://lkml.kernel.org/r/20240904-mm-generic-shadow-stack-guard-v2-0-a46b8b6dc0ed@kernel.org
Link: https://lkml.kernel.org/r/20240904-mm-generic-shadow-stack-guard-v2-1-a46b8b6dc0ed@kernel.org
Signed-off-by: Mark Brown <[email protected]>
Acked-by: Lorenzo Stoakes <[email protected]>
Reviewed-by: Liam R. Howlett <[email protected]>
Acked-by: Helge Deller <[email protected]> [parisc]
Cc: Alexander Gordeev <[email protected]>
Cc: Andreas Larsson <[email protected]>
Cc: Borislav Petkov <[email protected]>
Cc: Christian Borntraeger <[email protected]>
Cc: Christophe Leroy <[email protected]>
Cc: Chris Zankel <[email protected]>
Cc: Dave Hansen <[email protected]>
Cc: David S. Miller <[email protected]>
Cc: "Edgecombe, Rick P" <[email protected]>
Cc: Gerald Schaefer <[email protected]>
Cc: Guo Ren <[email protected]>
Cc: Heiko Carstens <[email protected]>
Cc: "H. Peter Anvin" <[email protected]>
Cc: Huacai Chen <[email protected]>
Cc: Ingo Molnar <[email protected]>
Cc: Ivan Kokshaysky <[email protected]>
Cc: James Bottomley <[email protected]>
Cc: John Paul Adrian Glaubitz <[email protected]>
Cc: Matt Turner <[email protected]>
Cc: Max Filippov <[email protected]>
Cc: Michael Ellerman <[email protected]>
Cc: Naveen N Rao <[email protected]>
Cc: Nicholas Piggin <[email protected]>
Cc: Richard Henderson <[email protected]>
Cc: Rich Felker <[email protected]>
Cc: Russell King <[email protected]>
Cc: Sven Schnelle <[email protected]>
Cc: Thomas Bogendoerfer <[email protected]>
Cc: Thomas Gleixner <[email protected]>
Cc: Vasily Gorbik <[email protected]>
Cc: Vineet Gupta <[email protected]>
Cc: Vlastimil Babka <[email protected]>
Cc: WANG Xuerui <[email protected]>
Cc: Yoshinori Sato <[email protected]>
Signed-off-by: Andrew Morton <[email protected]>

show more ...

xtensa: remove vma linked list walks

Use the VMA iterator instead. Since VMA can no longer be NULL in the
loop, then deal with out-of-memory outside the loop. This means a
slightly longer run time

xtensa: remove vma linked list walks

Use the VMA iterator instead. Since VMA can no longer be NULL in the
loop, then deal with out-of-memory outside the loop. This means a
slightly longer run time in the failure case (-ENOMEM) - it will run to
the end of the VMAs before erroring instead of in the middle of the loop.

Link: https://lkml.kernel.org/r/[email protected]
Signed-off-by: Matthew Wilcox (Oracle) <[email protected]>
Signed-off-by: Liam R. Howlett <[email protected]>
Reviewed-by: Davidlohr Bueso <[email protected]>
Tested-by: Yu Zhao <[email protected]>
Cc: Catalin Marinas <[email protected]>
Cc: David Hildenbrand <[email protected]>
Cc: David Howells <[email protected]>
Cc: SeongJae Park <[email protected]>
Cc: Sven Schnelle <[email protected]>
Cc: Vlastimil Babka <[email protected]>
Cc: Will Deacon <[email protected]>
Signed-off-by: Andrew Morton <[email protected]>

show more ...

xtensa: syscalls: switch to generic syscalltbl.sh

Many architectures duplicate similar shell scripts.

This commit converts xtensa to use scripts/syscalltbl.sh.

Signed-off-by: Masahiro Yamada <masa

xtensa: syscalls: switch to generic syscalltbl.sh

Many architectures duplicate similar shell scripts.

This commit converts xtensa to use scripts/syscalltbl.sh.

Signed-off-by: Masahiro Yamada <[email protected]>
Message-Id: <[email protected]>
Signed-off-by: Max Filippov <[email protected]>

show more ...

xtensa: stop filling syscall array with sys_ni_syscall

arch/xtensa/kernel/syscalls/syscalltbl.sh fills missing syscall numbers
with sys_ni_syscall.

So, the generated arch/xtensa/include/generated/a

xtensa: stop filling syscall array with sys_ni_syscall

arch/xtensa/kernel/syscalls/syscalltbl.sh fills missing syscall numbers
with sys_ni_syscall.

So, the generated arch/xtensa/include/generated/asm/syscall_table.h
has no hole.

Hence, the line:

[0 ... __NR_syscalls - 1] = (syscall_t)&sys_ni_syscall,

is meaningless.

The number of generated __SYSCALL() macros is the same as __NR_syscalls
(this is 442 as of v5.11).

Hence, the array size, [__NR_syscalls] is unneeded.

The designated initializer, '[nr] =', is also unneeded.

This file does not need to know __NR_syscalls. Drop the unneeded
<asm/unistd.h> include directive.

Signed-off-by: Masahiro Yamada <[email protected]>
Message-Id: <[email protected]>
Signed-off-by: Max Filippov <[email protected]>

show more ...

xtensa: implement syscall tracepoints

Add TIF_SYSCALL_TRACEPOINT flag definition; add _TIF_SYSCALL_TRACEPOINT
to _TIF_WORK_MASK. Call trace_sys_enter from do_syscall_trace_enter and
trace_sys_exit f

xtensa: implement syscall tracepoints

Add TIF_SYSCALL_TRACEPOINT flag definition; add _TIF_SYSCALL_TRACEPOINT
to _TIF_WORK_MASK. Call trace_sys_enter from do_syscall_trace_enter and
trace_sys_exit from do_syscall_trace_leave when TIF_SYSCALL_TRACEPOINT
flag is set.
Add declaration of sys_call_table to arch/xtensa/include/asm/syscall.h
Add definition of NR_syscalls to arch/xtensa/include/asm/unistd.h
Select HAVE_SYSCALL_TRACEPOINTS.

This change allows tracing each syscall entry and exit through the
ftrace mechanism.

Signed-off-by: Max Filippov <[email protected]>

show more ...

xtensa: generate uapi header and syscall table header files

System call table generation script must be run to gener-
ate unistd_32.h and syscall_table.h files. This patch will
have changes which wi

xtensa: generate uapi header and syscall table header files

System call table generation script must be run to gener-
ate unistd_32.h and syscall_table.h files. This patch will
have changes which will invokes the script.

This patch will generate unistd_32.h and syscall_table.h
files by the syscall table generation script invoked by
xtensa/Makefile and the generated files against the removed
files must be identical.

The generated uapi header file will be included in uapi/-
asm/unistd.h and generated system call table header file
will be included by kernel/syscall.c file.

Signed-off-by: Firoz Khan <[email protected]>
Signed-off-by: Max Filippov <[email protected]>

show more ...

xtensa: add __NR_syscalls along with __NR_syscall_count

__NR_syscall_count macro holds the number of system call
exist in xtensa architecture. We have to change the value
of __NR_syscall_count, if w

xtensa: add __NR_syscalls along with __NR_syscall_count

__NR_syscall_count macro holds the number of system call
exist in xtensa architecture. We have to change the value
of __NR_syscall_count, if we add or delete a system call.

One of the patch in this patch series has a script which
will generate a uapi header based on syscall.tbl file.
The syscall.tbl file contains the total number of system
calls information. So we have two option to update __NR-
_syscall_count value.

1. Update __NR_syscall_count in asm/unistd.h manually by
counting the no.of system calls. No need to update __NR-
_syscall_count until we either add a new system call or
delete existing system call.

2. We can keep this feature it above mentioned script,
that will count the number of syscalls and keep it in
a generated file. In this case we don't need to expli-
citly update __NR_syscall_count in asm/unistd.h file.

The 2nd option will be the recommended one. For that, I
added the __NR_syscalls macro in uapi/asm/unistd.h.
The macro __NR_syscalls also added for making the name
convention same across all architecture. While __NR_syscalls
isn't strictly part of the uapi, having it as part of the
generated header to simplifies the implementation. We also
need to enclose this macro with #ifdef __KERNEL__ to avoid
side effects.

Signed-off-by: Firoz Khan <[email protected]>
Signed-off-by: Max Filippov <[email protected]>
[Max: Drop __NR_syscall_count completely, use __NR_syscalls instead]

show more ...

mm: add ksys_fadvise64_64() helper; remove in-kernel call to sys_fadvise64_64()

Using the ksys_fadvise64_64() helper allows us to avoid the in-kernel
calls to the sys_fadvise64_64() syscall. The ksy

mm: add ksys_fadvise64_64() helper; remove in-kernel call to sys_fadvise64_64()

Using the ksys_fadvise64_64() helper allows us to avoid the in-kernel
calls to the sys_fadvise64_64() syscall. The ksys_ prefix denotes that
this function is meant as a drop-in replacement for the syscall. In
particular, it uses the same calling convention as ksys_fadvise64_64().

Some compat stubs called sys_fadvise64(), which then just passed through
the arguments to sys_fadvise64_64(). Get rid of this indirection, and call
ksys_fadvise64_64() directly.

This patch is part of a series which removes in-kernel calls to syscalls.
On this basis, the syscall entry path can be streamlined. For details, see
http://lkml.kernel.org/r/[email protected]

Cc: Andrew Morton <[email protected]>
Cc: [email protected]
Signed-off-by: Dominik Brodowski <[email protected]>

show more ...

mm: larger stack guard gap, between vmas

Stack guard page is a useful feature to reduce a risk of stack smashing
into a different mapping. We have been using a single page gap which
is sufficient to

mm: larger stack guard gap, between vmas

Stack guard page is a useful feature to reduce a risk of stack smashing
into a different mapping. We have been using a single page gap which
is sufficient to prevent having stack adjacent to a different mapping.
But this seems to be insufficient in the light of the stack usage in
userspace. E.g. glibc uses as large as 64kB alloca() in many commonly
used functions. Others use constructs liks gid_t buffer[NGROUPS_MAX]
which is 256kB or stack strings with MAX_ARG_STRLEN.

This will become especially dangerous for suid binaries and the default
no limit for the stack size limit because those applications can be
tricked to consume a large portion of the stack and a single glibc call
could jump over the guard page. These attacks are not theoretical,
unfortunatelly.

Make those attacks less probable by increasing the stack guard gap
to 1MB (on systems with 4k pages; but make it depend on the page size
because systems with larger base pages might cap stack allocations in
the PAGE_SIZE units) which should cover larger alloca() and VLA stack
allocations. It is obviously not a full fix because the problem is
somehow inherent, but it should reduce attack space a lot.

One could argue that the gap size should be configurable from userspace,
but that can be done later when somebody finds that the new 1MB is wrong
for some special case applications. For now, add a kernel command line
option (stack_guard_gap) to specify the stack gap size (in page units).

Implementation wise, first delete all the old code for stack guard page:
because although we could get away with accounting one extra page in a
stack vma, accounting a larger gap can break userspace - case in point,
a program run with "ulimit -S -v 20000" failed when the 1MB gap was
counted for RLIMIT_AS; similar problems could come with RLIMIT_MLOCK
and strict non-overcommit mode.

Instead of keeping gap inside the stack vma, maintain the stack guard
gap as a gap between vmas: using vm_start_gap() in place of vm_start
(or vm_end_gap() in place of vm_end if VM_GROWSUP) in just those few
places which need to respect the gap - mainly arch_get_unmapped_area(),
and and the vma tree's subtree_gap support for that.

Original-patch-by: Oleg Nesterov <[email protected]>
Original-patch-by: Michal Hocko <[email protected]>
Signed-off-by: Hugh Dickins <[email protected]>
Acked-by: Michal Hocko <[email protected]>
Tested-by: Helge Deller <[email protected]> # parisc
Signed-off-by: Linus Torvalds <[email protected]>

show more ...

sched/headers: Prepare for new header dependencies before moving more code to <linux/sched/mm.h>

We are going to split more MM APIs out of <linux/sched.h>, which
will have to be picked up from a cou

sched/headers: Prepare for new header dependencies before moving more code to <linux/sched/mm.h>

We are going to split more MM APIs out of <linux/sched.h>, which
will have to be picked up from a couple of .c files.

The APIs that we are going to move are:

arch_pick_mmap_layout()
arch_get_unmapped_area()
arch_get_unmapped_area_topdown()
mm_update_next_owner()

Include the header in the files that are going to need it.

Acked-by: Linus Torvalds <[email protected]>
Cc: Mike Galbraith <[email protected]>
Cc: Peter Zijlstra <[email protected]>
Cc: Thomas Gleixner <[email protected]>
Cc: [email protected]
Signed-off-by: Ingo Molnar <[email protected]>

show more ...

Replace <asm/uaccess.h> with <linux/uaccess.h> globally

This was entirely automated, using the script by Al:

PATT='^[[:blank:]]*#[[:blank:]]*include[[:blank:]]*<asm/uaccess.h>'
sed -i -e "s!$PA

Replace <asm/uaccess.h> with <linux/uaccess.h> globally

This was entirely automated, using the script by Al:

PATT='^[[:blank:]]*#[[:blank:]]*include[[:blank:]]*<asm/uaccess.h>'
sed -i -e "s!$PATT!#include <linux/uaccess.h>!" \
$(git grep -l "$PATT"|grep -v ^include/linux/uaccess.h)

to do the replacement at the end of the merge window.

Requested-by: Al Viro <[email protected]>
Signed-off-by: Linus Torvalds <[email protected]>

show more ...

xtensa: nommu: don't provide arch_get_unmapped_area

Nommu unconditionally provides arch_get_unmapped_area that always
returns -ENOMEM.

Signed-off-by: Max Filippov <[email protected]>

xtensa: avoid mmap cache aliasing

Provide arch_get_unmapped_area function aligning shared memory mapping
addresses to the biggest of the page size or the cache way size. That
guarantees that corresp

xtensa: avoid mmap cache aliasing

Provide arch_get_unmapped_area function aligning shared memory mapping
addresses to the biggest of the page size or the cache way size. That
guarantees that corresponding virtual addresses of shared mappings are
cached by the same cache sets.

Signed-off-by: Max Filippov <[email protected]>
Signed-off-by: Chris Zankel <[email protected]>

show more ...

xtensa: clean up files to make them code-style compliant

Remove heading and trailing spaces, trim trailing lines, and wrap lines
that are longer than 80 characters.

Signed-off-by: Chris Zankel <chr

xtensa: clean up files to make them code-style compliant

Remove heading and trailing spaces, trim trailing lines, and wrap lines
that are longer than 80 characters.

Signed-off-by: Chris Zankel <[email protected]>

show more ...

xtensa: allow multi-inclusion for uapi/unistd.h

Xtensa implements a method that allows to generate a arbitrary output
for each system call by defining the __SYSCALL(number, function, num_args).
This

xtensa: allow multi-inclusion for uapi/unistd.h

Xtensa implements a method that allows to generate a arbitrary output
for each system call by defining the __SYSCALL(number, function, num_args).
This usually requires to include uapi/unistd.h twice. Instead of removing
the guard agains multiple inclusion entirely, allow to include unistd.h again
only if __SYSCALL is defined. Note that __SYSCALL gets always undefined at
the end of the file.

Signed-off-by: Chris Zankel <[email protected]>

show more ...

UAPI: (Scripted) Disintegrate arch/xtensa/include/asm

UAPI: (Scripted) Disintegrate arch/xtensa/include/asm

Signed-off-by: David Howells <[email protected]>
Acked-by: Arnd Bergmann <[email protected]

UAPI: (Scripted) Disintegrate arch/xtensa/include/asm

UAPI: (Scripted) Disintegrate arch/xtensa/include/asm

Signed-off-by: David Howells <[email protected]>
Acked-by: Arnd Bergmann <[email protected]>
Acked-by: Thomas Gleixner <[email protected]>
Acked-by: Michael Kerrisk <[email protected]>
Acked-by: Paul E. McKenney <[email protected]>
Acked-by: Dave Jones <[email protected]>
Signed-off-by: Max Filippov <[email protected]>
Signed-off-by: Chris Zankel <[email protected]>

show more ...

ipc: add COMPAT_SHMLBA support

If the SHMLBA definition for a native task differs from the definition for
a compat task, the do_shmat() function would need to handle both.

This patch introduces COM

ipc: add COMPAT_SHMLBA support

If the SHMLBA definition for a native task differs from the definition for
a compat task, the do_shmat() function would need to handle both.

This patch introduces COMPAT_SHMLBA, which is used by the compat shmat
syscall when calling the ipc code and allows architectures such as AArch64
(where the native SHMLBA is 64k but the compat (AArch32) definition is
16k) to provide the correct semantics for compat IPC system calls.

Cc: David S. Miller <[email protected]>
Cc: Chris Zankel <[email protected]>
Cc: Arnd Bergmann <[email protected]>
Acked-by: Catalin Marinas <[email protected]>
Signed-off-by: Will Deacon <[email protected]>
Signed-off-by: Andrew Morton <[email protected]>
Signed-off-by: Linus Torvalds <[email protected]>

show more ...

sanitize do_pipe_flags() callers in arch

* hpux_pipe() - no need to take BKL
* sys32_pipe() in arch/x86/ia32 and xtensa_pipe() in arch/xtensa -
no need at all, since both functions are open-coded s

sanitize do_pipe_flags() callers in arch

* hpux_pipe() - no need to take BKL
* sys32_pipe() in arch/x86/ia32 and xtensa_pipe() in arch/xtensa -
no need at all, since both functions are open-coded sys_pipe()

Signed-off-by: Al Viro <[email protected]>

show more ...

Unify sys_mmap*

New helper - sys_mmap_pgoff(); switch syscalls to using it.

Acked-by: David S. Miller <[email protected]>
Signed-off-by: Al Viro <[email protected]>

flag parameters: pipe

This patch introduces the new syscall pipe2 which is like pipe but it also
takes an additional parameter which takes a flag value. This patch implements
the handling of O_CLOE

flag parameters: pipe

This patch introduces the new syscall pipe2 which is like pipe but it also
takes an additional parameter which takes a flag value. This patch implements
the handling of O_CLOEXEC for the flag. I did not add support for the new
syscall for the architectures which have a special sys_pipe implementation. I
think the maintainers of those archs have the chance to go with the unified
implementation but that's up to them.

The implementation introduces do_pipe_flags. I did that instead of changing
all callers of do_pipe because some of the callers are written in assembler.
I would probably screw up changing the assembly code. To avoid breaking code
do_pipe is now a small wrapper around do_pipe_flags. Once all callers are
changed over to do_pipe_flags the old do_pipe function can be removed.

The following test must be adjusted for architectures other than x86 and
x86-64 and in case the syscall numbers changed.

~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
#include <fcntl.h>
#include <stdio.h>
#include <unistd.h>
#include <sys/syscall.h>

#ifndef __NR_pipe2
# ifdef __x86_64__
# define __NR_pipe2 293
# elif defined __i386__
# define __NR_pipe2 331
# else
# error "need __NR_pipe2"
# endif
#endif

int
main (void)
{
int fd[2];
if (syscall (__NR_pipe2, fd, 0) != 0)
{
puts ("pipe2(0) failed");
return 1;
}
for (int i = 0; i < 2; ++i)
{
int coe = fcntl (fd[i], F_GETFD);
if (coe == -1)
{
puts ("fcntl failed");
return 1;
}
if (coe & FD_CLOEXEC)
{
printf ("pipe2(0) set close-on-exit for fd[%d]\n", i);
return 1;
}
}
close (fd[0]);
close (fd[1]);

if (syscall (__NR_pipe2, fd, O_CLOEXEC) != 0)
{
puts ("pipe2(O_CLOEXEC) failed");
return 1;
}
for (int i = 0; i < 2; ++i)
{
int coe = fcntl (fd[i], F_GETFD);
if (coe == -1)
{
puts ("fcntl failed");
return 1;
}
if ((coe & FD_CLOEXEC) == 0)
{
printf ("pipe2(O_CLOEXEC) does not set close-on-exit for fd[%d]\n", i);
return 1;
}
}
close (fd[0]);
close (fd[1]);

puts ("OK");

return 0;
}
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

Signed-off-by: Ulrich Drepper <[email protected]>
Acked-by: Davide Libenzi <[email protected]>
Cc: Michael Kerrisk <[email protected]>
Cc: <[email protected]>
Signed-off-by: Andrew Morton <[email protected]>
Signed-off-by: Linus Torvalds <[email protected]>

show more ...

[XTENSA] Fix fadvise64_64

Xtensa passes long long arguments in a even/odd register pair,
so we also need to shuffle the arguments when passed through the
system call to avoid an empty argument regis

[XTENSA] Fix fadvise64_64

Xtensa passes long long arguments in a even/odd register pair,
so we also need to shuffle the arguments when passed through the
system call to avoid an empty argument register.

Signed-off-by: Chris Zankel <[email protected]>

show more ...

[PATCH] Xtensa: Add ktermios and minor filename fix

The kernel termios (ktermios) changes were somehow missed for Xtensa. This
patch adds the ktermios structure and also includes some minor file na

[PATCH] Xtensa: Add ktermios and minor filename fix

The kernel termios (ktermios) changes were somehow missed for Xtensa. This
patch adds the ktermios structure and also includes some minor file name
fix that was missed in the syscall patch.

Signed-off-by: Chris Zankel <[email protected]>
Acked-by: Alan Cox <[email protected]>
Signed-off-by: Andrew Morton <[email protected]>
Signed-off-by: Linus Torvalds <[email protected]>

show more ...

[PATCH] xtensa: fix system call interface

This is a long outstanding patch to finally fix the syscall interface. The
constants used for the system calls are those we have provided in our libc
patch

[PATCH] xtensa: fix system call interface

This is a long outstanding patch to finally fix the syscall interface. The
constants used for the system calls are those we have provided in our libc
patches. This patch also fixes the shmbuf and stat structure, and fcntl
definitions.

Signed-off-by: Chris Zankel <[email protected]>
Signed-off-by: Andrew Morton <[email protected]>
Signed-off-by: Linus Torvalds <[email protected]>

show more ...