arm: Rely on generic printing of preemption model

__die() invokes later __show_regs() -> show_regs_print_info() which
prints the current preemption model.
Remove it from the initial line.

Signed-of

arm: Rely on generic printing of preemption model

__die() invokes later __show_regs() -> show_regs_print_info() which
prints the current preemption model.
Remove it from the initial line.

Signed-off-by: Sebastian Andrzej Siewior <[email protected]>
Signed-off-by: Peter Zijlstra (Intel) <[email protected]>
Reviewed-by: "Russell King (Oracle)" <[email protected]>
Link: https://lore.kernel.org/r/[email protected]

show more ...

ARM: fix cacheflush with PAN

It seems that the cacheflush syscall got broken when PAN for LPAE was
implemented. User access was not enabled around the cache maintenance
instructions, causing them to

ARM: fix cacheflush with PAN

It seems that the cacheflush syscall got broken when PAN for LPAE was
implemented. User access was not enabled around the cache maintenance
instructions, causing them to fault.

Fixes: 7af5b901e847 ("ARM: 9358/2: Implement PAN for LPAE by TTBR0 page table walks disablement")
Reported-by: Michał Pecio <[email protected]>
Tested-by: Michał Pecio <[email protected]>
Signed-off-by: Russell King (Oracle) <[email protected]>

show more ...

fix missing vmalloc.h includes

Patch series "Memory allocation profiling", v6.

Overview:
Low overhead [1] per-callsite memory allocation profiling. Not just for
debug kernels, overhead low enough t

fix missing vmalloc.h includes

Patch series "Memory allocation profiling", v6.

Overview:
Low overhead [1] per-callsite memory allocation profiling. Not just for
debug kernels, overhead low enough to be deployed in production.

Example output:
root@moria-kvm:~# sort -rn /proc/allocinfo
127664128 31168 mm/page_ext.c:270 func:alloc_page_ext
56373248 4737 mm/slub.c:2259 func:alloc_slab_page
14880768 3633 mm/readahead.c:247 func:page_cache_ra_unbounded
14417920 3520 mm/mm_init.c:2530 func:alloc_large_system_hash
13377536 234 block/blk-mq.c:3421 func:blk_mq_alloc_rqs
11718656 2861 mm/filemap.c:1919 func:__filemap_get_folio
9192960 2800 kernel/fork.c:307 func:alloc_thread_stack_node
4206592 4 net/netfilter/nf_conntrack_core.c:2567 func:nf_ct_alloc_hashtable
4136960 1010 drivers/staging/ctagmod/ctagmod.c:20 [ctagmod] func:ctagmod_start
3940352 962 mm/memory.c:4214 func:alloc_anon_folio
2894464 22613 fs/kernfs/dir.c:615 func:__kernfs_new_node
...

Usage:
kconfig options:
- CONFIG_MEM_ALLOC_PROFILING
- CONFIG_MEM_ALLOC_PROFILING_ENABLED_BY_DEFAULT
- CONFIG_MEM_ALLOC_PROFILING_DEBUG
adds warnings for allocations that weren't accounted because of a
missing annotation

sysctl:
/proc/sys/vm/mem_profiling

Runtime info:
/proc/allocinfo

Notes:

[1]: Overhead
To measure the overhead we are comparing the following configurations:
(1) Baseline with CONFIG_MEMCG_KMEM=n
(2) Disabled by default (CONFIG_MEM_ALLOC_PROFILING=y &&
CONFIG_MEM_ALLOC_PROFILING_BY_DEFAULT=n)
(3) Enabled by default (CONFIG_MEM_ALLOC_PROFILING=y &&
CONFIG_MEM_ALLOC_PROFILING_BY_DEFAULT=y)
(4) Enabled at runtime (CONFIG_MEM_ALLOC_PROFILING=y &&
CONFIG_MEM_ALLOC_PROFILING_BY_DEFAULT=n && /proc/sys/vm/mem_profiling=1)
(5) Baseline with CONFIG_MEMCG_KMEM=y && allocating with __GFP_ACCOUNT
(6) Disabled by default (CONFIG_MEM_ALLOC_PROFILING=y &&
CONFIG_MEM_ALLOC_PROFILING_BY_DEFAULT=n) && CONFIG_MEMCG_KMEM=y
(7) Enabled by default (CONFIG_MEM_ALLOC_PROFILING=y &&
CONFIG_MEM_ALLOC_PROFILING_BY_DEFAULT=y) && CONFIG_MEMCG_KMEM=y

Performance overhead:
To evaluate performance we implemented an in-kernel test executing
multiple get_free_page/free_page and kmalloc/kfree calls with allocation
sizes growing from 8 to 240 bytes with CPU frequency set to max and CPU
affinity set to a specific CPU to minimize the noise. Below are results
from running the test on Ubuntu 22.04.2 LTS with 6.8.0-rc1 kernel on
56 core Intel Xeon:

kmalloc pgalloc
(1 baseline) 6.764s 16.902s
(2 default disabled) 6.793s (+0.43%) 17.007s (+0.62%)
(3 default enabled) 7.197s (+6.40%) 23.666s (+40.02%)
(4 runtime enabled) 7.405s (+9.48%) 23.901s (+41.41%)
(5 memcg) 13.388s (+97.94%) 48.460s (+186.71%)
(6 def disabled+memcg) 13.332s (+97.10%) 48.105s (+184.61%)
(7 def enabled+memcg) 13.446s (+98.78%) 54.963s (+225.18%)

Memory overhead:
Kernel size:

text data bss dec diff
(1) 26515311 18890222 17018880 62424413
(2) 26524728 19423818 16740352 62688898 264485
(3) 26524724 19423818 16740352 62688894 264481
(4) 26524728 19423818 16740352 62688898 264485
(5) 26541782 18964374 16957440 62463596 39183

Memory consumption on a 56 core Intel CPU with 125GB of memory:
Code tags: 192 kB
PageExts: 262144 kB (256MB)
SlabExts: 9876 kB (9.6MB)
PcpuExts: 512 kB (0.5MB)

Total overhead is 0.2% of total memory.

Benchmarks:

Hackbench tests run 100 times:
hackbench -s 512 -l 200 -g 15 -f 25 -P
baseline disabled profiling enabled profiling
avg 0.3543 0.3559 (+0.0016) 0.3566 (+0.0023)
stdev 0.0137 0.0188 0.0077


hackbench -l 10000
baseline disabled profiling enabled profiling
avg 6.4218 6.4306 (+0.0088) 6.5077 (+0.0859)
stdev 0.0933 0.0286 0.0489

stress-ng tests:
stress-ng --class memory --seq 4 -t 60
stress-ng --class cpu --seq 4 -t 60
Results posted at: https://evilpiepirate.org/~kent/memalloc_prof_v4_stress-ng/

[2] https://lore.kernel.org/all/[email protected]/


This patch (of 37):

The next patch drops vmalloc.h from a system header in order to fix a
circular dependency; this adds it to all the files that were pulling it in
implicitly.

[[email protected]: fix arch/alpha/lib/memcpy.c]
Link: https://lkml.kernel.org/r/[email protected]
[[email protected]: fix arch/x86/mm/numa_32.c]
Link: https://lkml.kernel.org/r/[email protected]
[[email protected]: a few places were depending on sizes.h]
Link: https://lkml.kernel.org/r/[email protected]
[[email protected]: fix mm/kasan/hw_tags.c]
Link: https://lkml.kernel.org/r/[email protected]
[[email protected]: fix arc build]
Link: https://lkml.kernel.org/r/[email protected]
Link: https://lkml.kernel.org/r/[email protected]
Link: https://lkml.kernel.org/r/[email protected]
Signed-off-by: Kent Overstreet <[email protected]>
Signed-off-by: Suren Baghdasaryan <[email protected]>
Signed-off-by: Arnd Bergmann <[email protected]>
Reviewed-by: Pasha Tatashin <[email protected]>
Tested-by: Kees Cook <[email protected]>
Cc: Alexander Viro <[email protected]>
Cc: Alex Gaynor <[email protected]>
Cc: Alice Ryhl <[email protected]>
Cc: Andreas Hindborg <[email protected]>
Cc: Benno Lossin <[email protected]>
Cc: "Björn Roy Baron" <[email protected]>
Cc: Boqun Feng <[email protected]>
Cc: Christoph Lameter <[email protected]>
Cc: Dennis Zhou <[email protected]>
Cc: Gary Guo <[email protected]>
Cc: Miguel Ojeda <[email protected]>
Cc: Peter Zijlstra <[email protected]>
Cc: Tejun Heo <[email protected]>
Cc: Vlastimil Babka <[email protected]>
Cc: Wedson Almeida Filho <[email protected]>
Signed-off-by: Andrew Morton <[email protected]>

show more ...

ARM: 9349/1: unwind: Add missing "Call trace:" line

Every other architecture in Linux includes the line "Call trace:" before
backtraces. In some cases ARM would print "Backtrace:", but this was
only

ARM: 9349/1: unwind: Add missing "Call trace:" line

Every other architecture in Linux includes the line "Call trace:" before
backtraces. In some cases ARM would print "Backtrace:", but this was
only via 1 specific call path, and wasn't included in CPU Oops nor things
like KASAN, UBSAN, etc that called dump_stack(). Regularize this line
so CI systems and other things (like LKDTM) that depend on parsing
"Call trace:" out of dmesg will see it for ARM.

Before this patch:

UBSAN: array-index-out-of-bounds in ../drivers/misc/lkdtm/bugs.c:376:16
index 8 is out of range for type 'char [8]'
CPU: 0 PID: 1402 Comm: cat Not tainted 6.7.0-rc2 #1
Hardware name: Generic DT based system
dump_backtrace from show_stack+0x20/0x24
r7:00000042 r6:00000000 r5:60070013 r4:80cf5d7c
show_stack from dump_stack_lvl+0x88/0x98
dump_stack_lvl from dump_stack+0x18/0x1c
r7:00000042 r6:00000008 r5:00000008 r4:80fab118
dump_stack from ubsan_epilogue+0x10/0x3c
ubsan_epilogue from __ubsan_handle_out_of_bounds+0x80/0x84
...

After this patch:

UBSAN: array-index-out-of-bounds in ../drivers/misc/lkdtm/bugs.c:376:16
index 8 is out of range for type 'char [8]'
CPU: 0 PID: 1402 Comm: cat Not tainted 6.7.0-rc2 #1
Hardware name: Generic DT based system
Call trace:
dump_backtrace from show_stack+0x20/0x24
r7:00000042 r6:00000000 r5:60070013 r4:80cf5d7c
show_stack from dump_stack_lvl+0x88/0x98
dump_stack_lvl from dump_stack+0x18/0x1c
r7:00000042 r6:00000008 r5:00000008 r4:80fab118
dump_stack from ubsan_epilogue+0x10/0x3c
ubsan_epilogue from __ubsan_handle_out_of_bounds+0x80/0x84
...

Link: https://lore.kernel.org/r/[email protected]

Reported-by: Mark Brown <[email protected]>
Cc: Ard Biesheuvel <[email protected]>
Cc: Arnd Bergmann <[email protected]>
Cc: Linus Walleij <[email protected]>
Cc: Vladimir Murzin <[email protected]>
Cc: Zhen Lei <[email protected]>
Cc: Keith Packard <[email protected]>
Cc: Haibo Li <[email protected]>
Cc: <[email protected]>
Reviewed-by: Mark Brown <[email protected]>
Reviewed-by: Linus Walleij <[email protected]>
Reviewed-by: AngeloGioacchino Del Regno <[email protected]>
Signed-off-by: Kees Cook <[email protected]>
Signed-off-by: Russell King (Oracle) <[email protected]>

show more ...

ARM: 9302/1: traps: hide unused functions on NOMMU

A couple of functions in this file are only used on MMU-enabled
builds, and never even declared otherwise, causing these build
warnings:

arch/arm/

ARM: 9302/1: traps: hide unused functions on NOMMU

A couple of functions in this file are only used on MMU-enabled
builds, and never even declared otherwise, causing these build
warnings:

arch/arm/kernel/traps.c:759:6: error: no previous prototype for '__pte_error' [-Werror=missing-prototypes]
arch/arm/kernel/traps.c:764:6: error: no previous prototype for '__pmd_error' [-Werror=missing-prototypes]
arch/arm/kernel/traps.c:769:6: error: no previous prototype for '__pgd_error' [-Werror=missing-prototypes]

Protect these in an #ifdef to avoid the warnings and save a little
bit of .text space.

Reviewed-by: Kees Cook <[email protected]>
Signed-off-by: Arnd Bergmann <[email protected]>
Signed-off-by: Russell King (Oracle) <[email protected]>

show more ...

ARM: 9277/1: Make the dumped instructions are consistent with the disassembled ones

In ARM, the mapping of instruction memory is always little-endian, except
some BE-32 supported ARM architectures.

ARM: 9277/1: Make the dumped instructions are consistent with the disassembled ones

In ARM, the mapping of instruction memory is always little-endian, except
some BE-32 supported ARM architectures. Such as ARMv7-R, its instruction
endianness may be BE-32. Of course, its data endianness will also be BE-32
mode. Due to two negatives make a positive, the instruction stored in the
register after reading is in little-endian format. But for the case of
BE-8, the instruction endianness is LE, the instruction stored in the
register after reading is in big-endian format, which is inconsistent
with the disassembled one.

For example:
The content of disassembly:
c0429ee8: e3500000 cmp r0, #0
c0429eec: 159f2044 ldrne r2, [pc, #68]
c0429ef0: 108f2002 addne r2, pc, r2
c0429ef4: 1882000a stmne r2, {r1, r3}
c0429ef8: e7f000f0 udf #0

The output of undefined instruction exception:
Internal error: Oops - undefined instruction: 0 [#1] SMP ARM
... ...
Code: 000050e3 44209f15 02208f10 0a008218 (f000f0e7)

This inconveniences the checking of instructions. What's worse is that,
for somebody who don't know about this, might think the instructions are
all broken.

So, when CONFIG_CPU_ENDIAN_BE8=y, let's convert the instructions to
little-endian format before they are printed. The conversion result is
as follows:
Code: e3500000 159f2044 108f2002 1882000a (e7f000f0)

Signed-off-by: Zhen Lei <[email protected]>
Signed-off-by: Russell King (Oracle) <[email protected]>

show more ...

ARM: 9276/1: Refactor dump_instr()

1. Rename local variable 'val16' to 'tmp'. So that the processing
statements of thumb and arm can be aligned.
2. Fix two sparse check warnings: (add __user for

ARM: 9276/1: Refactor dump_instr()

1. Rename local variable 'val16' to 'tmp'. So that the processing
statements of thumb and arm can be aligned.
2. Fix two sparse check warnings: (add __user for type conversion)
warning: incorrect type in initializer (different address spaces)
expected unsigned short [noderef] __user *register __p
got unsigned short [usertype] *
3. Prepare for the next patch to avoid repeated judgment.
Before:
if (!user_mode(regs)) {
if (thumb)
else
} else {
if (thumb)
else
}

After:
if (thumb) {
if (user_mode(regs))
else
} else {
if (user_mode(regs))
else
}

Signed-off-by: Zhen Lei <[email protected]>
Signed-off-by: Russell King (Oracle) <[email protected]>

show more ...

ARM: 9224/1: Dump the stack traces based on the parameter 'regs' of show_regs()

Function show_regs() is usually called in interrupt handler or exception
handler, it prints the registers specified by

ARM: 9224/1: Dump the stack traces based on the parameter 'regs' of show_regs()

Function show_regs() is usually called in interrupt handler or exception
handler, it prints the registers specified by the parameter 'regs', then
dump the stack traces. Although not explicitly documented, dump the stack
traces based on'regs' seems to make the most sense. Although dump_stack()
can finally dump the desired content, because 'regs' are saved by the
entry of current interrupt or exception. In the following example we can
see: 1) The backtrace of interrupt or exception handler is not expected,
it causes confusion. 2) Something is printed repeatedly. The line with
the kernel version "CPU: 0 PID: 70 Comm: test0 Not tainted 5.19.0+ #8",
the registers saved in "Exception stack" which 'regs' actually point to.

For example:
rcu: INFO: rcu_sched self-detected stall on CPU
rcu: 0-....: (499 ticks this GP) idle=379/1/0x40000002 softirq=91/91 fqs=249
(t=500 jiffies g=-911 q=13 ncpus=4)
CPU: 0 PID: 70 Comm: test0 Not tainted 5.19.0+ #8
Hardware name: ARM-Versatile Express
PC is at ktime_get+0x4c/0xe8
LR is at ktime_get+0x4c/0xe8
pc : 8019a474 lr : 8019a474 psr: 60000013
sp : cabd1f28 ip : 00000001 fp : 00000005
r10: 527bf1b8 r9 : 431bde82 r8 : d7b634db
r7 : 0000156e r6 : 61f234f8 r5 : 00000001 r4 : 80ca86c0
r3 : ffffffff r2 : fe5bce0b r1 : 00000000 r0 : 01a431f4
Flags: nZCv IRQs on FIQs on Mode SVC_32 ISA ARM Segment none
Control: 10c5387d Table: 6121406a DAC: 00000051
CPU: 0 PID: 70 Comm: test0 Not tainted 5.19.0+ #8 <-----------start----------
Hardware name: ARM-Versatile Express |
unwind_backtrace from show_stack+0x10/0x14 |
show_stack from dump_stack_lvl+0x40/0x4c |
dump_stack_lvl from rcu_dump_cpu_stacks+0x10c/0x134 |
rcu_dump_cpu_stacks from rcu_sched_clock_irq+0x780/0xaf4 |
rcu_sched_clock_irq from update_process_times+0x54/0x74 |
update_process_times from tick_periodic+0x3c/0xd4 |
tick_periodic from tick_handle_periodic+0x20/0x80 worthless
tick_handle_periodic from twd_handler+0x30/0x40 or
twd_handler from handle_percpu_devid_irq+0x8c/0x1c8 duplicated
handle_percpu_devid_irq from generic_handle_domain_irq+0x24/0x34 |
generic_handle_domain_irq from gic_handle_irq+0x74/0x88 |
gic_handle_irq from generic_handle_arch_irq+0x34/0x44 |
generic_handle_arch_irq from call_with_stack+0x18/0x20 |
call_with_stack from __irq_svc+0x98/0xb0 |
Exception stack(0xcabd1ed8 to 0xcabd1f20) |
1ec0: 01a431f4 00000000 |
1ee0: fe5bce0b ffffffff 80ca86c0 00000001 61f234f8 0000156e d7b634db 431bde82 |
1f00: 527bf1b8 00000005 00000001 cabd1f28 8019a474 8019a474 60000013 ffffffff |
__irq_svc from ktime_get+0x4c/0xe8 <---------end--------------
ktime_get from test_task+0x44/0x110
test_task from kthread+0xd8/0xf4
kthread from ret_from_fork+0x14/0x2c
Exception stack(0xcabd1fb0 to 0xcabd1ff8)
1fa0: 00000000 00000000 00000000 00000000
1fc0: 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000
1fe0: 00000000 00000000 00000000 00000000 00000013 00000000

After replacing dump_stack() with dump_backtrace():
rcu: INFO: rcu_sched self-detected stall on CPU
rcu: 0-....: (500 ticks this GP) idle=8f7/1/0x40000002 softirq=129/129 fqs=241
(t=500 jiffies g=-915 q=13 ncpus=4)
CPU: 0 PID: 69 Comm: test0 Not tainted 5.19.0+ #9
Hardware name: ARM-Versatile Express
PC is at ktime_get+0x4c/0xe8
LR is at ktime_get+0x4c/0xe8
pc : 8019a494 lr : 8019a494 psr: 60000013
sp : cabddf28 ip : 00000001 fp : 00000002
r10: 0779cb48 r9 : 431bde82 r8 : d7b634db
r7 : 00000a66 r6 : e835ab70 r5 : 00000001 r4 : 80ca86c0
r3 : ffffffff r2 : ff337d39 r1 : 00000000 r0 : 00cc82c6
Flags: nZCv IRQs on FIQs on Mode SVC_32 ISA ARM Segment none
Control: 10c5387d Table: 611d006a DAC: 00000051
ktime_get from test_task+0x44/0x110
test_task from kthread+0xd8/0xf4
kthread from ret_from_fork+0x14/0x2c
Exception stack(0xcabddfb0 to 0xcabddff8)
dfa0: 00000000 00000000 00000000 00000000
dfc0: 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000
dfe0: 00000000 00000000 00000000 00000000 00000013 00000000

Signed-off-by: Zhen Lei <[email protected]>
Signed-off-by: Russell King (Oracle) <[email protected]>

show more ...

ARM: 9232/1: Replace this_cpu_* with raw_cpu_* in handle_bad_stack()

The hardware automatically disable the IRQ interrupt before jumping to the
interrupt or exception vector. Therefore, the preempt_

ARM: 9232/1: Replace this_cpu_* with raw_cpu_* in handle_bad_stack()

The hardware automatically disable the IRQ interrupt before jumping to the
interrupt or exception vector. Therefore, the preempt_disable() operation
in this_cpu_read() after macro expansion is unnecessary. In fact, function
this_cpu_read() may trigger scheduling, see pseudocode below.

Pseudocode of this_cpu_read(xx):
preempt_disable_notrace();
raw_cpu_read(xx);
if (unlikely(__preempt_count_dec_and_test()))
__preempt_schedule_notrace();

Therefore, use raw_cpu_* instead of this_cpu_* to eliminate potential
hazards. At the very least, it reduces a few lines of assembly code.

Signed-off-by: Zhen Lei <[email protected]>
Signed-off-by: Russell King (Oracle) <[email protected]>

show more ...

ARM: 9221/1: traps: print un-hashed user pc on undefined instruction

When user undefined instruction debug is enabled pc value is hashed like
kernel pointers for security reason. But the security be

ARM: 9221/1: traps: print un-hashed user pc on undefined instruction

When user undefined instruction debug is enabled pc value is hashed like
kernel pointers for security reason. But the security benefit of this
hash is very limited because the code goes on to call __show_regs() that
prints the plain pointer value. pc is a user pointer anyway, so the
kernel does not leak anything. The only result is confusion about the
difference between the pc value on the first printed line, and the value
that __show_regs() prints.

Always print the plain value of pc.

Signed-off-by: Baruch Siach <[email protected]>
Signed-off-by: Russell King (Oracle) <[email protected]>

show more ...

ARM: Revert "unwind: dump exception stack from calling frame"

After simplifying the stack switch code in the IRQ exception handler by
deferring the actual stack switch to call_with_stack(), we no lo

ARM: Revert "unwind: dump exception stack from calling frame"

After simplifying the stack switch code in the IRQ exception handler by
deferring the actual stack switch to call_with_stack(), we no longer
need to special case the way we dump the exception stack, since it will
always be at the top of whichever stack was active when the exception
was taken.

So revert this special handling for the ARM unwinder.

This reverts commit 4ab6827081c63b83011a18d8e27f621ed34b1194.

Signed-off-by: Ard Biesheuvel <[email protected]>
Signed-off-by: Russell King (Oracle) <[email protected]>

show more ...

ARM: Spectre-BHB workaround

Workaround the Spectre BHB issues for Cortex-A15, Cortex-A57,
Cortex-A72, Cortex-A73 and Cortex-A75. We also include Brahma B15 as
well to be safe, which is affected by S

ARM: Spectre-BHB workaround

Workaround the Spectre BHB issues for Cortex-A15, Cortex-A57,
Cortex-A72, Cortex-A73 and Cortex-A75. We also include Brahma B15 as
well to be safe, which is affected by Spectre V2 in the same ways as
Cortex-A15.

Reviewed-by: Catalin Marinas <[email protected]>
Signed-off-by: Russell King (Oracle) <[email protected]>

show more ...

ARM: early traps initialisation

Provide a couple of helpers to copy the vectors and stubs, and also
to flush the copied vectors and stubs.

Acked-by: Catalin Marinas <[email protected]>
Signed

ARM: early traps initialisation

Provide a couple of helpers to copy the vectors and stubs, and also
to flush the copied vectors and stubs.

Acked-by: Catalin Marinas <[email protected]>
Signed-off-by: Russell King (Oracle) <[email protected]>

show more ...

uaccess: fix type mismatch warnings from access_ok()

On some architectures, access_ok() does not do any argument type
checking, so replacing the definition with a generic one causes
a few warnings f

uaccess: fix type mismatch warnings from access_ok()

On some architectures, access_ok() does not do any argument type
checking, so replacing the definition with a generic one causes
a few warnings for harmless issues that were never caught before.

Fix the ones that I found either through my own test builds or
that were reported by the 0-day bot.

Reported-by: kernel test robot <[email protected]>
Reviewed-by: Christoph Hellwig <[email protected]>
Acked-by: Dinh Nguyen <[email protected]>
Signed-off-by: Arnd Bergmann <[email protected]>

show more ...

ARM: mm: make vmalloc_seq handling SMP safe

Rework the vmalloc_seq handling so it can be used safely under SMP, as
we started using it to ensure that vmap'ed stacks are guaranteed to be
mapped by th

ARM: mm: make vmalloc_seq handling SMP safe

Rework the vmalloc_seq handling so it can be used safely under SMP, as
we started using it to ensure that vmap'ed stacks are guaranteed to be
mapped by the active mm before switching to a task, and here we need to
ensure that changes to the page tables are visible to other CPUs when
they observe a change in the sequence count.

Since LPAE needs none of this, fold a check against it into the
vmalloc_seq counter check after breaking it out into a separate static
inline helper.

Given that vmap'ed stacks are now also supported on !SMP configurations,
let's drop the WARN() that could potentially now fire spuriously.

Signed-off-by: Ard Biesheuvel <[email protected]>

show more ...

ARM: reduce the information printed in call traces

A while back, Linus complained about the numeric values printed by the
ARM backtracing code. Printing these values does not make sense if one
does

ARM: reduce the information printed in call traces

A while back, Linus complained about the numeric values printed by the
ARM backtracing code. Printing these values does not make sense if one
does not have access to the kernel ELF image (as is normally the case
when helping a third party on a mailing list), but if one does, they
can be very useful to find the code, rather than searching for the
function name, and then doing hex math to work out where the backtrace
entry is referring to.

Provide an option to control whether this information is included,
which will only be visible if EXPERT is enabled.

Signed-off-by: Russell King <[email protected]>

show more ...

exit: Add and use make_task_dead.

There are two big uses of do_exit. The first is it's design use to be
the guts of the exit(2) system call. The second use is to terminate
a task after something c

exit: Add and use make_task_dead.

There are two big uses of do_exit. The first is it's design use to be
the guts of the exit(2) system call. The second use is to terminate
a task after something catastrophic has happened like a NULL pointer
in kernel code.

Add a function make_task_dead that is initialy exactly the same as
do_exit to cover the cases where do_exit is called to handle
catastrophic failure. In time this can probably be reduced to just a
light wrapper around do_task_dead. For now keep it exactly the same so
that there will be no behavioral differences introducing this new
concept.

Replace all of the uses of do_exit that use it for catastraphic
task cleanup with make_task_dead to make it clear what the code
is doing.

As part of this rename rewind_stack_do_exit
rewind_stack_and_make_dead.

Signed-off-by: "Eric W. Biederman" <[email protected]>

show more ...

ARM: implement THREAD_INFO_IN_TASK for uniprocessor systems

On UP systems, only a single task can be 'current' at the same time,
which means we can use a global variable to track it. This means we c

ARM: implement THREAD_INFO_IN_TASK for uniprocessor systems

On UP systems, only a single task can be 'current' at the same time,
which means we can use a global variable to track it. This means we can
also enable THREAD_INFO_IN_TASK for those systems, as in that case,
thread_info is accessed via current rather than the other way around,
removing the need to store thread_info at the base of the task stack.
This, in turn, permits us to enable IRQ stacks and vmap'ed stacks on UP
systems as well.

To partially mitigate the performance overhead of this arrangement, use
a ADD/ADD/LDR sequence with the appropriate PC-relative group
relocations to load the value of current when needed. This means that
accessing current will still only require a single load as before,
avoiding the need for a literal to carry the address of the global
variable in each function. However, accessing thread_info will now
require this load as well.

Acked-by: Linus Walleij <[email protected]>
Acked-by: Nicolas Pitre <[email protected]>
Signed-off-by: Ard Biesheuvel <[email protected]>
Tested-by: Marc Zyngier <[email protected]>
Tested-by: Vladimir Murzin <[email protected]> # ARMv7M

show more ...

ARM: implement support for vmap'ed stacks

Wire up the generic support for managing task stack allocations via vmalloc,
and implement the entry code that detects whether we faulted because of a
stack

ARM: implement support for vmap'ed stacks

Wire up the generic support for managing task stack allocations via vmalloc,
and implement the entry code that detects whether we faulted because of a
stack overrun (or future stack overrun caused by pushing the pt_regs array)

While this adds a fair amount of tricky entry asm code, it should be
noted that it only adds a TST + branch to the svc_entry path. The code
implementing the non-trivial handling of the overflow stack is emitted
out-of-line into the .text section.

Since on ARM, we rely on do_translation_fault() to keep PMD level page
table entries that cover the vmalloc region up to date, we need to
ensure that we don't hit such a stale PMD entry when accessing the
stack. So we do a dummy read from the new stack while still running from
the old one on the context switch path, and bump the vmalloc_seq counter
when PMD level entries in the vmalloc range are modified, so that the MM
switch fetches the latest version of the entries.

Note that we need to increase the per-mode stack by 1 word, to gain some
space to stash a GPR until we know it is safe to touch the stack.
However, due to the cacheline alignment of the struct, this does not
actually increase the memory footprint of the struct stack array at all.

Signed-off-by: Ard Biesheuvel <[email protected]>
Tested-by: Keith Packard <[email protected]>
Tested-by: Marc Zyngier <[email protected]>
Tested-by: Vladimir Murzin <[email protected]> # ARMv7M

show more ...

ARM: implement IRQ stacks

Now that we no longer rely on the stack pointer to access the current
task struct or thread info, we can implement support for IRQ stacks
cleanly as well.

Define a per-CPU

ARM: implement IRQ stacks

Now that we no longer rely on the stack pointer to access the current
task struct or thread info, we can implement support for IRQ stacks
cleanly as well.

Define a per-CPU IRQ stack and switch to this stack when taking an IRQ,
provided that we were not already using that stack in the interrupted
context. This is never the case for IRQs taken from user space, but ones
taken while running in the kernel could fire while one taken from user
space has not completed yet.

Signed-off-by: Ard Biesheuvel <[email protected]>
Acked-by: Linus Walleij <[email protected]>
Tested-by: Keith Packard <[email protected]>
Acked-by: Nick Desaulniers <[email protected]>
Tested-by: Marc Zyngier <[email protected]>
Tested-by: Vladimir Murzin <[email protected]> # ARMv7M

show more ...

ARM: unwind: dump exception stack from calling frame

The existing code that dumps the contents of the pt_regs structure
passed to __entry routines does so while unwinding the callee frame, and
deref

ARM: unwind: dump exception stack from calling frame

The existing code that dumps the contents of the pt_regs structure
passed to __entry routines does so while unwinding the callee frame, and
dereferences the stack pointer as a struct pt_regs*. This will no longer
work when we enable support for IRQ or overflow stacks, because the
struct pt_regs may live on the task stack, while we are executing from
another stack.

The unwinder has access to this information, but only while unwinding
the calling frame. So let's combine the exception stack dumping code
with the handling of the calling frame as well. By printing it before
dumping the caller/callee addresses, the output order is preserved.

Signed-off-by: Ard Biesheuvel <[email protected]>
Reviewed-by: Arnd Bergmann <[email protected]>
Acked-by: Linus Walleij <[email protected]>
Tested-by: Keith Packard <[email protected]>
Tested-by: Marc Zyngier <[email protected]>
Tested-by: Vladimir Murzin <[email protected]> # ARMv7M

show more ...

ARM: export dump_mem() to other objects

The unwind info based stack unwinder will make its own call to
dump_mem() to dump the exception stack, so give it external linkage.

Signed-off-by: Ard Bieshe

ARM: export dump_mem() to other objects

The unwind info based stack unwinder will make its own call to
dump_mem() to dump the exception stack, so give it external linkage.

Signed-off-by: Ard Biesheuvel <[email protected]>
Reviewed-by: Arnd Bergmann <[email protected]>
Acked-by: Linus Walleij <[email protected]>
Tested-by: Keith Packard <[email protected]>
Tested-by: Marc Zyngier <[email protected]>
Tested-by: Vladimir Murzin <[email protected]> # ARMv7M

show more ...

ARM: 9125/1: fix incorrect use of get_kernel_nofault()

Commit 344179fc7ef4 ("ARM: 9106/1: traps: use get_kernel_nofault instead
of set_fs()") replaced an occurrence of __get_user() with
get_kernel_n

ARM: 9125/1: fix incorrect use of get_kernel_nofault()

Commit 344179fc7ef4 ("ARM: 9106/1: traps: use get_kernel_nofault instead
of set_fs()") replaced an occurrence of __get_user() with
get_kernel_nofault(), but inverted the sense of the conditional in the
process, resulting in no values to be printed at all.

I.e., every exception stack now looks like this:

Exception stack(0xc18d1fb0 to 0xc18d1ff8)
1fa0: ???????? ???????? ???????? ????????
1fc0: ???????? ???????? ???????? ???????? ???????? ???????? ???????? ????????
1fe0: ???????? ???????? ???????? ???????? ???????? ????????

which is rather unhelpful.

Fixes: 344179fc7ef4 ("ARM: 9106/1: traps: use get_kernel_nofault instead of set_fs()")
Signed-off-by: Ard Biesheuvel <[email protected]>
Reviewed-by: Arnd Bergmann <[email protected]>
Signed-off-by: Russell King (Oracle) <[email protected]>

show more ...

ARM: 9125/1: fix incorrect use of get_kernel_nofault()

Commit 344179fc7ef4 ("ARM: 9106/1: traps: use get_kernel_nofault instead
of set_fs()") replaced an occurrence of __get_user() with
get_kernel_n

ARM: 9125/1: fix incorrect use of get_kernel_nofault()

Commit 344179fc7ef4 ("ARM: 9106/1: traps: use get_kernel_nofault instead
of set_fs()") replaced an occurrence of __get_user() with
get_kernel_nofault(), but inverted the sense of the conditional in the
process, resulting in no values to be printed at all.

I.e., every exception stack now looks like this:

Exception stack(0xc18d1fb0 to 0xc18d1ff8)
1fa0: ???????? ???????? ???????? ????????
1fc0: ???????? ???????? ???????? ???????? ???????? ???????? ???????? ????????
1fe0: ???????? ???????? ???????? ???????? ???????? ????????

which is rather unhelpful.

Fixes: 344179fc7ef4 ("ARM: 9106/1: traps: use get_kernel_nofault instead of set_fs()")
Signed-off-by: Ard Biesheuvel <[email protected]>
Reviewed-by: Arnd Bergmann <[email protected]>
Signed-off-by: Russell King (Oracle) <[email protected]>

show more ...

trap: cleanup trap_init()

There are some empty trap_init() definitions in different ARCHs, Introduce
a new weak trap_init() function to clean them up.

Link: https://lkml.kernel.org/r/20210812123602

trap: cleanup trap_init()

There are some empty trap_init() definitions in different ARCHs, Introduce
a new weak trap_init() function to clean them up.

Link: https://lkml.kernel.org/r/[email protected]
Signed-off-by: Kefeng Wang <[email protected]>
Acked-by: Russell King (Oracle) <[email protected]> [arm32]
Acked-by: Vineet Gupta [arc]
Acked-by: Michael Ellerman <[email protected]> [powerpc]
Cc: Yoshinori Sato <[email protected]>
Cc: Ley Foon Tan <[email protected]>
Cc: Jonas Bonn <[email protected]>
Cc: Stefan Kristiansson <[email protected]>
Cc: Stafford Horne <[email protected]>
Cc: James E.J. Bottomley <[email protected]>
Cc: Helge Deller <[email protected]>
Cc: Benjamin Herrenschmidt <[email protected]>
Cc: Paul Mackerras <[email protected]>
Cc: Paul Walmsley <[email protected]>
Cc: Jeff Dike <[email protected]>
Cc: Richard Weinberger <[email protected]>
Cc: Anton Ivanov <[email protected]>
Signed-off-by: Andrew Morton <[email protected]>
Signed-off-by: Linus Torvalds <[email protected]>

show more ...