ARM: 9381/1: kasan: clear stale stack poison

We found below OOB crash:

[ 33.452494] ==================================================================
[ 33.453513] BUG: KASAN: stack-out-of-boun

ARM: 9381/1: kasan: clear stale stack poison

We found below OOB crash:

[ 33.452494] ==================================================================
[ 33.453513] BUG: KASAN: stack-out-of-bounds in refresh_cpu_vm_stats.constprop.0+0xcc/0x2ec
[ 33.454660] Write of size 164 at addr c1d03d30 by task swapper/0/0
[ 33.455515]
[ 33.455767] CPU: 0 PID: 0 Comm: swapper/0 Tainted: G O 6.1.25-mainline #1
[ 33.456880] Hardware name: Generic DT based system
[ 33.457555] unwind_backtrace from show_stack+0x18/0x1c
[ 33.458326] show_stack from dump_stack_lvl+0x40/0x4c
[ 33.459072] dump_stack_lvl from print_report+0x158/0x4a4
[ 33.459863] print_report from kasan_report+0x9c/0x148
[ 33.460616] kasan_report from kasan_check_range+0x94/0x1a0
[ 33.461424] kasan_check_range from memset+0x20/0x3c
[ 33.462157] memset from refresh_cpu_vm_stats.constprop.0+0xcc/0x2ec
[ 33.463064] refresh_cpu_vm_stats.constprop.0 from tick_nohz_idle_stop_tick+0x180/0x53c
[ 33.464181] tick_nohz_idle_stop_tick from do_idle+0x264/0x354
[ 33.465029] do_idle from cpu_startup_entry+0x20/0x24
[ 33.465769] cpu_startup_entry from rest_init+0xf0/0xf4
[ 33.466528] rest_init from arch_post_acpi_subsys_init+0x0/0x18
[ 33.467397]
[ 33.467644] The buggy address belongs to stack of task swapper/0/0
[ 33.468493] and is located at offset 112 in frame:
[ 33.469172] refresh_cpu_vm_stats.constprop.0+0x0/0x2ec
[ 33.469917]
[ 33.470165] This frame has 2 objects:
[ 33.470696] [32, 76) 'global_zone_diff'
[ 33.470729] [112, 276) 'global_node_diff'
[ 33.471294]
[ 33.472095] The buggy address belongs to the physical page:
[ 33.472862] page:3cd72da8 refcount:1 mapcount:0 mapping:00000000 index:0x0 pfn:0x41d03
[ 33.473944] flags: 0x1000(reserved|zone=0)
[ 33.474565] raw: 00001000 ed741470 ed741470 00000000 00000000 00000000 ffffffff 00000001
[ 33.475656] raw: 00000000
[ 33.476050] page dumped because: kasan: bad access detected
[ 33.476816]
[ 33.477061] Memory state around the buggy address:
[ 33.477732] c1d03c00: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
[ 33.478630] c1d03c80: 00 00 00 00 00 00 00 00 f1 f1 f1 f1 00 00 00 00
[ 33.479526] >c1d03d00: 00 04 f2 f2 f2 f2 00 00 00 00 00 00 f1 f1 f1 f1
[ 33.480415] ^
[ 33.481195] c1d03d80: 00 00 00 00 00 00 00 00 00 00 04 f3 f3 f3 f3 f3
[ 33.482088] c1d03e00: f3 f3 f3 f3 00 00 00 00 00 00 00 00 00 00 00 00
[ 33.482978] ==================================================================

We find the root cause of this OOB is that arm does not clear stale stack
poison in the case of cpuidle.

This patch refer to arch/arm64/kernel/sleep.S to resolve this issue.

From cited commit [1] that explain the problem

Functions which the compiler has instrumented for KASAN place poison on
the stack shadow upon entry and remove this poison prior to returning.

In the case of cpuidle, CPUs exit the kernel a number of levels deep in
C code. Any instrumented functions on this critical path will leave
portions of the stack shadow poisoned.

If CPUs lose context and return to the kernel via a cold path, we
restore a prior context saved in __cpu_suspend_enter are forgotten, and
we never remove the poison they placed in the stack shadow area by
functions calls between this and the actual exit of the kernel.

Thus, (depending on stackframe layout) subsequent calls to instrumented
functions may hit this stale poison, resulting in (spurious) KASAN
splats to the console.

To avoid this, clear any stale poison from the idle thread for a CPU
prior to bringing a CPU online.

From cited commit [2]

Extend to check for CONFIG_KASAN_STACK

[1] commit 0d97e6d8024c ("arm64: kasan: clear stale stack poison")
[2] commit d56a9ef84bd0 ("kasan, arm64: unpoison stack only with CONFIG_KASAN_STACK")

Signed-off-by: Boy Wu <[email protected]>
Reviewed-by: Mark Rutland <[email protected]>
Acked-by: Andrey Ryabinin <[email protected]>
Reviewed-by: Linus Walleij <[email protected]>
Fixes: 5615f69bc209 ("ARM: 9016/2: Initialize the mapping of KASan shadow memory")
Signed-off-by: Russell King (Oracle) <[email protected]>

show more ...

ARM: mm: switch to swapper_pg_dir early for vmap'ed stack

When onlining a CPU, switch to swapper_pg_dir as soon as possible so
that it is guaranteed that the vmap'ed stack is mapped before it is
use

ARM: mm: switch to swapper_pg_dir early for vmap'ed stack

When onlining a CPU, switch to swapper_pg_dir as soon as possible so
that it is guaranteed that the vmap'ed stack is mapped before it is
used.

Signed-off-by: Ard Biesheuvel <[email protected]>

show more ...

ARM: percpu: add SMP_ON_UP support

Permit the use of the TPIDRPRW system register for carrying the per-CPU
offset in generic SMP configurations that also target non-SMP capable
ARMv6 cores. This use

ARM: percpu: add SMP_ON_UP support

Permit the use of the TPIDRPRW system register for carrying the per-CPU
offset in generic SMP configurations that also target non-SMP capable
ARMv6 cores. This uses the SMP_ON_UP code patching framework to turn all
TPIDRPRW accesses into reads/writes of entry #0 in the __per_cpu_offset
array.

While at it, switch over some existing direct TPIDRPRW accesses in asm
code to invocations of a new helper that is patched in the same way when
necessary.

Note that CPU_V6+SMP without SMP_ON_UP results in a kernel that does not
boot on v6 CPUs without SMP extensions, so add this dependency to
Kconfig as well.

Acked-by: Linus Walleij <[email protected]>
Acked-by: Nicolas Pitre <[email protected]>
Signed-off-by: Ard Biesheuvel <[email protected]>
Tested-by: Marc Zyngier <[email protected]>
Tested-by: Vladimir Murzin <[email protected]> # ARMv7M

show more ...

ARM: implement support for vmap'ed stacks

Wire up the generic support for managing task stack allocations via vmalloc,
and implement the entry code that detects whether we faulted because of a
stack

ARM: implement support for vmap'ed stacks

Wire up the generic support for managing task stack allocations via vmalloc,
and implement the entry code that detects whether we faulted because of a
stack overrun (or future stack overrun caused by pushing the pt_regs array)

While this adds a fair amount of tricky entry asm code, it should be
noted that it only adds a TST + branch to the svc_entry path. The code
implementing the non-trivial handling of the overflow stack is emitted
out-of-line into the .text section.

Since on ARM, we rely on do_translation_fault() to keep PMD level page
table entries that cover the vmalloc region up to date, we need to
ensure that we don't hit such a stale PMD entry when accessing the
stack. So we do a dummy read from the new stack while still running from
the old one on the context switch path, and bump the vmalloc_seq counter
when PMD level entries in the vmalloc range are modified, so that the MM
switch fetches the latest version of the entries.

Note that we need to increase the per-mode stack by 1 word, to gain some
space to stash a GPR until we know it is safe to touch the stack.
However, due to the cacheline alignment of the struct, this does not
actually increase the memory footprint of the struct stack array at all.

Signed-off-by: Ard Biesheuvel <[email protected]>
Tested-by: Keith Packard <[email protected]>
Tested-by: Marc Zyngier <[email protected]>
Tested-by: Vladimir Murzin <[email protected]> # ARMv7M

show more ...

ARM: sleep.S: use PC-relative insn sequence for sleep_save_sp/mpidr_hash

Replace the open coded PC relative offset calculations with adr_l and
ldr_l invocations. This removes some open coded PC rela

ARM: sleep.S: use PC-relative insn sequence for sleep_save_sp/mpidr_hash

Replace the open coded PC relative offset calculations with adr_l and
ldr_l invocations. This removes some open coded PC relative arithmetic,
avoids literal pools on v7+, and slightly reduces the footprint of the
code. Note that ALT_SMP() expects a single instruction so move the macro
invocation after it.

Reviewed-by: Nicolas Pitre <[email protected]>
Signed-off-by: Ard Biesheuvel <[email protected]>

show more ...

ARM: 8847/1: pm: fix HYP/SVC mode mismatch when MCPM is used

MCPM does a soft reset of the CPUs and uses common cpu_resume() routine to
perform low-level platform initialization. This results in a t

ARM: 8847/1: pm: fix HYP/SVC mode mismatch when MCPM is used

MCPM does a soft reset of the CPUs and uses common cpu_resume() routine to
perform low-level platform initialization. This results in a try to install
HYP stubs for the second time for each CPU and results in false HYP/SVC
mode mismatch detection. The HYP stubs are already installed at the
beginning of the kernel initialization on the boot CPU (head.S) or in the
secondary_startup() for other CPUs. To fix this issue MCPM code should use
a cpu_resume() routine without HYP stubs installation.

This change fixes HYP/SVC mode mismatch on Samsung Exynos5422-based Odroid
XU3/XU4/HC1 boards.

Fixes: 3721924c8154 ("ARM: 8081/1: MCPM: provide infrastructure to allow for MCPM loopback")
Signed-off-by: Marek Szyprowski <[email protected]>
Acked-by: Nicolas Pitre <[email protected]>
Tested-by: Anand Moon <[email protected]>
Signed-off-by: Russell King <[email protected]>

show more ...

License cleanup: add SPDX GPL-2.0 license identifier to files with no license

Many source files in the tree are missing licensing information, which
makes it harder for compliance tools to determine

License cleanup: add SPDX GPL-2.0 license identifier to files with no license

Many source files in the tree are missing licensing information, which
makes it harder for compliance tools to determine the correct license.

By default all files without license information are under the default
license of the kernel, which is GPL version 2.

Update the files which contain no license information with the 'GPL-2.0'
SPDX license identifier. The SPDX identifier is a legally binding
shorthand, which can be used instead of the full boiler plate text.

This patch is based on work done by Thomas Gleixner and Kate Stewart and
Philippe Ombredanne.

How this work was done:

Patches were generated and checked against linux-4.14-rc6 for a subset of
the use cases:
- file had no licensing information it it.
- file was a */uapi/* one with no licensing information in it,
- file was a */uapi/* one with existing licensing information,

Further patches will be generated in subsequent months to fix up cases
where non-standard license headers were used, and references to license
had to be inferred by heuristics based on keywords.

The analysis to determine which SPDX License Identifier to be applied to
a file was done in a spreadsheet of side by side results from of the
output of two independent scanners (ScanCode & Windriver) producing SPDX
tag:value files created by Philippe Ombredanne. Philippe prepared the
base worksheet, and did an initial spot review of a few 1000 files.

The 4.13 kernel was the starting point of the analysis with 60,537 files
assessed. Kate Stewart did a file by file comparison of the scanner
results in the spreadsheet to determine which SPDX license identifier(s)
to be applied to the file. She confirmed any determination that was not
immediately clear with lawyers working with the Linux Foundation.

Criteria used to select files for SPDX license identifier tagging was:
- Files considered eligible had to be source code files.
- Make and config files were included as candidates if they contained >5
lines of source
- File already had some variant of a license header in it (even if <5
lines).

All documentation files were explicitly excluded.

The following heuristics were used to determine which SPDX license
identifiers to apply.

- when both scanners couldn't find any license traces, file was
considered to have no license information in it, and the top level
COPYING file license applied.

For non */uapi/* files that summary was:

SPDX license identifier # files
---------------------------------------------------|-------
GPL-2.0 11139

and resulted in the first patch in this series.

If that file was a */uapi/* path one, it was "GPL-2.0 WITH
Linux-syscall-note" otherwise it was "GPL-2.0". Results of that was:

SPDX license identifier # files
---------------------------------------------------|-------
GPL-2.0 WITH Linux-syscall-note 930

and resulted in the second patch in this series.

- if a file had some form of licensing information in it, and was one
of the */uapi/* ones, it was denoted with the Linux-syscall-note if
any GPL family license was found in the file or had no licensing in
it (per prior point). Results summary:

SPDX license identifier # files
---------------------------------------------------|------
GPL-2.0 WITH Linux-syscall-note 270
GPL-2.0+ WITH Linux-syscall-note 169
((GPL-2.0 WITH Linux-syscall-note) OR BSD-2-Clause) 21
((GPL-2.0 WITH Linux-syscall-note) OR BSD-3-Clause) 17
LGPL-2.1+ WITH Linux-syscall-note 15
GPL-1.0+ WITH Linux-syscall-note 14
((GPL-2.0+ WITH Linux-syscall-note) OR BSD-3-Clause) 5
LGPL-2.0+ WITH Linux-syscall-note 4
LGPL-2.1 WITH Linux-syscall-note 3
((GPL-2.0 WITH Linux-syscall-note) OR MIT) 3
((GPL-2.0 WITH Linux-syscall-note) AND MIT) 1

and that resulted in the third patch in this series.

- when the two scanners agreed on the detected license(s), that became
the concluded license(s).

- when there was disagreement between the two scanners (one detected a
license but the other didn't, or they both detected different
licenses) a manual inspection of the file occurred.

- In most cases a manual inspection of the information in the file
resulted in a clear resolution of the license that should apply (and
which scanner probably needed to revisit its heuristics).

- When it was not immediately clear, the license identifier was
confirmed with lawyers working with the Linux Foundation.

- If there was any question as to the appropriate license identifier,
the file was flagged for further research and to be revisited later
in time.

In total, over 70 hours of logged manual review was done on the
spreadsheet to determine the SPDX license identifiers to apply to the
source files by Kate, Philippe, Thomas and, in some cases, confirmation
by lawyers working with the Linux Foundation.

Kate also obtained a third independent scan of the 4.13 code base from
FOSSology, and compared selected files where the other two scanners
disagreed against that SPDX file, to see if there was new insights. The
Windriver scanner is based on an older version of FOSSology in part, so
they are related.

Thomas did random spot checks in about 500 files from the spreadsheets
for the uapi headers and agreed with SPDX license identifier in the
files he inspected. For the non-uapi files Thomas did random spot checks
in about 15000 files.

In initial set of patches against 4.14-rc6, 3 files were found to have
copy/paste license identifier errors, and have been fixed to reflect the
correct identifier.

Additionally Philippe spent 10 hours this week doing a detailed manual
inspection and review of the 12,461 patched files from the initial patch
version early this week with:
- a full scancode scan run, collecting the matched texts, detected
license ids and scores
- reviewing anything where there was a license detected (about 500+
files) to ensure that the applied SPDX license was correct
- reviewing anything where there was no detection but the patch license
was not GPL-2.0 WITH Linux-syscall-note to ensure that the applied
SPDX license was correct

This produced a worksheet with 20 files needing minor correction. This
worksheet was then exported into 3 different .csv files for the
different types of files to be modified.

These .csv files were then reviewed by Greg. Thomas wrote a script to
parse the csv files and add the proper SPDX tag to the file, in the
format that the file expected. This script was further refined by Greg
based on the output to detect more types of files automatically and to
distinguish between header and source .c files (which need different
comment types.) Finally Greg ran the script using the .csv files to
generate the patches.

Reviewed-by: Kate Stewart <[email protected]>
Reviewed-by: Philippe Ombredanne <[email protected]>
Reviewed-by: Thomas Gleixner <[email protected]>
Signed-off-by: Greg Kroah-Hartman <[email protected]>

show more ...

ARM: align .data section

Robert Jarzmik reports that his PXA25x system fails to boot with 4.12,
failing at __flush_whole_cache in arch/arm/mm/proc-xscale.S:215:

0xc0019e20 <+0>: ldr r1,

ARM: align .data section

Robert Jarzmik reports that his PXA25x system fails to boot with 4.12,
failing at __flush_whole_cache in arch/arm/mm/proc-xscale.S:215:

0xc0019e20 <+0>: ldr r1, [pc, #788]
0xc0019e24 <+4>: ldr r0, [r1] <== here

with r1 containing 0xc06f82cd, which is the address of "clean_addr".
Examination of the System.map shows:

c06f22c8 D user_pmd_table
c06f22cc d __warned.19178
c06f22cd d clean_addr

indicating that a .data.unlikely section has appeared just before the
.data section from proc-xscale.S. According to objdump -h, it appears
that our assembly files default their .data alignment to 2**0, which
is bad news if the preceding .data section size is not power-of-2
aligned at link time.

Add the appropriate .align directives to all assembly files in arch/arm
that are missing them where we require an appropriate alignment.

Reported-by: Robert Jarzmik <[email protected]>
Tested-by: Robert Jarzmik <[email protected]>
Signed-off-by: Russell King <[email protected]>

show more ...

ARM: fix new BSYM() usage introduced via for-arm-soc branch

Commit 32e55a777f83 ("ARM: 8389/1: Add cpu_resume_arm() for firmwares
that resume in ARM state") needed to introduce a new usage of BSYM()

ARM: fix new BSYM() usage introduced via for-arm-soc branch

Commit 32e55a777f83 ("ARM: 8389/1: Add cpu_resume_arm() for firmwares
that resume in ARM state") needed to introduce a new usage of BSYM()
to fix a problem with a previous patch. This in turn causes a conflict
with the "bsym" branch which removes this symbol, replacing it with a
'badr' assembly macro. Fix this up.

Signed-off-by: Russell King <[email protected]>

show more ...

ARM: fix EFM32 build breakage caused by cpu_resume_arm

Fix:
arch/arm/kernel/sleep.S:121: Error: selected processor does not support ARM opcodes
arch/arm/kernel/sleep.S:123: Error: attempt to use an

ARM: fix EFM32 build breakage caused by cpu_resume_arm

Fix:
arch/arm/kernel/sleep.S:121: Error: selected processor does not support ARM opcodes
arch/arm/kernel/sleep.S:123: Error: attempt to use an ARM instruction on a Thumb-only processor -- `adr r9,1f+1'
arch/arm/kernel/sleep.S:124: Error: attempt to use an ARM instruction on a Thumb-only processor -- `bx r9'

Signed-off-by: Russell King <[email protected]>

show more ...

ARM: 8389/1: Add cpu_resume_arm() for firmwares that resume in ARM state

Some platforms always enter the kernel in the ARM state even if
the kernel is compiled for THUMB2. Add a small wrapper on top

ARM: 8389/1: Add cpu_resume_arm() for firmwares that resume in ARM state

Some platforms always enter the kernel in the ARM state even if
the kernel is compiled for THUMB2. Add a small wrapper on top of
cpu_resume() that switches into THUMB2 state.

This provides the functionality to fix a problem reported by Kevin
Hilman on next-20150601 where the ifc6410 fails to boot a THUMB2
kernel because the platform's firmware always enters the kernel in
ARM mode from deep idle states.

(rmk: tweaked to work without BSYM->badr changes.)

Reported-by: Kevin Hilman <[email protected]>
Cc: Ard Biesheuvel <[email protected]>
Cc: Lina Iyer <[email protected]>
Signed-off-by: Stephen Boyd <[email protected]>
Signed-off-by: Russell King <[email protected]>

show more ...

ARM: replace BSYM() with badr assembly macro

BSYM() was invented to allow us to work around a problem with the
assembler, where local symbols resolved by the assembler for the 'adr'
instruction did

ARM: replace BSYM() with badr assembly macro

BSYM() was invented to allow us to work around a problem with the
assembler, where local symbols resolved by the assembler for the 'adr'
instruction did not take account of their ISA.

Since we don't want BSYM() used elsewhere, replace BSYM() with a new
macro 'badr', which is like the 'adr' pseudo-op, but with the BSYM()
mechanics integrated into it. This ensures that the BSYM()-ification
is only used in conjunction with 'adr'.

Acked-by: Dave Martin <[email protected]>
Acked-by: Nicolas Pitre <[email protected]>
Signed-off-by: Russell King <[email protected]>

show more ...

ARM: 8324/1: move cpu_resume() to .text section

Move cpu_resume() to the .text section where it belongs. Change
the adr reference to sleep_save_sp to an explicit PC relative
reference so sleep_save_

ARM: 8324/1: move cpu_resume() to .text section

Move cpu_resume() to the .text section where it belongs. Change
the adr reference to sleep_save_sp to an explicit PC relative
reference so sleep_save_sp itself can remain in .data.

This helps prevent linker failure on large kernels, as the code
in the .data section may be too far away to be in range for normal
b/bl instructions.

Reviewed-by: Nicolas Pitre <[email protected]>
Tested-by: Sudeep Holla <[email protected]>
Signed-off-by: Ard Biesheuvel <[email protected]>
Signed-off-by: Russell King <[email protected]>

show more ...

ARM: convert all "mov.* pc, reg" to "bx reg" for ARMv6+

ARMv6 and greater introduced a new instruction ("bx") which can be used
to return from function calls. Recent CPUs perform better when the
"b

ARM: convert all "mov.* pc, reg" to "bx reg" for ARMv6+

ARMv6 and greater introduced a new instruction ("bx") which can be used
to return from function calls. Recent CPUs perform better when the
"bx lr" instruction is used rather than the "mov pc, lr" instruction,
and this sequence is strongly recommended to be used by the ARM
architecture manual (section A.4.1.1).

We provide a new macro "ret" with all its variants for the condition
code which will resolve to the appropriate instruction.

Rather than doing this piecemeal, and miss some instances, change all
the "mov pc" instances to use the new macro, with the exception of
the "movs" instruction and the kprobes code. This allows us to detect
the "mov pc, lr" case and fix it up - and also gives us the possibility
of deploying this for other registers depending on the CPU selection.

Reported-by: Will Deacon <[email protected]>
Tested-by: Stephen Warren <[email protected]> # Tegra Jetson TK1
Tested-by: Robert Jarzmik <[email protected]> # mioa701_bootresume.S
Tested-by: Andrew Lunn <[email protected]> # Kirkwood
Tested-by: Shawn Guo <[email protected]>
Tested-by: Tony Lindgren <[email protected]> # OMAPs
Tested-by: Gregory CLEMENT <[email protected]> # Armada XP, 375, 385
Acked-by: Sekhar Nori <[email protected]> # DaVinci
Acked-by: Christoffer Dall <[email protected]> # kvm/hyp
Acked-by: Haojian Zhuang <[email protected]> # PXA3xx
Acked-by: Stefano Stabellini <[email protected]> # Xen
Tested-by: Uwe Kleine-König <[email protected]> # ARMv7M
Tested-by: Simon Horman <[email protected]> # Shmobile
Signed-off-by: Russell King <[email protected]>

show more ...

ARM: 8053/1: kernel: sleep: restore HYP mode configuration in cpu_resume

On CPUs with virtualization extensions the kernel installs HYP mode
configuration on both primary and secondary cpus upon col

ARM: 8053/1: kernel: sleep: restore HYP mode configuration in cpu_resume

On CPUs with virtualization extensions the kernel installs HYP mode
configuration on both primary and secondary cpus upon cold boot.

On platforms where CPUs are shutdown in idle paths (ie CPU core gating),
when a CPU resumes from low-power states it currently does not execute
code that reinstalls the HYP configuration, which means that the kernel
cannot run eg KVM properly on such machines.

This patch, mirroring cold-boot behaviour, executes position independent
code that reinstalls HYP configuration and drops to SVC mode safely on
warmboot, so that deep idle states can be enabled in kernel running as
hosts on platforms with power management HW.

Cc: Christoffer Dall <[email protected]>
Cc: Dave Martin <[email protected]>
Cc: Marc Zyngier <[email protected]>
Cc: Nicolas Pitre <[email protected]>
Signed-off-by: Lorenzo Pieralisi <[email protected]>
Acked-by: Marc Zyngier <[email protected]>
Reviewed-by: Dave Martin <[email protected]>
Signed-off-by: Russell King <[email protected]>

show more ...

ARM: set BE8 if LE in head code

If we are booting in LE and compiled for BE8, then add code to
set the state to bE8. Since the instruction stream is always LE,
we do not need to do anything special

ARM: set BE8 if LE in head code

If we are booting in LE and compiled for BE8, then add code to
set the state to bE8. Since the instruction stream is always LE,
we do not need to do anything special to the instruction.

Also ensure that the secondary processors are started in the same mode.

Note, we do add about 20 bytes to the kernel image, but it seems easier
to do this than adding another configuration to change.

Signed-off-by: Ben Dooks <[email protected]>
Reviewed-by: Dave Martin <[email protected]>
Tested-by: Thomas Petazzoni <[email protected]>

show more ...

ARM: suspend: use hash of cpu_logical_map value to index into save array

Currently we hash the MPIDR of the CPU being suspended to determine which
entry in the sleep_save_sp array to use. In some si

ARM: suspend: use hash of cpu_logical_map value to index into save array

Currently we hash the MPIDR of the CPU being suspended to determine which
entry in the sleep_save_sp array to use. In some situations, such as when
we want to resume on another physical CPU, the MPIDR of another CPU should
be used instead.

So let's use the value of cpu_logical_map(smp_processor_id()) in place
of the MPIDR in the suspend path. This will result in the same index
being used as with the previous code unless the caller has modified
cpu_logical_map() beforehand with the MPIDR of the physical CPU the
suspending logical CPU will resume on.

Consequently, if doing a physical CPU migration, cpu_logical_map() must
be updated appropriately somewhere between cpu_pm_enter() and
cpu_suspend().

The register allocation in __cpu_suspend is reworked in order to better
accommodate the additional argument.

Signed-off-by: Nicolas Pitre <[email protected]>
Acked-by: Lorenzo Pieralisi <[email protected]>
Reviewed-by: Dave Martin <[email protected]>

show more ...

ARM: kernel: implement stack pointer save array through MPIDR hashing

Current implementation of cpu_{suspend}/cpu_{resume} relies on the MPIDR
to index the array of pointers where the context is sav

ARM: kernel: implement stack pointer save array through MPIDR hashing

Current implementation of cpu_{suspend}/cpu_{resume} relies on the MPIDR
to index the array of pointers where the context is saved and restored.
The current approach works as long as the MPIDR can be considered a
linear index, so that the pointers array can simply be dereferenced by
using the MPIDR[7:0] value.
On ARM multi-cluster systems, where the MPIDR may not be a linear index,
to properly dereference the stack pointer array, a mapping function should
be applied to it so that it can be used for arrays look-ups.

This patch adds code in the cpu_{suspend}/cpu_{resume} implementation
that relies on shifting and ORing hashing method to map a MPIDR value to a
set of buckets precomputed at boot to have a collision free mapping from
MPIDR to context pointers.

The hashing algorithm must be simple, fast, and implementable with few
instructions since in the cpu_resume path the mapping is carried out with
the MMU off and the I-cache off, hence code and data are fetched from DRAM
with no-caching available. Simplicity is counterbalanced with a little
increase of memory (allocated dynamically) for stack pointers buckets, that
should be anyway fairly limited on most systems.

Memory for context pointers is allocated in a early_initcall with
size precomputed and stashed previously in kernel data structures.
Memory for context pointers is allocated through kmalloc; this
guarantees contiguous physical addresses for the allocated memory which
is fundamental to the correct functioning of the resume mechanism that
relies on the context pointer array to be a chunk of contiguous physical
memory. Virtual to physical address conversion for the context pointer
array base is carried out at boot to avoid fiddling with virt_to_phys
conversions in the cpu_resume path which is quite fragile and should be
optimized to execute as few instructions as possible.
Virtual and physical context pointer base array addresses are stashed in a
struct that is accessible from assembly using values generated through the
asm-offsets.c mechanism.

Cc: Will Deacon <[email protected]>
Cc: Catalin Marinas <[email protected]>
Cc: Russell King <[email protected]>
Cc: Colin Cross <[email protected]>
Cc: Santosh Shilimkar <[email protected]>
Cc: Daniel Lezcano <[email protected]>
Cc: Amit Kucheria <[email protected]>
Signed-off-by: Lorenzo Pieralisi <[email protected]>
Reviewed-by: Dave Martin <[email protected]>
Reviewed-by: Nicolas Pitre <[email protected]>
Tested-by: Shawn Guo <[email protected]>
Tested-by: Kevin Hilman <[email protected]>
Tested-by: Stephen Warren <[email protected]>

show more ...

Disintegrate asm/system.h for ARM

Disintegrate asm/system.h for ARM.

Signed-off-by: David Howells <[email protected]>
cc: Russell King <[email protected]>
cc: [email protected]

Disintegrate asm/system.h for ARM

Disintegrate asm/system.h for ARM.

Signed-off-by: David Howells <[email protected]>
cc: Russell King <[email protected]>
cc: [email protected]

show more ...

ARM: LPAE: add ISBs around MMU enabling code

Before we enable the MMU, we must ensure that the TTBR registers contain
sane values. After the MMU has been enabled, we jump to the *virtual*
address of

ARM: LPAE: add ISBs around MMU enabling code

Before we enable the MMU, we must ensure that the TTBR registers contain
sane values. After the MMU has been enabled, we jump to the *virtual*
address of the following function, so we also need to ensure that the
SCTLR write has taken effect.

This patch adds ISB instructions around the SCTLR write to ensure the
visibility of the above.

Signed-off-by: Will Deacon <[email protected]>
Signed-off-by: Catalin Marinas <[email protected]>

show more ...

ARM: suspend: use idmap_pgd instead of suspend_pgd

The ARM CPU suspend code requires cpu_resume_mmu to be identity mapped
in order to re-enable the MMU when coming out of suspend. Currently,
this is

ARM: suspend: use idmap_pgd instead of suspend_pgd

The ARM CPU suspend code requires cpu_resume_mmu to be identity mapped
in order to re-enable the MMU when coming out of suspend. Currently,
this is accomplished by maintaining a suspend_pgd with the relevant
mapping put in place at init time.

This patch replaces the use of suspend_pgd with the new idmap_pgd.
cpu_resume_mmu is placed in the .idmap.text section so that it is
included in the identity map.

Reviewed-by: Catalin Marinas <[email protected]>
Acked-by: Dave Martin <[email protected]>
Tested-by: Lorenzo Pieralisi <[email protected]>
Signed-off-by: Will Deacon <[email protected]>

show more ...

ARM: pm: convert some assembly to C

Convert some of the sleep.S guts to C code, which makes it easier to
use our macros and to add L2 cache handling. We provide a helper
function, __cpu_suspend_sav

ARM: pm: convert some assembly to C

Convert some of the sleep.S guts to C code, which makes it easier to
use our macros and to add L2 cache handling. We provide a helper
function, __cpu_suspend_save(), which deals with saving the common
state, setting up for resume, and flushing caches.

The remainder left as assembly code is the saving of the CPU general
purpose registers, and allocating space on the stack to save the CPU
specific registers and resume state.

Tested-by: Santosh Shilimkar <[email protected]>
Tested-by: Shawn Guo <[email protected]>
Tested-by: Lorenzo Pieralisi <[email protected]>
Signed-off-by: Russell King <[email protected]>

show more ...

ARM: pm: get rid of cpu_resume_turn_mmu_on

We don't require cpu_resume_turn_mmu_on as we can combine the ldr
instruction with the following code provided we ensure that
cpu_resume_mmu is aligned for

ARM: pm: get rid of cpu_resume_turn_mmu_on

We don't require cpu_resume_turn_mmu_on as we can combine the ldr
instruction with the following code provided we ensure that
cpu_resume_mmu is aligned for older CPUs. Note that we also align
to a 32-byte boundary to ensure that the code can't cross a section
boundary.

Tested-by: Santosh Shilimkar <[email protected]>
Tested-by: Shawn Guo <[email protected]>
Tested-by: Lorenzo Pieralisi <[email protected]>
Signed-off-by: Russell King <[email protected]>

show more ...

ARM: pm: only use preallocated page table during resume

Only use the preallocated page table during the resume, not while
suspending. This avoids the overhead of having to switch unnecessarily
to t

ARM: pm: only use preallocated page table during resume

Only use the preallocated page table during the resume, not while
suspending. This avoids the overhead of having to switch unnecessarily
to the resume page table in the suspend path.

Tested-by: Santosh Shilimkar <[email protected]>
Tested-by: Shawn Guo <[email protected]>
Tested-by: Lorenzo Pieralisi <[email protected]>
Signed-off-by: Russell King <[email protected]>

show more ...

ARM: pm: preallocate a page table for suspend/resume

Preallocate a page table and setup an identity mapping for the MMU
enable code. This means we don't have to "borrow" a page table to
do this, av

ARM: pm: preallocate a page table for suspend/resume

Preallocate a page table and setup an identity mapping for the MMU
enable code. This means we don't have to "borrow" a page table to
do this, avoiding complexities with L2 cache coherency.

Tested-by: Santosh Shilimkar <[email protected]>
Tested-by: Shawn Guo <[email protected]>
Tested-by: Lorenzo Pieralisi <[email protected]>
Signed-off-by: Russell King <[email protected]>

show more ...