x86/kexec: Disable RET on kexec

All the invocations unroll to __x86_return_thunk and this file
must be PIC independent.

This fixes kexec on 64-bit AMD boxes.

[ bp: Fix 32-bit build. ]

Reported-

x86/kexec: Disable RET on kexec

All the invocations unroll to __x86_return_thunk and this file
must be PIC independent.

This fixes kexec on 64-bit AMD boxes.

[ bp: Fix 32-bit build. ]

Reported-by: Edward Tran <[email protected]>
Reported-by: Awais Tanveer <[email protected]>
Suggested-by: Ankur Arora <[email protected]>
Signed-off-by: Konrad Rzeszutek Wilk <[email protected]>
Signed-off-by: Alexandre Chartre <[email protected]>
Signed-off-by: Borislav Petkov <[email protected]>

show more ...

x86: Prepare asm files for straight-line-speculation

Replace all ret/retq instructions with RET in preparation of making
RET a macro. Since AS is case insensitive it's a big no-op without
RET define

x86: Prepare asm files for straight-line-speculation

Replace all ret/retq instructions with RET in preparation of making
RET a macro. Since AS is case insensitive it's a big no-op without
RET defined.

find arch/x86/ -name \*.S | while read file
do
sed -i 's/\<ret[q]*\>/RET/' $file
done

Signed-off-by: Peter Zijlstra (Intel) <[email protected]>
Signed-off-by: Borislav Petkov <[email protected]>
Link: https://lore.kernel.org/r/[email protected]

show more ...

x86: Fix various typos in comments, take #2

Fix another ~42 single-word typos in arch/x86/ code comments,
missed a few in the first pass, in particular in .S files.

Signed-off-by: Ingo Molnar <ming

x86: Fix various typos in comments, take #2

Fix another ~42 single-word typos in arch/x86/ code comments,
missed a few in the first pass, in particular in .S files.

Signed-off-by: Ingo Molnar <[email protected]>
Cc: Borislav Petkov <[email protected]>
Cc: Thomas Gleixner <[email protected]>
Cc: Peter Zijlstra <[email protected]>
Cc: Paolo Bonzini <[email protected]>
Cc: Bjorn Helgaas <[email protected]>
Cc: [email protected]

show more ...

x86/asm: Annotate relocate_kernel_{32,64}.c

There are functions in relocate_kernel_{32,64}.c which are not
annotated. This makes automatic annotations on them rather hard. So
annotate all the functi

x86/asm: Annotate relocate_kernel_{32,64}.c

There are functions in relocate_kernel_{32,64}.c which are not
annotated. This makes automatic annotations on them rather hard. So
annotate all the functions now.

Note that these are not C-like functions, so FUNC is not used. Instead
CODE markers are used. Also the functions are not aligned, so the
NOALIGN versions are used:

- SYM_CODE_START_NOALIGN
- SYM_CODE_START_LOCAL_NOALIGN
- SYM_CODE_END

The result is:
0000 108 NOTYPE GLOBAL DEFAULT 1 relocate_kernel
006c 165 NOTYPE LOCAL DEFAULT 1 identity_mapped
0146 127 NOTYPE LOCAL DEFAULT 1 swap_pages
0111 53 NOTYPE LOCAL DEFAULT 1 virtual_mapped

Signed-off-by: Jiri Slaby <[email protected]>
Signed-off-by: Borislav Petkov <[email protected]>
Cc: Alexios Zavras <[email protected]>
Cc: Allison Randal <[email protected]>
Cc: Enrico Weigelt <[email protected]>
Cc: Greg Kroah-Hartman <[email protected]>
Cc: "H. Peter Anvin" <[email protected]>
Cc: Ingo Molnar <[email protected]>
Cc: [email protected]
Cc: Thomas Gleixner <[email protected]>
Cc: x86-ml <[email protected]>
Link: https://lkml.kernel.org/r/[email protected]

show more ...

treewide: Replace GPLv2 boilerplate/reference with SPDX - rule 230

Based on 2 normalized pattern(s):

this source code is licensed under the gnu general public license
version 2 see the file cop

treewide: Replace GPLv2 boilerplate/reference with SPDX - rule 230

Based on 2 normalized pattern(s):

this source code is licensed under the gnu general public license
version 2 see the file copying for more details

this source code is licensed under general public license version 2
see

extracted by the scancode license scanner the SPDX license identifier

GPL-2.0-only

has been chosen to replace the boilerplate/reference in 52 file(s).

Signed-off-by: Thomas Gleixner <[email protected]>
Reviewed-by: Enrico Weigelt <[email protected]>
Reviewed-by: Allison Randal <[email protected]>
Reviewed-by: Alexios Zavras <[email protected]>
Cc: [email protected]
Link: https://lkml.kernel.org/r/[email protected]
Signed-off-by: Greg Kroah-Hartman <[email protected]>

show more ...

x86/asm: Optimize unnecessarily wide TEST instructions

By the nature of the TEST operation, it is often possible to test
a narrower part of the operand:

"testl $3, mem" -> "testb $3, mem",

x86/asm: Optimize unnecessarily wide TEST instructions

By the nature of the TEST operation, it is often possible to test
a narrower part of the operand:

"testl $3, mem" -> "testb $3, mem",
"testq $3, %rcx" -> "testb $3, %cl"

This results in shorter instructions, because the TEST instruction
has no sign-entending byte-immediate forms unlike other ALU ops.

Note that this change does not create any LCP (Length-Changing Prefix)
stalls, which happen when adding a 0x66 prefix, which happens when
16-bit immediates are used, which changes such TEST instructions:

[test_opcode] [modrm] [imm32]

to:

[0x66] [test_opcode] [modrm] [imm16]

where [imm16] has a *different length* now: 2 bytes instead of 4.
This confuses the decoder and slows down execution.

REX prefixes were carefully designed to almost never hit this case:
adding REX prefix does not change instruction length except MOVABS
and MOV [addr],RAX instruction.

This patch does not add instructions which would use a 0x66 prefix,
code changes in assembly are:

-48 f7 07 01 00 00 00 testq $0x1,(%rdi)
+f6 07 01 testb $0x1,(%rdi)
-48 f7 c1 01 00 00 00 test $0x1,%rcx
+f6 c1 01 test $0x1,%cl
-48 f7 c1 02 00 00 00 test $0x2,%rcx
+f6 c1 02 test $0x2,%cl
-41 f7 c2 01 00 00 00 test $0x1,%r10d
+41 f6 c2 01 test $0x1,%r10b
-48 f7 c1 04 00 00 00 test $0x4,%rcx
+f6 c1 04 test $0x4,%cl
-48 f7 c1 08 00 00 00 test $0x8,%rcx
+f6 c1 08 test $0x8,%cl

Linus further notes:

"There are no stalls from using 8-bit instruction forms.

Now, changing from 64-bit or 32-bit 'test' instructions to 8-bit ones
*could* cause problems if it ends up having forwarding issues, so that
instead of just forwarding the result, you end up having to wait for
it to be stable in the L1 cache (or possibly the register file). The
forwarding from the store buffer is simplest and most reliable if the
read is done at the exact same address and the exact same size as the
write that gets forwarded.

But that's true only if:

(a) the write was very recent and is still in the write queue. I'm
not sure that's the case here anyway.

(b) on at least most Intel microarchitectures, you have to test a
different byte than the lowest one (so forwarding a 64-bit write
to a 8-bit read ends up working fine, as long as the 8-bit read
is of the low 8 bits of the written data).

A very similar issue *might* show up for registers too, not just
memory writes, if you use 'testb' with a high-byte register (where
instead of forwarding the value from the original producer it needs to
go through the register file and then shifted). But it's mainly a
problem for store buffers.

But afaik, the way Denys changed the test instructions, neither of the
above issues should be true.

The real problem for store buffer forwarding tends to be "write 8
bits, read 32 bits". That can be really surprisingly expensive,
because the read ends up having to wait until the write has hit the
cacheline, and we might talk tens of cycles of latency here. But
"write 32 bits, read the low 8 bits" *should* be fast on pretty much
all x86 chips, afaik."

Signed-off-by: Denys Vlasenko <[email protected]>
Acked-by: Andy Lutomirski <[email protected]>
Acked-by: Linus Torvalds <[email protected]>
Cc: Borislav Petkov <[email protected]>
Cc: Frederic Weisbecker <[email protected]>
Cc: H. Peter Anvin <[email protected]>
Cc: H. Peter Anvin <[email protected]>
Cc: Kees Cook <[email protected]>
Cc: Oleg Nesterov <[email protected]>
Cc: Steven Rostedt <[email protected]>
Cc: Will Drewry <[email protected]>
Link: http://lkml.kernel.org/r/[email protected]
Signed-off-by: Ingo Molnar <[email protected]>

show more ...

x86, asm, cleanup: Replace open-coded control register values with symbolic

Clean up an unnecessary open-coded control register values.

Signed-off-by: H. Peter Anvin <[email protected]>
Link: htt

x86, asm, cleanup: Replace open-coded control register values with symbolic

Clean up an unnecessary open-coded control register values.

Signed-off-by: H. Peter Anvin <[email protected]>
Link: http://lkml.kernel.org/n/[email protected]

show more ...

kexec, x86: Fix incorrect jump back address if not preserving context

In kexec jump support, jump back address passed to the kexeced
kernel via function calling ABI, that is, the function call
retur

kexec, x86: Fix incorrect jump back address if not preserving context

In kexec jump support, jump back address passed to the kexeced
kernel via function calling ABI, that is, the function call
return address is the jump back entry.

Furthermore, jump back entry == 0 should be used to signal that
the jump back or preserve context is not enabled in the original
kernel.

But in the current implementation the stack position used for
function call return address is not cleared context
preservation is disabled. The patch fixes this bug.

Reported-and-tested-by: Yin Kangkai <[email protected]>
Signed-off-by: Huang Ying <[email protected]>
Cc: Eric W. Biederman <[email protected]>
Cc: Vivek Goyal <[email protected]>
Cc: <[email protected]>
Link: http://lkml.kernel.org/r/[email protected]
Signed-off-by: Ingo Molnar <[email protected]>

show more ...

x86, kexec: fix kexec x86 coding style

Impact: Cleanup

Fix some coding style issue for kexec x86.

Signed-off-by: Huang Ying <[email protected]>
Signed-off-by: H. Peter Anvin <[email protected]>

x86: use _types.h headers in asm where available

In general, the only definitions that assembly files can use
are in _types.S headers (where available), so convert them.

Signed-off-by: Jeremy Fitzh

x86: use _types.h headers in asm where available

In general, the only definitions that assembly files can use
are in _types.S headers (where available), so convert them.

Signed-off-by: Jeremy Fitzhardinge <[email protected]>

show more ...

kexec/i386: setup kexec page table in C

Impact: change the kexec bootstrap code implementation from assembly to C

This patch transforms the kexec page tables setup code from assembler
code to C cod

kexec/i386: setup kexec page table in C

Impact: change the kexec bootstrap code implementation from assembly to C

This patch transforms the kexec page tables setup code from assembler
code to C code in machine_kexec_prepare. This improves readability and
reduces code line number.

Signed-off-by: Huang Ying <[email protected]>
Signed-off-by: Ingo Molnar <[email protected]>

show more ...

kexec/i386: remove PAGE_SIZE alignment from relocate_kernel

Impact: save kernel .text by loosening kexec page alignment

This patch removes PAGE_SIZE alignment from relocate_kernel(). Before
kexec j

kexec/i386: remove PAGE_SIZE alignment from relocate_kernel

Impact: save kernel .text by loosening kexec page alignment

This patch removes PAGE_SIZE alignment from relocate_kernel(). Before
kexec jump patches are merged, control page is mapped to
relocate_kernel in kexec page tables, so relocate_kernel must be
PAGE_SIZE aligned. Now, control page is mapped to identity mapped
address, so relocate_kernel need not to be PAGE_SIZE aligned any
more. This can reduce a few KB from kernel text segement.

Signed-off-by: Huang Ying <[email protected]>
Signed-off-by: Ingo Molnar <[email protected]>

show more ...

kexec jump: check code size in control page

Kexec/Kexec-jump require code size in control page is less than
PAGE_SIZE/2. This patch add link-time checking for this.

ASSERT() of ld link script is u

kexec jump: check code size in control page

Kexec/Kexec-jump require code size in control page is less than
PAGE_SIZE/2. This patch add link-time checking for this.

ASSERT() of ld link script is used as the link-time checking mechanism.

[[email protected]: build fix]
Signed-off-by: Huang Ying <[email protected]>
Cc: Pavel Machek <[email protected]>
Cc: "Rafael J. Wysocki" <[email protected]>
Cc: "Eric W. Biederman" <[email protected]>
Acked-by: Vivek Goyal <[email protected]>
Cc: Ingo Molnar <[email protected]>
Signed-off-by: Andrew Morton <[email protected]>
Signed-off-by: Linus Torvalds <[email protected]>

show more ...

kexec jump

This patch provides an enhancement to kexec/kdump. It implements the
following features:

- Backup/restore memory used by the original kernel before/after
kexec.

- Save/restore CPU st

kexec jump

This patch provides an enhancement to kexec/kdump. It implements the
following features:

- Backup/restore memory used by the original kernel before/after
kexec.

- Save/restore CPU state before/after kexec.

The features of this patch can be used as a general method to call program in
physical mode (paging turning off). This can be used to call BIOS code under
Linux.

kexec-tools needs to be patched to support kexec jump. The patches and
the precompiled kexec can be download from the following URL:

source: http://khibernation.sourceforge.net/download/release_v10/kexec-tools/kexec-tools-src_git_kh10.tar.bz2
patches: http://khibernation.sourceforge.net/download/release_v10/kexec-tools/kexec-tools-patches_git_kh10.tar.bz2
binary: http://khibernation.sourceforge.net/download/release_v10/kexec-tools/kexec_git_kh10

Usage example of calling some physical mode code and return:

1. Compile and install patched kernel with following options selected:

CONFIG_X86_32=y
CONFIG_KEXEC=y
CONFIG_PM=y
CONFIG_KEXEC_JUMP=y

2. Build patched kexec-tool or download the pre-built one.

3. Build some physical mode executable named such as "phy_mode"

4. Boot kernel compiled in step 1.

5. Load physical mode executable with /sbin/kexec. The shell command
line can be as follow:

/sbin/kexec --load-preserve-context --args-none phy_mode

6. Call physical mode executable with following shell command line:

/sbin/kexec -e

Implementation point:

To support jumping without reserving memory. One shadow backup page (source
page) is allocated for each page used by kexeced code image (destination
page). When do kexec_load, the image of kexeced code is loaded into source
pages, and before executing, the destination pages and the source pages are
swapped, so the contents of destination pages are backupped. Before jumping
to the kexeced code image and after jumping back to the original kernel, the
destination pages and the source pages are swapped too.

C ABI (calling convention) is used as communication protocol between
kernel and called code.

A flag named KEXEC_PRESERVE_CONTEXT for sys_kexec_load is added to
indicate that the loaded kernel image is used for jumping back.

Now, only the i386 architecture is supported.

Signed-off-by: Huang Ying <[email protected]>
Acked-by: Vivek Goyal <[email protected]>
Cc: "Eric W. Biederman" <[email protected]>
Cc: Pavel Machek <[email protected]>
Cc: Nigel Cunningham <[email protected]>
Cc: "Rafael J. Wysocki" <[email protected]>
Cc: Ingo Molnar <[email protected]>
Cc: Thomas Gleixner <[email protected]>
Signed-off-by: Andrew Morton <[email protected]>
Signed-off-by: Linus Torvalds <[email protected]>

show more ...

x86: relocate_kernel - use predefined macroses for page attributes

Signed-off-by: Cyrill Gorcunov <[email protected]>
Signed-off-by: Ingo Molnar <[email protected]>

x86: relocate_kernel - use predefined macroses for processor state

Signed-off-by: Cyrill Gorcunov <[email protected]>
Signed-off-by: Ingo Molnar <[email protected]>

x86: relocate_kernel - use PAGE_SIZE instead of numeric constant

Signed-off-by: Cyrill Gorcunov <[email protected]>
Signed-off-by: Ingo Molnar <[email protected]>

x86: relocate_kernel_32.S - clear register in more elegant way

Signed-off-by: Cyrill Gorcunov <[email protected]>
Signed-off-by: Ingo Molnar <[email protected]>

x86: relocate_kernel - use predefined PAGE_SIZE instead of own alias

This patch does clean up relocate_kernel_(32|64).S a bit by getting rid
of local PAGE_ALIGNED macro. We should use well-known PAG

x86: relocate_kernel - use predefined PAGE_SIZE instead of own alias

This patch does clean up relocate_kernel_(32|64).S a bit by getting rid
of local PAGE_ALIGNED macro. We should use well-known PAGE_SIZE instead

Signed-off-by: Cyrill Gorcunov <[email protected]>
Signed-off-by: Ingo Molnar <[email protected]>

show more ...

i386: move kernel

Signed-off-by: Thomas Gleixner <[email protected]>
Signed-off-by: Ingo Molnar <[email protected]>