kbuild: Fix signing issue for external modules

When running the sign script the kernel is within the source directory
of external modules. This caused issues when the kernel uses relative
paths, lik

kbuild: Fix signing issue for external modules

When running the sign script the kernel is within the source directory
of external modules. This caused issues when the kernel uses relative
paths, like:

make[5]: Entering directory '/build/client/devel/kernel/work/linux-2.6'
make[6]: Entering directory '/build/client/devel/addmodules/vtx/work/vtx'
INSTALL /build/client/devel/addmodules/vtx/_/lib/modules/6.13.0-devel+/extra/vtx.ko
SIGN /build/client/devel/addmodules/vtx/_/lib/modules/6.13.0-devel+/extra/vtx.ko
/bin/sh: 1: scripts/sign-file: not found
DEPMOD /build/client/devel/addmodules/vtx/_/lib/modules/6.13.0-devel+

Working around it by using absolute pathes here.

Fixes: 13b25489b6f8 ("kbuild: change working directory to external module directory with M=")
Signed-off-by: Torsten Hilbrich <[email protected]>
Signed-off-by: Masahiro Yamada <[email protected]>

show more ...

kbuild: remove extmod_prefix, MODORDER, MODULES_NSDEPS variables

With the previous changes, $(extmod_prefix), $(MODORDER), and
$(MODULES_NSDEPS) are constant. (empty, modules.order, and
modules.nsde

kbuild: remove extmod_prefix, MODORDER, MODULES_NSDEPS variables

With the previous changes, $(extmod_prefix), $(MODORDER), and
$(MODULES_NSDEPS) are constant. (empty, modules.order, and
modules.nsdeps, respectively).

Remove these variables and hard-code their values.

Signed-off-by: Masahiro Yamada <[email protected]>
Reviewed-by: Nicolas Schier <[email protected]>

show more ...

kbuild: add $(objtree)/ prefix to some in-kernel build artifacts

$(objtree) refers to the top of the output directory of kernel builds.

This commit adds the explicit $(objtree)/ prefix to build art

kbuild: add $(objtree)/ prefix to some in-kernel build artifacts

$(objtree) refers to the top of the output directory of kernel builds.

This commit adds the explicit $(objtree)/ prefix to build artifacts
needed for building external modules.

This change has no immediate impact, as the top-level Makefile
currently defines:

objtree := .

This commit prepares for supporting the building of external modules
in a different directory.

Signed-off-by: Masahiro Yamada <[email protected]>
Reviewed-by: Nicolas Schier <[email protected]>

show more ...

kbuild: add install target for modules.builtin.ranges

When CONFIG_BUILTIN_MODULE_RANGES is enabled, the modules.builtin.ranges
file should be installed in the module install location.

Signed-off-by

kbuild: add install target for modules.builtin.ranges

When CONFIG_BUILTIN_MODULE_RANGES is enabled, the modules.builtin.ranges
file should be installed in the module install location.

Signed-off-by: Kris Van Hees <[email protected]>
Reviewed-by: Nick Alcock <[email protected]>
Tested-by: Sam James <[email protected]>
Reviewed-by: Sami Tolvanen <[email protected]>
Tested-by: Sami Tolvanen <[email protected]>
Signed-off-by: Masahiro Yamada <[email protected]>

show more ...

kbuild: modinst: remove the multithread option from zstd compression

Parallel execution is supported by GNU Make:

$ make -j<N> modules_install

It is questionable to enable multithreading within

kbuild: modinst: remove the multithread option from zstd compression

Parallel execution is supported by GNU Make:

$ make -j<N> modules_install

It is questionable to enable multithreading within each zstd process
by default.

If you still want to do it, you can use the environment variable:

$ ZSTD_NBTHREADS=<N> make modules_install

Signed-off-by: Masahiro Yamada <[email protected]>
Reviewed-by: Nicolas Schier <[email protected]>
Tested-by: Sedat Dilek <[email protected]>

show more ...

module: Split modules_install compression and in-kernel decompression

The kernel configuration allows specifying a module compression mode. If
one is selected then each module gets compressed during

module: Split modules_install compression and in-kernel decompression

The kernel configuration allows specifying a module compression mode. If
one is selected then each module gets compressed during
'make modules_install' and additionally one can also enable support for
a respective direct in-kernel decompression support. This means that the
decompression support cannot be enabled without the automatic compression.

Some distributions, such as the (open)SUSE family, use a signer service for
modules. A build runs on a worker machine but signing is done by a separate
locked-down server that is in possession of the signing key. The build
invokes 'make modules_install' to create a modules tree, collects
information about the modules, asks the signer service for their signature,
appends each signature to the respective module and compresses all modules.

When using this arrangment, the 'make modules_install' step produces
unsigned+uncompressed modules and the distribution's own build recipe takes
care of signing and compression later.

The signing support can be currently enabled without automatically signing
modules during 'make modules_install'. However, the in-kernel decompression
support can be selected only after first enabling automatic compression
during this step.

To allow only enabling the in-kernel decompression support without the
automatic compression during 'make modules_install', separate the
compression options similarly to the signing options, as follows:

> Enable loadable module support
[*] Module compression
Module compression type (GZIP) --->
[*] Automatically compress all modules
[ ] Support in-kernel module decompression

* "Module compression" (MODULE_COMPRESS) is a new main switch for the
compression/decompression support. It replaces MODULE_COMPRESS_NONE.
* "Module compression type" (MODULE_COMPRESS_<type>) chooses the
compression type, one of GZ, XZ, ZSTD.
* "Automatically compress all modules" (MODULE_COMPRESS_ALL) is a new
option to enable module compression during 'make modules_install'. It
defaults to Y.
* "Support in-kernel module decompression" (MODULE_DECOMPRESS) enables
in-kernel decompression.

Signed-off-by: Petr Pavlu <[email protected]>
Acked-by: Masahiro Yamada <[email protected]>
Signed-off-by: Luis Chamberlain <[email protected]>

show more ...

kbuild: Use CRC32 and a 1MiB dictionary for XZ compressed modules

Kmod is now (since kmod commit 09c9f8c5df04 ("libkmod: Use kernel
decompression when available")) using the kernel decompressor, whe

kbuild: Use CRC32 and a 1MiB dictionary for XZ compressed modules

Kmod is now (since kmod commit 09c9f8c5df04 ("libkmod: Use kernel
decompression when available")) using the kernel decompressor, when
loading compressed modules.

However, the kernel XZ decompressor is XZ Embedded, which doesn't
handle CRC64 and dictionaries larger than 1MiB.

Use CRC32 and 1MiB dictionary when XZ compressing and installing
kernel modules.

Link: https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=1050582
Signed-off-by: Martin Nybo Andersen <[email protected]>
Signed-off-by: Masahiro Yamada <[email protected]>

show more ...

kbuild: avoid long argument lists in make modules_install

Running "make modules_install" may fail with

make[2]: execvp: /bin/sh: Argument list too long

if many modules are built and INSTALL_MOD_

kbuild: avoid long argument lists in make modules_install

Running "make modules_install" may fail with

make[2]: execvp: /bin/sh: Argument list too long

if many modules are built and INSTALL_MOD_PATH is long. This is because
scripts/Makefile.modinst creates all directories with one mkdir command.
Use $(foreach ...) instead to prevent an excessive argument list.

Fixes: 2dfec887c0fd ("kbuild: reduce the number of mkdir calls during modules_install")
Signed-off-by: Michal Kubecek <[email protected]>
Signed-off-by: Masahiro Yamada <[email protected]>

show more ...

kbuild: support modules_sign for external modules as well

The modules_sign target is currently only available for in-tree modules,
but it actually works for external modules as well.

Move the modul

kbuild: support modules_sign for external modules as well

The modules_sign target is currently only available for in-tree modules,
but it actually works for external modules as well.

Move the modules_sign rule to the common part.

Signed-off-by: Masahiro Yamada <[email protected]>
Reviewed-by: Nicolas Schier <[email protected]>

show more ...

kbuild: support 'make modules_sign' with CONFIG_MODULE_SIG_ALL=n

Commit d890f510c8e4 ("MODSIGN: Add modules_sign make target") introduced
'make modules_sign' to manually sign modules.

Some time lat

kbuild: support 'make modules_sign' with CONFIG_MODULE_SIG_ALL=n

Commit d890f510c8e4 ("MODSIGN: Add modules_sign make target") introduced
'make modules_sign' to manually sign modules.

Some time later, commit d9d8d7ed498e ("MODSIGN: Add option to not sign
modules during modules_install") introduced CONFIG_MODULE_SIG_ALL.
If it was disabled, mod_sign_cmd was set to no-op ('true' command).
It affected not only 'make modules_install' but also 'make modules_sign'.
With CONFIG_MODULE_SIG_ALL=n, neither modules_install nor modules_sign
is able to sign modules.

Kbuild has kept that behavior, and nobody has complained about it, but
I think it is weird.

CONFIG_MODULE_SIG_ALL=n should turn off signing only for modules_install.
If users want to sign modules manually, modules_sign should be offered.

Signed-off-by: Masahiro Yamada <[email protected]>
Reviewed-by: Nicolas Schier <[email protected]>

show more ...

kbuild: move more module installation code to scripts/Makefile.modinst

Move more relevant code to scripts/Makefile.modinst.

Signed-off-by: Masahiro Yamada <[email protected]>
Reviewed-by: Nicola

kbuild: move more module installation code to scripts/Makefile.modinst

Move more relevant code to scripts/Makefile.modinst.

Signed-off-by: Masahiro Yamada <[email protected]>
Reviewed-by: Nicolas Schier <[email protected]>

show more ...

kbuild: reduce the number of mkdir calls during modules_install

Calling 'mkdir' for every module results in redundant syscalls.

Use $(sort ...) to drop the duplicated directories.

Signed-off-by: M

kbuild: reduce the number of mkdir calls during modules_install

Calling 'mkdir' for every module results in redundant syscalls.

Use $(sort ...) to drop the duplicated directories.

Signed-off-by: Masahiro Yamada <[email protected]>
Reviewed-by: Nicolas Schier <[email protected]>

show more ...

kbuild: move depmod rule to scripts/Makefile.modinst

depmod is a part of the module installation.

scripts/Makefile.modinst is a better place to run it.

Signed-off-by: Masahiro Yamada <masahiroy@ke

kbuild: move depmod rule to scripts/Makefile.modinst

depmod is a part of the module installation.

scripts/Makefile.modinst is a better place to run it.

Signed-off-by: Masahiro Yamada <[email protected]>
Reviewed-by: Nicolas Schier <[email protected]>

show more ...

kbuild: modinst: Fix build error when CONFIG_MODULE_SIG_KEY is a PKCS#11 URI

When CONFIG_MODULE_SIG_KEY is PKCS#11 URI (pkcs11:*), signing of modules
fails:

scripts/sign-file sha256 /.../linux/pk

kbuild: modinst: Fix build error when CONFIG_MODULE_SIG_KEY is a PKCS#11 URI

When CONFIG_MODULE_SIG_KEY is PKCS#11 URI (pkcs11:*), signing of modules
fails:

scripts/sign-file sha256 /.../linux/pkcs11:token=foo;object=bar;pin-value=1111 certs/signing_key.x509 /.../kernel/crypto/tcrypt.ko
Usage: scripts/sign-file [-dp] <hash algo> <key> <x509> <module> [<dest>]
scripts/sign-file -s <raw sig> <hash algo> <x509> <module> [<dest>]

First, we need to avoid adding the $(srctree)/ prefix to the URL.

Second, since the kconfig string values no longer include quotes, we need to add
them again when passing a PKCS#11 URI to sign-file. This avoids
splitting by the shell if the URI contains semicolons.

Fixes: 4db9c2e3d055 ("kbuild: stop using config_filename in scripts/Makefile.modsign")
Fixes: 129ab0d2d9f3 ("kbuild: do not quote string values in include/config/auto.conf")
Signed-off-by: Jan Luebbe <[email protected]>
Signed-off-by: Masahiro Yamada <[email protected]>

show more ...

kbuild: Modify default INSTALL_MOD_DIR from extra to updates

The default INSTALL_MOD_DIR of using the /lib/modules/$(uname -r)/extra
directory for external modules assumes distributions will have so

kbuild: Modify default INSTALL_MOD_DIR from extra to updates

The default INSTALL_MOD_DIR of using the /lib/modules/$(uname -r)/extra
directory for external modules assumes distributions will have something
like /etc/depmod.d/dist.conf with:

search updates extra built-in

However, only some Red Hat releases have and use the "extra" stuff for
years now. Meanwhile, the depmod.c tool in kmod has *forever* used
the "updates" directory as part of the search path by default *if*
your distribution does not have any depmod.d configuration.

If you compile and install an external module today, even upstream
kernel mock drivers (tools/testing/cxl) the modules_install target
will pick up the new drivers but will not allow override of drivers
from updates to override built-in ones.

Since module-init-tools was deprecated over 11 years ago and now kmod
has since its inception used the "updates" directory as part of its
default search path to allow overrides, and since the "extra" stuff
was in practice only used by Red Hat stuff, use the more distro
agnostic override path "updates" to allow external modules to
also override proper production kernel modules.

This would allow mocking drivers tools to not have to muck with
depmod.d config files or assume that your distro will have extra
on a configuration file over built-in.

With today's default you end up actually *crashing* Linux when
trying to load cxl_test with the default "extra" [0] directory being
used. This fixes that and allows other mocking drivers to do
less work.

[0] https://lkml.kernel.org/r/[email protected]

Signed-off-by: Luis Chamberlain <[email protected]>
Acked-by: Dan Williams <[email protected]>
Acked-by: Masahiro Yamada <[email protected]>
Reviewed-by: Nicolas Schier <[email protected]>

show more ...

kbuild: change module.order to list *.o instead of *.ko

scripts/Makefile.build replaces the suffix .o with .ko, then
scripts/Makefile.modpost calls the sed command to change .ko back
to the original

kbuild: change module.order to list *.o instead of *.ko

scripts/Makefile.build replaces the suffix .o with .ko, then
scripts/Makefile.modpost calls the sed command to change .ko back
to the original .o suffix.

Instead of converting the suffixes back-and-forth, store the .o paths
in modules.order, and replace it with .ko in 'make modules_install'.

This avoids the unneeded sed command.

Signed-off-by: Masahiro Yamada <[email protected]>
Reviewed-by: Luis Chamberlain <[email protected]>

show more ...

kbuild: add read-file macro

Since GNU Make 4.2, $(file ...) supports the read operater '<', which
is useful to read a file without forking a new process. No warning is
shown even if the input file i

kbuild: add read-file macro

Since GNU Make 4.2, $(file ...) supports the read operater '<', which
is useful to read a file without forking a new process. No warning is
shown even if the input file is missing.

For older Make versions, it falls back to the cat command.

Signed-off-by: Masahiro Yamada <[email protected]>
Reviewed-by: Nicolas Schier <[email protected]>
Reviewed-by: Alexander Lobakin <[email protected]>
Tested-by: Alexander Lobakin <[email protected]>

show more ...

kbuild: do not sort after reading modules.order

modules.order lists modules in the deterministic order (that is why
"modules order"), and there is no duplication in the list.

$(sort ) is pointless.

kbuild: do not sort after reading modules.order

modules.order lists modules in the deterministic order (that is why
"modules order"), and there is no duplication in the list.

$(sort ) is pointless.

Signed-off-by: Masahiro Yamada <[email protected]>
Reviewed-by: Nicolas Schier <[email protected]>

show more ...

kbuild: error out if $(INSTALL_MOD_PATH) contains % or :

If the directory path given to INSTALL_MOD_PATH contains % or :,
the module_install fails.

% is used in pattern rules, and : as the separato

kbuild: error out if $(INSTALL_MOD_PATH) contains % or :

If the directory path given to INSTALL_MOD_PATH contains % or :,
the module_install fails.

% is used in pattern rules, and : as the separator of dependencies.

Bail out with a clearer error message.

Signed-off-by: Masahiro Yamada <[email protected]>
Reviewed-by: Nicolas Schier <[email protected]>

show more ...

kbuild: remove unused cmd_none in scripts/Makefile.modinst

Commit 65ce9c38326e ("kbuild: move module strip/compression code into
scripts/Makefile.modinst") added this unused code.

Perhaps, I though

kbuild: remove unused cmd_none in scripts/Makefile.modinst

Commit 65ce9c38326e ("kbuild: move module strip/compression code into
scripts/Makefile.modinst") added this unused code.

Perhaps, I thought cmd_none was useful for CONFIG_MODULE_COMPRESS_NONE,
but I did not use it after all.

Signed-off-by: Masahiro Yamada <[email protected]>
Reviewed-by: Nick Desaulniers <[email protected]>

show more ...

kbuild: do not quote string values in include/config/auto.conf

The previous commit fixed up all shell scripts to not include
include/config/auto.conf.

Now that include/config/auto.conf is only incl

kbuild: do not quote string values in include/config/auto.conf

The previous commit fixed up all shell scripts to not include
include/config/auto.conf.

Now that include/config/auto.conf is only included by Makefiles,
we can change it into a more Make-friendly form.

Previously, Kconfig output string values enclosed with double-quotes
(both in the .config and include/config/auto.conf):

CONFIG_X="foo bar"

Unlike shell, Make handles double-quotes (and single-quotes as well)
verbatim. We must rip them off when used.

There are some patterns:

[1] $(patsubst "%",%,$(CONFIG_X))
[2] $(CONFIG_X:"%"=%)
[3] $(subst ",,$(CONFIG_X))
[4] $(shell echo $(CONFIG_X))

These are not only ugly, but also fragile.

[1] and [2] do not work if the value contains spaces, like
CONFIG_X=" foo bar "

[3] does not work correctly if the value contains double-quotes like
CONFIG_X="foo\"bar"

[4] seems to work better, but has a cost of forking a process.

Anyway, quoted strings were always PITA for our Makefiles.

This commit changes Kconfig to stop quoting in include/config/auto.conf.

These are the string type symbols referenced in Makefiles or scripts:

ACPI_CUSTOM_DSDT_FILE
ARC_BUILTIN_DTB_NAME
ARC_TUNE_MCPU
BUILTIN_DTB_SOURCE
CC_IMPLICIT_FALLTHROUGH
CC_VERSION_TEXT
CFG80211_EXTRA_REGDB_KEYDIR
EXTRA_FIRMWARE
EXTRA_FIRMWARE_DIR
EXTRA_TARGETS
H8300_BUILTIN_DTB
INITRAMFS_SOURCE
LOCALVERSION
MODULE_SIG_HASH
MODULE_SIG_KEY
NDS32_BUILTIN_DTB
NIOS2_DTB_SOURCE
OPENRISC_BUILTIN_DTB
SOC_CANAAN_K210_DTB_SOURCE
SYSTEM_BLACKLIST_HASH_LIST
SYSTEM_REVOCATION_KEYS
SYSTEM_TRUSTED_KEYS
TARGET_CPU
UNUSED_KSYMS_WHITELIST
XILINX_MICROBLAZE0_FAMILY
XILINX_MICROBLAZE0_HW_VER
XTENSA_VARIANT_NAME

I checked them one by one, and fixed up the code where necessary.

Signed-off-by: Masahiro Yamada <[email protected]>

show more ...

kbuild: stop using config_filename in scripts/Makefile.modsign

Toward the goal of removing the config_filename macro, drop
the double-quotes and add $(srctree)/ prefix in an ad hoc way.

Signed-off-

kbuild: stop using config_filename in scripts/Makefile.modsign

Toward the goal of removing the config_filename macro, drop
the double-quotes and add $(srctree)/ prefix in an ad hoc way.

Signed-off-by: Masahiro Yamada <[email protected]>
Reviewed-by: Nicolas Schier <[email protected]>

show more ...

kbuild: add support for zstd compressed modules

kmod 28 supports modules compressed in zstd format so let's add this
possibility to kernel.

Signed-off-by: Piotr Gorski <[email protected]>
R

kbuild: add support for zstd compressed modules

kmod 28 supports modules compressed in zstd format so let's add this
possibility to kernel.

Signed-off-by: Piotr Gorski <[email protected]>
Reviewed-by: Oleksandr Natalenko <[email protected]>
Signed-off-by: Masahiro Yamada <[email protected]>

show more ...

kbuild: merge scripts/Makefile.modsign to scripts/Makefile.modinst

scripts/Makefile.modsign is a subset of scripts/Makefile.modinst,
and duplicates the code. Let's merge them.

By the way, you do no

kbuild: merge scripts/Makefile.modsign to scripts/Makefile.modinst

scripts/Makefile.modsign is a subset of scripts/Makefile.modinst,
and duplicates the code. Let's merge them.

By the way, you do not need to run 'make modules_sign' explicitly
because modules are signed as a part of 'make modules_install' when
CONFIG_MODULE_SIG_ALL=y. If CONFIG_MODULE_SIG_ALL=n, mod_sign_cmd is
set to 'true', so 'make modules_sign' is not functional.

In my understanding, the reason of still keeping this is to handle
corner cases like commit 64178cb62c32 ("builddeb: fix stripped module
signatures if CONFIG_DEBUG_INFO and CONFIG_MODULE_SIG_ALL are set").

Signed-off-by: Masahiro Yamada <[email protected]>

show more ...

kbuild: move module strip/compression code into scripts/Makefile.modinst

Both mod_strip_cmd and mod_compress_cmd are only used in
scripts/Makefile.modinst, hence there is no good reason to define th

kbuild: move module strip/compression code into scripts/Makefile.modinst

Both mod_strip_cmd and mod_compress_cmd are only used in
scripts/Makefile.modinst, hence there is no good reason to define them
in the top Makefile. Move the relevant code to scripts/Makefile.modinst.

Also, show separate log messages for each of install, strip, sign, and
compress.

Signed-off-by: Masahiro Yamada <[email protected]>

show more ...