kbuild: Fix comment typo in scripts/Makefile.modpost

Change comment "create one <module>.mod.c file pr. module"
to "create one <module>.mod.c file per module"

Signed-off-by: Ramji Jiyani <ramjiyani

kbuild: Fix comment typo in scripts/Makefile.modpost

Change comment "create one <module>.mod.c file pr. module"
to "create one <module>.mod.c file per module"

Signed-off-by: Ramji Jiyani <[email protected]>
Signed-off-by: Masahiro Yamada <[email protected]>

show more ...

kbuild: Fix TRIM_UNUSED_KSYMS with LTO_CLANG

With CONFIG_LTO_CLANG, we currently link modules into native
code just before modpost, which means with TRIM_UNUSED_KSYMS
enabled, we still look at the L

kbuild: Fix TRIM_UNUSED_KSYMS with LTO_CLANG

With CONFIG_LTO_CLANG, we currently link modules into native
code just before modpost, which means with TRIM_UNUSED_KSYMS
enabled, we still look at the LLVM bitcode in the .o files when
generating the list of used symbols. As the bitcode doesn't
yet have calls to compiler intrinsics and llvm-nm doesn't see
function references that only exist in function-level inline
assembly, we currently need a whitelist for TRIM_UNUSED_KSYMS to
work with LTO.

This change moves module LTO linking to happen earlier, and
thus avoids the issue with LLVM bitcode and TRIM_UNUSED_KSYMS
entirely, allowing us to also drop the whitelist from
gen_autoksyms.sh.

Link: https://github.com/ClangBuiltLinux/linux/issues/1369
Signed-off-by: Sami Tolvanen <[email protected]>
Reviewed-by: Alexander Lobakin <[email protected]>
Tested-by: Alexander Lobakin <[email protected]>
Signed-off-by: Masahiro Yamada <[email protected]>

show more ...

kbuild: fix false-positive modpost warning when all symbols are trimmed

Nathan reports that the mips defconfig emits the following warning:

WARNING: modpost: Symbol info of vmlinux is missing. Un

kbuild: fix false-positive modpost warning when all symbols are trimmed

Nathan reports that the mips defconfig emits the following warning:

WARNING: modpost: Symbol info of vmlinux is missing. Unresolved symbol check will be entirely skipped.

This false-positive happens when CONFIG_TRIM_UNUSED_KSYMS is enabled,
but no CONFIG option is set to 'm'.

Commit a0590473c5e6 ("nfs: fix PNFS_FLEXFILE_LAYOUT Kconfig default")
turned the last 'm' into 'y' for the mips defconfig, and uncovered
this issue.

In this case, the module feature itself is enabled, but we have no
module to build. As a result, CONFIG_TRIM_UNUSED_KSYMS drops all the
instances of EXPORT_SYMBOL. Then, modpost wrongly assumes vmlinux is
missing because vmlinux.symvers is empty. (As another false-positive
case, you can create a module that does not use any symbol of vmlinux).

The current behavior is to entirely suppress the unresolved symbol
warnings when vmlinux is missing just because there are too many.
I found the origin of this code in the historical git tree. [1]

If this is a matter of noisiness, I think modpost can display the
first 10 warnings, and the number of suppressed warnings at the end.

You will get a bit noisier logs when you run 'make modules' without
vmlinux, but such warnings are better to show because you never know
the resulting modules are actually loadable or not.

This commit changes the following:

- If any of input *.symver files is missing, pass -w option to let
the module build keep going with warnings instead of errors.

- If there are too many (10+) unresolved symbol warnings, show only
the first 10, and also the number of suppressed warnings.

[1]: https://git.kernel.org/pub/scm/linux/kernel/git/history/history.git/commit/?id=1cc0e0529569bf6a94f6d49770aa6d4b599d2c46

Reported-by: Nathan Chancellor <[email protected]>
Signed-off-by: Masahiro Yamada <[email protected]>

show more ...

kbuild: do not set -w for vmlinux.o modpost

The -w option is meaningless for the first pass of modpost (vmlinux.o).

We know there are unresolved symbols in vmlinux.o, hence we skip
check_exports()

kbuild: do not set -w for vmlinux.o modpost

The -w option is meaningless for the first pass of modpost (vmlinux.o).

We know there are unresolved symbols in vmlinux.o, hence we skip
check_exports() and other checks when mod->is_vmlinux is set.

See the following part in the for-loop.

if (mod->is_vmlinux || mod->from_dump)
continue;

Signed-off-by: Masahiro Yamada <[email protected]>

show more ...

kbuild: generate Module.symvers only when vmlinux exists

The external module build shows the following warning if Module.symvers
is missing in the kernel tree.

WARNING: Symbol version dump "Modul

kbuild: generate Module.symvers only when vmlinux exists

The external module build shows the following warning if Module.symvers
is missing in the kernel tree.

WARNING: Symbol version dump "Module.symvers" is missing.
Modules may not have dependencies or modversions.

I think this is an important heads-up because the resulting modules may
not work as expected. This happens when you did not build the entire
kernel tree, for example, you might have prepared the minimal setups
for external modules by 'make defconfig && make modules_preapre'.

A problem is that 'make modules' creates Module.symvers even without
vmlinux. In this case, that warning is suppressed since Module.symvers
already exists in spite of its incomplete content.

The incomplete (i.e. invalid) Module.symvers should not be created.

This commit changes the second pass of modpost to dump symbols into
modules-only.symvers. The final Module.symvers is created by
concatenating vmlinux.symvers and modules-only.symvers if both exist.

Module.symvers is supposed to collect symbols from both vmlinux and
modules. It might be a bit confusing, and I am not quite sure if it
is an official interface, but presumably it is difficult to rename it
because some tools (e.g. kmod) parse it.

Signed-off-by: Masahiro Yamada <[email protected]>

show more ...

kbuild: prefix $(srctree)/ to some included Makefiles

VPATH is used in Kbuild to make pattern rules search for prerequisites
in both $(objtree) and $(srctree). Some of *.c, *.S files are not real
so

kbuild: prefix $(srctree)/ to some included Makefiles

VPATH is used in Kbuild to make pattern rules search for prerequisites
in both $(objtree) and $(srctree). Some of *.c, *.S files are not real
sources, but generated by tools such as flex, bison, perl.

In contrast, I doubt the benefit of --include-dir=$(abs_srctree) because
it is always clear which Makefiles are real sources, and which are not.

So, my hope is to add $(srctree)/ prefix to all check-in Makefiles,
then remove --include-dir=$(abs_srctree) flag in the future.

I am touching only some Kbuild core parts for now. Treewide fixes will
be needed to achieve this goal.

Signed-off-by: Masahiro Yamada <[email protected]>

show more ...

kbuild: lto: fix module versioning

With CONFIG_MODVERSIONS, version information is linked into each
compilation unit that exports symbols. With LTO, we cannot use this
method as all C code is compil

kbuild: lto: fix module versioning

With CONFIG_MODVERSIONS, version information is linked into each
compilation unit that exports symbols. With LTO, we cannot use this
method as all C code is compiled into LLVM bitcode instead. This
change collects symbol versions into .symversions files and merges
them in link-vmlinux.sh where they are all linked into vmlinux.o at
the same time.

Signed-off-by: Sami Tolvanen <[email protected]>
Reviewed-by: Kees Cook <[email protected]>
Signed-off-by: Kees Cook <[email protected]>
Link: https://lore.kernel.org/r/[email protected]

show more ...

kbuild: add support for Clang LTO

This change adds build system support for Clang's Link Time
Optimization (LTO). With -flto, instead of ELF object files, Clang
produces LLVM bitcode, which is compi

kbuild: add support for Clang LTO

This change adds build system support for Clang's Link Time
Optimization (LTO). With -flto, instead of ELF object files, Clang
produces LLVM bitcode, which is compiled into native code at link
time, allowing the final binary to be optimized globally. For more
details, see:

https://llvm.org/docs/LinkTimeOptimization.html

The Kconfig option CONFIG_LTO_CLANG is implemented as a choice,
which defaults to LTO being disabled. To use LTO, the architecture
must select ARCH_SUPPORTS_LTO_CLANG and support:

- compiling with Clang,
- compiling all assembly code with Clang's integrated assembler,
- and linking with LLD.

While using CONFIG_LTO_CLANG_FULL results in the best runtime
performance, the compilation is not scalable in time or
memory. CONFIG_LTO_CLANG_THIN enables ThinLTO, which allows
parallel optimization and faster incremental builds. ThinLTO is
used by default if the architecture also selects
ARCH_SUPPORTS_LTO_CLANG_THIN:

https://clang.llvm.org/docs/ThinLTO.html

To enable LTO, LLVM tools must be used to handle bitcode files, by
passing LLVM=1 and LLVM_IAS=1 options to make:

$ make LLVM=1 LLVM_IAS=1 defconfig
$ scripts/config -e LTO_CLANG_THIN
$ make LLVM=1 LLVM_IAS=1

To prepare for LTO support with other compilers, common parts are
gated behind the CONFIG_LTO option, and LTO can be disabled for
specific files by filtering out CC_FLAGS_LTO.

Signed-off-by: Sami Tolvanen <[email protected]>
Reviewed-by: Kees Cook <[email protected]>
Signed-off-by: Kees Cook <[email protected]>
Link: https://lore.kernel.org/r/[email protected]

show more ...

kbuild: remove redundant FORCE definition in scripts/Makefile.modpost

The same code exists a few lines above.

Fixes: 436b2ac603d5 ("modpost: invoke modpost only when input files are updated")
Signe

kbuild: remove redundant FORCE definition in scripts/Makefile.modpost

The same code exists a few lines above.

Fixes: 436b2ac603d5 ("modpost: invoke modpost only when input files are updated")
Signed-off-by: Masahiro Yamada <[email protected]>

show more ...

modpost: move -d option in scripts/Makefile.modpost

Collect options for modules into a single place.

Signed-off-by: Masahiro Yamada <[email protected]>

modpost: remove -s option

The -s option was added by commit 8d8d8289df65 ("kbuild: do not do
section mismatch checks on vmlinux in 2nd pass").

Now that the second pass does not parse vmlinux, this

modpost: remove -s option

The -s option was added by commit 8d8d8289df65 ("kbuild: do not do
section mismatch checks on vmlinux in 2nd pass").

Now that the second pass does not parse vmlinux, this option is
unneeded.

Signed-off-by: Masahiro Yamada <[email protected]>

show more ...

modpost: show warning if any of symbol dump files is missing

If modpost fails to load a symbol dump file, it cannot check unresolved
symbols, hence module dependency will not be added. Nor CRCs can

modpost: show warning if any of symbol dump files is missing

If modpost fails to load a symbol dump file, it cannot check unresolved
symbols, hence module dependency will not be added. Nor CRCs can be added.

Currently, external module builds check only $(objtree)/Module.symvers,
but it should check files specified by KBUILD_EXTRA_SYMBOLS as well.

Move the warning message from the top Makefile to scripts/Makefile.modpost
and print the warning if any dump file is missing.

Signed-off-by: Masahiro Yamada <[email protected]>

show more ...

modpost: invoke modpost only when input files are updated

Currently, the second pass of modpost is always invoked when you run
'make' or 'make modules' even if none of modules is changed.

Use if_ch

modpost: invoke modpost only when input files are updated

Currently, the second pass of modpost is always invoked when you run
'make' or 'make modules' even if none of modules is changed.

Use if_changed to invoke it only when it is necessary.

Signed-off-by: Masahiro Yamada <[email protected]>

show more ...

modpost: generate vmlinux.symvers and reuse it for the second modpost

The full build runs modpost twice, first for vmlinux.o and second for
modules.

The first pass dumps all the vmlinux symbols int

modpost: generate vmlinux.symvers and reuse it for the second modpost

The full build runs modpost twice, first for vmlinux.o and second for
modules.

The first pass dumps all the vmlinux symbols into Module.symvers, but
the second pass parses vmlinux again instead of reusing the dump file,
presumably because it needs to avoid accumulating stale symbols.

Loading symbol info from a dump file is faster than parsing an ELF object.
Besides, modpost deals with various issues to parse vmlinux in the second
pass.

A solution is to make the first pass dumps symbols into a separate file,
vmlinux.symvers. The second pass reads it, and parses module .o files.
The merged symbol information is dumped into Module.symvers in the same
way as before.

This makes further modpost cleanups possible.

Also, it fixes the problem of 'make vmlinux', which previously overwrote
Module.symvers, throwing away module symbols.

I slightly touched scripts/link-vmlinux.sh so that vmlinux is re-linked
when you cross this commit. Otherwise, vmlinux.symvers would not be
generated.

Signed-off-by: Masahiro Yamada <[email protected]>

show more ...

modpost: refactor -i option calculation

Prepare to use -i for in-tree modpost as well.

Signed-off-by: Masahiro Yamada <[email protected]>

modpost: print symbol dump file as the build target in short log

The symbol dump *.symvers is the output of modpost. Print it in
the short log.

Signed-off-by: Masahiro Yamada <[email protected]>

modpost: re-add -e to set external_module flag

Previously, the -i option had two functions; load a symbol dump file,
and set the external_module flag.

I want to assign a dedicate option for each of

modpost: re-add -e to set external_module flag

Previously, the -i option had two functions; load a symbol dump file,
and set the external_module flag.

I want to assign a dedicate option for each of them.

Going forward, the -i is used to load a symbol dump file, and the -e
to set the external_module flag.

With this, we will be able to use -i for loading in-kernel symbols.

Signed-off-by: Masahiro Yamada <[email protected]>

show more ...

modpost: allow to pass -i option multiple times to remove -e option

Now that there is no difference between -i and -e, they can be unified.

Make modpost accept the -i option multiple times, then re

modpost: allow to pass -i option multiple times to remove -e option

Now that there is no difference between -i and -e, they can be unified.

Make modpost accept the -i option multiple times, then remove -e.

I will reuse -e for a different purpose.

Signed-off-by: Masahiro Yamada <[email protected]>

show more ...

modpost: pass -N option only for modules modpost

The built-in only code is not required to have MODULE_IMPORT_NS() to
use symbols. So, the namespace is not checked for vmlinux(.o).

Do not pass the

modpost: pass -N option only for modules modpost

The built-in only code is not required to have MODULE_IMPORT_NS() to
use symbols. So, the namespace is not checked for vmlinux(.o).

Do not pass the meaningless -N option to the first pass of modpost.

Signed-off-by: Masahiro Yamada <[email protected]>

show more ...

modpost: move -T option close to the modpost command

The '-T -' option reads the file list from stdin.

It is clearer to put it close to the piped command.

Signed-off-by: Masahiro Yamada <masahiroy

modpost: move -T option close to the modpost command

The '-T -' option reads the file list from stdin.

It is clearer to put it close to the piped command.

Signed-off-by: Masahiro Yamada <[email protected]>

show more ...

modpost: fix -i (--ignore-errors) MAKEFLAGS detection

$(filter -i,$(MAKEFLAGS)) works only in limited use-cases.

The representation of $(MAKEFLAGS) depends on various factors:
- GNU Make version

modpost: fix -i (--ignore-errors) MAKEFLAGS detection

$(filter -i,$(MAKEFLAGS)) works only in limited use-cases.

The representation of $(MAKEFLAGS) depends on various factors:
- GNU Make version (version 3.8x or version 4.x)
- The presence of other flags like -j

In my experiments, $(MAKEFLAGS) is expanded as follows:

* GNU Make 3.8x:

* without -j option:
--no-print-directory -Rri

* with -j option:
--no-print-directory -Rr --jobserver-fds=3,4 -j -i

* GNU Make 4.x:

* without -j option:
irR --no-print-directory

* with -j option:
irR -j --jobserver-fds=3,4 --no-print-directory

For GNU Make 4.x, the flags are grouped as 'irR', which does not work.

For the single thread build with GNU Make 3.8x, the flags are grouped
as '-Rri', which does not work either.

To make it work for all cases, do likewise as commit 6f0fa58e4596
("kbuild: simplify silent build (-s) detection").

BTW, since commit ff9b45c55b26 ("kbuild: modpost: read modules.order
instead of $(MODVERDIR)/*.mod"), you also need to pass -k option to
build final *.ko files. 'make -i -k' ignores compile errors in modules,
and build as many remaining *.ko as possible.

Please note this feature is kind of dangerous if other modules depend
on the broken module because the generated modules will lack the correct
module dependency or CRC. Honestly, I am not a big fan of it, but I am
keeping this feature.

Fixes: eed380f3f593 ("modpost: Optionally ignore secondary errors seen if a single module build fails")
Cc: Guenter Roeck <[email protected]>
Signed-off-by: Masahiro Yamada <[email protected]>

show more ...

kbuild: disallow multi-word in M= or KBUILD_EXTMOD

$(firstword ...) in scripts/Makefile.modpost was added by commit
3f3fd3c05585 ("[PATCH] kbuild: allow multi-word $M in Makefile.modpost")
to build

kbuild: disallow multi-word in M= or KBUILD_EXTMOD

$(firstword ...) in scripts/Makefile.modpost was added by commit
3f3fd3c05585 ("[PATCH] kbuild: allow multi-word $M in Makefile.modpost")
to build multiple external module directories.

It was a solution to resolve symbol dependencies when an external
module depends on another external module.

Commit 0d96fb20b7ed ("kbuild: Add new Kbuild variable
KBUILD_EXTRA_SYMBOLS") introduced another solution by passing symbol
info via KBUILD_EXTRA_SYMBOLS, then broke the multi-word M= support.

include $(if $(wildcard $(KBUILD_EXTMOD)/Kbuild), \
$(KBUILD_EXTMOD)/Kbuild, $(KBUILD_EXTMOD)/Makefile)

... does not work if KBUILD_EXTMOD contains multiple words.

This feature has been broken for more than a decade. Remove the
bitrotten code, and stop parsing if M or KBUILD_EXTMOD contains
multiple words.

As Documentation/kbuild/modules.rst explains, if your module depends
on another one, there are two solutions:
- add a common top-level Kbuild file
- use KBUILD_EXTRA_SYMBOLS

Signed-off-by: Masahiro Yamada <[email protected]>

show more ...

modpost: return error if module is missing ns imports and MODULE_ALLOW_MISSING_NAMESPACE_IMPORTS=n

Currently when CONFIG_MODULE_ALLOW_MISSING_NAMESPACE_IMPORTS=n, modpost
only warns when a module is

modpost: return error if module is missing ns imports and MODULE_ALLOW_MISSING_NAMESPACE_IMPORTS=n

Currently when CONFIG_MODULE_ALLOW_MISSING_NAMESPACE_IMPORTS=n, modpost
only warns when a module is missing namespace imports. Under this
configuration, such a module cannot be loaded into the kernel anyway, as
the module loader would reject it. We might as well return a build
error when a module is missing namespace imports under
CONFIG_MODULE_ALLOW_MISSING_NAMESPACE_IMPORTS=n, so that the build
warning does not go ignored/unnoticed.

Signed-off-by: Jessica Yu <[email protected]>
Signed-off-by: Masahiro Yamada <[email protected]>

show more ...

kbuild: remove 'Building modules, stage 2.' log

This log is displayed every time modules are built, but it is not
so important.

Signed-off-by: Masahiro Yamada <[email protected]>

scripts/nsdeps: support nsdeps for external module builds

scripts/nsdeps is written to take care of only in-tree modules.
Perhaps, this is not a bug, but just a design. At least,
Documentation/core-

scripts/nsdeps: support nsdeps for external module builds

scripts/nsdeps is written to take care of only in-tree modules.
Perhaps, this is not a bug, but just a design. At least,
Documentation/core-api/symbol-namespaces.rst focuses on in-tree modules.

Having said that, some people already tried nsdeps for external modules.
So, it would be nice to support it.

Reported-by: Steve French <[email protected]>
Reported-by: Jessica Yu <[email protected]>
Signed-off-by: Masahiro Yamada <[email protected]>
Tested-by: Jessica Yu <[email protected]>
Acked-by: Jessica Yu <[email protected]>
Reviewed-by: Matthias Maennich <[email protected]>
Tested-by: Matthias Maennich <[email protected]>

show more ...