scripts: handle BrokenPipeError for python scripts

In the follow-up of commit fb3041d61f68 ("kbuild: fix SIGPIPE error
message for AR=gcc-ar and AR=llvm-ar"), Kees Cook pointed out that
tools should

scripts: handle BrokenPipeError for python scripts

In the follow-up of commit fb3041d61f68 ("kbuild: fix SIGPIPE error
message for AR=gcc-ar and AR=llvm-ar"), Kees Cook pointed out that
tools should _not_ catch their own SIGPIPEs [1] [2].

Based on his feedback, LLVM was fixed [3].

However, Python's default behavior is to show noisy bracktrace when
SIGPIPE is sent. So, scripts written in Python are basically in the
same situation as the buggy llvm tools.

Example:

$ make -s allnoconfig
$ make -s allmodconfig
$ scripts/diffconfig .config.old .config | head -n1
-ALIX n
Traceback (most recent call last):
File "/home/masahiro/linux/scripts/diffconfig", line 132, in <module>
main()
File "/home/masahiro/linux/scripts/diffconfig", line 130, in main
print_config("+", config, None, b[config])
File "/home/masahiro/linux/scripts/diffconfig", line 64, in print_config
print("+%s %s" % (config, new_value))
BrokenPipeError: [Errno 32] Broken pipe

Python documentation [4] notes how to make scripts die immediately and
silently:

"""
Piping output of your program to tools like head(1) will cause a
SIGPIPE signal to be sent to your process when the receiver of its
standard output closes early. This results in an exception like
BrokenPipeError: [Errno 32] Broken pipe. To handle this case,
wrap your entry point to catch this exception as follows:

import os
import sys

def main():
try:
# simulate large output (your code replaces this loop)
for x in range(10000):
print("y")
# flush output here to force SIGPIPE to be triggered
# while inside this try block.
sys.stdout.flush()
except BrokenPipeError:
# Python flushes standard streams on exit; redirect remaining output
# to devnull to avoid another BrokenPipeError at shutdown
devnull = os.open(os.devnull, os.O_WRONLY)
os.dup2(devnull, sys.stdout.fileno())
sys.exit(1) # Python exits with error code 1 on EPIPE

if __name__ == '__main__':
main()

Do not set SIGPIPE’s disposition to SIG_DFL in order to avoid
BrokenPipeError. Doing that would cause your program to exit
unexpectedly whenever any socket connection is interrupted while
your program is still writing to it.
"""

Currently, tools/perf/scripts/python/intel-pt-events.py seems to be the
only script that fixes the issue that way.

tools/perf/scripts/python/compaction-times.py uses another approach
signal.signal(signal.SIGPIPE, signal.SIG_DFL) but the Python
documentation clearly says "Don't do it".

I cannot fix all Python scripts since there are so many.
I fixed some in the scripts/ directory.

[1]: https://lore.kernel.org/all/202211161056.1B9611A@keescook/
[2]: https://github.com/llvm/llvm-project/issues/59037
[3]: https://github.com/llvm/llvm-project/commit/4787efa38066adb51e2c049499d25b3610c0877b
[4]: https://docs.python.org/3/library/signal.html#note-on-sigpipe

Signed-off-by: Masahiro Yamada <[email protected]>
Reviewed-by: Nick Desaulniers <[email protected]>
Reviewed-by: Nicolas Schier <[email protected]>

show more ...

checkkconfigsymbols.py: Remove skipping of help lines in parse_kconfig_file

When parsing Kconfig files to find symbol definitions and references,
lines after a 'help' line are skipped until a new co

checkkconfigsymbols.py: Remove skipping of help lines in parse_kconfig_file

When parsing Kconfig files to find symbol definitions and references,
lines after a 'help' line are skipped until a new config definition
starts.

However, Kconfig statements can actually be after a help section, as
long as these have shallower indentation. These are skipped by the
parser.

This means that symbols referenced in this kind of statements are
ignored by this function and thus are not considered undefined
references in case the symbol is not defined.

Remove the 'skip' logic entirely, as it is not needed if we just use the
STMT regex to find the end of help lines.

However, this means that keywords that appear as part of the help
message (i.e. with the same indentation as the help lines) it will be
considered as a reference/definition. This can happen now as well, but
only with REGEX_KCONFIG_DEF lines. Also, the keyword must have a SYMBOL
after it, which probably means that someone referenced a config in the
help so it seems like a bonus :)

The real solution is to keep track of the indentation when a the first
help line in encountered and then handle DEF and STMT lines only if the
indentation is shallower.

Signed-off-by: Ariel Marcovitch <[email protected]>
Signed-off-by: Masahiro Yamada <[email protected]>

show more ...

checkkconfigsymbols.py: Forbid passing 'HEAD' to --commit

As opposed to the --diff option, --commit can get ref names instead of
commit hashes.

When using the --commit option, the script resets the

checkkconfigsymbols.py: Forbid passing 'HEAD' to --commit

As opposed to the --diff option, --commit can get ref names instead of
commit hashes.

When using the --commit option, the script resets the working directory
to the commit before the given ref, by adding '~' to the end of the ref.

However, the 'HEAD' ref is relative, and so when the working directory
is reset to 'HEAD~', 'HEAD' points to what was 'HEAD~'. Then when the
script resets to 'HEAD' it actually stays in the same commit. In this
case, the script won't report any cases because there is no diff between
the cases of the two refs.

Prevent the user from using HEAD refs.

A better solution might be to resolve the refs before doing the
reset, but for now just disallow such refs.

Signed-off-by: Ariel Marcovitch <[email protected]>
Signed-off-by: Masahiro Yamada <[email protected]>

show more ...

checkkconfigsymbols.py: Fix the '--ignore' option

It seems like the implementation of the --ignore option is broken.

In check_symbols_helper, when going through the list of files, a file is
added t

checkkconfigsymbols.py: Fix the '--ignore' option

It seems like the implementation of the --ignore option is broken.

In check_symbols_helper, when going through the list of files, a file is
added to the list of source files to check if it matches the ignore
pattern. Instead, as stated in the comment below this condition, the
file should be added if it doesn't match the pattern.

This means that when providing an ignore pattern, the only files that
will be checked will be the ones we want the ignore, in addition to the
Kconfig files that don't match the pattern (the check in
parse_kconfig_files is done right)

Signed-off-by: Ariel Marcovitch <[email protected]>
Signed-off-by: Masahiro Yamada <[email protected]>

show more ...

kconfig: remove '---help---' support

The conversion is done. No more user of '---help---'.

Cc: Ulf Magnusson <[email protected]>
Signed-off-by: Masahiro Yamada <[email protected]>

treewide: Replace GPLv2 boilerplate/reference with SPDX - rule 166

Based on 1 normalized pattern(s):

licensed under the terms of the gnu gpl license version 2

extracted by the scancode license s

treewide: Replace GPLv2 boilerplate/reference with SPDX - rule 166

Based on 1 normalized pattern(s):

licensed under the terms of the gnu gpl license version 2

extracted by the scancode license scanner the SPDX license identifier

GPL-2.0-only

has been chosen to replace the boilerplate/reference in 62 file(s).

Signed-off-by: Thomas Gleixner <[email protected]>
Reviewed-by: Allison Randal <[email protected]>
Reviewed-by: Kate Stewart <[email protected]>
Reviewed-by: Richard Fontana <[email protected]>
Cc: [email protected]
Link: https://lkml.kernel.org/r/[email protected]
Signed-off-by: Greg Kroah-Hartman <[email protected]>

show more ...

checkkconfigsymbols.py: support Kconfig's 'imply' statement

Support the new imply statement in Kconfig. The imply statement has
been added by commit 237e3ad0f195 ("Kconfig: Introduce the "imply"
ke

checkkconfigsymbols.py: support Kconfig's 'imply' statement

Support the new imply statement in Kconfig. The imply statement has
been added by commit 237e3ad0f195 ("Kconfig: Introduce the "imply"
keyword") and is a weak version of a select, but the target symbol can
still be turned off.

Signed-off-by: Andreas Ziegler <[email protected]>
Signed-off-by: Valentin Rothberg <[email protected]>
Signed-off-by: Greg Kroah-Hartman <[email protected]>

show more ...

checkkconfigsymbols.py: don't sort similar symbols

Don't sort the list of string-similar Kconfig symbols alphabetically to
preserve the correct order of string similarity.

Signed-off-by: Valentin R

checkkconfigsymbols.py: don't sort similar symbols

Don't sort the list of string-similar Kconfig symbols alphabetically to
preserve the correct order of string similarity.

Signed-off-by: Valentin Rothberg <[email protected]>
Signed-off-by: Greg Kroah-Hartman <[email protected]>

show more ...

checkkconfigsymbols.py: support git's "^" syntax

Support git's "^" syntax for diffing two commits, for instance via
"--diff HEAD^^^..HEAD".

Signed-off-by: Michael Ellermann <[email protected]>
Sig

checkkconfigsymbols.py: support git's "^" syntax

Support git's "^" syntax for diffing two commits, for instance via
"--diff HEAD^^^..HEAD".

Signed-off-by: Michael Ellermann <[email protected]>
Signed-off-by: Valentin Rothberg <[email protected]>
Signed-off-by: Greg Kroah-Hartman <[email protected]>

show more ...

checkkconfigsymbols: use arglist instead of cmd string

Splitting a command string could lead to unintended arguments. Use an
argument list in the execute() function instead.

Signed-off-by: Valenti

checkkconfigsymbols: use arglist instead of cmd string

Splitting a command string could lead to unintended arguments. Use an
argument list in the execute() function instead.

Signed-off-by: Valentin Rothberg <[email protected]>
Signed-off-by: Greg Kroah-Hartman <[email protected]>

show more ...

checkkconfigsymblos: consistent symbol terminology

'symbol' and 'feature' are used synonymously to refer to Kconfig symbols
(configs, menus, etc.). Use the term 'symbol' to have a consistent
termin

checkkconfigsymblos: consistent symbol terminology

'symbol' and 'feature' are used synonymously to refer to Kconfig symbols
(configs, menus, etc.). Use the term 'symbol' to have a consistent
terminology and to make the code more comprehensible.

Signed-off-by: Valentin Rothberg <[email protected]>
Signed-off-by: Greg Kroah-Hartman <[email protected]>

show more ...

checkkconfigsymbols.py: fix pylint and pep8 warnings

Fix pylint and pep8 warnings to have a consistent syntax and style.

Signed-off-by: Valentin Rothberg <[email protected]>
Signed-off-by:

checkkconfigsymbols.py: fix pylint and pep8 warnings

Fix pylint and pep8 warnings to have a consistent syntax and style.

Signed-off-by: Valentin Rothberg <[email protected]>
Signed-off-by: Greg Kroah-Hartman <[email protected]>

show more ...

checkkconfigsymbols: use ArgumentParser

Replace the deprecated OptionParser with ArgumentParser, as recommended
by pylint.

Signed-off-by: Valentin Rothberg <[email protected]>
Signed-off-b

checkkconfigsymbols: use ArgumentParser

Replace the deprecated OptionParser with ArgumentParser, as recommended
by pylint.

Signed-off-by: Valentin Rothberg <[email protected]>
Signed-off-by: Greg Kroah-Hartman <[email protected]>

show more ...

checkkconfigsymbols.py: port to Python 3

Python 2 is slowly dying, so port the script to Python 3.

Signed-off-by: Valentin Rothberg <[email protected]>
Signed-off-by: Greg Kroah-Hartman <g

checkkconfigsymbols.py: port to Python 3

Python 2 is slowly dying, so port the script to Python 3.

Signed-off-by: Valentin Rothberg <[email protected]>
Signed-off-by: Greg Kroah-Hartman <[email protected]>

show more ...

checkkconfigsymbols.py: avoid shell injection

Use subprocess and set shell to False to avoid potential shell
injections.

Reported-by: Bernd Dietzel <[email protected]>
Signed-off-by: Valentin Rothb

checkkconfigsymbols.py: avoid shell injection

Use subprocess and set shell to False to avoid potential shell
injections.

Reported-by: Bernd Dietzel <[email protected]>
Signed-off-by: Valentin Rothberg <[email protected]>
Signed-off-by: Greg Kroah-Hartman <[email protected]>

show more ...

checkkconfigsymbols.py: add --no-color option, don't print color to non-TTY

Only print the ANSI colour escape codes if stdout is a TTY. Useful if
redirecting output to a file or piping to another sc

checkkconfigsymbols.py: add --no-color option, don't print color to non-TTY

Only print the ANSI colour escape codes if stdout is a TTY. Useful if
redirecting output to a file or piping to another script.

Also add a new option, --no-color, if the user wants to disable colour
output for whatever reason.

Signed-off-by: Andrew Donnellan <[email protected]>
Acked-by: Valentin Rothberg <[email protected]>
Signed-off-by: Greg Kroah-Hartman <[email protected]>

show more ...

checkkconfigsymbols.py: Fix typo in help message

Fix a typo in the help message for the -d parameter by removing one 'm'.

Signed-off-by: Andreas Ziegler <[email protected]>
Acked-by: Valentin

checkkconfigsymbols.py: Fix typo in help message

Fix a typo in the help message for the -d parameter by removing one 'm'.

Signed-off-by: Andreas Ziegler <[email protected]>
Acked-by: Valentin Rothberg <[email protected]>
Signed-off-by: Greg Kroah-Hartman <[email protected]>

show more ...

checkkconfigsymbols.py: find similar symbols

Add support to find string-similar symbols. When option --sim SYM is
specified, checkkconfigsymbols.py will print at most 10 symbols defined
in Kconfig

checkkconfigsymbols.py: find similar symbols

Add support to find string-similar symbols. When option --sim SYM is
specified, checkkconfigsymbols.py will print at most 10 symbols defined
in Kconfig that are string similar to SYM in the following format:
Similar symbols: $COMMA_SEPARATED_LIST_OF_SYMBOLS

Note, if no similar symbols are found it is indicated as follows:
Similar symbols: no similar symbols found

Since the implemented functionality is also useful when searching the
entire source or when diffing two commits, a list of similar symbols is
printed unconditionally with the other data. In order to make the
output more readable, the format now looks as follows:

$UNDEFINED_SYMBOL
Referencing files: $COMMA_SEPARATED_LIST_OF_FILES
Similar symbols: $COMMA_SEPARATED_LIST_OF_SYMBOLS
[Optional with '--find']
Commits changing symbol:
- $COMMIT_1_HASH ("$COMMIT_1_MESSAGE")
- $COMMIT_2_HASH ("$COMMIT_2_MESSAGE")
or
- no commit found

Signed-off-by: Valentin Rothberg <[email protected]>
Signed-off-by: Greg Kroah-Hartman <[email protected]>

show more ...

checkkconfigsymbols.py: multiprocessing of files

Distribute the parsing of source and Kconfig files on all available
cores to speed up processing.

Signed-off-by: Valentin Rothberg <valentinrothberg

checkkconfigsymbols.py: multiprocessing of files

Distribute the parsing of source and Kconfig files on all available
cores to speed up processing.

Signed-off-by: Valentin Rothberg <[email protected]>
Signed-off-by: Greg Kroah-Hartman <[email protected]>

show more ...

scripts/checkkconfigsymbols.py: support default statements

Until now, checkkonfigsymbols.py did not check default statements for
references on missing Kconfig symbols (i.e., undefined Kconfig option

scripts/checkkconfigsymbols.py: support default statements

Until now, checkkonfigsymbols.py did not check default statements for
references on missing Kconfig symbols (i.e., undefined Kconfig options).
Hence, add support to parse and check the Kconfig default statement.

Signed-off-by: Valentin Rothberg <[email protected]>
Signed-off-by: Greg Kroah-Hartman <[email protected]>

show more ...

checkkconfigsymbols.py: colored output

Color output to make it more readable. Symbols will be printed yellow,
relevant commits (see --find) red.

Signed-off-by: Valentin Rothberg <valentinrothberg@

checkkconfigsymbols.py: colored output

Color output to make it more readable. Symbols will be printed yellow,
relevant commits (see --find) red.

Signed-off-by: Valentin Rothberg <[email protected]>
Acked-by: Stefan Hengelein <[email protected]>
Acked-by: Andreas Ruprecht <[email protected]>
Signed-off-by: Greg Kroah-Hartman <[email protected]>

show more ...

checkkconfigsymbols.py: find relevant commits

Add option -f/--find to find relevant commits when using the --diff
option. --find is useful in case a user wants to check commits that
potentially cau

checkkconfigsymbols.py: find relevant commits

Add option -f/--find to find relevant commits when using the --diff
option. --find is useful in case a user wants to check commits that
potentially cause a Kconfig symbol to be missing. This is done via 'git
log -G $SYMBOL' (i.e., to get a list of commits that change $SYMBOL).
The relevant commits are printed below the "SYMBOL\tFILES" line,
followed by an empty line to increase readability.

Signed-off-by: Valentin Rothberg <[email protected]>
Acked-by: Stefan Hengelein <[email protected]>
Acked-by: Andreas Ruprecht <[email protected]>
Signed-off-by: Greg Kroah-Hartman <[email protected]>

show more ...

checkkconfigsymbols.py: set python2 as default interpreter

Some more recent distributions set the default interpreter to python3,
causing the script to break since it's written for python2.

Signed-

checkkconfigsymbols.py: set python2 as default interpreter

Some more recent distributions set the default interpreter to python3,
causing the script to break since it's written for python2.

Signed-off-by: Valentin Rothberg <[email protected]>
Signed-off-by: Greg Kroah-Hartman <[email protected]>

show more ...

checkkconfigsymbols.py: add option -i to ignore files

Sometimes a user might be interested to filter certain reports (e.g.,
the many defconfigs). Now, this can be achieved by specifying a Python
re

checkkconfigsymbols.py: add option -i to ignore files

Sometimes a user might be interested to filter certain reports (e.g.,
the many defconfigs). Now, this can be achieved by specifying a Python
regex with -i / --ignore.

Signed-off-by: Valentin Rothberg <[email protected]>
Signed-off-by: Greg Kroah-Hartman <[email protected]>

show more ...

checkkconfigsymbols.py: fix sorted output

Commit b1a3f243485f ("checkkconfigsymbols.py: make it Git aware")
mistakenly removed to print undefined Kconfig symbols in alphabetical
order. Furthermore,

checkkconfigsymbols.py: fix sorted output

Commit b1a3f243485f ("checkkconfigsymbols.py: make it Git aware")
mistakenly removed to print undefined Kconfig symbols in alphabetical
order. Furthermore, the script does not print anything anymore when the
entire tree is checked (i.e., when no commit is specified).

This patch restores the sorted output and adds the missing print for the
default case. Additionally, the file lists are now sorted as well which
(a) makes it easier to read and (b) makes the output deterministic.

Signed-off-by: Valentin Rothberg <[email protected]>
Signed-off-by: Greg Kroah-Hartman <[email protected]>

show more ...