|
Revision tags: v6.15, v6.15-rc7, v6.15-rc6, v6.15-rc5, v6.15-rc4, v6.15-rc3, v6.15-rc2, v6.15-rc1, v6.14 |
|
| #
14e594a1 |
| 19-Mar-2025 |
Rae Moar <[email protected]> |
kunit: tool: fix count of tests if late test plan
Fix test count with late test plan.
For example, TAP version 13 ok 1 test1 1..4
Returns a count of 1 passed, 1 crashed (because it expects t
kunit: tool: fix count of tests if late test plan
Fix test count with late test plan.
For example, TAP version 13 ok 1 test1 1..4
Returns a count of 1 passed, 1 crashed (because it expects tests after the test plan): returning the total count of 2 tests
Change this to be 1 passed, 1 error: total count of 1 test
Link: https://lore.kernel.org/r/[email protected] Signed-off-by: Rae Moar <[email protected]> Reviewed-by: David Gow <[email protected]> Signed-off-by: Shuah Khan <[email protected]>
show more ...
|
|
Revision tags: v6.14-rc7 |
|
| #
2e0cf2b3 |
| 13-Mar-2025 |
Rae Moar <[email protected]> |
kunit: tool: add test to check parsing late test plan
Add test to check for the infinite loop caused by the inability to parse a late test plan.
The test parses the following output: TAP version 1
kunit: tool: add test to check parsing late test plan
Add test to check for the infinite loop caused by the inability to parse a late test plan.
The test parses the following output: TAP version 13 ok 4 test4 1..4
Link: https://lore.kernel.org/r/[email protected] Signed-off-by: Rae Moar <[email protected]> Reviewed-by: David Gow <[email protected]> Signed-off-by: Shuah Khan <[email protected]>
show more ...
|
|
Revision tags: v6.14-rc6, v6.14-rc5, v6.14-rc4, v6.14-rc3, v6.14-rc2, v6.14-rc1, v6.13, v6.13-rc7, v6.13-rc6, v6.13-rc5, v6.13-rc4, v6.13-rc3, v6.13-rc2, v6.13-rc1, v6.12 |
|
| #
3c67a2c0 |
| 13-Nov-2024 |
Rae Moar <[email protected]> |
kunit: tool: print failed tests only
Add flag --failed to kunit.py to print only failed tests. This printing is done after running is over.
This patch also adds the method print_test() that will al
kunit: tool: print failed tests only
Add flag --failed to kunit.py to print only failed tests. This printing is done after running is over.
This patch also adds the method print_test() that will also print your Test object. Before, all printing of tests occurred during parsing. This method could be useful in the future when converting to/from KTAP to this pretty-print output.
Link: https://lore.kernel.org/r/[email protected] Signed-off-by: Rae Moar <[email protected]> Reviewed-by: David Gow <[email protected]> Signed-off-by: Shuah Khan <[email protected]>
show more ...
|
| #
062a9dd9 |
| 13-Nov-2024 |
David Gow <[email protected]> |
kunit: tool: Only print the summary
Allow only printing the summary at the end of a test run, rather than all individual test results. This summary will list a few failing tests if there are any.
T
kunit: tool: Only print the summary
Allow only printing the summary at the end of a test run, rather than all individual test results. This summary will list a few failing tests if there are any.
To use:
./tools/testing/kunit/kunit.py run --summary
Link: https://lore.kernel.org/r/[email protected] Signed-off-by: Rae Moar <[email protected]> Signed-off-by: David Gow <[email protected]> Signed-off-by: Shuah Khan <[email protected]>
show more ...
|
|
Revision tags: v6.12-rc7, v6.12-rc6, v6.12-rc5, v6.12-rc4, v6.12-rc3, v6.12-rc2, v6.12-rc1, v6.11, v6.11-rc7, v6.11-rc6, v6.11-rc5, v6.11-rc4, v6.11-rc3, v6.11-rc2, v6.11-rc1, v6.10, v6.10-rc7, v6.10-rc6, v6.10-rc5, v6.10-rc4, v6.10-rc3, v6.10-rc2, v6.10-rc1, v6.9, v6.9-rc7, v6.9-rc6, v6.9-rc5, v6.9-rc4, v6.9-rc3, v6.9-rc2, v6.9-rc1, v6.8, v6.8-rc7, v6.8-rc6, v6.8-rc5, v6.8-rc4, v6.8-rc3, v6.8-rc2, v6.8-rc1, v6.7, v6.7-rc8, v6.7-rc7, v6.7-rc6, v6.7-rc5 |
|
| #
6eb0ea28 |
| 07-Dec-2023 |
Rae Moar <[email protected]> |
kunit: tool: add test for parsing attributes
Add test for parsing attributes to kunit_tool_test.py. Test checks attributes are parsed and saved in the test logs.
This test also checks that the attr
kunit: tool: add test for parsing attributes
Add test for parsing attributes to kunit_tool_test.py. Test checks attributes are parsed and saved in the test logs.
This test also checks that the attributes have not interfered with the parsing of other test information, specifically the suite header as the test plan was being incorrectely parsed.
Signed-off-by: Rae Moar <[email protected]> Reviewed-by: David Gow <[email protected]> Signed-off-by: Shuah Khan <[email protected]>
show more ...
|
|
Revision tags: v6.7-rc4, v6.7-rc3, v6.7-rc2, v6.7-rc1, v6.6, v6.6-rc7, v6.6-rc6, v6.6-rc5, v6.6-rc4, v6.6-rc3, v6.6-rc2, v6.6-rc1, v6.5, v6.5-rc7, v6.5-rc6, v6.5-rc5, v6.5-rc4 |
|
| #
723c8258 |
| 25-Jul-2023 |
Rae Moar <[email protected]> |
kunit: tool: Add command line interface to filter and report attributes
Add ability to kunit.py to filter attributes and report a list of tests including attributes without running tests.
Add flag
kunit: tool: Add command line interface to filter and report attributes
Add ability to kunit.py to filter attributes and report a list of tests including attributes without running tests.
Add flag "--filter" to input filters on test attributes. Tests will be filtered out if they do not match all inputted filters.
Example: --filter speed=slow (This filter would run only the tests that are marked as slow)
Filters have operations: <, >, <=, >=, !=, and =. But note that the characters < and > are often interpreted by the shell, so they may need to be quoted or escaped.
Example: --filter "speed>slow" or --filter speed\>slow (This filter would run only the tests that have the speed faster than slow.
Additionally, multiple filters can be used.
Example: --filter "speed=slow, module!=example" (This filter would run only the tests that have the speed slow and are not in the "example" module)
Note if the user wants to skip filtered tests instead of not running/showing them use the "--filter_action=skip" flag instead.
Expose the output of kunit.action=list option with flag "--list_tests" to output a list of tests. Additionally, add flag "--list_tests_attr" to output a list of tests and their attributes. These flags are useful to see tests and test attributes without needing to run tests.
Example of the output of "--list_tests_attr": example example.test_1 example.test_2 # example.test_2.speed: slow
This output includes a suite, example, with two test cases, test_1 and test_2. And in this instance test_2 has been marked as slow.
Reviewed-by: David Gow <[email protected]> Signed-off-by: Rae Moar <[email protected]> Signed-off-by: Shuah Khan <[email protected]>
show more ...
|
|
Revision tags: v6.5-rc3, v6.5-rc2, v6.5-rc1, v6.4, v6.4-rc7, v6.4-rc6, v6.4-rc5, v6.4-rc4, v6.4-rc3, v6.4-rc2, v6.4-rc1, v6.3, v6.3-rc7, v6.3-rc6, v6.3-rc5, v6.3-rc4, v6.3-rc3 |
|
| #
126901ba |
| 16-Mar-2023 |
Daniel Latypov <[email protected]> |
kunit: tool: remove unused imports and variables
We don't run a linter regularly over kunit.py code (the default settings on most don't like kernel style, e.g. tabs) so some of these imports didn't
kunit: tool: remove unused imports and variables
We don't run a linter regularly over kunit.py code (the default settings on most don't like kernel style, e.g. tabs) so some of these imports didn't get removed when they stopped being used.
Signed-off-by: Daniel Latypov <[email protected]> Reviewed-by: David Gow <[email protected]> Signed-off-by: Shuah Khan <[email protected]>
show more ...
|
|
Revision tags: v6.3-rc2, v6.3-rc1, v6.2, v6.2-rc8, v6.2-rc7, v6.2-rc6, v6.2-rc5, v6.2-rc4, v6.2-rc3, v6.2-rc2, v6.2-rc1, v6.1, v6.1-rc8 |
|
| #
c2bb92bc |
| 30-Nov-2022 |
Daniel Latypov <[email protected]> |
kunit: tool: make parser preserve whitespace when printing test log
Currently, kunit_parser.py is stripping all leading whitespace to make parsing easier. But this means we can't accurately show ker
kunit: tool: make parser preserve whitespace when printing test log
Currently, kunit_parser.py is stripping all leading whitespace to make parsing easier. But this means we can't accurately show kernel output for failing tests or when the kernel crashes.
Embarassingly, this affects even KUnit's own output, e.g. [13:40:46] Expected 2 + 1 == 2, but [13:40:46] 2 + 1 == 3 (0x3) [13:40:46] not ok 1 example_simple_test [13:40:46] [FAILED] example_simple_test
After this change, here's what the output in context would look like [13:40:46] =================== example (4 subtests) =================== [13:40:46] # example_simple_test: initializing [13:40:46] # example_simple_test: EXPECTATION FAILED at lib/kunit/kunit-example-test.c:29 [13:40:46] Expected 2 + 1 == 2, but [13:40:46] 2 + 1 == 3 (0x3) [13:40:46] [FAILED] example_simple_test [13:40:46] [SKIPPED] example_skip_test [13:40:46] [SKIPPED] example_mark_skipped_test [13:40:46] [PASSED] example_all_expect_macros_test [13:40:46] # example: initializing suite [13:40:46] # example: pass:1 fail:1 skip:2 total:4 [13:40:46] # Totals: pass:1 fail:1 skip:2 total:4 [13:40:46] ===================== [FAILED] example =====================
This example shows one minor cosmetic defect this approach has. The test counts lines prevent us from dedenting the suite-level output. But at the same time, any form of non-KUnit output would do the same unless it happened to be indented as well.
Signed-off-by: Daniel Latypov <[email protected]> Reviewed-by: David Gow <[email protected]> Signed-off-by: Shuah Khan <[email protected]>
show more ...
|
| #
5937e0c0 |
| 29-Nov-2022 |
Daniel Latypov <[email protected]> |
kunit: tool: don't include KTAP headers and the like in the test log
We print the "test log" on failure. This is meant to be all the kernel output that happened during the test.
But we also include
kunit: tool: don't include KTAP headers and the like in the test log
We print the "test log" on failure. This is meant to be all the kernel output that happened during the test.
But we also include the special KTAP lines in it, which are often redundant.
E.g. we include the "not ok" line in the log, right before we print that the test case failed... [13:51:48] Expected 2 + 1 == 2, but [13:51:48] 2 + 1 == 3 (0x3) [13:51:48] not ok 1 example_simple_test [13:51:48] [FAILED] example_simple_test
More full example after this patch: [13:51:48] =================== example (4 subtests) =================== [13:51:48] # example_simple_test: initializing [13:51:48] # example_simple_test: EXPECTATION FAILED at lib/kunit/kunit-example-test.c:29 [13:51:48] Expected 2 + 1 == 2, but [13:51:48] 2 + 1 == 3 (0x3) [13:51:48] [FAILED] example_simple_test
Signed-off-by: Daniel Latypov <[email protected]> Reviewed-by: David Gow <[email protected]> Signed-off-by: Shuah Khan <[email protected]>
show more ...
|
|
Revision tags: v6.1-rc7 |
|
| #
434498a6 |
| 23-Nov-2022 |
Rae Moar <[email protected]> |
kunit: tool: parse KTAP compliant test output
Change the KUnit parser to be able to parse test output that complies with the KTAP version 1 specification format found here: https://kernel.org/doc/ht
kunit: tool: parse KTAP compliant test output
Change the KUnit parser to be able to parse test output that complies with the KTAP version 1 specification format found here: https://kernel.org/doc/html/latest/dev-tools/ktap.html. Ensure the parser is able to parse tests with the original KUnit test output format as well.
KUnit parser now accepts any of the following test output formats:
Original KUnit test output format:
TAP version 14 1..1 # Subtest: kunit-test-suite 1..3 ok 1 - kunit_test_1 ok 2 - kunit_test_2 ok 3 - kunit_test_3 # kunit-test-suite: pass:3 fail:0 skip:0 total:3 # Totals: pass:3 fail:0 skip:0 total:3 ok 1 - kunit-test-suite
KTAP version 1 test output format:
KTAP version 1 1..1 KTAP version 1 1..3 ok 1 kunit_test_1 ok 2 kunit_test_2 ok 3 kunit_test_3 ok 1 kunit-test-suite
New KUnit test output format (changes made in the next patch of this series):
KTAP version 1 1..1 KTAP version 1 # Subtest: kunit-test-suite 1..3 ok 1 kunit_test_1 ok 2 kunit_test_2 ok 3 kunit_test_3 # kunit-test-suite: pass:3 fail:0 skip:0 total:3 # Totals: pass:3 fail:0 skip:0 total:3 ok 1 kunit-test-suite
Signed-off-by: Rae Moar <[email protected]> Reviewed-by: Daniel Latypov <[email protected]> Reviewed-by: David Gow <[email protected]> Signed-off-by: Shuah Khan <[email protected]>
show more ...
|
|
Revision tags: v6.1-rc6, v6.1-rc5 |
|
| #
0a7d5c30 |
| 11-Nov-2022 |
Daniel Latypov <[email protected]> |
kunit: tool: tweak error message when no KTAP found
We currently tell people we "couldn't find any KTAP output" with no indication as to what this might mean.
After this patch, we get:
$ ./tools/t
kunit: tool: tweak error message when no KTAP found
We currently tell people we "couldn't find any KTAP output" with no indication as to what this might mean.
After this patch, we get:
$ ./tools/testing/kunit/kunit.py parse /dev/null ============================================================ [ERROR] Test: <missing>: Could not find any KTAP output. Did any KUnit tests run? ============================================================ Testing complete. Ran 0 tests: errors: 1
Note: we could try and generate a more verbose message like > Please check .kunit/test.log to see the raw kernel output. or the like, but we'd need to know what the build dir was to know where test.log actually lives.
This patch tries to make a more minimal improvement.
Signed-off-by: Daniel Latypov <[email protected]> Reviewed-by: David Gow <[email protected]> Signed-off-by: Shuah Khan <[email protected]>
show more ...
|
|
Revision tags: v6.1-rc4 |
|
| #
101e32a0 |
| 03-Nov-2022 |
Daniel Latypov <[email protected]> |
kunit: tool: remove redundant file.close() call in unit test
We're using a `with` block above, so the file object is already closed.
Signed-off-by: Daniel Latypov <[email protected]> Reviewed-by:
kunit: tool: remove redundant file.close() call in unit test
We're using a `with` block above, so the file object is already closed.
Signed-off-by: Daniel Latypov <[email protected]> Reviewed-by: David Gow <[email protected]> Signed-off-by: Shuah Khan <[email protected]>
show more ...
|
| #
05d9d2c3 |
| 03-Nov-2022 |
Daniel Latypov <[email protected]> |
kunit: tool: unit tests all check parser errors, standardize formatting a bit
Let's verify that the parser isn't reporting any errors for valid inputs.
This change also * does result.status checkin
kunit: tool: unit tests all check parser errors, standardize formatting a bit
Let's verify that the parser isn't reporting any errors for valid inputs.
This change also * does result.status checking on one line * makes sure we consistently do it outside of the `with` block
Signed-off-by: Daniel Latypov <[email protected]> Reviewed-by: David Gow <[email protected]> Signed-off-by: Shuah Khan <[email protected]>
show more ...
|
| #
f473dd94 |
| 03-Nov-2022 |
Daniel Latypov <[email protected]> |
kunit: tool: make TestCounts a dataclass
Since we're using Python 3.7+, we can use dataclasses to tersen the code.
It also lets us create pre-populated TestCounts() objects and compare them in our
kunit: tool: make TestCounts a dataclass
Since we're using Python 3.7+, we can use dataclasses to tersen the code.
It also lets us create pre-populated TestCounts() objects and compare them in our unit test. (Before, you could only create empty ones).
Signed-off-by: Daniel Latypov <[email protected]> Reviewed-by: David Gow <[email protected]> Signed-off-by: Shuah Khan <[email protected]>
show more ...
|
|
Revision tags: v6.1-rc3 |
|
| #
f19dd011 |
| 28-Oct-2022 |
Daniel Latypov <[email protected]> |
kunit: tool: print summary of failed tests if a few failed out of a lot
E.g. all the hw_breakpoint tests are failing right now. So if I run `kunit.py run --altests --arch=x86_64`, then I see > Testi
kunit: tool: print summary of failed tests if a few failed out of a lot
E.g. all the hw_breakpoint tests are failing right now. So if I run `kunit.py run --altests --arch=x86_64`, then I see > Testing complete. Ran 408 tests: passed: 392, failed: 9, skipped: 7
Seeing which 9 tests failed out of the hundreds is annoying. If my terminal doesn't have scrollback support, I have to resort to looking at `.kunit/test.log` for the `not ok` lines.
Teach kunit.py to print a summarized list of failures if the # of tests reachs an arbitrary threshold (>=100 tests).
To try and keep the output from being too long/noisy, this new logic a) just reports "parent_test failed" if every child test failed b) won't print anything if there are >10 failures (also arbitrary).
With this patch, we get an extra line of output showing: > Testing complete. Ran 408 tests: passed: 392, failed: 9, skipped: 7 > Failures: hw_breakpoint
This also works with parameterized tests, e.g. if I add a fake failure > Failures: kcsan.test_atomic_builtins_missing_barrier.threads=6
Note: we didn't have enough tests for this to be a problem before. But with commit 980ac3ad0512 ("kunit: tool: rename all_test_uml.config, use it for --alltests"), --alltests works and thus running >100 tests will probably become more common.
Signed-off-by: Daniel Latypov <[email protected]> Reviewed-by: David Gow <[email protected]> Signed-off-by: Shuah Khan <[email protected]>
show more ...
|
| #
3ffdcf7e |
| 28-Oct-2022 |
Daniel Latypov <[email protected]> |
kunit: tool: make unit test not print parsed testdata to stdout
Currently, if you run $ ./tools/testing/kunit/kunit_tool_test.py you'll see a lot of output from the parser as we feed it testdata.
T
kunit: tool: make unit test not print parsed testdata to stdout
Currently, if you run $ ./tools/testing/kunit/kunit_tool_test.py you'll see a lot of output from the parser as we feed it testdata.
This makes the output hard to read and fairly confusing, esp. since our testdata includes example failures, which get printed out in red.
Silence that output so real failures are easier to see.
Signed-off-by: Daniel Latypov <[email protected]> Reviewed-by: David Gow <[email protected]> Signed-off-by: Shuah Khan <[email protected]>
show more ...
|
|
Revision tags: v6.1-rc2, v6.1-rc1, v6.0, v6.0-rc7, v6.0-rc6, v6.0-rc5, v6.0-rc4 |
|
| #
980ac3ad |
| 02-Sep-2022 |
Daniel Latypov <[email protected]> |
kunit: tool: rename all_test_uml.config, use it for --alltests
Context: 1. all_tests_uml.config used to be UML specific back when users to manually specify CONFIG_VIRTIO_UML=y to enable CONFIG_PC
kunit: tool: rename all_test_uml.config, use it for --alltests
Context: 1. all_tests_uml.config used to be UML specific back when users to manually specify CONFIG_VIRTIO_UML=y to enable CONFIG_PCI=y. 2. --alltests used allyesconfig along with a curated list of options to disable. It's only ever worked for brief periods of time and has perennially been broken due to compile issues.
Now all_tests_uml.config should work across ~all architectures. Let's instead use this to implement --alltests.
Note: if anyone was using all_tests_uml.config, this change breaks them. I think that's unlikely since it was added in 5.19 and was a lot to type: --kunitconfig=tools/testing/kunit/configs/all_tests_uml.config. We could make it a symlink to the new name, but I don't think the caution is warranted here.
Signed-off-by: Daniel Latypov <[email protected]> Reviewed-by: David Gow <[email protected]> Signed-off-by: Shuah Khan <[email protected]>
show more ...
|
|
Revision tags: v6.0-rc3, v6.0-rc2, v6.0-rc1, v5.19, v5.19-rc8, v5.19-rc7, v5.19-rc6 |
|
| #
6fc3a863 |
| 08-Jul-2022 |
David Gow <[email protected]> |
kunit: tool: Enable virtio/PCI by default on UML
There are several tests which depend on PCI, and hence need a bunch of extra options to run under UML. This makes it awkward to give configuration in
kunit: tool: Enable virtio/PCI by default on UML
There are several tests which depend on PCI, and hence need a bunch of extra options to run under UML. This makes it awkward to give configuration instructions (whether in documentation, or as part of a .kunitconfig file), as two separate, incompatible sets of config options are required for UML and "most other architectures".
For non-UML architectures, it's possible to add default kconfig options via the qemu_config python files, but there's no equivalent for UML. Add a new tools/testing/kunit/configs/arch_uml.config file containing extra kconfig options to use on UML.
Tested-by: José Expósito <[email protected]> Reviewed-by: Daniel Latypov <[email protected]> Signed-off-by: David Gow <[email protected]> Reviewed-by: Brendan Higgins <[email protected]> Signed-off-by: Daniel Latypov <[email protected]> Signed-off-by: Shuah Khan <[email protected]>
show more ...
|
| #
53b46621 |
| 08-Jul-2022 |
Daniel Latypov <[email protected]> |
kunit: tool: make --kunitconfig repeatable, blindly concat
It's come up a few times that it would be useful to have --kunitconfig be repeatable [1][2].
This could be done before with a bit of shell
kunit: tool: make --kunitconfig repeatable, blindly concat
It's come up a few times that it would be useful to have --kunitconfig be repeatable [1][2].
This could be done before with a bit of shell-fu, e.g. $ find fs/ -name '.kunitconfig' -exec cat {} + | \ ./tools/testing/kunit/kunit.py run --kunitconfig=/dev/stdin or equivalently: $ cat fs/ext4/.kunitconfig fs/fat/.kunitconfig | \ ./tools/testing/kunit/kunit.py run --kunitconfig=/dev/stdin
But this can be fairly clunky to use in practice.
And having explicit support in kunit.py opens the door to having more config fragments of interest, e.g. options for PCI on UML [1], UML coverage [2], variants of tests [3]. There's another argument to be made that users can just use multiple --kconfig_add's, but this gets very clunky very fast (e.g. [2]).
Note: there's a big caveat here that some kconfig options might be incompatible. We try to give a clearish error message in the simple case where the same option appears multiple times with conflicting values, but more subtle ones (e.g. mutually exclusive options) will be potentially very confusing for the user. I don't know we can do better.
Note 2: if you want to combine a --kunitconfig with the default, you either have to do to specify the current build_dir > --kunitconfig=.kunit --kunitconfig=additional.config or > --kunitconfig=tools/testing/kunit/configs/default.config --kunitconifg=additional.config each of which have their downsides (former depends on --build_dir, doesn't work if you don't have a .kunitconfig yet), etc.
Example with conflicting values: > $ ./tools/testing/kunit/kunit.py config --kunitconfig=lib/kunit --kunitconfig=/dev/stdin <<EOF > CONFIG_KUNIT_TEST=n > CONFIG_KUNIT=m > EOF > ... > kunit_kernel.ConfigError: Multiple values specified for 2 options in kunitconfig: > CONFIG_KUNIT=y > vs from /dev/stdin > CONFIG_KUNIT=m > > CONFIG_KUNIT_TEST=y > vs from /dev/stdin > # CONFIG_KUNIT_TEST is not set
[1] https://lists.freedesktop.org/archives/dri-devel/2022-June/357616.html [2] https://lore.kernel.org/linux-kselftest/CAFd5g45f3X3xF2vz2BkTHRqOC4uW6GZxtUUMaP5mwwbK8uNVtA@mail.gmail.com/ [3] https://lore.kernel.org/linux-kselftest/CANpmjNOdSy6DuO6CYZ4UxhGxqhjzx4tn0sJMbRqo2xRFv9kX6Q@mail.gmail.com/
Signed-off-by: Daniel Latypov <[email protected]> Reviewed-by: Brendan Higgins <[email protected]> Signed-off-by: Shuah Khan <[email protected]>
show more ...
|
|
Revision tags: v5.19-rc5 |
|
| #
8a7c6f85 |
| 27-Jun-2022 |
Daniel Latypov <[email protected]> |
kunit: tool: refactor internal kconfig handling, allow overriding
Currently, you cannot ovewrwrite what's in your kunitconfig via --kconfig_add. Nor can you override something in a qemu_config via e
kunit: tool: refactor internal kconfig handling, allow overriding
Currently, you cannot ovewrwrite what's in your kunitconfig via --kconfig_add. Nor can you override something in a qemu_config via either means.
This patch makes it so we have this level of priority * --kconfig_add * kunitconfig file (the default or the one from --kunitconfig) * qemu_config
The rationale for this order is that the more "dynamic" sources of kconfig options should take priority.
--kconfig_add is obviously the most dynamic. And for kunitconfig, users probably tweak the file manually or specify --kunitconfig more often than they delve into qemu_config python files.
And internally, we convert the kconfigs from a python list into a set or dict fairly often. We should just use a dict internally. We exposed the set transform in the past since we didn't define __eq__, so also take the chance to shore up the kunit_kconfig.Kconfig interface.
Example =======
Let's consider the unrealistic example where someone would want to disable CONFIG_KUNIT. I.e. they run $ ./tools/testing/kunit/kunit.py config --kconfig_add=CONFIG_KUNIT=n
Before ------ We'd write the following > # CONFIG_KUNIT is not set > CONFIG_KUNIT_ALL_TESTS=y > CONFIG_KUNIT_TEST=y > CONFIG_KUNIT=y > CONFIG_KUNIT_EXAMPLE_TEST=y
And we'd error out with > ERROR:root:Not all Kconfig options selected in kunitconfig were in the generated .config. > This is probably due to unsatisfied dependencies. > Missing: # CONFIG_KUNIT is not set
After ----- We'd write the following > # CONFIG_KUNIT is not set > CONFIG_KUNIT_TEST=y > CONFIG_KUNIT_ALL_TESTS=y > CONFIG_KUNIT_EXAMPLE_TEST=y
And we'd error out with > ERROR:root:Not all Kconfig options selected in kunitconfig were in the generated .config. > This is probably due to unsatisfied dependencies. > Missing: CONFIG_KUNIT_EXAMPLE_TEST=y, CONFIG_KUNIT_TEST=y, CONFIG_KUNIT_ALL_TESTS=y
Signed-off-by: Daniel Latypov <[email protected]> Reviewed-by: David Gow <[email protected]> Reviewed-by: Brendan Higgins <[email protected]> Signed-off-by: Shuah Khan <[email protected]>
show more ...
|
|
Revision tags: v5.19-rc4, v5.19-rc3, v5.19-rc2, v5.19-rc1, v5.18 |
|
| #
a9333bd3 |
| 18-May-2022 |
Daniel Latypov <[email protected]> |
kunit: tool: introduce --qemu_args
Example usage: $ ./tools/testing/kunit/kunit.py run --arch=x86_64 \ --kconfig_add=CONFIG_SMP=y --qemu_args='-smp 8'
Looking in the test.log, one can see > smp:
kunit: tool: introduce --qemu_args
Example usage: $ ./tools/testing/kunit/kunit.py run --arch=x86_64 \ --kconfig_add=CONFIG_SMP=y --qemu_args='-smp 8'
Looking in the test.log, one can see > smp: Bringing up secondary CPUs ... > .... node #0, CPUs: #1 #2 #3 #4 #5 #6 #7 > smp: Brought up 1 node, 8 CPUs
This flag would allow people to make tweaks like this without having to create custom qemu_config files.
For consistency with --kernel_args, we allow users to repeat this argument, e.g. you can tack on a --qemu_args='-m 2048', or you could just append it to the first string ('-smp 8 -m 2048').
Signed-off-by: Daniel Latypov <[email protected]> Reviewed-by: David Gow <[email protected]> Reviewed-by: Brendan Higgins <[email protected]> Signed-off-by: Shuah Khan <[email protected]>
show more ...
|
| #
e756dbeb |
| 16-May-2022 |
Daniel Latypov <[email protected]> |
kunit: tool: refactoring printing logic into kunit_printer.py
Context: * kunit_kernel.py is importing kunit_parser.py just to use the print_with_timestamp() function * the parser is directly print
kunit: tool: refactoring printing logic into kunit_printer.py
Context: * kunit_kernel.py is importing kunit_parser.py just to use the print_with_timestamp() function * the parser is directly printing to stdout, which will become an issue if we ever try to run multiple kernels in parallel
This patch introduces a kunit_printer.py file and migrates callers of kunit_parser.print_with_timestamp() to call kunit_printer.stdout.print_with_timestamp() instead.
Future changes: If we want to support showing results for parallel runs, we could then create new Printer's that don't directly write to stdout and refactor the code to pass around these Printer objects.
Signed-off-by: Daniel Latypov <[email protected]> Reviewed-by: David Gow <[email protected]> Reviewed-by: Brendan Higgins <[email protected]> Signed-off-by: Shuah Khan <[email protected]>
show more ...
|
| #
8a04930f |
| 16-May-2022 |
Daniel Latypov <[email protected]> |
kunit: tool: redo how we construct and mock LinuxSourceTree
Our main function currently has an optional `linux` argument which is used to by our unit tests to inject a mock. We currently have the sa
kunit: tool: redo how we construct and mock LinuxSourceTree
Our main function currently has an optional `linux` argument which is used to by our unit tests to inject a mock. We currently have the same code copy-pasted several times to do if not linux: linux = MakeRealInstance(cli_args.foo, cli_args.bar, ...)
But in python, dependency injection isn't necessary or idiomatic when we can just use mock.patch() to mock things out.
This change 1. adds a helper to create a LinuxSourceTree from the cli_args 2. drops the `linux` parameter in favor of mocking the __init__ func.
Signed-off-by: Daniel Latypov <[email protected]> Reviewed-by: David Gow <[email protected]> Reviewed-by: Brendan Higgins <[email protected]> Signed-off-by: Shuah Khan <[email protected]>
show more ...
|
| #
76f0d6f5 |
| 16-May-2022 |
Daniel Latypov <[email protected]> |
kunit: tool: drop unused load_config argument
It's always set to true except in one test case. And in that test case it can safely be set to true anyways.
Signed-off-by: Daniel Latypov <dlatypov@go
kunit: tool: drop unused load_config argument
It's always set to true except in one test case. And in that test case it can safely be set to true anyways.
Signed-off-by: Daniel Latypov <[email protected]> Reviewed-by: David Gow <[email protected]> Reviewed-by: Brendan Higgins <[email protected]> Signed-off-by: Shuah Khan <[email protected]>
show more ...
|
|
Revision tags: v5.18-rc7 |
|
| #
0453f984 |
| 09-May-2022 |
Daniel Latypov <[email protected]> |
kunit: tool: misc cleanups
This primarily comes from running pylint over kunit tool code and ignoring some warnings we don't care about. If we ever got a fully clean setup, we could add this to run_
kunit: tool: misc cleanups
This primarily comes from running pylint over kunit tool code and ignoring some warnings we don't care about. If we ever got a fully clean setup, we could add this to run_checks.py, but we're not there yet.
Fix things like * Drop unused imports * check `is None`, not `== None` (see PEP 8) * remove redundant parens around returns * remove redundant `else` / convert `elif` to `if` where appropriate * rename make_arch_qemuconfig() param to base_kunitconfig (this is the name used in the subclass, and it's a better one) * kunit_tool_test: check the exit code for SystemExit (could be 0)
Signed-off-by: Daniel Latypov <[email protected]> Reviewed-by: David Gow <[email protected]> Reviewed-by: Brendan Higgins <[email protected]> Signed-off-by: Shuah Khan <[email protected]>
show more ...
|