Lines Matching refs:tests
9 :program:`lit` [*options*] [*tests*]
19 :program:`lit` should be run with one or more *tests* to run specified on the
21 search for tests (see :ref:`test-discovery`).
24 tests have been run :program:`lit` will print summary information on the number
25 of tests which passed or failed (see :ref:`test-status-results`). The
26 :program:`lit` program will execute with a non-zero exit code if any tests
33 :program:`lit` also includes a number of options for controlling how tests are
61 Run ``N`` tests in parallel. By default, this is automatically chosen to
86 Show less output, for example don't show information on tests that pass.
107 Show more information about all tests, for example the entire test
116 Show the names of unsupported tests.
120 Show the names of tests that were expected to fail.
129 Specify an additional ``PATH`` to use when searching for executables in tests.
133 Run individual tests under valgrind (using the memcheck tool). The
139 failure in) certain tests.
151 certain tests.
153 .. option:: --time-tests
155 Track the wall time individual tests take to execute and includes the results
156 in the summary output. This is useful for determining which tests in a test
161 Exit with status zero even if some tests fail.
173 By default, `lit` will run failing tests first, then run tests in descending
183 Run the tests in a random order, not failing/slowest first. Deprecated,
192 .. option:: --max-tests=N
194 Run at most ``N`` tests and then terminate.
198 Spend at most ``N`` seconds (approximately) running tests and then terminate.
204 Divide the set of selected tests into ``M`` equal-sized subsets or
214 Define the order in which tests are run. The supported values are:
216 - lexical - tests will be run in lexical order according to the test file
219 - random - tests will be run in random order.
221 - smart - tests that failed previously will be run first, then the remaining
222 tests, all in descending execution time order. This is the default as it
240 Run only those tests whose name matches the regular expression specified in
247 Filter out those tests whose name matches the regular expression specified in
254 Treat those tests whose name is in the semicolon separated list ``LIST`` as
267 In this case, all of the following tests are treated as ``XFAIL``:
286 Do not treat the specified tests as ``XFAIL``. The environment variable
306 .. option:: --show-tests
308 List all of the discovered tests and exit.
323 The inputs passed to :program:`lit` can be either individual tests, or entire
324 directories or hierarchies of tests to run. When :program:`lit` starts up, the
325 first thing it does is convert the inputs into a complete list of tests to run
333 how to find and run the tests inside the test suite.
336 list of inputs adding tests for individual files and recursively searching for
337 tests in directories.
339 This behavior makes it easy to specify a subset of tests to run, while still
340 allowing the test suite configuration to control exactly how tests are
341 interpreted. In addition, :program:`lit` always identifies tests by the test
360 tests containing an ``ALLOW_RETRIES:`` annotation.
370 The test succeeded, but it was expected to fail. This is used for tests which
386 which can report unsupported tests.
393 Depending on the test format tests may produce additional information about
406 arbitrary tests, and to expose a single convenient interface to these
407 tests. :program:`lit` itself doesn't know how to run tests, rather this logic is
413 As described in :ref:`test-discovery`, tests are always located inside a *test
414 suite*. Test suites serve to define the format of the tests they contain, the
415 logic for finding those tests, and any additional information to run the tests.
444 discover and run tests in the test suite. Generally this will be a builtin test
448 builds this is the directory that will be scanned for tests.
451 the object directory. This is where tests will be run and temporary output files
455 tests in the suite.
457 **standalone_tests** When true, mark a directory with tests expected to be run
462 **suffixes** For **lit** test formats which scan directories for tests, this
468 **unsupported** Mark an unsupported directory, all tests within it will be
488 directory (following *test_source_root*) looking for tests. When :program:`lit`
497 "virtual tests" which have a path that contains both the path to the actual
511 be used to define subdirectories of optional tests, or to change other
612 The following is an example of a test run output which consists of four tests A,