Lines Matching refs:test
8 tests to the in-tree DPDK test suites.
10 The DPDK test suite model is loosely based on the xUnit model,
11 where tests are grouped into test suites, and suites are run by runners.
21 The primary tools are the `dpdk-test` application,
22 and the ``meson test`` infrastructure.
26 There exists a bit of confusion with the test suite and test case separation
27 with respect to `dpdk-test` and ``meson test``.
28 Both have a concept of test suite and test case.
30 A test suite is a group of test cases,
31 and a test case represents the steps needed to test a particular set of code.
33 to denote a Meson test suite / case.
36 Running a test
39 DPDK tests are run via the main test runner, the `dpdk-test` app.
40 The `dpdk-test` app is a command-line interface that facilitates
41 running various tests or test suites.
45 that allows launching specific test suites.
46 This is the default operating mode of `dpdk-test` and can be done by::
48 $ ./build/app/test/dpdk-test --dpdk-options-here
61 At the prompt, simply type the name of the test suite you wish to run
67 assigning a specific test suite name to the environment variable ``DPDK_TEST``
68 before invoking the `dpdk-test` command, such as::
70 $ DPDK_TEST=version_autotest ./build/app/test/dpdk-test --dpdk-options-here
86 The above shows running a specific test case.
91 to providing the test suite name in an environment variable.
92 The unit test app can accept test suite names via command line arguments::
94 $ ./build/app/test/dpdk-test --dpdk-options-here version_autotest version_autotest
111 The primary benefit here is specifying multiple test names,
114 Additionally, it is possible to specify additional test parameters
117 This isn't currently used in the Meson test suites.
120 Running test cases via Meson
124 without needing to remember or look up each test suite name,
125 the build system includes a standard way of executing the Meson test suites.
126 After building via ``ninja``, the ``meson test`` command
127 with no arguments will execute the Meson test suites.
129 There are five pre-configured Meson test suites.
130 The first is the **fast** test suite, which is the largest group of test cases.
133 These test suites can take longer to run and do performance evaluations.
134 The third is the **driver** test suite,
140 The Meson test suites can be selected by adding the ``--suite`` option
141 to the ``meson test`` command.
142 Ex: ``meson test --suite fast-tests``::
144 $ meson test -C build --suite fast-tests
146 [2543/2543] Linking target app/test/dpdk-test.
155 The ``meson test`` command can also execute individual Meson test cases
156 via the command line by adding the test names as an argument::
158 $ meson test -C build version_autotest
160 [2543/2543] Linking target app/test/dpdk-test.
164 Note that these test cases must be known to Meson
165 for the ``meson test`` command to run them.
166 Simply adding a new test to the `dpdk-test` application isn't enough.
167 See the section `Adding a suite or test case to Meson`_ for more details.
170 Adding tests to dpdk-test application
178 The DPDK test application supports two layers of tests:
179 #. *test cases* which are individual tests
180 #. *test suites* which are groups of test cases
182 To add a new test suite to the DPDK test application,
183 create a new test file for that suite
184 (ex: see *app/test/test_version.c* for the ``version_autotest`` test suite).
185 There are two important functions for interacting with the test harness:
188 Registers a test command with the name `command_name`
193 Returns a runner for a full test suite object,
194 which contains a test suite name, setup, tear down,
196 and vector of unit test cases.
198 Each test suite has a setup and tear down function
199 that runs at the beginning and end of the test suite execution.
200 Each unit test has a similar function for test case setup and tear down.
202 Each test suite may use a nested list of sub-testsuites,
204 This support allows for better granularity when designing test suites.
205 The sub-testsuites list can also be used in parallel with the vector of test cases,
206 in this case the test cases will be run,
208 To see an example of a test suite using sub-testsuites,
209 see *app/test/test_cryptodev.c*.
212 of the appropriate unit test suite structure.
213 An example of both a test suite and a case:
225 #include "test.h"
242 TEST_CASES_END(), /**< NULL terminate unit test array */
254 that can be used to create a complete test suite with test case.
257 of the unit test suite structure, for example:
287 TEST_CASES_END(), /**< NULL terminate unit test array */
303 &end_testsuite /**< NULL test suite to indicate end of list */
321 Designing a test
332 by returning a test result failure, using the ``TEST_FAILED`` error code.
334 has occurred in a test routine.
336 where the test has catastrophically failed.
352 See `lib/librte_eal/include/rte_test.h` for a list of defined test assertions.
354 Sometimes it is important to indicate that a test needs to be skipped,
355 either because the environment isn't able to support running the test,
357 The test suite supports returning a result of ``TEST_SKIPPED``
358 during test case setup, or during test case execution
359 to indicate that the preconditions of the test aren't available.
362 $ meson test -C build --suite fast-tests
364 [2543/2543] Linking target app/test/dpdk-test.
383 $ meson test -C build --suite fast-tests
389 This can be used to assist in test development.
392 Adding a suite or test case to Meson
395 Adding to one of the Meson test suites involves
396 editing the appropriate Meson build file `app/test/meson.build`
397 and adding the command to the correct test suite class.
398 Once added, the new test will be run
401 A user or developer can confirm that a test is known to Meson
404 $ meson test -C build --list
409 Some of these test suites are run during continuous integration tests,
412 In general, when a test is added to the `dpdk-test` application,
413 it probably should be added to a Meson test suite,
415 Preference is to add tests to the Meson test suites.
422 via EAL arguments, as this is not automatically done by the test::
424 $ ./build/app/test/dpdk-test --vdev crypto_aesni_mb
425 $ meson test -C build --suite driver-tests \
426 --test-args="--vdev crypto_aesni_mb"
431 This vdev will be created automatically by the test app,