MFC r345203,r345205,r345353,r345645,r345708,r345709,r345735,r345770,r346081,r346270,r346574,r346576:

r345203:

Initial googlemock/googletest integration into the build/FreeBSD test suite

This initi

MFC r345203,r345205,r345353,r345645,r345708,r345709,r345735,r345770,r346081,r346270,r346574,r346576:

r345203:

Initial googlemock/googletest integration into the build/FreeBSD test suite

This initial integration takes googlemock/googletest release 1.8.1, integrates
the library, tests, and sample unit tests into the build.

googlemock/googletest's inclusion is optionally available via `MK_GOOGLETEST`.
`MK_GOOGLETEST` is dependent on `MK_TESTS` and is enabled by default when
built with a C++11 capable toolchain.

Google tests can be specified via the `GTESTS` variable, which, in comparison
with the other test drivers, is more simplified/streamlined, as Googletest only
supports C++ tests; not raw C or shell tests (C tests can be written in C++
using the standard embedding methods).

No dependent libraries are assumed for the tests. One must specify `gmock`,
`gmock_main`, `gtest`, or `gtest_main`, via `LIBADD` for the program.

More information about googlemock and googletest can be found on the
Googletest [project page](https://github.com/google/googletest), and the
[GoogleMock](https://github.com/google/googletest/blob/v1.8.x/googlemock/docs/Documentation.md)
and
[GoogleTest](https://github.com/google/googletest/tree/v1.8.x/googletest/docs)
docs.

These tests are originally integrated into the build as plain driver tests, but
will be natively integrated into Kyua in a later version.

Known issues/Errata:
* [WhenDynamicCastToTest.AmbiguousCast fails on FreeBSD](https://github.com/google/googletest/issues/2172)

r345205:

Integrate cddl/usr.sbin/zfds/tests into the FreeBSD test suite

This change integrates the unit tests for zfsd into the test suite using the
integration method described in r345203.

This change removes the `LOCALBASE` includes added for the port version of
googlemock/googletest, as well as unnecessary `LIBADD`/`DPADD` and `CXXFLAGS`
defines, which are included in the `GTEST_CXXFLAGS` variable, as part of
r345203.

r345353 (by asomers):

googletest: backport GTEST_SKIP to googletest 1.8.1

This commit backports revisions 00938b2b228f3b70d3d9e51f29a1505bdad43f1e and
59f90a338bce2376b540ee239cf4e269bf6d68ad from googletest's master branch to
our included version of googletest, which is based on 1.8.1. It adds the
GTEST_SKIP feature, which is very useful for a project like FreeBSD where
some tests depend on particular system configurations.

Obtained from: github.com/google/googletest

r345645:

Spam CXXFLAGS with `-I${DESTDIR}/usr/include/private`, instead of GTEST_CXXFLAGS

This makes it easier for googletest users to leverage googletest, instead of
forcing them to plug GTEST_CXXFLAGS into CXXFLAGS manually (resulting in
unnecessary duplication).

I will be following this up with a more proper fix in src.libnames.mk, as
src.libnames.mk should be automatically adding this directory to
CFLAGS/CXXFLAGS when private libraries are referenced. Not doing so can result
in mismatches between base-provided private library's and ports-provided
library's headers.

While here, tweak the comment to clarify what the intent is behind spamming
CXXFLAGS.

r345708:

Standardize `-std=c++* as `CXXSTD`

CXXSTD was added as the C++ analogue to CSTD.

CXXSTD defaults to `-std=c++11` with supporting compilers; `-std=gnu++98`,
otherwise for older versions of g++.

This change standardizes the CXXSTD variable, originally added to
googletest.test.inc.mk as part of r345203.

As part of this effort, convert all `CXXFLAGS+= -std=*` calls to use `CXXSTD`.

Notes:

This value is not sanity checked in bsd.sys.mk, however, given the two
most used C++ compilers on FreeBSD (clang++ and g++) support both modes, it is
likely to work with both toolchains. This method will be refined in the future
to support more variants of C++, as not all versions of clang++ and g++ (for
instance) support C++14, C++17, etc.

Any manual appending of `-std=*` to `CXXFLAGS` should be replaced with CXXSTD.
Example:

Before this commit:
```
CXXFLAGS+= -std=c++14
```

After this commit:
```
CXXSTD= c++14
```

Relnotes: yes
Tested with: make tinderbox

r345709:

Allow users to override CSTD/CXXSTD on a per-prog basis

The current logic for CSTD/CXXSTD requires homogenity as far as the
supported C/C++ standards, which is a sensible default. However, when
dealing with differing versions of C++, some code may compile with C++11, but
not C++17 (for instance). So in order to avoid having people convert over their
code to the new standard, give the users the ability to specify the standard on
a per-program basis.

This will allow a user to override the supporting standard for a set of
programs, mixing C++11 with C++14 (for instance).

Apprved by: emaste (mentor)

r345735:

Allow programs to set `NO_SHARED` on a per-PROG basis

This is particularly useful when installing programs for tests that need to be
linked statically, e.g., mini-me from capsicum-test, which is linked statically
to avoid the dynamic library lookup in the upstream project.

r345770:

Import proof-of-concept for handling `GTEST_SKIP()` in `Environment::SetUp`

Per the upstream pull-request [1]:

```
gtest prior to this change would completely ignore `GTEST_SKIP()` if
called in `Environment::SetUp()`, instead of bailing out early, unlike
`Test::SetUp()`, which would cause the tests themselves to be skipped.
The only way (prior to this change) to skip the tests would be to
trigger a fatal error via `GTEST_FAIL()`.

Desirable behavior, in this case, when dealing with
`Environment::SetUp()` is to check for prerequisites on a system
(example, kernel supports a particular featureset, e.g., capsicum), and
skip the tests. The alternatives prior to this change would be
undesirable:

- Failing sends the wrong message to the test user, as the result of the
tests is indeterminate, not failed.
- Having to add per-test class abstractions that override `SetUp()` to
test for the capsicum feature set, then skip all of the tests in their
respective SetUp fixtures, would be a lot of human and computational
work; checking for the feature would need to be done for all of the
tests, instead of once for all of the tests.

For those reasons, making `Environment::SetUp()` handle `GTEST_SKIP()`,
by not executing the testcases, is the most desirable solution.

In order to properly diagnose what happened when running the tests if
they are skipped, print out the diagnostics in an ad hoc manner.

Update the documentation to note this change and integrate a new test,
gtest_skip_in_environment_setup_test, into the test suite.

This change addresses #2189.

Signed-off-by: Enji Cooper <[email protected]>
```

The goal with my merging in this change is to avoid requiring extensive
refactoring/retesting of test suites when ensuring prerequisites are met,
e.g., checking for a CAPABILITIES-enabled kernel before running capsicum-test
(see D19758 for more details).

The proof-of-concept is being imported before accepted by the upstream
project due to the fact that the upstream project is undergoing a potential
development freeze and the maintainers aren't responding to my PR.

1. https://github.com/google/googletest/pull/2203

r346081 (by trasz):

Make zfsd(8) build obey CFLAGS.

Obtained from: CheriBSD

r346270 (by trasz):

Drop -g from CFLAGS for zfsd(8). No idea why it was ever there.

r346574:

Rework CXXSTD setting via r345708

This change allows the user to once again override the C++ standard, restoring
high-level pre-r345708 behavior.

This also unbreaks building lib/ofed/libibnetdisc/Makefile with a non-C++11
capable compiler, e.g., g++ 4.2.1, as the library supported being built with
older C++ standards.

r346576:

Fix up CXXSTD support originally added in r345708

r345708 worked for the base system, but unfortunately, caused a lot of
disruption for third-party packages that relied on C++, since bsd.sys.mk is
used by applications outside the base system. The defaults picked didn't match
the compiler's defaults and broke some builds that didn't specify a standard,
as well as some that overrode the value by setting `-std=gnu++14` (for
example) manually.

This change takes a more relaxed approach to appending `-std=${CXXSTD}` to
CXXFLAGS, by only doing so when the value is specified, as opposed to
overriding the standard set by an end-user. This avoids the need for having
to bake NOP default into bsd.sys.mk for supported compiler-toolchain
versions.

In order to make this change possible, add CXXSTD to Makefile snippets which
relied on the default value (c++11) added in r345708.

show more ...

Revert r348136

I accidentally committed some unrelated local changes to
`.../tests/sys/opencrypto` along with this MFC set.

MFC r345203,r345205,r345353,r345645,r345708,r345709,r345735,r345770,r346574,r346576:

r345203:

Initial googlemock/googletest integration into the build/FreeBSD test suite

This initial integration t

MFC r345203,r345205,r345353,r345645,r345708,r345709,r345735,r345770,r346574,r346576:

r345203:

Initial googlemock/googletest integration into the build/FreeBSD test suite

This initial integration takes googlemock/googletest release 1.8.1, integrates
the library, tests, and sample unit tests into the build.

googlemock/googletest's inclusion is optionally available via `MK_GOOGLETEST`.
`MK_GOOGLETEST` is dependent on `MK_TESTS` and is enabled by default when
built with a C++11 capable toolchain.

Google tests can be specified via the `GTESTS` variable, which, in comparison
with the other test drivers, is more simplified/streamlined, as Googletest only
supports C++ tests; not raw C or shell tests (C tests can be written in C++
using the standard embedding methods).

No dependent libraries are assumed for the tests. One must specify `gmock`,
`gmock_main`, `gtest`, or `gtest_main`, via `LIBADD` for the program.

More information about googlemock and googletest can be found on the
Googletest [project page](https://github.com/google/googletest), and the
[GoogleMock](https://github.com/google/googletest/blob/v1.8.x/googlemock/docs/Documentation.md)
and
[GoogleTest](https://github.com/google/googletest/tree/v1.8.x/googletest/docs)
docs.

These tests are originally integrated into the build as plain driver tests, but
will be natively integrated into Kyua in a later version.

Known issues/Errata:
* [WhenDynamicCastToTest.AmbiguousCast fails on FreeBSD](https://github.com/google/googletest/issues/2172)

r345205:

Integrate cddl/usr.sbin/zfds/tests into the FreeBSD test suite

This change integrates the unit tests for zfsd into the test suite using the
integration method described in r345203.

This change removes the `LOCALBASE` includes added for the port version of
googlemock/googletest, as well as unnecessary `LIBADD`/`DPADD` and `CXXFLAGS`
defines, which are included in the `GTEST_CXXFLAGS` variable, as part of
r345203.

r345353 (by asomers):

googletest: backport GTEST_SKIP to googletest 1.8.1

This commit backports revisions 00938b2b228f3b70d3d9e51f29a1505bdad43f1e and
59f90a338bce2376b540ee239cf4e269bf6d68ad from googletest's master branch to
our included version of googletest, which is based on 1.8.1. It adds the
GTEST_SKIP feature, which is very useful for a project like FreeBSD where
some tests depend on particular system configurations.

Obtained from: github.com/google/googletest

r345645:

Spam CXXFLAGS with `-I${DESTDIR}/usr/include/private`, instead of GTEST_CXXFLAGS

This makes it easier for googletest users to leverage googletest, instead of
forcing them to plug GTEST_CXXFLAGS into CXXFLAGS manually (resulting in
unnecessary duplication).

I will be following this up with a more proper fix in src.libnames.mk, as
src.libnames.mk should be automatically adding this directory to
CFLAGS/CXXFLAGS when private libraries are referenced. Not doing so can result
in mismatches between base-provided private library's and ports-provided
library's headers.

While here, tweak the comment to clarify what the intent is behind spamming
CXXFLAGS.

r345708:

Standardize `-std=c++* as `CXXSTD`

CXXSTD was added as the C++ analogue to CSTD.

CXXSTD defaults to `-std=c++11` with supporting compilers; `-std=gnu++98`,
otherwise for older versions of g++.

This change standardizes the CXXSTD variable, originally added to
googletest.test.inc.mk as part of r345203.

As part of this effort, convert all `CXXFLAGS+= -std=*` calls to use `CXXSTD`.

Notes:

This value is not sanity checked in bsd.sys.mk, however, given the two
most used C++ compilers on FreeBSD (clang++ and g++) support both modes, it is
likely to work with both toolchains. This method will be refined in the future
to support more variants of C++, as not all versions of clang++ and g++ (for
instance) support C++14, C++17, etc.

Any manual appending of `-std=*` to `CXXFLAGS` should be replaced with CXXSTD.
Example:

Before this commit:
```
CXXFLAGS+= -std=c++14
```

After this commit:
```
CXXSTD= c++14
```

Relnotes: yes
Tested with: make tinderbox

r345709:

Allow users to override CSTD/CXXSTD on a per-prog basis

The current logic for CSTD/CXXSTD requires homogenity as far as the
supported C/C++ standards, which is a sensible default. However, when
dealing with differing versions of C++, some code may compile with C++11, but
not C++17 (for instance). So in order to avoid having people convert over their
code to the new standard, give the users the ability to specify the standard on
a per-program basis.

This will allow a user to override the supporting standard for a set of
programs, mixing C++11 with C++14 (for instance).

Approved by: emaste (mentor)

r345735:

Allow programs to set `NO_SHARED` on a per-PROG basis

This is particularly useful when installing programs for tests that need to be
linked statically, e.g., mini-me from capsicum-test, which is linked statically
to avoid the dynamic library lookup in the upstream project.

r345770:

Import proof-of-concept for handling `GTEST_SKIP()` in `Environment::SetUp`

Per the upstream pull-request [1]:

```
gtest prior to this change would completely ignore `GTEST_SKIP()` if
called in `Environment::SetUp()`, instead of bailing out early, unlike
`Test::SetUp()`, which would cause the tests themselves to be skipped.
The only way (prior to this change) to skip the tests would be to
trigger a fatal error via `GTEST_FAIL()`.

Desirable behavior, in this case, when dealing with
`Environment::SetUp()` is to check for prerequisites on a system
(example, kernel supports a particular featureset, e.g., capsicum), and
skip the tests. The alternatives prior to this change would be
undesirable:

- Failing sends the wrong message to the test user, as the result of the
tests is indeterminate, not failed.
- Having to add per-test class abstractions that override `SetUp()` to
test for the capsicum feature set, then skip all of the tests in their
respective SetUp fixtures, would be a lot of human and computational
work; checking for the feature would need to be done for all of the
tests, instead of once for all of the tests.

For those reasons, making `Environment::SetUp()` handle `GTEST_SKIP()`,
by not executing the testcases, is the most desirable solution.

In order to properly diagnose what happened when running the tests if
they are skipped, print out the diagnostics in an ad hoc manner.

Update the documentation to note this change and integrate a new test,
gtest_skip_in_environment_setup_test, into the test suite.

This change addresses #2189.

Signed-off-by: Enji Cooper <[email protected]>
```

The goal with my merging in this change is to avoid requiring extensive
refactoring/retesting of test suites when ensuring prerequisites are met,
e.g., checking for a CAPABILITIES-enabled kernel before running capsicum-test
(see D19758 for more details).

The proof-of-concept is being imported before accepted by the upstream
project due to the fact that the upstream project is undergoing a potential
development freeze and the maintainers aren't responding to my PR.

1. https://github.com/google/googletest/pull/2203

r346574:

Rework CXXSTD setting via r345708

This change allows the user to once again override the C++ standard, restoring
high-level pre-r345708 behavior.

This also unbreaks building lib/ofed/libibnetdisc/Makefile with a non-C++11
capable compiler, e.g., g++ 4.2.1, as the library supported being built with
older C++ standards.

r346576:

Fix up CXXSTD support originally added in r345708

r345708 worked for the base system, but unfortunately, caused a lot of
disruption for third-party packages that relied on C++, since bsd.sys.mk is
used by applications outside the base system. The defaults picked didn't match
the compiler's defaults and broke some builds that didn't specify a standard,
as well as some that overrode the value by setting `-std=gnu++14` (for
example) manually.

This change takes a more relaxed approach to appending `-std=${CXXSTD}` to
CXXFLAGS, by only doing so when the value is specified, as opposed to
overriding the standard set by an end-user. This avoids the need for having
to bake NOP default into bsd.sys.mk for supported compiler-toolchain
versions.

In order to make this change possible, add CXXSTD to Makefile snippets which
relied on the default value (c++11) added in r345708.

show more ...

Add kyua TAP test integration examples

The examples are patterned loosely after the ATF examples, similar
to the plain test examples.

MFC after: 1 month

share/examples/tests/Makefile: clean up example snippets/documentation

- TESTSDIR doesn't need to be specified after r289158.
- Including bsd.own.mk isn't required since no MK_<foo> knobs are being

share/examples/tests/Makefile: clean up example snippets/documentation

- TESTSDIR doesn't need to be specified after r289158.
- Including bsd.own.mk isn't required since no MK_<foo> knobs are being
manipulated.
- TESTS_SUBDIRS should be written out in an append format, one entry
per line, to provide a better, more conflict resistant example.

MFC after: 1 month

show more ...

Merge ^/user/ngie/release-pkg-fix-tests to unbreak how test files are installed
after r298107

Summary of changes:

- Replace all instances of FILES/TESTS with ${PACKAGE}FILES. This ensures that
na

Merge ^/user/ngie/release-pkg-fix-tests to unbreak how test files are installed
after r298107

Summary of changes:

- Replace all instances of FILES/TESTS with ${PACKAGE}FILES. This ensures that
namespacing is kept with FILES appropriately, and that this shouldn't need
to be repeated if the namespace changes -- only the definition of PACKAGE
needs to be changed
- Allow PACKAGE to be overridden by callers instead of forcing it to always be
`tests`. In the event we get to the point where things can be split up
enough in the base system, it would make more sense to group the tests
with the blocks they're a part of, e.g. byacc with byacc-tests, etc
- Remove PACKAGE definitions where possible, i.e. where FILES wasn't used
previously.
- Remove unnecessary TESTSPACKAGE definitions; this has been elided into
bsd.tests.mk
- Remove unnecessary BINDIRs used previously with ${PACKAGE}FILES;
${PACKAGE}FILESDIR is now automatically defined in bsd.test.mk.
- Fix installation of files under data/ subdirectories in lib/libc/tests/hash
and lib/libc/tests/net/getaddrinfo
- Remove unnecessary .include <bsd.own.mk>s (some opportunistic cleanup)

Document the proposed changes in share/examples/tests/tests/... via examples
so it's clear that ${PACKAGES}FILES is the suggested way forward in terms of
replacing FILES. share/mk/bsd.README didn't seem like the appropriate method
of communicating that info.

MFC after: never probably
X-MFC with: r298107
PR: 209114
Relnotes: yes
Tested with: buildworld, installworld, checkworld; buildworld, packageworld
Sponsored by: EMC / Isilon Storage Division

show more ...

Fix including Kyuafile in packaged base system.

Fix a related typo while here.

Note, this change results in the Kyuafile inclusion in the runtime
package, which needs to be fixed, however addresses

Fix including Kyuafile in packaged base system.

Fix a related typo while here.

Note, this change results in the Kyuafile inclusion in the runtime
package, which needs to be fixed, however addresses the PR as far
as I can tell in my tests.

PR: 209114
Submitted by: ngie
Sponsored by: The FreeBSD Foundation

show more ...

First pass to fix the 'tests' packages.

Sponsored by: The FreeBSD Foundation

Add some sample test programs.

This change adds some sample test cases to share/examples/tests/
demonstrating the basic usage of the atf and plain interfaces.

These test programs are fully-function

Add some sample test programs.

This change adds some sample test cases to share/examples/tests/
demonstrating the basic usage of the atf and plain interfaces.

These test programs are fully-functional and are installed as part
of the test suite, which guarantees that the sample code remains
correct. However, they currently mostly serve as a placeholder for
additional examples and may be incomplete (depending on how you
look at them). I will see what else can be useful while working on
documentation.

As a bonus, the addition of these tests exercise the *.test.mk files,
one of which (plain.test.mk) was not yet in use, and also demonstrates
that it's possible to mix different kinds of test programs into the
same test suite.

Approved by: rpaulo (mentor)

show more ...