riscv: actually enable sanitizers

1. Backport https://github.com/llvm/llvm-project/commit/b475ce39e8b1de3a70ea242473f136a567be46e3.
2. Enable libclang_rt for riscv.

Previous commit missed it.

Revi

riscv: actually enable sanitizers

1. Backport https://github.com/llvm/llvm-project/commit/b475ce39e8b1de3a70ea242473f136a567be46e3.
2. Enable libclang_rt for riscv.

Previous commit missed it.

Reviewed by: dim
Differential Revision: https://reviews.freebsd.org/D34543
Approved by: re (gjb)

(cherry picked from commit 3781e77995e9b81afcbf7b3dc84b33b864c48794)
(cherry picked from commit e58a146c5d24172b53f18b953b3d29ee25bdb90c)

show more ...

Add libfido2 to the build

From https://github.com/Yubico/libfido2:

libfido2 provides library functionality and command-line tools to
communicate with a FIDO device over USB, and to verify a

Add libfido2 to the build

From https://github.com/Yubico/libfido2:

libfido2 provides library functionality and command-line tools to
communicate with a FIDO device over USB, and to verify attestation
and assertion signatures.

libfido2 supports the FIDO U2F (CTAP 1) and FIDO 2.0 (CTAP 2)
protocols.

libfido2 will be used by ssh to support FIDO/U2F keys. It is currently
intended only for use by ssh, and so is installed as a PRIVATELIB and is
placed in the ssh pkgbase package.

This is currently disabled for the 32-bit library build as libfido2 is
not compatible with the COMPAT_32BIT hack in usb_ioctl.h.

Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D32448

(cherry picked from commit 7b1e19ad78c6a3f84f81cb1a16a39500f0337062)
(cherry picked from commit 93942379cced89ad4ac653f262ac8277a8550853)

show more ...

Add libcbor to the build

From https://github.com/PJK/libcbor:

libcbor is a C library for parsing and generating CBOR, the general-
purpose schema-less binary data format.

libcbor will be u

Add libcbor to the build

From https://github.com/PJK/libcbor:

libcbor is a C library for parsing and generating CBOR, the general-
purpose schema-less binary data format.

libcbor will be used by ssh to support FIDO/U2F keys. It is currently
intended only for use by ssh, and so is installed as a PRIVATELIB and is
placed in the ssh pkgbase package.

cbor_export.h and configuration.h were generated by the upstream CMake
build. We could create them with bmake rules instead (as NetBSD has
done) but this is a fine start.

This is currently disabled for the 32-bit library build as libfido2 is
not compatible with the COMPAT_32BIT hack in usb_ioctl.h, and there is
no need for libcbor without libfido2.

Reviewed by: kevans
MFC after: 2 weeks
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D32347

(cherry picked from commit 2e85df652caef859c532b7e1e8a178c75f1a4a92)

show more ...

Add WITH_LLVM_BINUTILS to install LLVM binutils instead of Elftoolchain

When WITH_LLVM_BINUTILS is set, we will install the LLVM binutils as
ar/ranlib/nm/objcopy/etc. instead of the elftoolchain one

Add WITH_LLVM_BINUTILS to install LLVM binutils instead of Elftoolchain

When WITH_LLVM_BINUTILS is set, we will install the LLVM binutils as
ar/ranlib/nm/objcopy/etc. instead of the elftoolchain ones.
Having the LLVM binutils instead of the elftoolchain ones allows us to use
features such as LTO that depend on binutils that understand LLVM IR.
Another benefit will be an improved user-experience when compiling with
AddressSanitizer, since ASAN does not symbolize backtraces correctly if
addr2line is elftoolchain addr2line instead of llvm-symbolizer.
See https://lists.freebsd.org/archives/freebsd-toolchain/2021-July/000062.html
for more details.

This is currently off by default but will be turned on by default at some
point in the near future.

Reviewed By: emaste

Differential Revision: https://reviews.freebsd.org/D31060

(cherry picked from commit 021385aba56279febcfdcc64d23673a0106ae45d)

show more ...

Don't build sanitizer runtimes under WITHOUT_CXX

In the past we built the sanitizer runtimes when building Clang
(and using Clang as the compiler) but 7676b388adbc changed this to
be conditional onl

Don't build sanitizer runtimes under WITHOUT_CXX

In the past we built the sanitizer runtimes when building Clang
(and using Clang as the compiler) but 7676b388adbc changed this to
be conditional only on using Clang, to make the runtimes available
for external Clang.

They fail to build when WITHOUT_CXX is set though, so add MK_CXX
as part of the condition.

Reported by: Michael Dexter, Build Option Survey
Reviewed by: imp, jrtc27
Fixes: 7676b388adbc ("Always build the sanitizer runtimes...")
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D32731

(cherry picked from commit ad09e2c8cfbc2cf6f2b8826c121d6de8b3bfe96d)

show more ...

Simplify and speed up the kyua build

Instead of having multiple kyua libraries, just include the files as part
of usr.bin/kyua. Previously, we would build each kyua source up to four
times: once as

Simplify and speed up the kyua build

Instead of having multiple kyua libraries, just include the files as part
of usr.bin/kyua. Previously, we would build each kyua source up to four
times: once as a .o file and once as a .pieo. Additionally, the kyua
libraries might be built again for compat32. As all the kyua libraries
amount to 102 C++ sources the build time is significant (especially when
using an assertions enabled compiler). This change ensures that we build
306 fewer .cpp source files as part of buildworld.

Reviewed By: brooks
MFC after: 1 week
Differential Revision: https://reviews.freebsd.org/D30967

Remove lib/kyua from the build

I forgot to include this line in 2eb9ad427475190e87d026de4ca7fc296947f2a0.

Reported by: Jenkins CI
MFC after: 1 week
Fixes: 2eb9ad427475190ei ("Simplify and speed up the kyua build")

Revert "Remove lib/kyua from the build"

Accidentally removed it from the wrong file...

This reverts commit 8ec4ba8a76aaf256942ac5996d45cc29022641ae.

Remove lib/kyua from the build

I forgot to include this line in 2eb9ad427475190e87d026de4ca7fc296947f2a0.

Reported by: Jenkins CI
MFC after: 1 week
Fixes: 2eb9ad427475190ei ("Simplify and speed up the kyua build")

(cherry picked from commit 2eb9ad427475190e87d026de4ca7fc296947f2a0)
(cherry picked from commit 8ec4ba8a76aaf256942ac5996d45cc29022641ae)
(cherry picked from commit 89da04fcaa33f100a60d06770f7424e5b7f1312f)
(cherry picked from commit c951566915886330612bee880d6ece0d65bf9f5d)

show more ...

pfctl: Move ioctl abstraction functions into libpfctl

Introduce a library to wrap the pf ioctl interface.

MFC after: 4 weeks
Sponsored by: Rubicon Communications, LLC ("Netgate")
Differential Revis

pfctl: Move ioctl abstraction functions into libpfctl

Introduce a library to wrap the pf ioctl interface.

MFC after: 4 weeks
Sponsored by: Rubicon Communications, LLC ("Netgate")
Differential Revision: https://reviews.freebsd.org/D29562

(cherry picked from commit 0d71f9f36e6c1849fdaf6c1edc1178be07357034)

show more ...

Always build the sanitizer runtimes when compiling with clang

This allows instrumenting e.g. test binaries even when compiling with an
external clang (e.g. CROSS_TOOLCHAIN=llvm11). I have some upcom

Always build the sanitizer runtimes when compiling with clang

This allows instrumenting e.g. test binaries even when compiling with an
external clang (e.g. CROSS_TOOLCHAIN=llvm11). I have some upcoming patches
that allow building the entire base system with ASan/UBSan/etc.
instrumentation and this is required in preparation for this.

Reviewed By: dim, emaste
Differential Revision: https://reviews.freebsd.org/D28532

(cherry picked from commit 7676b388adbc81a2ad46b43852cd9bc7ac7fad7e)

show more ...

build: remove LIBPTHREAD/LIBTHR build options

WITHOUT_LIBTHR has been broken for a little over five years now, since the
xz 5.2.0 update introduced a hard liblzma dependency on libthr, and building

build: remove LIBPTHREAD/LIBTHR build options

WITHOUT_LIBTHR has been broken for a little over five years now, since the
xz 5.2.0 update introduced a hard liblzma dependency on libthr, and building
a useful system without threading support is becoming increasingly more
difficult.

Additionally, in the five plus years that it's been broken more reverse
dependencies have cropped up in libzstd, libsqlite3, and libcrypto (among
others) that make it more and more difficult to reconcile the effort needed
to fix these options.

Remove the broken options.

PR: 252760

(cherry picked from commit 123ae3045dd21badb93ce52445e18e364b3ac807)
(cherry picked from commit 251a6ddfbdcd72e0de922e8320d2f0cc6806a423)

show more ...

Import zstd 1.4.8

flua: Add a libjail module

libjail is pretty small, so it makes for a good proof of concept demonstrating
how a system library can be wrapped to create a loadable Lua module for flua.

* Introduce 3

flua: Add a libjail module

libjail is pretty small, so it makes for a good proof of concept demonstrating
how a system library can be wrapped to create a loadable Lua module for flua.

* Introduce 3lua section for man pages
* Add libjail module

Reviewed by: kevans, manpages
Relnotes: yes
Differential Revision: https://reviews.freebsd.org/D26080

show more ...

Add virtio-9p (aka VirtFS) filesystem sharing to bhyve.

VirtFS allows sharing an arbitrary directory tree between bhyve virtual
machine and the host. Current implementation has a fairly complete sup

Add virtio-9p (aka VirtFS) filesystem sharing to bhyve.

VirtFS allows sharing an arbitrary directory tree between bhyve virtual
machine and the host. Current implementation has a fairly complete support
for 9P2000.L protocol, except for the extended attribute support. It has
been verified to work with the qemu-kvm hypervisor.

Reviewed by: rgrimes, emaste, jhb, trasz
Approved by: trasz (mentor)
MFC after: 1 month
Relnotes: yes
Sponsored by: Conclusive Engineering (development), vStack.com (funding)
Differential Revision: https://reviews.freebsd.org/D10335

show more ...

lib: add libnetmap

This changeset introduces the new libnetmap library for writing
netmap applications.
Before libnetmap, applications could either use the kernel API
directly (e.g. NIOCREGIF/NIOCCT

lib: add libnetmap

This changeset introduces the new libnetmap library for writing
netmap applications.
Before libnetmap, applications could either use the kernel API
directly (e.g. NIOCREGIF/NIOCCTRL) or the simple header-only-library
netmap_user.h (e.g. nm_open(), nm_close(), nm_mmap() etc.)

The new library offers more functionalities than netmap_user.h:
- Support for complex netmap options, such as external memory
allocators or per-buffer offsets. This opens the way to future
extensions.
- More flexibility in the netmap port bind options, such as
non-numeric names for pipes, or the ability to specify the netmap
allocator that must be used for a given port.
- Automatic tracking of the netmap memory regions in use across the
open ports.

At the moment there is no man page, but the libnetmap.h header file
has in-depth documentation.

Reviewed by: hrs
MFC after: 2 weeks
Differential Revision: https://reviews.freebsd.org/D26171

show more ...

Move libsqlite3 to the top of the SUBDIR list

In parallel builds, this should allow sqlite to start building earlier and
increase parallelism when building lib/. Looking at htop output during
buildw

Move libsqlite3 to the top of the SUBDIR list

In parallel builds, this should allow sqlite to start building earlier and
increase parallelism when building lib/. Looking at htop output during
buildworld/tinderbox, there are long phases where only one CPU is active
optimizing the massive sqlite3.c file since the build of libsqlite3 is
started quite late.

Reviewed By: emaste
Differential Revision: https://reviews.freebsd.org/D26169

show more ...

Ensure libregex is built in time for googletest

In lib/Makefile, we document the dependency with SUBDIR_DEPEND

For buildworld orchestration, just prebuild libregex if GOOGLETEST is
enabled. googlet

Ensure libregex is built in time for googletest

In lib/Makefile, we document the dependency with SUBDIR_DEPEND

For buildworld orchestration, just prebuild libregex if GOOGLETEST is
enabled. googletest will get built in a later pass.

show more ...

rename in-tree libevent v1 to libevent1

r316063 installed pf's embedded libevent as a private lib, with headers
in /usr/include/private/event. Unfortunately we also have a copy of
libevent v2 inclu

rename in-tree libevent v1 to libevent1

r316063 installed pf's embedded libevent as a private lib, with headers
in /usr/include/private/event. Unfortunately we also have a copy of
libevent v2 included in ntp, which needed to be updated for compatibility
with OpenSSL 1.1.

As unadorned 'libevent' generally refers to libevent v2, be explicit that
this one is libevent v1.

Reviewed by: vangyzen (earlier)
Differential Revision: https://reviews.freebsd.org/D17275

show more ...

Import Zstd 1.4.5

Add an internal liblua and use it in flua.

The new liblua will be used in a forthcoming import of kyua.

Reviewed by: kevans
Obtained from: CheriBSD
Sponsored by: DARPA
Differential Revision: https:

Add an internal liblua and use it in flua.

The new liblua will be used in a forthcoming import of kyua.

Reviewed by: kevans
Obtained from: CheriBSD
Sponsored by: DARPA
Differential Revision: https://reviews.freebsd.org/D24090

show more ...

retire the LLVM_LIBUNWIND option

LLVM's libunwind is used on all FreeBSD-supported CPU architectures and
is a required component.

Reviewed by: brooks (earlier)
Sponsored by: The FreeBSD Foundation

retire the LLVM_LIBUNWIND option

LLVM's libunwind is used on all FreeBSD-supported CPU architectures and
is a required component.

Reviewed by: brooks (earlier)
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D23123

show more ...

Remove excluding libproc and librtld_db on sparc64.

Provide libssp based on libc

For libssp.so, rebuild stack_protector.c with FORTIFY_SOURCE stubs that just
abort built into it.

For libssp_nonshared.a, steal stack_protector_compat.c from
^/lib/libc

Provide libssp based on libc

For libssp.so, rebuild stack_protector.c with FORTIFY_SOURCE stubs that just
abort built into it.

For libssp_nonshared.a, steal stack_protector_compat.c from
^/lib/libc/secure and massage it to maintain that __stack_chk_fail_local
is a hidden symbol.

libssp is now built unconditionally regardless of {WITH,WITHOUT}_SSP in the
build environment, and the gcclibs version has been disconnected from the
build in favor of this one.

PR: 242950 (exp-run)
Reviewed by: kib, emaste, pfg, Oliver Pinter (earlier version)
Also discussed with: kan
MFC after: 2 weeks
Differential Revision: https://reviews.freebsd.org/D22943

show more ...

libclang_rt: enable on powerpc*

Summary:
Enable on powerpc64 and in lib/libclang_rt/Makefile change
MACHINE_CPUARCH to MACHINE_ARCH because on powerpc64
MACHINE_ARCH==MACHINE_CPUARCH so the 32-bit l

libclang_rt: enable on powerpc*

Summary:
Enable on powerpc64 and in lib/libclang_rt/Makefile change
MACHINE_CPUARCH to MACHINE_ARCH because on powerpc64
MACHINE_ARCH==MACHINE_CPUARCH so the 32-bit library overwrites 64-bit
library during installworld.

This patch doesn't enable any other libclang_rt libraries because they
need to be separately ported.

I have verified that games/julius (which fails on powerpc64 elfv2
without this change because of no libclang_rt profiling library) builds.

Test Plan: Ship it, test on powerpc and powerpcspe

Submitted by: pkubaj
Reviewed by: dim, jhibbits
Differential Revision: https://reviews.freebsd.org/D22425
MFC after: 1 month
X-MFC-With: r353358

show more ...

Vendor import of NetBSD's external/bsd/blacklist @ 20191106

Sponsored by: The FreeBSD Foundation

Import Zstd 1.4.4

Introduce stats(3), a flexible statistics gathering API.

This provides a framework to define a template describing
a set of "variables of interest" and the intended way for
the framework to maintain

Introduce stats(3), a flexible statistics gathering API.

This provides a framework to define a template describing
a set of "variables of interest" and the intended way for
the framework to maintain them (for example the maximum, sum,
t-digest, or a combination thereof). Afterwards the user
code feeds in the raw data, and the framework maintains
these variables inside a user-provided, opaque stats blobs.
The framework also provides a way to selectively extract the
stats from the blobs. The stats(3) framework can be used in
both userspace and the kernel.

See the stats(3) manual page for details.

This will be used by the upcoming TCP statistics gathering code,
https://reviews.freebsd.org/D20655.

The stats(3) framework is disabled by default for now, except
in the NOTES kernel (for QA); it is expected to be enabled
in amd64 GENERIC after a cool down period.

Reviewed by: sef (earlier version)
Obtained from: Netflix
Relnotes: yes
Sponsored by: Klara Inc, Netflix
Differential Revision: https://reviews.freebsd.org/D20477

show more ...