libprocstat kstack: fix race with thread creation

When collecting kernel stacks for a target process, if the process
adds a thread between the two calls to sysctl, ignore the additional
threads. Pr

libprocstat kstack: fix race with thread creation

When collecting kernel stacks for a target process, if the process
adds a thread between the two calls to sysctl, ignore the additional
threads. Previously, procstat would print only a useless error
message. Now, it prints a consistent snapshot of the stacks.
We know that snapshot is already stale, but it could still be stale
even with a more complex fix to reallocate and retry, so such a fix
is hardly worth the effort.

Reported by: [email protected]
MFC after: 1 week
Sponsored by: Dell EMC Isilon

(cherry picked from commit 427f12f150e875c40acb84f292a80bfa0b90a1a2)

show more ...

procstat_getfiles_sysctl: do not require non-null ki_fd

PR: 260174

(cherry picked from commit 7a9423d6f360e3758ca67fbb25d309140ea93670)

Style

(cherry picked from commit 0ea3e4a27bc0d1b0fdf876c3ef89d790e29ad5b1)

Decode and report native eventfd descriptors from libprocstat and procstat.

Submitted by: [email protected]
Reviewed by: markj (previous version)
MFC after: 2 weeks
Differential

Decode and report native eventfd descriptors from libprocstat and procstat.

Submitted by: [email protected]
Reviewed by: markj (previous version)
MFC after: 2 weeks
Differential Revision: https://reviews.freebsd.org/D26668

show more ...

Remove the cloned file descriptors for /dev/crypto.

Crypto file descriptors were added in the original OCF import as a way
to provide per-open data (specifically the list of symmetric
sessions). Ho

Remove the cloned file descriptors for /dev/crypto.

Crypto file descriptors were added in the original OCF import as a way
to provide per-open data (specifically the list of symmetric
sessions). However, this gives a bit of a confusing API where one has
to open /dev/crypto and then invoke an ioctl to obtain a second file
descriptor. This also does not match the API used with /dev/crypto on
other BSDs or with Linux's /dev/crypto driver.

Character devices have gained support for per-open data via cdevpriv
since OCF was imported, so use cdevpriv to simplify the userland API
by permitting ioctls directly on /dev/crypto descriptors.

To provide backwards compatibility, CRIOGET now opens another
/dev/crypto descriptor via kern_openat() rather than dup'ing the
existing file descriptor. This preserves prior semantics in case
CRIOGET is invoked multiple times on a single file descriptor.

Reviewed by: markj
Relnotes: yes
Sponsored by: Chelsio Communications
Differential Revision: https://reviews.freebsd.org/D27302

show more ...

Split out cwd/root/jail, cmask state from filedesc table

No functional change intended.

Tracking these structures separately for each proc enables future work to
correctly emulate clone(2) in linux

Split out cwd/root/jail, cmask state from filedesc table

No functional change intended.

Tracking these structures separately for each proc enables future work to
correctly emulate clone(2) in linux(4).

__FreeBSD_version is bumped (to 1300130) for consumption by, e.g., lsof.

Reviewed by: kib
Discussed with: markj, mjg
Differential Revision: https://reviews.freebsd.org/D27037

show more ...

Merge OpenZFS support in to HEAD.

The primary benefit is maintaining a completely shared
code base with the community allowing FreeBSD to receive
new features sooner and with less effort.

I would a

Merge OpenZFS support in to HEAD.

The primary benefit is maintaining a completely shared
code base with the community allowing FreeBSD to receive
new features sooner and with less effort.

I would advise against doing 'zpool upgrade'
or creating indispensable pools using new
features until this change has had a month+
to soak.

Work on merging FreeBSD support in to what was
at the time "ZFS on Linux" began in August 2018.
I first publicly proposed transitioning FreeBSD
to (new) OpenZFS on December 18th, 2018. FreeBSD
support in OpenZFS was finally completed in December
2019. A CFT for downstreaming OpenZFS support in
to FreeBSD was first issued on July 8th. All issues
that were reported have been addressed or, for
a couple of less critical matters there are
pull requests in progress with OpenZFS. iXsystems
has tested and dogfooded extensively internally.
The TrueNAS 12 release is based on OpenZFS with
some additional features that have not yet made
it upstream.

Improvements include:
project quotas, encrypted datasets,
allocation classes, vectorized raidz,
vectorized checksums, various command line
improvements, zstd compression.

Thanks to those who have helped along the way:
Ryan Moeller, Allan Jude, Zack Welch, and many
others.

Sponsored by: iXsystems, Inc.
Differential Revision: https://reviews.freebsd.org/D25872

show more ...

libprocstat: fix kvm filedesc access after introduction of fdescenttbl

libprocstat: fix reading of file descriptor table via kvm

This seems to have been broken since r247602 (from year 2013!).
Can be easily tested with
fstat -N /boot/kernel/kernel -M /var/crash/vmcor

libprocstat: fix reading of file descriptor table via kvm

This seems to have been broken since r247602 (from year 2013!).
Can be easily tested with
fstat -N /boot/kernel/kernel -M /var/crash/vmcore.last

MFC after: 1 week
Sponsored by: Panzura

show more ...

fd: use smr for managing struct pwd

This has a side effect of eliminating filedesc slock/sunlock during path
lookup, which in turn removes contention vs concurrent modifications to the fd
table.

Re

fd: use smr for managing struct pwd

This has a side effect of eliminating filedesc slock/sunlock during path
lookup, which in turn removes contention vs concurrent modifications to the fd
table.

Reviewed by: markj, kib
Differential Revision: https://reviews.freebsd.org/D23889

show more ...

fd: move vnodes out of filedesc into a dedicated structure

The new structure is copy-on-write. With the assumption that path lookups are
significantly more frequent than chdirs and chrooting this is

fd: move vnodes out of filedesc into a dedicated structure

The new structure is copy-on-write. With the assumption that path lookups are
significantly more frequent than chdirs and chrooting this is a win.

This provides stable root and jail root vnodes without the need to reference
them on lookup, which in turn means less work on globally shared structures.
Note this also happens to fix a bug where jail vnode was never referenced,
meaning subsequent access on lookup could run into use-after-free.

Reviewed by: kib
Differential Revision: https://reviews.freebsd.org/D23884

show more ...

procstat: read lo_name instead of now removed v_tag

Define a vm_map method for user-space for advancing from a map entry
to its successor in cases where examining a map entry requires a
helper like kvm_read_all. Use that method, with kvm_read_all, to

Define a vm_map method for user-space for advancing from a map entry
to its successor in cases where examining a map entry requires a
helper like kvm_read_all. Use that method, with kvm_read_all, to fix
procstat_getfiles_kvm, which tries to find the successor now without
using such a helper. This addresses a problem introduced by r355491.

Reviewed by: markj (previous version)
Discussed with: kib
Differential Revision: https://reviews.freebsd.org/D22728

show more ...

Fix a type error in fixing libprocstat to be compatible with vm_map changes.

Approved by: markj
Differential Revision: https://reviews.freebsd.org/D22726

r355491 broke compilation of libprocstat.c. Change that code to use
new methods for accessing first, next map entries.

Approved by: kib
Differential Revision: https://reviews.freebsd.org/D22725

Print type designator 'D' for the KF_TYPE_DEV files.

No type-specific data is provided by the kernel.

Sponsored by: Mellanox Technologies
MFC after: 1 week

libprocstat: fix memory leak

Free the rlimits array on the happy path in procstat_getrlimit_core().

Reported by: Coverity
CID: 1373328
Sponsored by: Dell EMC

spdx: initial adoption of licensing ID tags.

The Software Package Data Exchange (SPDX) group provides a specification
to make it easier for automated tools to detect and summarize well known
opensou

spdx: initial adoption of licensing ID tags.

The Software Package Data Exchange (SPDX) group provides a specification
to make it easier for automated tools to detect and summarize well known
opensource licenses. We are gradually adopting the specification, noting
that the tags are considered only advisory and do not, in any way,
superceed or replace the license texts.

Special thanks to Wind River for providing access to "The Duke of
Highlander" tool: an older (2014) run over FreeBSD tree was useful as a
starting point.

Initially, only tag files that use BSD 4-Clause "Original" license.

RelNotes: yes
Differential Revision: https://reviews.freebsd.org/D13133

show more ...

Make procstat(1) recognize process descriptors, so that it shows
"P" instead of "?" in "procstat -af" output. Note that there are
still a few more DTYPE_* kinds we don't decode yet.

Reported by: rwa

Make procstat(1) recognize process descriptors, so that it shows
"P" instead of "?" in "procstat -af" output. Note that there are
still a few more DTYPE_* kinds we don't decode yet.

Reported by: rwatson
MFC after: 2 weeks
Sponsored by: DARPA, AFRL
Differential Revision: https://reviews.freebsd.org/D12426

show more ...

Hide struct socket and struct unpcb from the userland.

Violators may define _WANT_SOCKET and _WANT_UNPCB respectively and
are not guaranteed for stability of the structures. The violators
list is t

Hide struct socket and struct unpcb from the userland.

Violators may define _WANT_SOCKET and _WANT_UNPCB respectively and
are not guaranteed for stability of the structures. The violators
list is the the usual one: libprocstat(3) and netstat(1) internally
and lsof in ports.

In struct xunpcb remove the inclusion of kernel structure and add
a bunch of spare fields. The xsocket already has socket not included,
but add there spares as well. Embed xsockbuf into xsocket.

Sort declarations in sys/socketvar.h to separate kernel only from
userland available ones.

PR: 221820 (exp-run)

show more ...

procstat_getptlwpinfo(..): clarify the fact that KVM/SYSCTL support
isn't supported

This will make the error message reported in bug 220023 a bit more
intuitive for end-users that don't have access

procstat_getptlwpinfo(..): clarify the fact that KVM/SYSCTL support
isn't supported

This will make the error message reported in bug 220023 a bit more
intuitive for end-users that don't have access to the source code to
decode the procstat->type argument.

MFC after: 1 month
MFC with: r316286
PR: 220023

show more ...

procstat(1): Add TCP socket send/recv buffer size

Add TCP socket send and receive buffer size to procstat -f output.

Reviewed by: kib, markj
Sponsored by: Dell EMC Isilon
Differential Revision: htt

procstat(1): Add TCP socket send/recv buffer size

Add TCP socket send and receive buffer size to procstat -f output.

Reviewed by: kib, markj
Sponsored by: Dell EMC Isilon
Differential Revision: https://reviews.freebsd.org/D10689

show more ...

Commit the 64-bit inode project.

Extend the ino_t, dev_t, nlink_t types to 64-bit ints. Modify
struct dirent layout to add d_off, increase the size of d_fileno
to 64-bits, increase the size of d_na

Commit the 64-bit inode project.

Extend the ino_t, dev_t, nlink_t types to 64-bit ints. Modify
struct dirent layout to add d_off, increase the size of d_fileno
to 64-bits, increase the size of d_namlen to 16-bits, and change
the required alignment. Increase struct statfs f_mntfromname[] and
f_mntonname[] array length MNAMELEN to 1024.

ABI breakage is mitigated by providing compatibility using versioned
symbols, ingenious use of the existing padding in structures, and
by employing other tricks. Unfortunately, not everything can be
fixed, especially outside the base system. For instance, third-party
APIs which pass struct stat around are broken in backward and
forward incompatible ways.

Kinfo sysctl MIBs ABI is changed in backward-compatible way, but
there is no general mechanism to handle other sysctl MIBS which
return structures where the layout has changed. It was considered
that the breakage is either in the management interfaces, where we
usually allow ABI slip, or is not important.

Struct xvnode changed layout, no compat shims are provided.

For struct xtty, dev_t tty device member was reduced to uint32_t.
It was decided that keeping ABI compat in this case is more useful
than reporting 64-bit dev_t, for the sake of pstat.

Update note: strictly follow the instructions in UPDATING. Build
and install the new kernel with COMPAT_FREEBSD11 option enabled,
then reboot, and only then install new world.

Credits: The 64-bit inode project, also known as ino64, started life
many years ago as a project by Gleb Kurtsou (gleb). Kirk McKusick
(mckusick) then picked up and updated the patch, and acted as a
flag-waver. Feedback, suggestions, and discussions were carried
by Ed Maste (emaste), John Baldwin (jhb), Jilles Tjoelker (jilles),
and Rick Macklem (rmacklem). Kris Moore (kris) performed an initial
ports investigation followed by an exp-run by Antoine Brodin (antoine).
Essential and all-embracing testing was done by Peter Holm (pho).
The heavy lifting of coordinating all these efforts and bringing the
project to completion were done by Konstantin Belousov (kib).

Sponsored by: The FreeBSD Foundation (emaste, kib)
Differential revision: https://reviews.freebsd.org/D10439

show more ...

Reorder includes to placate MIPS build.

Reported by: markj
Sponsored by: Dell EMC Isilon

Add support for capturing 'struct ptrace_lwpinfo' for signals
resulting in a process dumping core in the corefile.

Also extend procstat to view select members of 'struct ptrace_lwpinfo'
from the con

Add support for capturing 'struct ptrace_lwpinfo' for signals
resulting in a process dumping core in the corefile.

Also extend procstat to view select members of 'struct ptrace_lwpinfo'
from the contents of the note.

Sponsored by: Dell EMC Isilon

show more ...