mac: cheaper check for ifnet_create_mbuf and ifnet_check_transmit

Sponsored by: Rubicon Communications, LLC ("Netgate")

(cherry picked from commit f77697dd9f31df85cd86370888606c81833f7c8a)

mac: cheaper check for mac_vnode_check_readlink

cache: combine fast path enabled status into one flag

Tested by: pho

pipe: allow for lockless pipe_stat

pipes get stated all thet time and this avoidably contributed to contention.
The pipe lock is only held to accomodate MAC and to check the type.

Since normally th

pipe: allow for lockless pipe_stat

pipes get stated all thet time and this avoidably contributed to contention.
The pipe lock is only held to accomodate MAC and to check the type.

Since normally there is no probe for pipe stat depessimize this by having the
flag.

The pipe_state field gets modified with locks held all the time and it's not
feasible to convert them to use atomic store. Move the type flag away to a
separate variable as a simple cleanup and to provide stable field to read.
Use short for both fields to avoid growing the struct.

While here short-circuit MAC for pipe_poll as well.

show more ...

vfs: add a cheaper entry for mac_vnode_check_access

vfs: elide MAC-induced locking on rename if there are no relevant hoooks

vfs: fix vn_poll performance with either MAC or AUDIT

The code would unconditionally lock the vnode to audit or call the
mac hoook, even if neither want to do anything. Pre-check the state
to avoid

vfs: fix vn_poll performance with either MAC or AUDIT

The code would unconditionally lock the vnode to audit or call the
mac hoook, even if neither want to do anything. Pre-check the state
to avoid locking in the common case of nothing to do.

Note this code should not be normally executed anyway as vnodes are
always return ready. However, poll1/2 from will-it-scale use regular
files for benchmarking, presumably to focus on the interface itself
as the vnode handler is not supposed to do almost anything.

This in particular fixes poll2 which passes 128 fds.

$ ./poll2_processes -s 10
before: 134411
after: 271572

show more ...

mac_policy: Remove mac_policy_sx

This lock was made unnecessary by the addition of mac_policy_rms in r356120.

Reviewed by: mjg, kib
Differential Revision: https://reviews.freebsd.org/D24283

Mark more nodes as CTLFLAG_MPSAFE or CTLFLAG_NEEDGIANT (17 of many)

r357614 added CTLFLAG_NEEDGIANT to make it easier to find nodes that are
still not MPSAFE (or already are but aren’t properly mark

Mark more nodes as CTLFLAG_MPSAFE or CTLFLAG_NEEDGIANT (17 of many)

r357614 added CTLFLAG_NEEDGIANT to make it easier to find nodes that are
still not MPSAFE (or already are but aren’t properly marked).
Use it in preparation for a general review of all nodes.

This is non-functional change that adds annotations to SYSCTL_NODE and
SYSCTL_PROC nodes using one of the soon-to-be-required flags.

Mark all obvious cases as MPSAFE. All entries that haven't been marked
as MPSAFE before are by default marked as NEEDGIANT

Approved by: kib (mentor, blanket)
Commented by: kib, gallatin, melifaro
Differential Revision: https://reviews.freebsd.org/D23718

show more ...

vfs: use mac fastpath for lookup, open, read, write, mmap

mac: implement fast path for checks

All checking routines walk a linked list of all modules in order to determine
if given hook is installed. This became a significant problem after mac_ntpd
started

mac: implement fast path for checks

All checking routines walk a linked list of all modules in order to determine
if given hook is installed. This became a significant problem after mac_ntpd
started being loaded by default.

Implement a way perform checks for select hooks by testing a boolean.

Use it for priv_check and priv_grant, which are constantly called from priv_check.

The real fix would use hotpatching, but the above provides a way to know when
to do it.

show more ...

mac: use a sleepable rmlock instead of an sx lock

If any non-static modules are loaded (and mac_ntpd tends to be), the lock is
taken all the time al over the kernel. On platforms like arm64 this res

mac: use a sleepable rmlock instead of an sx lock

If any non-static modules are loaded (and mac_ntpd tends to be), the lock is
taken all the time al over the kernel. On platforms like arm64 this results in
an avoidable significant performance degradation. Since write-locking is almost
never needed, use a primitive optimized towards read-locking.

Sample result of building the kernel on tmpfs 11 times:
stock 11142.80s user 6704.44s system 4924% cpu 6:02.42 total
patched 11118.95s user 2374.94s system 4547% cpu 4:56.71 total

show more ...

Require that MAC label buffers be able to store a non-empty string.

The buffer size may be used to initialize an sbuf in
MAC_POLICY_EXTERNALIZE, and without this constraint it's possible to
trigger

Require that MAC label buffers be able to store a non-empty string.

The buffer size may be used to initialize an sbuf in
MAC_POLICY_EXTERNALIZE, and without this constraint it's possible to
trigger an assertion failure in the sbuf code. With INVARIANTS
disabled, the first attempt to write to the sbuf will fail.

Reported by: pho
Reviewed by: delphij
MFC after: 1 week
Differential Revision: https://reviews.freebsd.org/D16527

show more ...

sys/security: minor spelling fixes.

No functional change.

Fix style issues around existing SDT probes.

- Use SDT_PROBE<N>() instead of SDT_PROBE(). This has no functional effect
at the moment, but will be needed for some future changes.
- Don't hardcode

Fix style issues around existing SDT probes.

- Use SDT_PROBE<N>() instead of SDT_PROBE(). This has no functional effect
at the moment, but will be needed for some future changes.
- Don't hardcode the module component of the probe identifier. This is
set automatically by the SDT framework.

MFC after: 1 week

show more ...

As constantly reported during kernel compilation, m_buflen is unsigned so
can never be < 0. Remove the expression, which can never be true.

MFC after: 1 week

dtrace sdt: remove the ugly sname parameter of SDT_PROBE_DEFINE

In its stead use the Solaris / illumos approach of emulating '-' (dash)
in probe names with '__' (two consecutive underscores).

Revie

dtrace sdt: remove the ugly sname parameter of SDT_PROBE_DEFINE

In its stead use the Solaris / illumos approach of emulating '-' (dash)
in probe names with '__' (two consecutive underscores).

Reviewed by: markj
MFC after: 3 weeks

show more ...

- For kernel compiled only with KDTRACE_HOOKS and not any lock debugging
option, unbreak the lock tracing release semantic by embedding
calls to LOCKSTAT_PROFILE_RELEASE_LOCK() direclty in the in

- For kernel compiled only with KDTRACE_HOOKS and not any lock debugging
option, unbreak the lock tracing release semantic by embedding
calls to LOCKSTAT_PROFILE_RELEASE_LOCK() direclty in the inlined
version of the releasing functions for mutex, rwlock and sxlock.
Failing to do so skips the lockstat_probe_func invokation for
unlocking.
- As part of the LOCKSTAT support is inlined in mutex operation, for
kernel compiled without lock debugging options, potentially every
consumer must be compiled including opt_kdtrace.h.
Fix this by moving KDTRACE_HOOKS into opt_global.h and remove the
dependency by opt_kdtrace.h for all files, as now only KDTRACE_FRAMES
is linked there and it is only used as a compile-time stub [0].

[0] immediately shows some new bug as DTRACE-derived support for debug
in sfxge is broken and it was never really tested. As it was not
including correctly opt_kdtrace.h before it was never enabled so it
was kept broken for a while. Fix this by using a protection stub,
leaving sfxge driver authors the responsibility for fixing it
appropriately [1].

Sponsored by: EMC / Isilon storage division
Discussed with: rstone
[0] Reported by: rstone
[1] Discussed with: philip

show more ...

Fix some typos that were causing probe argument types to show up as unknown.

Reviewed by: rwatson (mac provider)
Approved by: re (glebius)
MFC after: 1 week

Make the mac_policy_rm lock recursable, which allows reentrance into
the mac framework. It is needed when priv_check_cred(9) is called from
the mac callback, e.g. in the mac_portacl(4).

Reported by

Make the mac_policy_rm lock recursable, which allows reentrance into
the mac framework. It is needed when priv_check_cred(9) is called from
the mac callback, e.g. in the mac_portacl(4).

Reported by: az
Reviewed by: rwatson
Sponsored by: The FreeBSD Foundation
MFC after: 1 week
Approved by: re (gjb)

show more ...

put sys/systm.h at its proper place or add it if missing

Reported by: lstewart, tinderbox
Pointyhat to: avg, attilio
MFC after: 1 week
MFC with: r228430

sysctl(9) cleanup checkpoint: amd64 GENERIC builds cleanly.

Commit the security directory.

Add an extra comment to the SDT probes definition. This allows us to get
use '-' in probe names, matching the probe names in Solaris.[1]

Add userland SDT probes definitions to sys/sdt.h.

Sponsored

Add an extra comment to the SDT probes definition. This allows us to get
use '-' in probe names, matching the probe names in Solaris.[1]

Add userland SDT probes definitions to sys/sdt.h.

Sponsored by: The FreeBSD Foundation
Discussed with: rwaston [1]

show more ...

Mark MAC Framework sx and rm locks as NOWITNESS to suppress warnings that
might arise from WITNESS not understanding its locking protocol, which
should be deadlock-free. Currently these warnings gen

Mark MAC Framework sx and rm locks as NOWITNESS to suppress warnings that
might arise from WITNESS not understanding its locking protocol, which
should be deadlock-free. Currently these warnings generally don't occur,
but as object locking is pushed into policies for some object types, they
would otherwise occur more often.

Obtained from: TrustedBSD Project

show more ...

Add internal 'mac_policy_count' counter to the MAC Framework, which is a
count of the number of registered policies.

Rather than unconditionally locking sockets before passing them into MAC,
lock th

Add internal 'mac_policy_count' counter to the MAC Framework, which is a
count of the number of registered policies.

Rather than unconditionally locking sockets before passing them into MAC,
lock them in the MAC entry points only if mac_policy_count is non-zero.

This avoids locking overhead for a number of socket system calls when no
policies are registered, eliminating measurable overhead for the MAC
Framework for the socket subsystem when there are no active policies.

Possibly socket locks should be acquired by policies if they are required
for socket labels, which would further avoid locking overhead when there
are policies but they don't require labeling of sockets, or possibly
don't even implement socket controls.

Obtained from: TrustedBSD Project

show more ...