netmap: Make memory pools NUMA-aware

Each netmap adapter associated with a physical adapter is attached to a
netmap memory pool. contigmalloc() is used to allocate physically
contiguous memory for

netmap: Make memory pools NUMA-aware

Each netmap adapter associated with a physical adapter is attached to a
netmap memory pool. contigmalloc() is used to allocate physically
contiguous memory for the pool, but ideally we would ensure that all
such memory is allocated from the NUMA domain local to the adapter.

Augment netmap's memory pools with a NUMA domain ID, similar to how
IOMMU groups are handled in the Linux port. That is, when attaching to
a physical adapter, ensure that the associated memory pools are local to
the adapter's associated memory domain, creating new pools as needed.

Some types of ifnets do not have any defined NUMA affinity; in this case
the domain ID in question is the sentinel value -1.

Add a sysctl, dev.netmap.port_numa_affinity, which can be used to enable
the new behaviour. Keep it disabled by now to avoid surprises in case
netmap applications are relying on zero-copy optimizations to forward
packets between ports belonging to different NUMA domains.

Reviewed by: vmaffione
MFC after: 2 weeks
Sponsored by: Klara, Inc.
Differential Revision: https://reviews.freebsd.org/D46666

(cherry picked from commit 1bae9dc584272dd75dc4e04cb5d73be0e9fb562a)

show more ...

netmap: Ignore errors in CSB_WRITE()

The CSB_WRITE() and _READ() macros respectively write to and read from
userspace memory and so can in principle fault. However, we do not
check for errors and w

netmap: Ignore errors in CSB_WRITE()

The CSB_WRITE() and _READ() macros respectively write to and read from
userspace memory and so can in principle fault. However, we do not
check for errors and will proceed blindly if they fail. Add assertions
to verify that they do not.

This is in preparation for annotating copyin() and related functions
with __result_use_check.

Reviewed by: vmaffione
MFC after: 1 week
Differential Revision: https://reviews.freebsd.org/D43200

(cherry picked from commit 99efa2c88d93c6272a8f54b18d18d0fd9d60f137)

show more ...

sys: Remove $FreeBSD$: one-line .h pattern

Remove /^\s*\*+\s*\$FreeBSD\$.*$\n/

spdx: The BSD-2-Clause-FreeBSD identifier is obsolete, drop -FreeBSD

The SPDX folks have obsoleted the BSD-2-Clause-FreeBSD identifier. Catch
up to that fact and revert to their recommended match of

spdx: The BSD-2-Clause-FreeBSD identifier is obsolete, drop -FreeBSD

The SPDX folks have obsoleted the BSD-2-Clause-FreeBSD identifier. Catch
up to that fact and revert to their recommended match of BSD-2-Clause.

Discussed with: pfg
MFC After: 3 days
Sponsored by: Netflix

show more ...

netmap: Fix queue stalls with generic interfaces

In emulated mode, the FreeBSD netmap port attempts to perform zero-copy
transmission. This works as follows: the kernel ring is populated with
mbuf

netmap: Fix queue stalls with generic interfaces

In emulated mode, the FreeBSD netmap port attempts to perform zero-copy
transmission. This works as follows: the kernel ring is populated with
mbuf headers to which netmap buffers are attached. When transmitting,
the mbuf refcount is initialized to 2, and when the counter value has
been decremented to 1 netmap infers that the driver has freed the mbuf
and thus transmission is complete.

This scheme does not generalize to the situation where netmap is
attaching to a software interface which may transmit packets among
multiple "queues", as is the case with bridge or lagg interfaces. In
that case, we would be relying on backing hardware drivers to free
transmitted mbufs promptly, but this isn't guaranteed; a driver may
reasonably defer freeing a small number of transmitted buffers
indefinitely. If such a buffer ends up at the tail of a netmap transmit
ring, further transmits can end up blocked indefinitely.

Fix the problem by removing the zero-copy scheme (which is also not
implemented in the Linux port of netmap). Instead, the kernel ring is
populated with regular mbuf clusters into which netmap buffers are
copied by nm_os_generic_xmit_frame(). The refcounting scheme is
preserved, and this lets us avoid allocating a fresh cluster per
transmitted packet in the common case. If the transmit ring is full, a
callout is used to free the "stuck" mbuf, avoiding the queue deadlock
described above.

Furthermore, when recycling mbuf clusters, be sure to fully reinitialize
the mbuf header instead of simply re-setting M_PKTHDR. Some software
interfaces, like if_vlan, may set fields in the header which should be
reset before the mbuf is reused.

Reviewed by: vmaffione
MFC after: 1 month
Sponsored by: Zenarmor
Sponsored by: OPNsense
Sponsored by: Klara, Inc.
Differential Revision: https://reviews.freebsd.org/D38065

show more ...

netmap: get rid of save_if_input for emulated adapters

The save_if_input function pointer was meant to save the previous
value of ifp->if_input before replacing it with the emulated
adapter hook.
Ho

netmap: get rid of save_if_input for emulated adapters

The save_if_input function pointer was meant to save the previous
value of ifp->if_input before replacing it with the emulated
adapter hook.
However, the same pointer value is already stored in the if_input
field of the netmap_adapter struct, to be used for host TX ring processing.

Reuse the netmap_adapter if_input field to simplify the code
and save some space.

MFC after: 14 days

show more ...

netmap: get rid of WNA() macro

MFC after: 7 days

Mechanically convert netmap(4) to IfAPI

Reviewed by: vmaffione, zlei
Sponsored by: Juniper Networks, Inc.
Differential Revision: https://reviews.freebsd.org/D37814

netmap: Correct a comment

Reviewed by: vmaffione
MFC after: 1 week
Sponsored by: Zenarmor
Sponsored by: OPNsense
Sponsored by: Klara, Inc.
Differential Revision: https://reviews.freebsd.org/D38063

debug_put_get: don't crash on null pointers

MFC after: 7 days

netmap: drop compatibility FreeBSD code

Netmap users on FreeBSD are not supposed to import code from the
github netmap repository anymore. They should use the code that
is available in the src repo.

netmap: drop compatibility FreeBSD code

Netmap users on FreeBSD are not supposed to import code from the
github netmap repository anymore. They should use the code that
is available in the src repo. We can therefore drop the compatibility
code.

MFC after: 7 days

show more ...

netmap_update_config: update na->name to cope with reconfigurations

MFC after: 1 week

sys: Nuke double-semicolons

A distinct number of double-semicolons have ended up in FreeBSD. Take a
pass at getting rid of many of these harmless typos.

Reviewed by: emaste, rrs
Pull Request: http

sys: Nuke double-semicolons

A distinct number of double-semicolons have ended up in FreeBSD. Take a
pass at getting rid of many of these harmless typos.

Reviewed by: emaste, rrs
Pull Request: https://github.com/freebsd/freebsd-src/pull/609
Differential Revision: https://reviews.freebsd.org/D31716

show more ...

netmap: add a tunable for the maximum number of VALE switches

The new dev.netmap.max_bridges sysctl tunable can be set in
loader.conf(5) to change the default maximum number of VALE
switches that ca

netmap: add a tunable for the maximum number of VALE switches

The new dev.netmap.max_bridges sysctl tunable can be set in
loader.conf(5) to change the default maximum number of VALE
switches that can be created. Current defaults is 8.

MFC after: 2 weeks

show more ...

netmap: import changes from upstream

- make sure rings are disabled during resets
- introduce netmap_update_hostrings_mode(), with support
for multiple host rings
- always initialize ni_bufs_h

netmap: import changes from upstream

- make sure rings are disabled during resets
- introduce netmap_update_hostrings_mode(), with support
for multiple host rings
- always initialize ni_bufs_head in netmap_if
ni_bufs_head was not properly initialized when no external buffers were
requestedx and contained the ni_bufs_head from the last request. This
was causing spurious buffer frees when alternating between apps that
used external buffers and apps that did not use them.
- check na validitity under lock on detach
- netmap_mem: fix leak on error path
- nm_dispatch: fix compilation on Raspberry Pi

MFC after: 2 weeks

show more ...

netmap: make sure rings are disabled during resets

Explicitly disable ring synchronization before calling
callbacks that may result in a hardware reset.

Before this patch we relied on capturing the

netmap: make sure rings are disabled during resets

Explicitly disable ring synchronization before calling
callbacks that may result in a hardware reset.

Before this patch we relied on capturing the down/up events which,
however, may not be issued by all drivers.

show more ...

netmap: several typo fixes

No functional changes intended.

netmap: fix typo bug in netmap_compute_buf_len

netmap: add kernel support for the "offsets" feature

This feature enables applications to ask netmap to transmit or
receive packets starting at a user-specified offset from the
beginning of the netm

netmap: add kernel support for the "offsets" feature

This feature enables applications to ask netmap to transmit or
receive packets starting at a user-specified offset from the
beginning of the netmap buffer. This is meant to ease those
packet manipulation operations such as pushing or popping packet
headers, that may be useful to implement software switches,
routers and other packet processors.
To use the feature, drivers (e.g., iflib, vtnet, etc.) must have
explicit support. This change does not add support for any driver,
but introduces the necessary kernel changes. However, offsets support
is already included for VALE ports and pipes.

show more ...

netmap: simplify parameter passing

Changes imported from the netmap github.

netmap: introduce netmap_kring_on()

This function returns NULL if the ring identified by
queue id and direction is in netmap mode. Otherwise
return the corresponding kring.
Use this function to repl

netmap: introduce netmap_kring_on()

This function returns NULL if the ring identified by
queue id and direction is in netmap mode. Otherwise
return the corresponding kring.
Use this function to replace vtnet_netmap_queue_on().

MFC after: 1 week

show more ...

netmap: improve netmap(4) and vale(4) man pages

Clean up obsolete sysctl descriptions and add missing ones.

PR: 243838
Reviewed by: bcr
MFC after: 1 week
Differential Revision: https://reviews.fre

netmap: improve netmap(4) and vale(4) man pages

Clean up obsolete sysctl descriptions and add missing ones.

PR: 243838
Reviewed by: bcr
MFC after: 1 week
Differential Revision: https://reviews.freebsd.org/D23546

show more ...

netmap: disable passthrough with no hypervisor support

The netmap passthrough subsystem requires proper support in the
hypervisor. In particular, two PCI device ids (from the Red Hat
PCI vendor id 0

netmap: disable passthrough with no hypervisor support

The netmap passthrough subsystem requires proper support in the
hypervisor. In particular, two PCI device ids (from the Red Hat
PCI vendor id 0x1b36) need to be assigned to the two netmap
virtual devices. We then disable these devices until the ids have
not been assigned, in order to avoid conflicts with other
virtual devices emulated by upstream QEMU.

PR: 241774
MFC after: 3 days

show more ...

netmap: import changes from upstream (SHA 137f537eae513)

- Rework option processing.
- Use larger integers for memory size values in the
memory management code.

MFC after: 2 weeks

netmap: don't schedule kqueue notify task when kqueue is not used

This change adds a counter (kqueue_users) to keep track of how many
kqueue users are referencing a given struct nm_selinfo.
In this

netmap: don't schedule kqueue notify task when kqueue is not used

This change adds a counter (kqueue_users) to keep track of how many
kqueue users are referencing a given struct nm_selinfo.
In this way, nm_os_selwakeup() can schedule the kevent notification
task only when kqueue is actually being used.
This is important to avoid wasting CPU in the common case where
kqueue is not used.

Reviewed by: Aleksandr Fedorov <[email protected]>
MFC after: 1 week
Differential Revision: https://reviews.freebsd.org/D19177

show more ...