ena: Clear NS_MOREFRAG flag for last netmap slot

When processing packets within the rx-flow
ena_netmap_rx_load_desc doesn't know the number of descriptors, so it
sets NS_MOREFRAG to all the slots to

ena: Clear NS_MOREFRAG flag for last netmap slot

When processing packets within the rx-flow
ena_netmap_rx_load_desc doesn't know the number of descriptors, so it
sets NS_MOREFRAG to all the slots to indicate that there are more
fragments for this packet.
The code calls ena_netmap_rx_load_desc() for every descriptor in
this packet to map the relevant buffer into the netmap shared memory.
After ena_netmap_rx_load_desc() calls, we need to unset the NS_MOREFRAG
for the last fragment to indicate that this is the last fragment,
so we explicitly turn off NS_MOREFRAG flag.
Current code overrides all other flags and sets NS_BUF_CHANGED.
This patch unsets the relevant flag only.

Approved by: cperciva (mentor)
Sponsored by: Amazon, Inc.

(cherry picked from commit 2f17afd19a3534dc1755c52edb0c2f70ea0eb1e4)

show more ...

ena: Handle wrap around for prefetch in netmap

Netmap index wraps around based on the number of netmap kernel ring
slots.
Currently the driver prefetches the next slot using nm_i + 1 which may
be wr

ena: Handle wrap around for prefetch in netmap

Netmap index wraps around based on the number of netmap kernel ring
slots.
Currently the driver prefetches the next slot using nm_i + 1 which may
be wrong since it does not handle wrap around.
This patch fixes that by using the kernel API for fetching the next
netmap index.

Approved by: cperciva (mentor)
Sponsored by: Amazon, Inc.

(cherry picked from commit ce20b51cb71bfb548fcaafc4bacb8290460f03d5)

show more ...

ena: Properly unmap last socket chain in netmap

In case ena_com_prepare_tx() fails within the netmap tx flow,
the driver will unmap the last socket chain.
Currently, the driver unmaps the wrong sock

ena: Properly unmap last socket chain in netmap

In case ena_com_prepare_tx() fails within the netmap tx flow,
the driver will unmap the last socket chain.
Currently, the driver unmaps the wrong socket within
ena_netmap_unmap_last_socket_chain().

Illustration of the flow:

1- ena_netmap_tx_frames()
2- ena_netmap_tx_frame()
3- ena_netmap_tx_map_slots()
3.1- Map slot
3.2- Advance to the next socket
4- ena_com_prepare_tx()
4.1- ena_com_prepare_tx() fails
5- ena_netmap_unmap_last_socket_chain()

In step 5, where the driver unmaps the socket, the netmap
index already points at the next entry, meaning we're unmapping the
wrong socket in case ena_com_prepare_tx() fails.
In order to fix that, the driver should first update the netmap index to
point at the previous entry and only then update the socket parameters.

Approved by: cperciva (mentor)
Sponsored by: Amazon, Inc.

(cherry picked from commit f236e544a2ff685ae151f3232e3785a6a9aab035)

show more ...

ena: Add reset reason for corrupted RX cdescs

RX completion descriptors may sometimes contain errors due
to corruption. Upon identifying such a case, the driver will
trigger a reset with an explicit

ena: Add reset reason for corrupted RX cdescs

RX completion descriptors may sometimes contain errors due
to corruption. Upon identifying such a case, the driver will
trigger a reset with an explicit reset reason
ENA_REGS_RESET_RX_DESCRIPTOR_MALFORMED.

Approved by: cperciva (mentor)
Sponsored by: Amazon, Inc.

(cherry picked from commit 4af71159db3cd4a37055b2b3d982ec53703c5c3d)

show more ...

ena: Add reset reason for corrupted TX cdescs

TX completion descriptors may sometimes contain errors due
to corruption. Upon identifying such a case, the driver will
trigger a reset with an explicit

ena: Add reset reason for corrupted TX cdescs

TX completion descriptors may sometimes contain errors due
to corruption. Upon identifying such a case, the driver will
trigger a reset with an explicit reset reason
ENA_REGS_RESET_TX_DESCRIPTOR_MALFORMED.

Approved by: cperciva (mentor)
Sponsored by: Amazon, Inc.

(cherry picked from commit 38727218460008a500fbc18f08c90082ed678895)

show more ...

ena: Update license signatures to 2024

This commit updates all the license signatures to 2024.

Approved by: cperciva (mentor)
Sponsored by: Amazon, Inc.

(cherry picked from commit 8d6806cd08c093fc

ena: Update license signatures to 2024

This commit updates all the license signatures to 2024.

Approved by: cperciva (mentor)
Sponsored by: Amazon, Inc.

(cherry picked from commit 8d6806cd08c093fc001db1f94cf122368b8d1549)

show more ...

ena: Update the license dating to 2023

Some of the files are using outdated linceses.
Update the license to be 2023.

Approved by: cperciva (mentor)
MFC after: 2 weeks
Sponsored by: Amazon, Inc.

(c

ena: Update the license dating to 2023

Some of the files are using outdated linceses.
Update the license to be 2023.

Approved by: cperciva (mentor)
MFC after: 2 weeks
Sponsored by: Amazon, Inc.

(cherry picked from commit 246aa273244e91a30d70997a3be790a29f9eb29c)

show more ...

ena: Remove CQ tail pointer update API

This commit removes the usage of this API from the freebsd driver since
the relevant functionality is not supported by the device.

Approved by: cperciva (ment

ena: Remove CQ tail pointer update API

This commit removes the usage of this API from the freebsd driver since
the relevant functionality is not supported by the device.

Approved by: cperciva (mentor)
MFC after: 2 weeks
Sponsored by: Amazon, Inc.

(cherry picked from commit 2835752e075f2fa3edcb596df8306c570ec4cae6)

show more ...

sys: Remove $FreeBSD$: one-line .c pattern

Remove /^[\s*]*__FBSDID\("\$FreeBSD\$"\);?\s*\n/

Mechanically convert ena(4) to DrvAPI

Reviewed by: mw
Differential Revision: https://reviews.freebsd.org/D37837

ena: Fix invalid KASSERT test in netmap code

The KASSERT was originally added to ensure that the netmap Rx ring is
not NULL, however, it was checking for the opposite.

Obtained from: Semihalf
MFC a

ena: Fix invalid KASSERT test in netmap code

The KASSERT was originally added to ensure that the netmap Rx ring is
not NULL, however, it was checking for the opposite.

Obtained from: Semihalf
MFC after: 2 weeks
Sponsored by: Amazon, Inc.

show more ...

ena: Align names of constants

Most of the constants in ena.h file were prefixed with ENA_*, while
others did not have this prefix. Align the constants by prefixing the
remaining constants with ENA.

ena: Align names of constants

Most of the constants in ena.h file were prefixed with ENA_*, while
others did not have this prefix. Align the constants by prefixing the
remaining constants with ENA.

Obtained from: Semihalf
MFC after: 2 weeks
Sponsored by: Amazon, Inc.

show more ...

ena: Fix styling issues

Align code style with FreeBSD style(9) guidelines.

Obtained from: Semihalf
MFC after: 2 weeks
Sponsored by: Amazon, Inc.

ena: Extend debug prints for invalid req_id resets

Print information about qid if req_id is invalid. Add information about
qid and req_id if mbuf is invalid.

Obtained from: Semihalf
MFC after: 2 we

ena: Extend debug prints for invalid req_id resets

Print information about qid if req_id is invalid. Add information about
qid and req_id if mbuf is invalid.

Obtained from: Semihalf
MFC after: 2 weeks
Sponsored by: Amazon, Inc.

show more ...

ena: Add ena_ring_tx_doorbell() function

Add ena_ring_tx_doorbell function to remove code duplication.

Obtained from: Semihalf
MFC after: 2 weeks
Sponsored by: Amazon, Inc.

ena: Remove unused variable.

ena: Share ena_global_lock between driver instances

In order to use `ena_global_lock` in sysctl context, it must be kept
outside the driver instance's software context, as sysctls can be called
befo

ena: Share ena_global_lock between driver instances

In order to use `ena_global_lock` in sysctl context, it must be kept
outside the driver instance's software context, as sysctls can be called
before attach and after detach, leading to lock use before sx_init and
after sx_destroy otherwise.
Solve this issue by turning `ena_global_lock` into a file scope
variable, shared between all instances of the driver and associated
sysctl context, and in turn initialized/destroyed in dedicated
SYSINIT/SYSUNINIT functions.
As a side effect, this change also fixes existing race in the reset
routine, when simultaneously accessing sysctl exposed properties.

Obtained from: Semihalf
MFC after: 2 weeks
Sponsored by: Amazon, Inc.

show more ...

ena: Disable meta descriptor caching for netmap

If LLQ is being used, `ena_tx_ctx.meta_valid` must stay enabled. This
fixes netmap support on latest generation ENA HW and aligns it with the
core dri

ena: Disable meta descriptor caching for netmap

If LLQ is being used, `ena_tx_ctx.meta_valid` must stay enabled. This
fixes netmap support on latest generation ENA HW and aligns it with the
core driver behavior.

As netmap doesn't support any csum offloads, the
`adapter->disable_meta_caching` value can be simply passed to the HW.

Obtained from: Semihalf
MFC after: 2 weeks
Sponsored by: Amazon, Inc.

show more ...

ena: Trigger reset on ena_com_prepare_tx failure

All ena_com_prepare_tx errors other than ENA_COM_NO_MEM are fatal and
require device reset.

Obtained from: Semihalf
MFC after: 2 weeks
Sponsored by:

ena: Trigger reset on ena_com_prepare_tx failure

All ena_com_prepare_tx errors other than ENA_COM_NO_MEM are fatal and
require device reset.

Obtained from: Semihalf
MFC after: 2 weeks
Sponsored by: Amazon, Inc.

show more ...

Add SPDX license tag to the ENA driver files

Refering to guide: https://wiki.freebsd.org/SPDX the SPDX tag should not
replace the standard license text, however it should be added over the
standard

Add SPDX license tag to the ENA driver files

Refering to guide: https://wiki.freebsd.org/SPDX the SPDX tag should not
replace the standard license text, however it should be added over the
standard license text to make the automation easier.

Because of that, the old license was kept, but the SPDX tag was added
on top of every ENA driver file.

Submited by: Michal Krawczyk <[email protected]>
Obtained from: Semihalf
Sponsored by: Amazon, Inc
MFC after: 1 week
Differential revision: https://reviews.freebsd.org/D27117

show more ...

Update ENA driver version to v2.2.0

Driver version upgrade is connected with support for the new device
fetures, like Tx drops reporting or disabling meta caching.

Moreover, the driver configuratio

Update ENA driver version to v2.2.0

Driver version upgrade is connected with support for the new device
fetures, like Tx drops reporting or disabling meta caching.

Moreover, the driver configuration from the sysctl was reworked to
provide safer and better flow for configuring:
* number of IO queues (new feature),
* drbr size on Tx,
* Rx queue size.

Moreover, a lot of minor bug fixes and improvements were added.

Copyright date in the license of the modified files in this release was
updated to 2020.

Submitted by: Michal Krawczyk <[email protected]>
Obtained from: Semihalf
Sponsored by: Amazon, Inc.

show more ...

Create ENA IO queues with optional backoff

If requested size of IO queues is not supported try to decrease it until
finding the highest value that can be satisfied.

Submitted by: Maciej Bielski <m

Create ENA IO queues with optional backoff

If requested size of IO queues is not supported try to decrease it until
finding the highest value that can be satisfied.

Submitted by: Maciej Bielski <[email protected]>
Obtained from: Semihalf
Sponsored by: Amazon, Inc.

show more ...

Rework ENA Rx queue size configuration

This patch reworks how the Rx queue size is being reconfigured and how
the information from the device is being processed.

Reconfiguration of the queues and r

Rework ENA Rx queue size configuration

This patch reworks how the Rx queue size is being reconfigured and how
the information from the device is being processed.

Reconfiguration of the queues and reset of the device in order to make
the changes alive isn't the best approach. It can be done synchronously
and it will let to pass information if the reconfiguration was
successful to the user. It now is done in the ena_update_queue_size()
function.

To avoid reallocation of the ring buffer, statistic counters and the
reinitialization of the mutexes when only new size has to be assigned,
the io queues initialization function has been split into 2 stages:
basic, which is just copying appropriate fields and the advanced, which
allocates and inits more advanced structures for the IO rings.

Moreover, now the max allowed Rx and Tx ring size is being kept
statically in the adapter and the size of the variables holding those
values has been changed to uint32_t everywhere.

Information about IO queues size is now being logged in the up routine
instead of the attach.

Submitted by: Michal Krawczyk <[email protected]>
Obtained from: Semihalf
Sponsored by: Amazon, Inc.

show more ...

Use single global lock in the ENA driver

Currently, the driver had 2 global locks - one was sx lock used for
up/down synchronization and the second one was mutex, which was used
for link configurati

Use single global lock in the ENA driver

Currently, the driver had 2 global locks - one was sx lock used for
up/down synchronization and the second one was mutex, which was used
for link configuration and timer service callout.

It is better to have single lock for that. We cannot use mutex, as it
can sleep and cause witness errors in up/down configuration, so sx lock
seems to be the only choice.

Callout cannot use sx lock, but the timer service is MP safe, so we just
need to avoid race between ena_down() and ena_detach(). It can be
avoided by acquiring sx lock.

Simple macros were added that are encapsulating implementation of the
lock and makes the code cleaner.

Submitted by: Michal Krawczyk <[email protected]>
Obtained from: Semihalf
Sponsored by: Amazon, Inc.

show more ...

Add trigger reset function in the ENA driver

As the reset triggering is no longer a simple macro that was just
setting appropriate flag, the new function for triggering reset was
added. It improves

Add trigger reset function in the ENA driver

As the reset triggering is no longer a simple macro that was just
setting appropriate flag, the new function for triggering reset was
added. It improves code readability a lot, as we are avoiding additional
indentation.

Submitted by: Michal Krawczyk <[email protected]>
Obtained from: Semihalf
Sponsored by: Amazon, Inc.

show more ...