| c27f502e | 03-Jan-2022 |
Michal Krawczyk <[email protected]> |
ena: update ENA version to v2.5.0
Some of the changes in this release:
- IPv6 L4 checksum offload fixes.
- Optimization of the Tx req_id validation.
- Timer service adjustments.
- NUMA awareness for
ena: update ENA version to v2.5.0
Some of the changes in this release:
- IPv6 L4 checksum offload fixes.
- Optimization of the Tx req_id validation.
- Timer service adjustments.
- NUMA awareness for the kernel RSS mode.
Submitted by: Michal Krawczyk <[email protected]>
Obtained from: Semihalf
MFC after: 2 weeks
Sponsored by: Amazon, Inc.
(cherry picked from commit 8a5b4859c7fcefac7db611ef38450ef0a66d68a8)
show more ...
|
| 8875a587 | 03-Jan-2022 |
Dawid Gorecki <[email protected]> |
ena: do not call reset if device is unresponsive
If the device becomes unresponsive, the driver will not be able to
finish the reset process correctly. Timeout during version validation
indicates th
ena: do not call reset if device is unresponsive
If the device becomes unresponsive, the driver will not be able to
finish the reset process correctly. Timeout during version validation
indicates that the device is currently not responding. In that case
do not perform the reset and instead reschedule timer service. Because
of that the driver will continue trying to reset the device until it
succeeds or is detached.
Submitted by: Dawid Gorecki <[email protected]>
Obtained from: Semihalf
MFC after: 2 weeks
Sponsored by: Amazon, Inc.
(cherry picked from commit d10ec3ad7739a6f621d398d034632f68f647d72f)
show more ...
|
| ed4368c2 | 03-Jan-2022 |
Dawid Gorecki <[email protected]> |
ena: start timer service on attach
The timer service was started when the interface was brought up and it
was stopped when it was brought down. Since ena_up requires the device
to be responsive, tri
ena: start timer service on attach
The timer service was started when the interface was brought up and it
was stopped when it was brought down. Since ena_up requires the device
to be responsive, triggering the reset would become impossible if the
device became unresponsive with the interface down.
Since most of the functions in timer service already perform the check
to see if the device is running, this only requires starting the callout
in attach and stopping it when bringing the interface up or down to
avoid race between different admin queue calls.
Since callout functions for timer service are always called with the
same arguments, replace callout_{init,reset,drain} calls with
ENA_TIMER_{INIT,RESET,DRAIN} macros.
Submitted by: Dawid Gorecki <[email protected]>
Obtained from: Semihalf
MFC after: 2 weeks
Sponsored by: Amazon, Inc.
(cherry picked from commit 78554d0c707c9401dbae53fb8bc65d291a5a09a5)
show more ...
|
| dbc92a6d | 03-Jan-2022 |
Artur Rojek <[email protected]> |
ena: rework tx req_id validation logic
Since `ena_com_tx_comp_req_id_get` already checks for `req_id` validity,
the logic was exiting early, never giving `validate_tx_req_id` a chance
to trigger dev
ena: rework tx req_id validation logic
Since `ena_com_tx_comp_req_id_get` already checks for `req_id` validity,
the logic was exiting early, never giving `validate_tx_req_id` a chance
to trigger device reset.
Rewrite the logic so that device reset is called based on return value
of `ena_com_tx_comp_req_id_get` instead.
Submitted by: Artur Rojek <[email protected]>
Obtained from: Semihalf
MFC after: 2 weeks
Sponsored by: Amazon, Inc.
(cherry picked from commit b168d0c850f34e8178d815ba3a87de2657a02ca9)
show more ...
|
| eff45c1e | 03-Jan-2022 |
Dawid Gorecki <[email protected]> |
ena: properly handle IPv6 L4 checksum offload
ena_tx_csum function did not check if IPv6 checksum offload was
requested it only checked checksum offloading flags for IPv4 packets.
Because of that, w
ena: properly handle IPv6 L4 checksum offload
ena_tx_csum function did not check if IPv6 checksum offload was
requested it only checked checksum offloading flags for IPv4 packets.
Because of that, when encountering CSUM_IP6_* flags, the function simply
returned without actually setting checksum offloading in ena_ctx.
Check CUSM_IP6_* flags to enable IPv6 checksum offload.
Additionally, only IPv4 header was being parsed regardless of EtherType
field, because of that, value of L4 protocol read when actually trying
to send IPv6 packets was wrong. Use ip6_lasthdr function to get length
of all IPv6 headers and payload protocol.
Set the DF flag to 1 in order to allow the device to offload the IPv6
checksum calculation and achieve optimal performance.
Add CSUM6_OFFLOAD and CSUM_OFFLOAD definitions into ena_datapath.h.
Submitted by: Dawid Gorecki <[email protected]>
Obtained from: Semihalf
MFC after: 2 weeks
Sponsored by: Amazon, Inc.
(cherry picked from commit 2bbef9d95dc10a69a3c5813a517f6e8fe583539a)
show more ...
|
| 112c1187 | 23-Jul-2021 |
Marcin Wojtas <[email protected]> |
Upgrade ENA to v2.4.1
ena: Remove redundant declaration of ena_log_level.
GCC6 raises a -Wredundant-decl error due to duplicate declarations
in ena_fbsd_log.h and ena_plat.h.
Sponsored by: Chelsio
Upgrade ENA to v2.4.1
ena: Remove redundant declaration of ena_log_level.
GCC6 raises a -Wredundant-decl error due to duplicate declarations
in ena_fbsd_log.h and ena_plat.h.
Sponsored by: Chelsio Communications
(cherry picked from commit 8843787aa1bdbd10de6ba47a04489179ec2d2d3c)
ena: Avoid unnecessary mbuf collapses for LLQ condition
In case of Low-latency Queue, one small enough descriptor can be pushed
directly to the ENA hw, thus saving one fragment. Check for this
condition before performing collapse.
Obtained from: Semihalf
MFC after: 2 weeks
Sponsored by: Amazon, Inc.
(cherry picked from commit c81f8c26115a64b9a97ecdb2a64e824dd839ee73)
ena: Trigger reset on ena_com_prepare_tx failure
All ena_com_prepare_tx errors other than ENA_COM_NO_MEM are fatal and
require device reset.
Obtained from: Semihalf
MFC after: 2 weeks
Sponsored by: Amazon, Inc.
(cherry picked from commit 36130d2979d695dd439bc607feb00dcdb9a1937b)
ena: Prevent reset after device destruction
Check for ENA_FLAG_TRIGGER_RESET inside a locked context in order to
avoid potential race conditions with ena_destroy_device. This aligns the
reset task logic with the Linux driver.
Obtained from: Semihalf
MFC after: 2 weeks
Sponsored by: Amazon, Inc.
(cherry picked from commit 433ab9b6987b42b3e5b25b8b5dc7e5178c7ef9bb)
ena: Add extra log messages
Stay aligned with the Linux driver by adding the following logs:
* inform the user about retrying queue creation
* warn on non-empty ena_tx_buffer.mbuf prior to ena_tx_map_mbuf
Obtained from: Semihalf
MFC after: 2 weeks
Sponsored by: Amazon, Inc.
(cherry picked from commit 77160654a162b5faa8ad7a02e18d2bef2589f868)
ena: Add locking assertions
ENA silently assumed that ena_up, ena_down and ena_start_xmit routines
should be called within locked context. Driver's logic heavily assumes
on concurrent access to those routines, so for safety and better
documentation about this assumption, the locking assertions were added
to the above functions.
The assertion was added only for the main steps (skipping the helper
functions) which can be called from multiple places including the kernel
and the driver itself.
Obtained from: Semihalf
MFC after: 2 weeks
Sponsored by: Amazon, Inc.
(cherry picked from commit cb98c439d66c303353a9f4abbbe9ddb51559c638)
ena: Move RSS logic into its own source files
Delegate RSS related functionality into separate .c/.h files in
preparation for the full RSS support.
While at it, reorder functions and remove prototypes for ones with
internal linkage.
Obtained from: Semihalf
MFC after: 2 weeks
Sponsored by: Amazon, Inc.
(cherry picked from commit 986e7b9227668caf9620f207e3c1d708c87b634d)
ena: Disable meta descriptor caching for netmap
If LLQ is being used, `ena_tx_ctx.meta_valid` must stay enabled. This
fixes netmap support on latest generation ENA HW and aligns it with the
core driver behavior.
As netmap doesn't support any csum offloads, the
`adapter->disable_meta_caching` value can be simply passed to the HW.
Obtained from: Semihalf
MFC after: 2 weeks
Sponsored by: Amazon, Inc.
(cherry picked from commit a831466830de6ab55fc03170290b313157196e81)
ena: Share ena_global_lock between driver instances
In order to use `ena_global_lock` in sysctl context, it must be kept
outside the driver instance's software context, as sysctls can be called
before attach and after detach, leading to lock use before sx_init and
after sx_destroy otherwise.
Solve this issue by turning `ena_global_lock` into a file scope
variable, shared between all instances of the driver and associated
sysctl context, and in turn initialized/destroyed in dedicated
SYSINIT/SYSUNINIT functions.
As a side effect, this change also fixes existing race in the reset
routine, when simultaneously accessing sysctl exposed properties.
Obtained from: Semihalf
MFC after: 2 weeks
Sponsored by: Amazon, Inc.
(cherry picked from commit 07aff471c0de2de9a1dc5c7749c46b525bdd0201)
ena: Add missing statistics
Provide the following sysctl statistics in order to stay aligned with
the Linux driver:
* rx_ring.csum_good
* tx_ring.unmask_interrupt_num
Also rename the 'bad_csum' statistic name to 'csum_bad' for alignment.
Obtained from: Semihalf
MFC after: 2 weeks
Sponsored by: Amazon, Inc.
(cherry picked from commit 223c8cb12e951c63807300a0cbdc4a1569520b4b)
ena: Implement full RSS reconfiguration
Bind RX/TX queues and MSI-X vectors to matching CPUs based on the RSS
bucket entries.
Introduce sysctls for the following RSS functionality:
- rss.indir_table: indirection table mapping
- rss.indir_table_size: indirection table size
- rss.key: RSS hash key (if Toeplitz used)
Said sysctls are only available when compiled without `option RSS`, as
kernel-side RSS support currently doesn't offer RSS reconfiguration.
Migrate the hash algorithm from CRC32 to Toeplitz and change the initial
hash value to 0x0 in order to match the standard Toeplitz implementation.
Provide helpers for hash key inversion required for HW operations.
Obtained from: Semihalf
MFC after: 2 weeks
Sponsored by: Amazon, Inc.
(cherry picked from commit 6d1ef2abd330fac4057f092abbbdc28a568b4327)
ena: fix building in-kernel driver
When building ENA as compiled into the kernel, the driver would fail to
build. Resolve the problem by introducing the following changes:
1. Add missing `ena_rss.c` entry in `sys/conf/files`.
2. Prevent SYSCTL_ADD_INT from throwing an assert due to an extra
CTLTYPE_INT flag.
Fixes: 986e7b92276 ("ena: Move RSS logic into its own source files")
Fixes: 6d1ef2abd33 ("ena: Implement full RSS reconfiguration")
Obtained from: Semihalf
Sponsored by: Amazon, Inc.
MFC after: 1 week
(cherry picked from commit a3f0d18237bdcf272461d3b4b682de384c572144)
ena: Update driver version to v2.4.1
Some of the changes in this release:
* Hardware RSS hash key reconfiguration and indirection table
reconfiguration support.
* Full kernel RSS support.
* Extra statistic counters.
* Netmap support for ENAv3.
* Locking assertions.
* Extra log messages.
* Reset handling fixes.
Obtained from: Semihalf
MFC after: 2 weeks
Sponsored by: Amazon, Inc.
(cherry picked from commit 42c7760be3ea420668f625f2064ae347aa7e818e)
show more ...
|
| 7dee315e | 18-Nov-2020 |
Marcin Wojtas <[email protected]> |
Update ENA driver version to v2.3.0
The v2.3.0 introduces new ena_com layer, ENI metrics updates and SPDX
license tags.
Submitted by: Michal Krawczyk <[email protected]>
Obtained from: Semihalf
Sp
Update ENA driver version to v2.3.0
The v2.3.0 introduces new ena_com layer, ENI metrics updates and SPDX
license tags.
Submitted by: Michal Krawczyk <[email protected]>
Obtained from: Semihalf
Sponsored by: Amazon, Inc
MFC after: 1 week
Differential revision: https://reviews.freebsd.org/D27120
show more ...
|
| 7d2e6f20 | 18-Nov-2020 |
Marcin Wojtas <[email protected]> |
Rename descriptions of the supported ENA devices
Some of the PCI ID were described as ENA with LLQ support - it's not
fully accurate and because of that, their names were changed.
Instead of LLQ, u
Rename descriptions of the supported ENA devices
Some of the PCI ID were described as ENA with LLQ support - it's not
fully accurate and because of that, their names were changed.
Instead of LLQ, use RSERV0 for the description of those devices.
Submitted by: Michal Krawczyk <[email protected]>
Obtained from: Semihalf
Sponsored by: Amazon, Inc
MFC after: 1 week
Differential revision: https://reviews.freebsd.org/D27119
show more ...
|
| f180142c | 18-Nov-2020 |
Marcin Wojtas <[email protected]> |
Add ENI metrics for the ENA driver
The new HAL allows the driver to read extra ENI stats. Exact meaning of
each of them can be found in base/ena_defs/ena_admin_defs.h file and
structure ena_admin_en
Add ENI metrics for the ENA driver
The new HAL allows the driver to read extra ENI stats. Exact meaning of
each of them can be found in base/ena_defs/ena_admin_defs.h file and
structure ena_admin_eni_stats.
Those stats are being updated inside of the timer service, which is
executed every second.
ENI metrics are turned off by default. They can be enabled, using the
sysctl node: dev.ena.X.eni_metrics.update_delay
0 value in this node means that the update is turned off. Other values
determine how many seconds must pass, before ENI metrics will be
updated.
They can be acquired, using sysctl:
sysctl dev.ena.X.eni_metrics
Where X stands for the interface number.
Submitted by: Michal Krawczyk <[email protected]>
Obtained from: Semihalf
Sponsored by: Amazon, Inc
MFC after: 1 week
Differential revision: https://reviews.freebsd.org/D27118
show more ...
|
| 0835cc78 | 18-Nov-2020 |
Marcin Wojtas <[email protected]> |
Add SPDX license tag to the ENA driver files
Refering to guide: https://wiki.freebsd.org/SPDX the SPDX tag should not
replace the standard license text, however it should be added over the
standard
Add SPDX license tag to the ENA driver files
Refering to guide: https://wiki.freebsd.org/SPDX the SPDX tag should not
replace the standard license text, however it should be added over the
standard license text to make the automation easier.
Because of that, the old license was kept, but the SPDX tag was added
on top of every ENA driver file.
Submited by: Michal Krawczyk <[email protected]>
Obtained from: Semihalf
Sponsored by: Amazon, Inc
MFC after: 1 week
Differential revision: https://reviews.freebsd.org/D27117
show more ...
|
| c7444389 | 18-Nov-2020 |
Marcin Wojtas <[email protected]> |
Add Rx offsets support for the ENA driver
For the first descriptor in a chain the data may start at an offset.
It is optional feature of some devices, so the driver must ack that
it supports it.
Th
Add Rx offsets support for the ENA driver
For the first descriptor in a chain the data may start at an offset.
It is optional feature of some devices, so the driver must ack that
it supports it.
The data pointer of the mbuf is simply shifted by the given value.
Submitted by: Maciej Bielski <[email protected]>
Submitted by: Michal Krawczyk <[email protected]>
Obtained from: Semihalf
Sponsored by: Amazon, Inc
MFC after: 1 week
Differential revision: https://reviews.freebsd.org/D27116
show more ...
|
| 2287afd8 | 26-May-2020 |
Marcin Wojtas <[email protected]> |
Update ENA driver version to v2.2.0
Driver version upgrade is connected with support for the new device
fetures, like Tx drops reporting or disabling meta caching.
Moreover, the driver configuratio
Update ENA driver version to v2.2.0
Driver version upgrade is connected with support for the new device
fetures, like Tx drops reporting or disabling meta caching.
Moreover, the driver configuration from the sysctl was reworked to
provide safer and better flow for configuring:
* number of IO queues (new feature),
* drbr size on Tx,
* Rx queue size.
Moreover, a lot of minor bug fixes and improvements were added.
Copyright date in the license of the modified files in this release was
updated to 2020.
Submitted by: Michal Krawczyk <[email protected]>
Obtained from: Semihalf
Sponsored by: Amazon, Inc.
show more ...
|
| 7381a86f | 26-May-2020 |
Marcin Wojtas <[email protected]> |
Refactor ena_tx_map_mbuf() function
There is no guarantee from bus_dmamap_load_mbuf_sg() for matching
mbuf chain segments to dma physical segments.
This patch ensure correctly mapping to LLQ header
Refactor ena_tx_map_mbuf() function
There is no guarantee from bus_dmamap_load_mbuf_sg() for matching
mbuf chain segments to dma physical segments.
This patch ensure correctly mapping to LLQ header and DMA segments.
Submitted by: Ido Segev <[email protected]>
Obtained from: Amazon, Inc.
show more ...
|
| 9bf7da95 | 26-May-2020 |
Marcin Wojtas <[email protected]> |
Fix double-free bug within ena_detach()
There is ena_free_all_io_rings_resources() called twice on device
detach:
ena_detach():
ena_destroy_device():
/* First call */
ena_free_all_io_rings_resourc
Fix double-free bug within ena_detach()
There is ena_free_all_io_rings_resources() called twice on device
detach:
ena_detach():
ena_destroy_device():
/* First call */
ena_free_all_io_rings_resources()
/* Second call */
ena_free_all_io_rings_resources()
The double-free causes panic() on kldunload, for example.
As the ena_destroy_device() is also called by ena_reset_task() it is
better to stay unchanged. Thus, remove the "Second call" of the function.
Submitted by: Maciej Bielski <[email protected]>
Obtained from: Semihalf
Sponsored by: Amazon, Inc.
show more ...
|
| 0b432b70 | 26-May-2020 |
Marcin Wojtas <[email protected]> |
Allow disabling meta caching for ENA Tx path
Determined by a flag passed from the device. No metadata is set within
ena_tx_csum when caching is disabled.
Submitted by: Maciej Bielski <mba@semihalf
Allow disabling meta caching for ENA Tx path
Determined by a flag passed from the device. No metadata is set within
ena_tx_csum when caching is disabled.
Submitted by: Maciej Bielski <[email protected]>
Obtained from: Semihalf
Sponsored by: Amazon, Inc.
show more ...
|
| 9762a033 | 26-May-2020 |
Marcin Wojtas <[email protected]> |
Create ENA IO queues with optional backoff
If requested size of IO queues is not supported try to decrease it until
finding the highest value that can be satisfied.
Submitted by: Maciej Bielski <m
Create ENA IO queues with optional backoff
If requested size of IO queues is not supported try to decrease it until
finding the highest value that can be satisfied.
Submitted by: Maciej Bielski <[email protected]>
Obtained from: Semihalf
Sponsored by: Amazon, Inc.
show more ...
|
| 56d41ad5 | 26-May-2020 |
Marcin Wojtas <[email protected]> |
Add sysctl node for ENA IO queues number adjustment
By default, in ena_attach() the driver attempts to acquire
ena_adapter::max_num_io_queues MSI-X vectors for the purpose of IO
queues, however this
Add sysctl node for ENA IO queues number adjustment
By default, in ena_attach() the driver attempts to acquire
ena_adapter::max_num_io_queues MSI-X vectors for the purpose of IO
queues, however this is not guaranteed. The number of vectors acquired
depends also on system resources availability.
Regardless of that, enable the number of effectively used IO queues to
be further limited through the sysctl node.
Example: Assumming that there are 8 IO queues configured by default, the
command
$ sysctl dev.ena.0.io_queues_nb=4
will reduce the number of available IO queues to 4. Similarly, the value
can be also increased up to maximum supported value. A value higher than
maximum supported number of IO queues is ignored. Zero is ignored too.
Submitted by: Maciej Bielski <[email protected]>
Obtained from: Semihalf
Sponsored by: Amazon, Inc.
show more ...
|
| e2735b09 | 26-May-2020 |
Marcin Wojtas <[email protected]> |
Fix assumptions about number of IO queues in the ENA
Make the ena_adapter::num_io_queues a number of effectively used IO
queues. While the ena_adapter::max_num_io_queues is an upper-bound
specified
Fix assumptions about number of IO queues in the ENA
Make the ena_adapter::num_io_queues a number of effectively used IO
queues. While the ena_adapter::max_num_io_queues is an upper-bound
specified by the HW, the ena_adapter::num_io_queues may be lower than
that, depending on runtime system resources availability.
On reset, there are called ena_destroy_device() and then
ena_restore_device(). The latter calls, in turn, ena_enable_msix(),
which will attempt to re-acquire ena_adapter::max_num_io_queues of
MSIX vectors again.
Thus, the value of ena_adapter::num_io_queues may be different before
and after reset. For this reason, free the IO rings structures (drbr,
counters) in ena_destroy_device() and allocate again in
ena_restore_device().
Submitted by: Maciej Bielski <[email protected]>
Obtained from: Semihalf
Sponsored by: Amazon, Inc.
show more ...
|