net/ena: make Tx completion timeout configurable

The default missing Tx completion timeout was set to 5 seconds.
In order to provide users with the interface to control this timeout
to adjust it wit

net/ena: make Tx completion timeout configurable

The default missing Tx completion timeout was set to 5 seconds.
In order to provide users with the interface to control this timeout
to adjust it with the application's watchdog, the device argument for
controlling this value was added.

The parameter is called 'miss_txc_to' and can be modified using the
devargs interface:

./app -a <bdf>,miss_txc_to=UINT_NUMBER

This parameter accepts values from 0 to 60 and indicates number of
seconds after which the Tx packet will be considered as missing.

HW hints for the Tx completions timeout were removed to do not overwrite
parameter from the user. Also specifying default Tx completion timeout
value was moved from the configuration to init phase in order to
simplify default value assignment.

Signed-off-by: Michal Krawczyk <[email protected]>
Reviewed-by: Dawid Gorecki <[email protected]>
Reviewed-by: Shai Brandes <[email protected]>

show more ...

net/ena/base: make IO memzone unique per port

Originally, the ena_com memzone counter was shared by ports, which
caused the memzones to be harder to identify and could potentially
lead to race and b

net/ena/base: make IO memzone unique per port

Originally, the ena_com memzone counter was shared by ports, which
caused the memzones to be harder to identify and could potentially
lead to race and because of that the counter had to be atomic.

This atomic counter was global variable and it couldn't work in the
multiprocess implementation.

The memzone is now being identified by the local to port memzone counter
and the port ID - both of those information can be found in the shared
data, so it can be probed easily.

Signed-off-by: Michal Krawczyk <[email protected]>
Reviewed-by: Dawid Gorecki <[email protected]>
Reviewed-by: Shai Brandes <[email protected]>

show more ...

net/ena: proxy AQ calls to primary process

Due to how the ena_com compatibility layer is written, all AQ commands
triggering functions use stack to save results of AQ and then copy them
to user give

net/ena: proxy AQ calls to primary process

Due to how the ena_com compatibility layer is written, all AQ commands
triggering functions use stack to save results of AQ and then copy them
to user given function.
Therefore to keep the compatibility layer common, introduce ENA_PROXY
macro. It either calls the wrapped function directly (in primary
process) or proxies it to the primary via DPDK IPC mechanism. Since all
proxied calls are taken under a lock share the result data through
shared memory (in struct ena_adapter) to work around 256B IPC parameter
size limit.

New proxy calls can be added by
1. Adding a new message type at the end of enum ena_mp_req
2. Adding new message arguments to the struct ena_mp_body if needed
3. Defining proxy request descriptor with ENA_PROXY_DESC. Its arguments
include handlers for request preparation and response processing.
Any of those may be empty (aside of marking arguments as used).
4. Adding request handling logic to ena_mp_primary_handle()
5. Replacing proxied function calls with ENA_PROXY(adapter, <func>, ...)

Signed-off-by: Stanislaw Kardach <[email protected]>
Reviewed-by: Michal Krawczyk <[email protected]>
Reviewed-by: Dawid Gorecki <[email protected]>
Reviewed-by: Shai Brandes <[email protected]>

show more ...

net/ena: make link status change interrupt configurable

ENA uses AENQ for notification about various events, like LSC, keep
alive etc. By default it was enabling all AENQ that were supported by
both

net/ena: make link status change interrupt configurable

ENA uses AENQ for notification about various events, like LSC, keep
alive etc. By default it was enabling all AENQ that were supported by
both the driver and the device. As a result the LSC was always processed
even if the application turned it off explicitly.

As the DPDK provides application with the possibility to configure the
LSC, ENA should respect that. AENQ groups are now being updated upon
configure step, thus LSC can be activated or disabled between ENA PMD
reconfigurations. Moreover, the LSC capability for the device is being
determined dynamically.

Signed-off-by: Michal Krawczyk <[email protected]>
Reviewed-by: Dawid Gorecki <[email protected]>
Reviewed-by: Shai Brandes <[email protected]>

show more ...

net/ena: add extra Rx checksum related xstats

* Split 'bad_csum' Rx statistic into 'l3_csum_bad' and 'l4_csum_bad' to
be able to check which checksum was not calculated properly.
* Add l4_csum_goo

net/ena: add extra Rx checksum related xstats

* Split 'bad_csum' Rx statistic into 'l3_csum_bad' and 'l4_csum_bad' to
be able to check which checksum was not calculated properly.
* Add l4_csum_good statistic, which shows how many times L4 Rx checksum
was properly offloaded.

Signed-off-by: Michal Krawczyk <[email protected]>
Reviewed-by: Dawid Gorecki <[email protected]>
Reviewed-by: Shai Brandes <[email protected]>

show more ...

net/ena: remove unused offload variables

Those variables are being set, but never read. As they seem to be
leftover from the old offloads API and don't have any purpose right
now, they are simply be

net/ena: remove unused offload variables

Those variables are being set, but never read. As they seem to be
leftover from the old offloads API and don't have any purpose right
now, they are simply being removed.

Fixes: a4996bd89c42 ("ethdev: new Rx/Tx offloads API")
Cc: [email protected]

Signed-off-by: Michal Krawczyk <[email protected]>
Reviewed-by: Artur Rojek <[email protected]>
Reviewed-by: Dawid Gorecki <[email protected]>
Reviewed-by: Igor Chauskin <[email protected]>
Reviewed-by: Shai Brandes <[email protected]>

show more ...

net/ena: remove Tx mbuf linearization

The linearization of the mbuf isn't common practice for the PMD, as it
can expose it's capabilities to the upper layer using
rte_eth_dev_info_get().

Moreover,

net/ena: remove Tx mbuf linearization

The linearization of the mbuf isn't common practice for the PMD, as it
can expose it's capabilities to the upper layer using
rte_eth_dev_info_get().

Moreover, the rte_eth_tx_prepare() function should also verify if the
number of segments inside the mbuf isn't too high.

Because of those 2 circumstances, it may be safer to avoid modifying
mbuf on PMD's Tx side and remove linearization at all. Instead, add
verification of the number of segments to the eth_ena_prep_pkts().

Signed-off-by: Michal Krawczyk <[email protected]>
Reviewed-by: Artur Rojek <[email protected]>
Reviewed-by: Dawid Gorecki <[email protected]>
Reviewed-by: Igor Chauskin <[email protected]>
Reviewed-by: Shai Brandes <[email protected]>

show more ...

fix spelling in comments and strings

The tool comes from https://github.com/jsoref

Signed-off-by: Josh Soref <[email protected]>
Signed-off-by: Thomas Monjalon <[email protected]>

ethdev: add namespace

Add 'RTE_ETH' namespace to all enums & macros in a backward compatible
way. The macros for backward compatibility can be removed in next LTS.
Also updated some struct names to

ethdev: add namespace

Add 'RTE_ETH' namespace to all enums & macros in a backward compatible
way. The macros for backward compatibility can be removed in next LTS.
Also updated some struct names to have 'rte_eth' prefix.

All internal components switched to using new names.

Syntax fixed on lines that this patch touches.

Signed-off-by: Ferruh Yigit <[email protected]>
Acked-by: Tyler Retzlaff <[email protected]>
Acked-by: Andrew Rybchenko <[email protected]>
Acked-by: Ajit Khaparde <[email protected]>
Acked-by: Jerin Jacob <[email protected]>
Acked-by: Wisam Jaddo <[email protected]>
Acked-by: Rosen Xu <[email protected]>
Acked-by: Chenbo Xia <[email protected]>
Acked-by: Hemant Agrawal <[email protected]>
Acked-by: Somnath Kotur <[email protected]>

show more ...

net/ena: check missing Tx completions

In some cases Tx descriptors may be uncompleted by the HW and as a
result they will never be released.

This patch adds checking for the missing Tx completions

net/ena: check missing Tx completions

In some cases Tx descriptors may be uncompleted by the HW and as a
result they will never be released.

This patch adds checking for the missing Tx completions to the ENA timer
service, so in order to use this feature, the application must call the
function rte_timer_manage().

Missing Tx completion reset threshold is determined dynamically, by
taking into consideration ring size and the default value.

Tx cleanup is associated with the Tx burst function. As DPDK
applications can call Tx burst function dynamically, time when last
cleanup was called must be traced to avoid false detection of the
missing Tx completion.

Signed-off-by: Michal Krawczyk <[email protected]>
Reviewed-by: Igor Chauskin <[email protected]>
Reviewed-by: Shai Brandes <[email protected]>

show more ...

net/ena: support Tx/Rx free thresholds

The caller can pass Tx or Rx free threshold value to the configuration
structure for each ring. It determines when the Tx/Rx function should
start cleaning up/

net/ena: support Tx/Rx free thresholds

The caller can pass Tx or Rx free threshold value to the configuration
structure for each ring. It determines when the Tx/Rx function should
start cleaning up/refilling the descriptors. ENA was ignoring this value
and doing it's own calculations.

Now the user can configure ENA's behavior using this parameter and if
this variable won't be set, the ENA will continue with the old behavior
and will use it's own threshold value.

The default value is not provided by the ENA in the ena_infos_get(), as
it's being determined dynamically, depending on the requested ring size.

Note that NULL check for Tx conf was removed from the function
ena_tx_queue_setup(), as at this place the configuration will be
either provided by the user or the default config will be used and it's
handled by the upper (rte_ethdev) layer.

Tx threshold shouldn't be used for the Tx cleanup budget as it can be
inadequate to the used burst. Now the PMD tries to release mbufs for the
ring until it will be depleted.

Signed-off-by: Michal Krawczyk <[email protected]>
Reviewed-by: Igor Chauskin <[email protected]>
Reviewed-by: Shai Brandes <[email protected]>

show more ...

net/ena: fix offload capabilities verification

ENA PMD has multiple checksum offload flags, which are more discrete
than the DPDK offload capabilities flags.
As the driver wasn't storing it's intern

net/ena: fix offload capabilities verification

ENA PMD has multiple checksum offload flags, which are more discrete
than the DPDK offload capabilities flags.
As the driver wasn't storing it's internal checksum offload capabilities
and was relying only on the DPDK capabilities, not all scenarios could
be properly covered (like when to prepare pseudo header checksum and
when not).

Moreover, the user could request offload capability, which isn't
supported by the HW and the PMD would quietly ignore the issue.

This commit reworks eth_ena_prep_pkts() function to perform additional
checks and to properly reflect the HW requirements. With the
RTE_LIBRTE_ETHDEV_DEBUG enabled, the function will do even more
verifications, to help the user find any issues with the mbuf
configuration.

Fixes: b3fc5a1ae10d ("net/ena: add Tx preparation")
Cc: [email protected]

Signed-off-by: Michal Krawczyk <[email protected]>
Reviewed-by: Igor Chauskin <[email protected]>
Reviewed-by: Shai Brandes <[email protected]>

show more ...

net/ena: rework RSS configuration

Allow user to specify his own hash key and hash ctrl if the
device is supporting that. HW interprets the key in reverse byte order,
so the PMD reorders the key befo

net/ena: rework RSS configuration

Allow user to specify his own hash key and hash ctrl if the
device is supporting that. HW interprets the key in reverse byte order,
so the PMD reorders the key before passing it to the ena_com layer.

Default key is being set in random matter each time the device is being
initialized.

Moreover, make minor adjustments for reta size setting in terms
of returning error values.

RSS code was moved to ena_rss.c file to improve readability.

Signed-off-by: Michal Krawczyk <[email protected]>
Reviewed-by: Shai Brandes <[email protected]>
Reviewed-by: Shay Agroskin <[email protected]>
Reviewed-by: Amit Bernstein <[email protected]>

show more ...

net/ena: make ethdev references multi-process safe

rte_pci_device and rte_eth_dev are process-local structures. Therefore
ena_adapter::pdev and ena_adapter::rte_dev cannot be used universally.

Both

net/ena: make ethdev references multi-process safe

rte_pci_device and rte_eth_dev are process-local structures. Therefore
ena_adapter::pdev and ena_adapter::rte_dev cannot be used universally.

Both ena_timer_wd_callback and ena_interrupt_handler_rte needs access to
the rte_eth_dev, but as they are being setup and executed in the primary
process, it is safe to pass there the same pointer, which is used for
the device configuration.

In all other cases, except the eth_ena_dev_init(), the rte_eth_dev_data
is used instead.

Signed-off-by: Stanislaw Kardach <[email protected]>
Signed-off-by: Michal Krawczyk <[email protected]>
Reviewed-by: Igor Chauskin <[email protected]>
Reviewed-by: Shay Agroskin <[email protected]>

show more ...

net/ena: prevent double doorbell

Add per-tx-ring flag for packets that were pushed to HW but await
doorbell. That is to prevent a situation when a doorbell is sent due to
reaching Tx burst threshold

net/ena: prevent double doorbell

Add per-tx-ring flag for packets that were pushed to HW but await
doorbell. That is to prevent a situation when a doorbell is sent due to
reaching Tx burst threshold and next send fails (e.g., due to queue
full). In such case we shouldn't send another doorbell because there are
no actual packets waiting for transmission.

Fixes: c7519ea5eb8d ("net/ena: call additional doorbells if needed")
Cc: [email protected]

Signed-off-by: Igor Chauskin <[email protected]>
Reviewed-by: Michal Krawczyk <[email protected]>

show more ...

net/ena: expose ENI stats as additional xstats

New HAL allows driver to read extra ENI stats. Exact meaning of each of
them can be found in base/ena_defs/ena_admin_defs.h file and structure
ena_admi

net/ena: expose ENI stats as additional xstats

New HAL allows driver to read extra ENI stats. Exact meaning of each of
them can be found in base/ena_defs/ena_admin_defs.h file and structure
ena_admin_eni_stats.

The ena_eni_stats structure is exactly the same as ena_admin_eni_stats,
but it was required to be added for compatibility with xstats macros.

Reading ENI stats requires communication with the admin queue.

Signed-off-by: Michal Krawczyk <[email protected]>
Reviewed-by: Igor Chauskin <[email protected]>
Reviewed-by: Guy Tzalik <[email protected]>
Reviewed-by: Stephen Hemminger <[email protected]>

show more ...

net/ena: lock dynamic usages of admin queue

There are some cases, where the admin queue commands after the
configuration phase finished - for example, the application could ask
for the driver statis

net/ena: lock dynamic usages of admin queue

There are some cases, where the admin queue commands after the
configuration phase finished - for example, the application could ask
for the driver statistics from multiple cores at once.

As by the design, the admin queue is not multithread safe, the spinlock
was added to protect all usages of the admin queue after the
configuration is done.

Signed-off-by: Michal Krawczyk <[email protected]>
Reviewed-by: Igor Chauskin <[email protected]>
Reviewed-by: Guy Tzalik <[email protected]>
Reviewed-by: Stephen Hemminger <[email protected]>

show more ...

net/ena: use macros for ring index operations

To improve code readability, abstraction was added for operating on IO
rings indexes.

Driver was defining local variable for ring mask in each function

net/ena: use macros for ring index operations

To improve code readability, abstraction was added for operating on IO
rings indexes.

Driver was defining local variable for ring mask in each function that
needed to operate on the ring indexes. Now it is being stored in the
ring as this value won't change unless size of the ring will change and
macros for advancing indexes using the mask has been added.

Signed-off-by: Michal Krawczyk <[email protected]>
Reviewed-by: Igor Chauskin <[email protected]>
Reviewed-by: Guy Tzalik <[email protected]>

show more ...

net/ena: limit refill threshold by fixed value

Divider used for both Tx and Rx cleanup/refill threshold can cause too
big delay in case of the really big rings - for example if the 8k Rx
ring will b

net/ena: limit refill threshold by fixed value

Divider used for both Tx and Rx cleanup/refill threshold can cause too
big delay in case of the really big rings - for example if the 8k Rx
ring will be used, the refill won't trigger unless 1024 threshold will
be reached. It will also cause driver to try to allocate that much
descriptors.

Limiting it by fixed value - 256 in that case, would limit maximum
time spent in repopulate function.

Signed-off-by: Michal Krawczyk <[email protected]>
Reviewed-by: Igor Chauskin <[email protected]>
Reviewed-by: Guy Tzalik <[email protected]>

show more ...

net/ena: refactor Rx

* Split main Rx function into multiple ones - the body of the main
was very big and further there were 2 nested loops, which were
making the code hard to read
* Rework how t

net/ena: refactor Rx

* Split main Rx function into multiple ones - the body of the main
was very big and further there were 2 nested loops, which were
making the code hard to read
* Rework how the Rx mbuf chains are being created - Instead of having
while loop which has conditional check if it's first segment, handle
this segment outside the loop and if more fragments are existing,
process them inside.
* Initialize Rx mbuf using simple function - it's the common thing for
the 1st and next segments.
* Create structure for Rx buffer to align it with Tx path, other ENA
drivers and to make the variable name more descriptive - on DPDK, Rx
buffer must hold only mbuf, so initially array of mbufs was used as
the buffers. However, it was misleading, as it was named
"rx_buffer_info". To make it more clear, the structure holding mbuf
pointer was added and now there is possibility to expand it in the
future without reworking the driver.
* Remove redundant variables and conditional checks.

Signed-off-by: Michal Krawczyk <[email protected]>
Reviewed-by: Igor Chauskin <[email protected]>
Reviewed-by: Guy Tzalik <[email protected]>

show more ...

net/ena: disable meta caching

In the LLQ (Low-latency queue) mode, the device can indicate that meta
data descriptor caching is disabled. In that case the driver should send
valid meta descriptor on

net/ena: disable meta caching

In the LLQ (Low-latency queue) mode, the device can indicate that meta
data descriptor caching is disabled. In that case the driver should send
valid meta descriptor on every Tx packet.

Signed-off-by: Michal Krawczyk <[email protected]>
Reviewed-by: Igor Chauskin <[email protected]>
Reviewed-by: Guy Tzalik <[email protected]>

show more ...

net/ena: add Tx drops statistic

ENA device can report in the AENQ handler amount of Tx packets that were
dropped and not sent.

This statistic is showing global value for the device and because
rte_

net/ena: add Tx drops statistic

ENA device can report in the AENQ handler amount of Tx packets that were
dropped and not sent.

This statistic is showing global value for the device and because
rte_eth_stats is missing field that could indicate this value (it
isn't the Tx error), it is being presented as a extended statistic.

As the current design of extended statistics prevents tx_drops from
being an atomic variable and both tx_drops and rx_drops are only updated
from the AENQ handler, both were set as non-atomic for the alignment.

Signed-off-by: Michal Krawczyk <[email protected]>
Reviewed-by: Igor Chauskin <[email protected]>
Reviewed-by: Guy Tzalik <[email protected]>

show more ...

net/ena: support large LLQ headers

Default LLQ (Low-latency queue) maximum header size is 96 bytes and can
be too small for some types of packets - like IPv6 packets with multiple
extension. This ca

net/ena: support large LLQ headers

Default LLQ (Low-latency queue) maximum header size is 96 bytes and can
be too small for some types of packets - like IPv6 packets with multiple
extension. This can be fixed, by using large LLQ headers.

If the device supports larger LLQ headers, the user can activate them by
using device argument 'large_llq_hdr' with value '1'.

If the device isn't supporting this feature, the default value (96B)
will be used.

Signed-off-by: Michal Krawczyk <[email protected]>
Reviewed-by: Igor Chauskin <[email protected]>
Reviewed-by: Guy Tzalik <[email protected]>

show more ...

net/ena: refactor getting IO queues capabilities

Reading values from the device is about the maximum capabilities of the
device. Because of that, the names of the fields storing those values,
functi

net/ena: refactor getting IO queues capabilities

Reading values from the device is about the maximum capabilities of the
device. Because of that, the names of the fields storing those values,
functions and temporary variables, should be more descriptive in order
to improve self documentation of the code.

In connection with this, the way of getting maximum queue size could be
simplified - no hardcoded values are needed, as the device is going to
send it's capabilities anyway.

Signed-off-by: Michal Krawczyk <[email protected]>
Reviewed-by: Igor Chauskin <[email protected]>
Reviewed-by: Guy Tzalik <[email protected]>

show more ...

net/ena: ensure Rx buffer size is at least 1400B

Some of the ENA devices can't handle buffers which are smaller than a
1400B. Because of this limitation, size of the buffer is being checked
and limi

net/ena: ensure Rx buffer size is at least 1400B

Some of the ENA devices can't handle buffers which are smaller than a
1400B. Because of this limitation, size of the buffer is being checked
and limited during the Rx queue setup.

If it's below the allowed value, PMD won't finish it's configuration
successfully..

Signed-off-by: Michal Krawczyk <[email protected]>
Reviewed-by: Igor Chauskin <[email protected]>
Reviewed-by: Guy Tzalik <[email protected]>

show more ...