Fix packet forwarding performance in mlx5en(4) driver.

Approved by: so
Security: FreeBSD-EN-20:05.mlx5en

MFS r353405:
Fix regression issue after r352989:

As noted by the commit message, callouts are now persistant
and should not be in the auto-zero section of the RQ's and SQ's.
This fixes an assert whe

MFS r353405:
Fix regression issue after r352989:

As noted by the commit message, callouts are now persistant
and should not be in the auto-zero section of the RQ's and SQ's.
This fixes an assert when using the TX completion event
factor feature with mlx5en(4).

Found by: gallatin@
Sponsored by: Mellanox Technologies
Approved by: re (gjb)

show more ...

MFS r353184, r353186, r353188, r353190, r353192, r353194, r353196, r353198,
r353200, r353203, r353205, r353207, r353209, r353211, r353213, r353215,
r353217, r353219, r353221, r353223, r353225, r35322

MFS r353184, r353186, r353188, r353190, r353192, r353194, r353196, r353198,
r353200, r353203, r353205, r353207, r353209, r353211, r353213, r353215,
r353217, r353219, r353221, r353223, r353225, r353227, r353229, r353231,
r353233, r353235, r353237, r353239, r353241, r353243, r353245, r353247,
r353249, r353251, r353253, r353255, r353257, r353259, r353261, r353263,
r353265 and r353267:

Update mlx5core, mlx5en(4) and mlx5ib(4).

Sponsored by: Mellanox Technologies
Approved by: re (gjb)

MFC r352956:
Fix reported max SGE calculation in mlx5ib.

Add the 512 bytes limit of RDMA READ and the size of remote address to the max
SGE calculation.

Linux commit: 288c01b746aa

MFC r352957:
Update warning and error print formats in mlx5ib.

MFC r352958:
Make sure the number of IRQ vectors doesn't exceed 256 in mlx5core.
The "intr" field in "struct mlx5_ifc_eqc_bits" is only 8 bits wide.

MFC r352959:
Check return value of mlx5_vector2eqn() function in mlx5en.

MFC r352960:
Fix for missing cleanup code in error case in mlx5en.

MFC r352961:
Implement macro for asserting priv lock in mlx5en.

MFC r352962:
Add support for Multi-Physical Function Switch, MPFS, in mlx5en.

MPFS is a logical switch in the Mellanox device which forward packets
based on a hardware driven L2 address table, to one or more physical-
or virtual- functions. The physical- or virtual- function is required
to tell the MPFS by using the MPFS firmware commands, which unicast
MAC addresses it is requesting from the physical port's traffic.
Broadcast and multicast traffic however, is copied to all listening
physical- and virtual- functions and does not need a rule in the MPFS
switching table.

Linux commit: eeb66cdb682678bfd1f02a4547e3649b38ffea7e

MFC r352963:
Cleanup naming of IRQ vectors in mlx5en.
Remove unused IRQ naming functions and arrays.

MFC r352964:
Export channel IRQ number as part of the "hw_ctx_debug" sysctl(8) in mlx5en(4).

MFC r352965:
Correct and update some counter names in mlx5en(4).

MFC r352966:
Add port module event software counters in mlx5core.
While at it, fixup PME based on latest PRM defines.

MFC r352967:
Make the mlx5_vsc_wait_on_flag(9) function global.

MFC r352968:
Move mlx5_ifc_vsc_space_bits and mlx5_ifc_vsc_addr_bits to mlx5_ifc.h.

MFC r352969:
Use the MLX5_VSC_DOMAIN_SEMAPHORES constant instead of hand-rolled symbol
in mlx5core.

MFC r352970:
Define MLX5_VSC_DOMAIN_SCAN_CRSPACE.

MFC r352971:
Read rege map from crdump scan space in mlx5core.

MFC r352972:
Remove no longer needed fwdump register tables from mlx5core.

MFC r352973:
Add missing blank line at the end of the print in mlx5core.

MFC r352974:
Add proper print in case of 0x0 health syndrome in mlx5core.

In case of health counter fails to increment it indicates a bad device health.
In case when the syndrome indicated by firmware is 0x0, this indicates that
firmware is unable to respond to initialization segment reads.
Add proper print in this case.

MFC r352975:
Unify prints in mlx5core.

All prints in mlx5core should use on of the macros:
mlx5_core_err/dbg/warn

MFC r352976:
Unify prints in mlx5en(4).

All prints in mlx5en(4) should use on of the macros:
mlx5_en_err/dbg/warn

MFC r352977:
Sort the ports registers definitions numerically in mlx5core.

MFC r352978:
Add definition for the Port Buffer Status Register in mlx5core.

MFC r352979:
Update definitons for PPTB and PBMC registers layouts in mlx5core.

MFC r352980:
Add mlx5e_dbg() compatibility macro.

MFC r352981:
Import Linux code to query/set buffer state in mlx5en(4).

MFC r352982:
Add support for buffer parameter manipulations in mlx5en(4).

The following sysctls are added:
dev.mce.N.conf.qos.cable_length
dev.mce.N.conf.qos.buffers_size
dev.mce.N.conf.qos.buffers_prio

MFC r352983 and r353001:
Move EEPROM information query from a sysctl in mlx5en(4) to an ioctl
in mlx5core. The EEPROM information is not only a property of the
mlx5en(4) driver.

MFC r352984:
Add the ability to query the EEPROM information in mlx5tool(8).

MFC r352985:
Add sysctl(8) to get and set forward error correction, FEC, configuration
in mlx5en(4).

MFC r352986:
Return an error from ioctl(MLX5_FW_RESET) if reset was rejected in mlx5core.

MFC r352987:
Remove mkey_be from channel structure in mlx5en(4).
Use value from priv structure instead.
This saves some space in the channel structure.

MFC r352988:
Remove unused cpu field from channel structure in mlx5en(4).

MFC r352989:
Seal transmit path with regards to using destroyed mutex in mlx5en(4).

It may happen during link down that the running state may be observed
non-zero in the transmit routine, right before the running state is
cleared. This may end up using a destroyed mutex.

Make all channel mutexes and callouts persistant.

Preserve receive and send queue statistics during link toggle.

MFC r352991 and 353000:
Wait for FW readiness before initializing command interface in mlx5core.

Before attempting to initialize the command interface we must wait till
the fw_initializing bit is clear.

If we fail to meet this condition the hardware will drop our
configuration, specifically the descriptors page address. This scenario
can happen when the firmware is still executing an FLR flow and did not
finish yet so the driver needs to wait for that to finish.

Linux commits:
6c780a0267b8
b8a92577f4be.

MFC r352992:
Use size_t for byte_to_write variable when comparing to eeprom_info_out_len
which is also size_t in mlx5tool(8).

MFC r352993:
Randomize the delay when waiting for VSC flag in mlx5core.

The PRM suggests random 0 - 10ms to prevent multiple waiters on the same
interval in order to avoid starvation.

MFC r352994:
Improve mlx5_fwdump_prep logging in mlx5core.

MFC r352995:
Only update lossy buffers config when manual PFC configuration was done
in mlx5en(4).

MFC r352996:
Add print to show user a reason for rejecting buffer size change in mlx5en(4).

MFC r352997:
Print numeric error_type and module_status in mlx5core
in case the strings are not available.

MFC r352998:
Bump driver version for mlx5core, mlx5en(4) and mlx5ib(4).

show more ...