MFC r341172, r341270 (both by jhb@).r341172:Move CLIP table handling out of TOM and into the base driver.- Store the clip table in 'struct adapter' instead of in the TOM softc.- Init the clip t
MFC r341172, r341270 (both by jhb@).r341172:Move CLIP table handling out of TOM and into the base driver.- Store the clip table in 'struct adapter' instead of in the TOM softc.- Init the clip table during attach and teardown during detach.- While here, add a dev.<nexus>.<unit>.misc.clip sysctl to dump the CLIP table.This does mean that we update the clip table even if TOE is not enabled,but non-TOE things need the CLIP table anyway.Reviewed by: np, Krishnamraju Eraparaju @ ChelsioSponsored by: Chelsio CommunicationsDifferential Revision: https://reviews.freebsd.org/D18010r341270:Make most of the CLIP code conditional on #ifdef INET6.This fixes builds of kernels without INET6 such as LINT-NOINET6.Reported by: arybchikReviewed by: npSponsored by: Chelsio CommunicationsDifferential Revision: https://reviews.freebsd.org/D18384
show more ...
cxgbe(4): Replace T4_PKT_TIMESTAMP with something slightly less hackish.
cxgbe(4): Add code to deal with the chip's source MAC table (aka SMT).Submitted by: Krishnamraju Eraparaju @ ChelsioSponsored by: Chelsio Communications
cxgbe(4): Allocate offload Tx queues when a card has resourcesprovisioned for NIC_ETHOFLD and the kernel has option RATELIMIT.It is possible to use the chip's offload queues for normal NIC Tx and
cxgbe(4): Allocate offload Tx queues when a card has resourcesprovisioned for NIC_ETHOFLD and the kernel has option RATELIMIT.It is possible to use the chip's offload queues for normal NIC Tx andnot just TOE Tx. The difference is that these queues support out oforder processing of work requests and have a per-"flowid" mechanism fortracking credits between the driver and hardware. This allows Tx forany number of flows bound to different rate limits to be submitted to asingle Tx queue and the work requests for slow flows won't cause HOLblocking for the rest.Sponsored by: Chelsio Communications
cxgbe(4): Move all TCAM filter code into a separate file.Sponsored by: Chelsio Communications
cxgbe(4): Initial import of the "collect" component of Chelsio unifieddebug (cudbg) code, hooked up to the main driver via an ioctl.The ioctl can be used to collect the chip's internal state in a
cxgbe(4): Initial import of the "collect" component of Chelsio unifieddebug (cudbg) code, hooked up to the main driver via an ioctl.The ioctl can be used to collect the chip's internal state in acompressed dump file. These dumps can be decoded with the "view"component of cudbg.Obtained from: Chelsio CommunicationsMFC after: 2 monthsSponsored by: Chelsio Communications
cxgbe(4): Support routines for Tx traffic scheduling.- Create a new file, t4_sched.c, and move all of the code related to traffic management from t4_main.c and t4_sge.c to this file.- Track both
cxgbe(4): Support routines for Tx traffic scheduling.- Create a new file, t4_sched.c, and move all of the code related to traffic management from t4_main.c and t4_sge.c to this file.- Track both Channel Rate Limiter (ch_rl) and Class Rate Limiter (cl_rl) parameters in the PF driver.- Initialize all the cl_rl limiters with somewhat arbitrary default rates and provide routines to update them on the fly.- Provide routines to reserve and release traffic classes.MFC after: 1 monthSponsored by: Chelsio Communications
sys/modules: normalize .CURDIR-relative paths to SRCTOPThis simplifies make output/logicTested with: `cd sys/modules; make ALL_MODULES=` on amd64MFC after: 1 monthSponsored by: Dell EMC Isilon
Add a driver to create VF devices on Chelsio T4/T5 NICs.Chelsio NICs are a bit unique compared to some other NICs in that theyexpose different functionality on different physical functions. Inpa
Add a driver to create VF devices on Chelsio T4/T5 NICs.Chelsio NICs are a bit unique compared to some other NICs in that theyexpose different functionality on different physical functions. Inparticular, PF4 is used to manage the NIC interfaces ('t4nex' and 't5nex').However, PF4 is not able to create VF devices. Instead, VFs are onlysupported by physical functions 0 through 3. This commit adds 't4iov'and 't5iov' drivers that attach to PF0-3.One extra wrinkle is that the iov devices cannot enable SR-IOV until thefirwmare has been initialized by the main PF4 driver. To handle thiscase, a new t4_if kobj interface has been added to permit cross-callsbetween the PF drivers. The PF4 driver notifies sibling drivers when itis fully attached. It also requests sibling drivers to detach before itdetaches. Sibling drivers query the PF4 driver during their attachroutine to see if it is attached. If not, the sibling drivers defertheir attach actions until the PF4 driver informs them it is attached.VF devices are associated with a single port on the NIC. VF devicescreated from PF0 are associated with the first port on the NIC, VFsfrom PF1 are associated with the second port, etc. VF devices canonly be created from a PF device that has an associated port. Thus,on a 2-port card, VFs are only supported on PF0 and PF1.Reviewed by: np (earlier versions)MFC after: 1 monthSponsored by: Chelsio Communications
Add a 'show t4 tcb <nexus> <tid>' command to dump a TCB from DDB.This allows the contents of a TCB to be extracted from a T4/T5 card inDDB after a panic.
cxgbe(4): support for the kernel RSS option.You need PCBGROUP and RSS in the kernel config to use this.Relnotes: YesSponsored by: Chelsio Communications
Globally enable -fms-extensions when building kernel with gcc, and removethis option from all modules that enable it theirselves. In C mode -fms-extensions option enables anonymous structs and uni
Globally enable -fms-extensions when building kernel with gcc, and removethis option from all modules that enable it theirselves. In C mode -fms-extensions option enables anonymous structs and unions,allowing us to use this C11 feature in kernel. Of course, clang supportsit without any extra options.Reviewed by: dim
Make sure the compiler flag to get cxgbe(4) to compile with gcc is usedonly when gcc is being used. This is what r277225 should have been.Suggested by: dim@
Make cxgbe(4) buildable with the gcc in base.
cxgbe(4): major tx rework.a) Front load as much work as possible in if_transmit, before any driverlock or software queue has to get involved.b) Replace buf_ring with a brand new mp_ring (multipr
cxgbe(4): major tx rework.a) Front load as much work as possible in if_transmit, before any driverlock or software queue has to get involved.b) Replace buf_ring with a brand new mp_ring (multiproducer ring). Thisis specifically for the tx multiqueue model where one of the if_transmitproducer threads becomes the consumer and other producers carry on asusual. mp_ring is implemented as standalone code and it should bepossible to use it in any driver with tx multiqueue. It also has:- the ability to enqueue/dequeue multiple items. This might become significant if packet batching is ever implemented.- an abdication mechanism to allow a thread to give up writing tx descriptors and have another if_transmit thread take over. A thread that's writing tx descriptors can end up doing so for an unbounded time period if a) there are other if_transmit threads continuously feeding the sofware queue, and b) the chip keeps up with whatever the thread is throwing at it.- accurate statistics about interesting events even when the stats come at the expense of additional branches/conditional code.The NIC txq lock is uncontested on the fast path at this point. I'veleft it there for synchronization with the control events (interfaceup/down, modload/unload).c) Add support for "type 1" coalescing work request in the normal NIC txpath. This work request is optimized for frames with a single item inthe DMA gather list. These are very common when forwarding packets.Note that netmap tx in cxgbe already uses these "type 1" work requests.d) Do not request automatic cidx updates every 32 descriptors. Instead,request updates via bits in individual work requests (still every 32descriptors approximately). Also, request an automatic final updatewhen the queue idles after activity. This means NIC tx reclaim is stillperformed lazily but it will catch up quickly as soon as the queueidles. This seems to be the best middle ground and I'll probably dosomething similar for netmap tx as well.e) Implement a faster tx path for WRQs (used by TOE tx and controlqueues, _not_ by the normal NIC tx). Allow work requests to be writtendirectly to the hardware descriptor ring if room is available. I willconvert t4_tom and iw_cxgbe modules to this faster style gradually.MFC after: 2 months
Move most of the 15 variations on generating opt_inet.h andopt_inet6.h into kmod.mk by forcing almost everybody to eat the samedogfood. While at it, consolidate the opt_bpf.h and opt_mroute.htarge
Move most of the 15 variations on generating opt_inet.h andopt_inet6.h into kmod.mk by forcing almost everybody to eat the samedogfood. While at it, consolidate the opt_bpf.h and opt_mroute.htargets here too.
List one file per line in the Makefiles. This makes it easier to readdiffs when a file is added or removed.MFC after: 2 weeks
Improve compliance with style.Makefile(5).MFC after: 2 weeks
cxgbe(4): netmap support for Terminator 5 (T5) based 10G/40G cards.Netmap gets its own hardware-assisted virtual interface and won't takeover or disrupt the "normal" interface in any way. You can
cxgbe(4): netmap support for Terminator 5 (T5) based 10G/40G cards.Netmap gets its own hardware-assisted virtual interface and won't takeover or disrupt the "normal" interface in any way. You can use bothsimultaneously.For kernels with DEV_NETMAP, cxgbe(4) carves out an ncxl<N> interface(note the 'n' prefix) in the hardware to accompany each cxl<N>interface. These two ifnet's per port share the same wire but reallyare separate interfaces in the hardware and software. Each gets its ownL2 MAC addresses (unicast and multicast), MTU, checksum caps, etc. Youshould run netmap on the 'n' interfaces only, that's what they are for.With this, pkt-gen is able to transmit > 45Mpps out of a single 40G portof a T580 card. 2 port tx is at ~56Mpps total (28M + 28M) as of now.Single port receive is at 33Mpps but this is very much a work inprogress. I expect it to be closer to 40Mpps once done. In any casethe current effort can already saturate multiple 10G ports of a T5 cardat the smallest legal packet size. T4 gear is totally untested.trantor:~# ./pkt-gen -i ncxl0 -f tx -D 00:07:43:ab:cd:ef881.952141 main [1621] interface is ncxl0881.952250 extract_ip_range [275] range is 10.0.0.1:0 to 10.0.0.1:0881.952253 extract_ip_range [275] range is 10.1.0.1:0 to 10.1.0.1:0881.962540 main [1804] mapped 334980KB at 0x801dff000Sending on netmap:ncxl0: 4 queues, 1 threads and 1 cpus.10.0.0.1 -> 10.1.0.1 (00:00:00:00:00:00 -> 00:07:43:ab:cd:ef)881.962562 main [1882] Sending 512 packets every 0.000000000 s881.962563 main [1884] Wait 2 secs for phy reset884.088516 main [1886] Ready...884.088535 nm_open [457] overriding ifname ncxl0 ringid 0x0 flags 0x1884.088607 sender_body [996] start884.093246 sender_body [1064] drop copy885.090435 main_thread [1418] 45206353 pps (45289533 pkts in 1001840 usec)886.091600 main_thread [1418] 45322792 pps (45375593 pkts in 1001165 usec)887.092435 main_thread [1418] 45313992 pps (45351784 pkts in 1000834 usec)888.094434 main_thread [1418] 45315765 pps (45406397 pkts in 1002000 usec)889.095434 main_thread [1418] 45333218 pps (45378551 pkts in 1001000 usec)890.097434 main_thread [1418] 45315247 pps (45405877 pkts in 1002000 usec)891.099434 main_thread [1418] 45326515 pps (45417168 pkts in 1002000 usec)892.101434 main_thread [1418] 45333039 pps (45423705 pkts in 1002000 usec)893.103434 main_thread [1418] 45324105 pps (45414708 pkts in 1001999 usec)894.105434 main_thread [1418] 45318042 pps (45408723 pkts in 1002001 usec)895.106434 main_thread [1418] 45332430 pps (45377762 pkts in 1001000 usec)896.107434 main_thread [1418] 45338072 pps (45383410 pkts in 1001000 usec)...Relnotes: YesSponsored by: Chelsio Communications.
Use src.opts.mk in preference to bsd.own.mk except where we need stufffrom the latter.
Add support for packet-sniffing tracers to cxgbe(4). This works withall T4 and T5 based cards and is useful for analyzing TSO, LRO, TOE, andfor general purpose monitoring without tapping any cxgbe
Add support for packet-sniffing tracers to cxgbe(4). This works withall T4 and T5 based cards and is useful for analyzing TSO, LRO, TOE, andfor general purpose monitoring without tapping any cxgbe or cxl ifnetdirectly.Tracers on the T4/T5 chips provide access to Ethernet frames exactly asthey were received from or transmitted on the wire. On transmit, atracer will capture a frame after TSO segmentation, hw VLAN taginsertion, hw L3 & L4 checksum insertion, etc. It will also captureframes generated by the TCP offload engine (TOE traffic is normallyinvisible to the kernel). On receive, a tracer will capture a framebefore hw VLAN extraction, runt filtering, other badness filtering,before the steering/drop/L2-rewrite filters or the TOE have had a go atit, and of course before sw LRO in the driver.There are 4 tracers on a chip. A tracer can trace only in one direction(tx or rx). For now cxgbetool will set up tracers to capture the first128B of every transmitted or received frame on a given port. This is asmall subset of what the hardware can do. A pseudo ifnet with the samename as the nexus driver (t4nex0 or t5nex0) will be created for tracing.The data delivered to this ifnet is an additional copy made inside thechip. Normal delivery to cxgbe<n> or cxl<n> will be made as usual./* watch cxl0, which is the first port hanging off t5nex0. */# cxgbetool t5nex0 tracer 0 tx0 (watch what cxl0 is transmitting)# cxgbetool t5nex0 tracer 1 rx0 (watch what cxl0 is receiving)# cxgbetool t5nex0 tracer list# tcpdump -i t5nex0 <== all that cxl0 sees and puts on the wireIf you were doing TSO, a tcpdump on cxl0 may have shown you ~64K"frames" with no L3/L4 checksum but this will show you the frames thatwere actually transmitted./* all done */# cxgbetool t5nex0 tracer 0 disable# cxgbetool t5nex0 tracer 1 disable# cxgbetool t5nex0 tracer list# ifconfig t5nex0 destroy
Assume INET, INET6, and TCP_OFFLOAD when the driver is built out of tree andKERNBUILDDIR is not set.MFC after: 2 weeks
cxgbe(4): support for IPv6 TSO and LRO.Submitted by: bz (this is a modified version of that patch)
Fix make buildworld -DMODULES_WITH_WORLDSort opt_ srcs
L2 table code. This is enough to get the T4's switch + L2 rewritefilters working. (All other filters - switch without L2 info rewrite,steer, and drop - were already fully-functional).Some contr
L2 table code. This is enough to get the T4's switch + L2 rewritefilters working. (All other filters - switch without L2 info rewrite,steer, and drop - were already fully-functional).Some contrived examples of "switch" filters with L2 rewriting:# cxgbetool t4nex0 iport 0 dport 80 action switch vlan +9 eport 3Intercept all packets received on physical port 0 with TCP port 80 asdestination, insert a vlan tag with VID 9, and send them out of port 3.# cxgbetool t4nex0 sip 192.168.1.1/32 ivlan 5 action switch \ vlan =9 smac aa:bb:cc:dd:ee:ff eport 0Intercept all packets (received on any port) with source IP address192.168.1.1 and VLAN id 5, rewrite the VLAN id to 9, rewrite source macto aa:bb:cc:dd:ee:ff, and send it out of port 0.MFC after: 1 week
12