sys: Remove $FreeBSD$: one-line sh patternRemove /^\s*#[#!]?\s*\$FreeBSD\$.*$\n/
Add opt_kern_tls.h to the sources from t4_tom.ko.Missed in r353328.Sponsored by: Chelsio Communications
Support for TLS offload of TOE connections on T6 adapters.The TOE engine in Chelsio T6 adapters supports offloading of TLSencryption and TCP segmentation for offloaded connections. Socketsusing
Support for TLS offload of TOE connections on T6 adapters.The TOE engine in Chelsio T6 adapters supports offloading of TLSencryption and TCP segmentation for offloaded connections. Socketsusing TLS are required to use a set of custom socket options to uploadRX and TX keys to the NIC and to enable RX processing. Currentlythese socket options are implemented as TCP options in the vendorspecific range. A patched OpenSSL library will be made available in aport / package for use with the TLS TOE support.TOE sockets can either offload both transmit and reception of TLSrecords or just transmit. TLS offload (both RX and TX) is enabled bysetting the dev.t6nex.<x>.tls sysctl to 1 and requires TOE to beenabled on the relevant interface. Transmit offload can be used onany "normal" or TLS TOE socket by using the custom socket option toprogram a transmit key. This permits most TOE sockets totransparently offload TLS when applications use a patched SSL library(e.g. using LD_LIBRARY_PATH to request use of a patched OpenSSLlibrary). Receive offload can only be used with TOE sockets using theTLS mode. The dev.t6nex.0.toe.tls_rx_ports sysctl can be set to alist of TCP port numbers. Any connection with either a local orremote port number in that list will be created as a TLS socket ratherthan a plain TOE socket. Note that although this sysctl accepts anarbitrary list of port numbers, the sysctl(8) tool is only able to setsysctl nodes to a single value. A TLS socket will hang withoutreceiving data if used by an application that is not using a patchedSSL library. Thus, the tls_rx_ports node should be used with care.For a server mostly concerned with offloading TLS transmit, this nodeis not needed as plain TOE sockets will fall back to software cryptowhen using an unpatched SSL library.New per-interface statistics nodes are added giving counts of TLSpackets and payload bytes (payload bytes do not include TLS headers orauthentication tags/MACs) offloaded via the TOE engine, e.g.:dev.cc.0.stats.rx_tls_octets: 149dev.cc.0.stats.rx_tls_records: 13dev.cc.0.stats.tx_tls_octets: 26501823dev.cc.0.stats.tx_tls_records: 1620TLS transmit work requests are constructed by a new variant oft4_push_frames() called t4_push_tls_records() in tom/t4_tls.c.TLS transmit work requests require a buffer containing IVs. If theIVs are too large to fit into the work request, a separate buffer isallocated when constructing a work request. This buffer is associatedwith the transmit descriptor and freed when the descriptor is ACKed bythe adapter.Received TLS frames use two new CPL messages. The first message is aCPL_TLS_DATA containing the decryped payload of a single TLS record.The handler places the mbuf containing the received payload on anmbufq in the TOE pcb. The second message is a CPL_RX_TLS_CMP messagewhich includes a copy of the TLS header and indicates if there wereany errors. The handler for this message places the TLS header intothe socket buffer followed by the saved mbuf with the payload data.Both of these handlers are contained in tom/t4_tls.c.A few routines were exposed from t4_cpl_io.c for use by t4_tls.cincluding send_rx_credits(), a new send_rx_modulate(), andt4_close_conn().TLS keys for both transmit and receive are stored in onboard memoryin the NIC in the "TLS keys" memory region.In some cases a TLS socket can hang with pending data available in theNIC that is not delivered to the host. As a workaround, TLS socketsare more aggressive about sending CPL_RX_DATA_ACK messages anytime thatany data is read from a TLS socket. In addition, a fallback timer willperiodically send CPL_RX_DATA_ACK messages to the NIC for connectionsthat are still in the handshake phase. Once the connection hasfinished the handshake and programmed RX keys via the socket option,the timer is stopped.A new function select_ulp_mode() is used to determine what sub-mode agiven TOE socket should use (plain TOE, DDP, or TLS). The existingset_tcpddp_ulp_mode() function has been renamed to set_ulp_mode() andhandles initialization of TLS-specific state when necessary inaddition to DDP-specific state.Since TLS sockets do not receive individual TCP segments but alwaysreceive full TLS records, they can receive more data than is availablein the current window (e.g. if a 16k TLS record is received but thesocket buffer is itself 16k). To cope with this, just drop the windowto 0 when this happens, but track the overage and "eat" the overage asit is read from the socket buffer not opening the window (or addingrx_credits) for the overage bytes.Reviewed by: np (earlier version)Sponsored by: Chelsio CommunicationsDifferential Revision: https://reviews.freebsd.org/D14529
show more ...
cxgbe/t4_tom: Per-connection rate limiting for TCP sockets handled bythe TOE. For now this capability is always enabled in kernels withoptions RATELIMIT. t4_tom will check if_capenable once the b
cxgbe/t4_tom: Per-connection rate limiting for TCP sockets handled bythe TOE. For now this capability is always enabled in kernels withoptions RATELIMIT. t4_tom will check if_capenable once the base drivergets code to support rate limiting for any socket (TOE or not).This was tested with iperf3 and netperf ToT as they already supportSO_MAX_PACING_RATE sockopt. There is a bug in firmwares prior to1.16.45.0 that affects the BSD driver only and results in rate-limitingat an incorrect rate. This will resolve by itself as soon as 1.16.45.0or later firmware shows up in the driver.Relnotes: YesSponsored by: Chelsio Communications
sys/modules: normalize .CURDIR-relative paths to SRCTOPThis simplifies make output/logicTested with: `cd sys/modules; make ALL_MODULES=` on amd64MFC after: 1 monthSponsored by: Dell EMC Isilon
Move most of the 15 variations on generating opt_inet.h andopt_inet6.h into kmod.mk by forcing almost everybody to eat the samedogfood. While at it, consolidate the opt_bpf.h and opt_mroute.htarge
Move most of the 15 variations on generating opt_inet.h andopt_inet6.h into kmod.mk by forcing almost everybody to eat the samedogfood. While at it, consolidate the opt_bpf.h and opt_mroute.htargets here too.
List one file per line in the Makefiles. This makes it easier to readdiffs when a file is added or removed.MFC after: 2 weeks
Improve compliance with style.Makefile(5).MFC after: 2 weeks
Use src.opts.mk in preference to bsd.own.mk except where we need stufffrom the latter.
cxgbe(4): Updates to the hardware L2 table management code.- Add full support for IPv6 addresses.- Read the size of the L2 table during attach. Do not assume that PCIe physical function 4 of t
cxgbe(4): Updates to the hardware L2 table management code.- Add full support for IPv6 addresses.- Read the size of the L2 table during attach. Do not assume that PCIe physical function 4 of the card has all of the table to itself.- Use FNV instead of Jenkins to hash L3 addresses and drop the private copy of jhash.h from the driver.MFC after: 1 week
Support for TCP DDP (Direct Data Placement) in the T4 TOE module.Basically, this is automatic rx zero copy when feasible. TCP payload isDMA'd directly into the userspace buffer described by the u
Support for TCP DDP (Direct Data Placement) in the T4 TOE module.Basically, this is automatic rx zero copy when feasible. TCP payload isDMA'd directly into the userspace buffer described by the uio submittedin soreceive by an application.- Works with sockets that are being handled by the TCP offload engine of a T4 chip (you need t4_tom.ko module loaded after cxgbe, and an "ifconfig +toe" on the cxgbe interface).- Does not require any modification to the application.- Not enabled by default. Use hw.t4nex.<X>.toe.ddp="1" to enable it.
Assume INET, INET6, and TCP_OFFLOAD when the driver is built out of tree andKERNBUILDDIR is not set.MFC after: 2 weeks
- Updated TOE support in the kernel.- Stateful TCP offload drivers for Terminator 3 and 4 (T3 and T4) ASICs. These are available as t3_tom and t4_tom modules that augment cxgb(4) and cxgbe(4) r
- Updated TOE support in the kernel.- Stateful TCP offload drivers for Terminator 3 and 4 (T3 and T4) ASICs. These are available as t3_tom and t4_tom modules that augment cxgb(4) and cxgbe(4) respectively. The cxgb/cxgbe drivers continue to work as usual with or without these extra features.- iWARP driver for Terminator 3 ASIC (kernel verbs). T4 iWARP in the works and will follow soon.Build-tested with make universe.30s overview============What interfaces support TCP offload? Look for TOE4 and/or TOE6 in thecapabilities of an interface:# ifconfig -m | grep TOEEnable/disable TCP offload on an interface (just like any other ifnetcapability):# ifconfig cxgbe0 toe# ifconfig cxgbe0 -toeWhich connections are offloaded? Look for toe4 and/or toe6 in theoutput of netstat and sockstat:# netstat -np tcp | grep toe# sockstat -46c | grep toeReviewed by: bz, gnnSponsored by: Chelsio communications.MFC after: ~3 months (after 9.1, and after ensuring MFC is feasible)