1.. SPDX-License-Identifier: BSD-3-Clause 2 Copyright (c) 2015-2017 Atomic Rules LLC 3 All rights reserved. 4 5ARK Poll Mode Driver 6==================== 7 8The ARK PMD is a DPDK poll-mode driver for the Atomic Rules Arkville 9(ARK) family of devices. 10 11More information can be found at the `Atomic Rules website 12<http://atomicrules.com>`_. 13 14Overview 15-------- 16 17The Atomic Rules Arkville product is DPDK and AXI compliant product 18that marshals packets across a PCIe conduit between host DPDK mbufs and 19FPGA AXI streams. 20 21The ARK PMD, and the spirit of the overall Arkville product, 22has been to take the DPDK API/ABI as a fixed specification; 23then implement much of the business logic in FPGA RTL circuits. 24The approach of *working backwards* from the DPDK API/ABI and having 25the GPP host software *dictate*, while the FPGA hardware *copes*, 26results in significant performance gains over a naive implementation. 27 28While this document describes the ARK PMD software, it is helpful to 29understand what the FPGA hardware is and is not. The Arkville RTL 30component provides a single PCIe Physical Function (PF) supporting 31some number of RX/Ingress and TX/Egress Queues. The ARK PMD controls 32the Arkville core through a dedicated opaque Core BAR (CBAR). 33To allow users full freedom for their own FPGA application IP, 34an independent FPGA Application BAR (ABAR) is provided. 35 36One popular way to imagine Arkville's FPGA hardware aspect is as the 37FPGA PCIe-facing side of a so-called Smart NIC. The Arkville core does 38not contain any MACs, and is link-speed independent, as well as 39agnostic to the number of physical ports the application chooses to 40use. The ARK driver exposes the familiar PMD interface to allow packet 41movement to and from mbufs across multiple queues. 42 43However FPGA RTL applications could contain a universe of added 44functionality that an Arkville RTL core does not provide or can 45not anticipate. To allow for this expectation of user-defined 46innovation, the ARK PMD provides a dynamic mechanism of adding 47capabilities without having to modify the ARK PMD. 48 49The ARK PMD is intended to support all instances of the Arkville 50RTL Core, regardless of configuration, FPGA vendor, or target 51board. While specific capabilities such as number of physical 52hardware queue-pairs are negotiated; the driver is designed to 53remain constant over a broad and extendable feature set. 54 55Intentionally, Arkville by itself DOES NOT provide common NIC 56capabilities such as offload or receive-side scaling (RSS). 57These capabilities would be viewed as a gate-level "tax" on 58Green-box FPGA applications that do not require such function. 59Instead, they can be added as needed with essentially no 60overhead to the FPGA Application. 61 62The ARK PMD also supports optional user extensions, through dynamic linking. 63The ARK PMD user extensions are a feature of Arkville’s DPDK 64net/ark poll mode driver, allowing users to add their 65own code to extend the net/ark functionality without 66having to make source code changes to the driver. One motivation for 67this capability is that while DPDK provides a rich set of functions 68to interact with NIC-like capabilities (e.g. MAC addresses and statistics), 69the Arkville RTL IP does not include a MAC. Users can supply their 70own MAC or custom FPGA applications, which may require control from 71the PMD. The user extension is the means providing the control 72between the user's FPGA application and the existing DPDK features via 73the PMD. 74 75Device Parameters 76------------------- 77 78The ARK PMD supports device parameters that are used for packet 79routing and for internal packet generation and packet checking. This 80section describes the supported parameters. These features are 81primarily used for diagnostics, testing, and performance verification 82under the guidance of an Arkville specialist. The nominal use of 83Arkville does not require any configuration using these parameters. 84 85"Pkt_dir" 86 87The Packet Director controls connectivity between Arkville's internal 88hardware components. The features of the Pkt_dir are only used for 89diagnostics and testing; it is not intended for nominal use. The full 90set of features are not published at this level. 91 92Format: 93Pkt_dir=0x00110F10 94 95"Pkt_gen" 96 97The packet generator parameter takes a file as its argument. The file 98contains configuration parameters used internally for regression 99testing and are not intended to be published at this level. The 100packet generator is an internal Arkville hardware component. 101 102Format: 103Pkt_gen=./config/pg.conf 104 105"Pkt_chkr" 106 107The packet checker parameter takes a file as its argument. The file 108contains configuration parameters used internally for regression 109testing and are not intended to be published at this level. The 110packet checker is an internal Arkville hardware component. 111 112Format: 113Pkt_chkr=./config/pc.conf 114 115 116Data Path Interface 117------------------- 118 119Ingress RX and Egress TX operation is by the nominal DPDK API . 120The driver supports single-port, multi-queue for both RX and TX. 121 122Configuration Information 123------------------------- 124 125**DPDK Configuration Parameter** 126 127 * **RTE_LIBRTE_ARK_MIN_TX_PKTLEN** (default 0): Sets the minimum 128 packet length for tx packets to the FPGA. Packets less than this 129 length are padded to meet the requirement. This allows padding to 130 be offloaded or remain in host software. 131 132 133Building DPDK 134------------- 135 136See the :ref:`DPDK Getting Started Guide for Linux <linux_gsg>` for 137instructions on how to build DPDK. 138 139By default the ARK PMD library will be built into the DPDK library. 140 141For configuring and using UIO and VFIO frameworks, please also refer :ref:`the 142documentation that comes with DPDK suite <linux_gsg>`. 143 144To build with a non-zero minimum tx packet length, set the above macro in your 145CFLAGS environment prior to the meson build step. I.e., 146 147 export CFLAGS="-DRTE_LIBRTE_ARK_MIN_TX_PKTLEN=60" 148 meson build 149 150 151Supported ARK RTL PCIe Instances 152-------------------------------- 153 154ARK PMD supports the following Arkville RTL PCIe instances including: 155 156* ``1d6c:100d`` - AR-ARKA-FX0 [Arkville 32B DPDK Data Mover] 157* ``1d6c:100e`` - AR-ARKA-FX1 [Arkville 64B DPDK Data Mover] 158 159Supported Operating Systems 160--------------------------- 161 162Any Linux distribution fulfilling the conditions described in ``System Requirements`` 163section of :ref:`the DPDK documentation <linux_gsg>` or refer to *DPDK 164Release Notes*. ARM and PowerPC architectures are not supported at this time. 165 166 167Supported Features 168------------------ 169 170* Dynamic ARK PMD extensions 171* Multiple receive and transmit queues 172* Jumbo frames up to 9K 173* Hardware Statistics 174 175Unsupported Features 176-------------------- 177 178Features that may be part of, or become part of, the Arkville RTL IP that are 179not currently supported or exposed by the ARK PMD include: 180 181* PCIe SR-IOV Virtual Functions (VFs) 182* Arkville's Packet Generator Control and Status 183* Arkville's Packet Director Control and Status 184* Arkville's Packet Checker Control and Status 185* Arkville's Timebase Management 186 187Pre-Requisites 188-------------- 189 190#. Prepare the system as recommended by DPDK suite. This includes environment 191 variables, hugepages configuration, tool-chains and configuration 192 193#. Insert igb_uio kernel module using the command 'modprobe igb_uio' 194 195#. Bind the intended ARK device to igb_uio module 196 197At this point the system should be ready to run DPDK applications. Once the 198application runs to completion, the ARK PMD can be detached from igb_uio if necessary. 199 200Usage Example 201------------- 202 203Follow instructions available in the document 204:ref:`compiling and testing a PMD for a NIC <pmd_build_and_test>` to launch 205**testpmd** with Atomic Rules ARK devices managed by librte_net_ark. 206 207Example output: 208 209.. code-block:: console 210 211 [...] 212 EAL: PCI device 0000:01:00.0 on NUMA socket -1 213 EAL: probe driver: 1d6c:100e rte_ark_pmd 214 EAL: PCI memory mapped at 0x7f9b6c400000 215 PMD: eth_ark_dev_init(): Initializing 0:2:0.1 216 ARKP PMD CommitID: 378f3a67 217 Configuring Port 0 (socket 0) 218 Port 0: DC:3C:F6:00:00:01 219 Checking link statuses... 220 Port 0 Link Up - speed 100000 Mbps - full-duplex 221 Done 222 testpmd> 223