1.. SPDX-License-Identifier: BSD-3-Clause 2 Copyright(c) 2019 Intel Corporation. 3 4.. include:: <isonum.txt> 5 6IOAT Rawdev Driver for Intel\ |reg| QuickData Technology 7====================================================================== 8 9The ``ioat`` rawdev driver provides a poll-mode driver (PMD) for Intel\ |reg| 10QuickData Technology, part of Intel\ |reg| I/O Acceleration Technology 11`(Intel I/OAT) 12<https://www.intel.com/content/www/us/en/wireless-network/accel-technology.html>`_. 13This PMD, when used on supported hardware, allows data copies, for example, 14cloning packet data, to be accelerated by that hardware rather than having to 15be done by software, freeing up CPU cycles for other tasks. 16 17Hardware Requirements 18---------------------- 19 20On Linux, the presence of an Intel\ |reg| QuickData Technology hardware can 21be detected by checking the output of the ``lspci`` command, where the 22hardware will be often listed as "Crystal Beach DMA" or "CBDMA". For 23example, on a system with Intel\ |reg| Xeon\ |reg| CPU E5-2699 v4 @2.20GHz, 24lspci shows: 25 26.. code-block:: console 27 28 # lspci | grep DMA 29 00:04.0 System peripheral: Intel Corporation Xeon E7 v4/Xeon E5 v4/Xeon E3 v4/Xeon D Crystal Beach DMA Channel 0 (rev 01) 30 00:04.1 System peripheral: Intel Corporation Xeon E7 v4/Xeon E5 v4/Xeon E3 v4/Xeon D Crystal Beach DMA Channel 1 (rev 01) 31 00:04.2 System peripheral: Intel Corporation Xeon E7 v4/Xeon E5 v4/Xeon E3 v4/Xeon D Crystal Beach DMA Channel 2 (rev 01) 32 00:04.3 System peripheral: Intel Corporation Xeon E7 v4/Xeon E5 v4/Xeon E3 v4/Xeon D Crystal Beach DMA Channel 3 (rev 01) 33 00:04.4 System peripheral: Intel Corporation Xeon E7 v4/Xeon E5 v4/Xeon E3 v4/Xeon D Crystal Beach DMA Channel 4 (rev 01) 34 00:04.5 System peripheral: Intel Corporation Xeon E7 v4/Xeon E5 v4/Xeon E3 v4/Xeon D Crystal Beach DMA Channel 5 (rev 01) 35 00:04.6 System peripheral: Intel Corporation Xeon E7 v4/Xeon E5 v4/Xeon E3 v4/Xeon D Crystal Beach DMA Channel 6 (rev 01) 36 00:04.7 System peripheral: Intel Corporation Xeon E7 v4/Xeon E5 v4/Xeon E3 v4/Xeon D Crystal Beach DMA Channel 7 (rev 01) 37 38On a system with Intel\ |reg| Xeon\ |reg| Gold 6154 CPU @ 3.00GHz, lspci 39shows: 40 41.. code-block:: console 42 43 # lspci | grep DMA 44 00:04.0 System peripheral: Intel Corporation Sky Lake-E CBDMA Registers (rev 04) 45 00:04.1 System peripheral: Intel Corporation Sky Lake-E CBDMA Registers (rev 04) 46 00:04.2 System peripheral: Intel Corporation Sky Lake-E CBDMA Registers (rev 04) 47 00:04.3 System peripheral: Intel Corporation Sky Lake-E CBDMA Registers (rev 04) 48 00:04.4 System peripheral: Intel Corporation Sky Lake-E CBDMA Registers (rev 04) 49 00:04.5 System peripheral: Intel Corporation Sky Lake-E CBDMA Registers (rev 04) 50 00:04.6 System peripheral: Intel Corporation Sky Lake-E CBDMA Registers (rev 04) 51 00:04.7 System peripheral: Intel Corporation Sky Lake-E CBDMA Registers (rev 04) 52 53 54Compilation 55------------ 56 57For builds done with ``make``, the driver compilation is enabled by the 58``CONFIG_RTE_LIBRTE_PMD_IOAT_RAWDEV`` build configuration option. This is 59enabled by default in builds for x86 platforms, and disabled in other 60configurations. 61 62For builds using ``meson`` and ``ninja``, the driver will be built when the 63target platform is x86-based. 64 65Device Setup 66------------- 67 68The Intel\ |reg| QuickData Technology HW devices will need to be bound to a 69user-space IO driver for use. The script ``dpdk-devbind.py`` script 70included with DPDK can be used to view the state of the devices and to bind 71them to a suitable DPDK-supported kernel driver. When querying the status 72of the devices, they will appear under the category of "Misc (rawdev) 73devices", i.e. the command ``dpdk-devbind.py --status-dev misc`` can be 74used to see the state of those devices alone. 75 76Device Probing and Initialization 77~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 78 79Once bound to a suitable kernel device driver, the HW devices will be found 80as part of the PCI scan done at application initialization time. No vdev 81parameters need to be passed to create or initialize the device. 82 83Once probed successfully, the device will appear as a ``rawdev``, that is a 84"raw device type" inside DPDK, and can be accessed using APIs from the 85``rte_rawdev`` library. 86 87Using IOAT Rawdev Devices 88-------------------------- 89 90To use the devices from an application, the rawdev API can be used, along 91with definitions taken from the device-specific header file 92``rte_ioat_rawdev.h``. This header is needed to get the definition of 93structure parameters used by some of the rawdev APIs for IOAT rawdev 94devices, as well as providing key functions for using the device for memory 95copies. 96 97Getting Device Information 98~~~~~~~~~~~~~~~~~~~~~~~~~~~ 99 100Basic information about each rawdev device can be queried using the 101``rte_rawdev_info_get()`` API. For most applications, this API will be 102needed to verify that the rawdev in question is of the expected type. For 103example, the following code snippet can be used to identify an IOAT 104rawdev device for use by an application: 105 106.. code-block:: C 107 108 for (i = 0; i < count && !found; i++) { 109 struct rte_rawdev_info info = { .dev_private = NULL }; 110 found = (rte_rawdev_info_get(i, &info) == 0 && 111 strcmp(info.driver_name, 112 IOAT_PMD_RAWDEV_NAME_STR) == 0); 113 } 114 115When calling the ``rte_rawdev_info_get()`` API for an IOAT rawdev device, 116the ``dev_private`` field in the ``rte_rawdev_info`` struct should either 117be NULL, or else be set to point to a structure of type 118``rte_ioat_rawdev_config``, in which case the size of the configured device 119input ring will be returned in that structure. 120 121Device Configuration 122~~~~~~~~~~~~~~~~~~~~~ 123 124Configuring an IOAT rawdev device is done using the 125``rte_rawdev_configure()`` API, which takes the same structure parameters 126as the, previously referenced, ``rte_rawdev_info_get()`` API. The main 127difference is that, because the parameter is used as input rather than 128output, the ``dev_private`` structure element cannot be NULL, and must 129point to a valid ``rte_ioat_rawdev_config`` structure, containing the ring 130size to be used by the device. The ring size must be a power of two, 131between 64 and 4096. 132 133The following code shows how the device is configured in 134``test_ioat_rawdev.c``: 135 136.. code-block:: C 137 138 #define IOAT_TEST_RINGSIZE 512 139 struct rte_ioat_rawdev_config p = { .ring_size = -1 }; 140 struct rte_rawdev_info info = { .dev_private = &p }; 141 142 /* ... */ 143 144 p.ring_size = IOAT_TEST_RINGSIZE; 145 if (rte_rawdev_configure(dev_id, &info) != 0) { 146 printf("Error with rte_rawdev_configure()\n"); 147 return -1; 148 } 149 150Once configured, the device can then be made ready for use by calling the 151``rte_rawdev_start()`` API. 152 153Performing Data Copies 154~~~~~~~~~~~~~~~~~~~~~~~ 155 156To perform data copies using IOAT rawdev devices, the functions 157``rte_ioat_enqueue_copy()`` and ``rte_ioat_do_copies()`` should be used. 158Once copies have been completed, the completion will be reported back when 159the application calls ``rte_ioat_completed_copies()``. 160 161The ``rte_ioat_enqueue_copy()`` function enqueues a single copy to the 162device ring for copying at a later point. The parameters to that function 163include the physical addresses of both the source and destination buffers, 164as well as two "handles" to be returned to the user when the copy is 165completed. These handles can be arbitrary values, but two are provided so 166that the library can track handles for both source and destination on 167behalf of the user, e.g. virtual addresses for the buffers, or mbuf 168pointers if packet data is being copied. 169 170While the ``rte_ioat_enqueue_copy()`` function enqueues a copy operation on 171the device ring, the copy will not actually be performed until after the 172application calls the ``rte_ioat_do_copies()`` function. This function 173informs the device hardware of the elements enqueued on the ring, and the 174device will begin to process them. It is expected that, for efficiency 175reasons, a burst of operations will be enqueued to the device via multiple 176enqueue calls between calls to the ``rte_ioat_do_copies()`` function. 177 178The following code from ``test_ioat_rawdev.c`` demonstrates how to enqueue 179a burst of copies to the device and start the hardware processing of them: 180 181.. code-block:: C 182 183 struct rte_mbuf *srcs[32], *dsts[32]; 184 unsigned int j; 185 186 for (i = 0; i < RTE_DIM(srcs); i++) { 187 char *src_data; 188 189 srcs[i] = rte_pktmbuf_alloc(pool); 190 dsts[i] = rte_pktmbuf_alloc(pool); 191 srcs[i]->data_len = srcs[i]->pkt_len = length; 192 dsts[i]->data_len = dsts[i]->pkt_len = length; 193 src_data = rte_pktmbuf_mtod(srcs[i], char *); 194 195 for (j = 0; j < length; j++) 196 src_data[j] = rand() & 0xFF; 197 198 if (rte_ioat_enqueue_copy(dev_id, 199 srcs[i]->buf_iova + srcs[i]->data_off, 200 dsts[i]->buf_iova + dsts[i]->data_off, 201 length, 202 (uintptr_t)srcs[i], 203 (uintptr_t)dsts[i], 204 0 /* nofence */) != 1) { 205 printf("Error with rte_ioat_enqueue_copy for buffer %u\n", 206 i); 207 return -1; 208 } 209 } 210 rte_ioat_do_copies(dev_id); 211 212To retrieve information about completed copies, the API 213``rte_ioat_completed_copies()`` should be used. This API will return to the 214application a set of completion handles passed in when the relevant copies 215were enqueued. 216 217The following code from ``test_ioat_rawdev.c`` shows the test code 218retrieving information about the completed copies and validating the data 219is correct before freeing the data buffers using the returned handles: 220 221.. code-block:: C 222 223 if (rte_ioat_completed_copies(dev_id, 64, (void *)completed_src, 224 (void *)completed_dst) != RTE_DIM(srcs)) { 225 printf("Error with rte_ioat_completed_copies\n"); 226 return -1; 227 } 228 for (i = 0; i < RTE_DIM(srcs); i++) { 229 char *src_data, *dst_data; 230 231 if (completed_src[i] != srcs[i]) { 232 printf("Error with source pointer %u\n", i); 233 return -1; 234 } 235 if (completed_dst[i] != dsts[i]) { 236 printf("Error with dest pointer %u\n", i); 237 return -1; 238 } 239 240 src_data = rte_pktmbuf_mtod(srcs[i], char *); 241 dst_data = rte_pktmbuf_mtod(dsts[i], char *); 242 for (j = 0; j < length; j++) 243 if (src_data[j] != dst_data[j]) { 244 printf("Error with copy of packet %u, byte %u\n", 245 i, j); 246 return -1; 247 } 248 rte_pktmbuf_free(srcs[i]); 249 rte_pktmbuf_free(dsts[i]); 250 } 251 252 253Querying Device Statistics 254~~~~~~~~~~~~~~~~~~~~~~~~~~~ 255 256The statistics from the IOAT rawdev device can be got via the xstats 257functions in the ``rte_rawdev`` library, i.e. 258``rte_rawdev_xstats_names_get()``, ``rte_rawdev_xstats_get()`` and 259``rte_rawdev_xstats_by_name_get``. The statistics returned for each device 260instance are: 261 262* ``failed_enqueues`` 263* ``successful_enqueues`` 264* ``copies_started`` 265* ``copies_completed`` 266