1.. SPDX-License-Identifier: BSD-3-Clause 2 Copyright(c) 2017 Marvell International Ltd. 3 Copyright(c) 2017 Semihalf. 4 5.. _mvpp2_poll_mode_driver: 6 7MVPP2 Poll Mode Driver 8====================== 9 10The MVPP2 PMD (**librte_net_mvpp2**) provides poll mode driver support 11for the Marvell PPv2 (Packet Processor v2) 1/10 Gbps adapter. 12 13Detailed information about SoCs that use PPv2 can be obtained here: 14 15* https://www.marvell.com/embedded-processors/armada-70xx/ 16* https://www.marvell.com/embedded-processors/armada-80xx/ 17 18 19Features 20-------- 21 22Features of the MVPP2 PMD are: 23 24- Speed capabilities 25- Link status 26- Tx Queue start/stop 27- MTU update 28- Jumbo frame 29- Promiscuous mode 30- Allmulticast mode 31- Unicast MAC filter 32- Multicast MAC filter 33- RSS hash 34- VLAN filter 35- CRC offload 36- L3 checksum offload 37- L4 checksum offload 38- Packet type parsing 39- Basic stats 40- :ref:`Extended stats <extstats>` 41- RX flow control 42- Scattered TX frames 43- :ref:`QoS <qossupport>` 44- :ref:`Flow API <flowapi>` 45- :ref:`Traffic metering and policing <mtrapi>` 46- :ref:`Traffic Management API <tmapi>` 47 48Limitations 49----------- 50 51- Number of lcores is limited to 9 by MUSDK internal design. If more lcores 52 need to be allocated, locking will have to be considered. Number of available 53 lcores can be changed via ``MRVL_MUSDK_HIFS_RESERVED`` define in 54 ``mrvl_ethdev.c`` source file. 55 56- Flushing vlans added for filtering is not possible due to MUSDK missing 57 functionality. Current workaround is to reset board so that PPv2 has a 58 chance to start in a sane state. 59 60- MUSDK architecture does not support changing configuration in run time. 61 All necessary configurations should be done before first dev_start(). 62 63- RX queue start/stop is not supported. 64 65- Current implementation does not support replacement of buffers in the HW buffer pool 66 at run time, so it is responsibility of the application to ensure that MTU does not exceed the configured buffer size. 67 68- Configuring TX flow control currently is not supported. 69 70- In current implementation, mechanism for acknowledging transmitted packets (``tx_done_cleanup``) is not supported. 71 72- Running more than one DPDK-MUSDK application simultaneously is not supported. 73 74 75Prerequisites 76------------- 77 78- Custom Linux Kernel sources 79 80 .. code-block:: console 81 82 git clone https://github.com/MarvellEmbeddedProcessors/linux-marvell.git -b linux-4.4.120-armada-18.09 83 84- Out of tree `mvpp2x_sysfs` kernel module sources 85 86 .. code-block:: console 87 88 git clone https://github.com/MarvellEmbeddedProcessors/mvpp2x-marvell.git -b mvpp2x-armada-18.09 89 90- MUSDK (Marvell User-Space SDK) sources 91 92 .. code-block:: console 93 94 git clone https://github.com/MarvellEmbeddedProcessors/musdk-marvell.git -b musdk-armada-18.09 95 96 MUSDK is a light-weight library that provides direct access to Marvell's 97 PPv2 (Packet Processor v2). Alternatively prebuilt MUSDK library can be 98 requested from `Marvell Extranet <https://extranet.marvell.com>`_. Once 99 approval has been granted, library can be found by typing ``musdk`` in 100 the search box. 101 102 To get better understanding of the library one can consult documentation 103 available in the ``doc`` top level directory of the MUSDK sources. 104 105- DPDK environment 106 107 Follow the DPDK :ref:`Getting Started Guide for Linux <linux_gsg>` to setup 108 DPDK environment. 109 110 111Building DPDK 112------------- 113 114Driver needs precompiled MUSDK library during compilation. 115 116.. code-block:: console 117 118 export CROSS_COMPILE=<toolchain>/bin/aarch64-linux-gnu- 119 ./bootstrap 120 ./configure --host=aarch64-linux-gnu 121 make install 122 123MUSDK will be installed to `usr/local` under current directory. 124For the detailed build instructions please consult ``doc/musdk_get_started.txt``. 125 126The path to the MUSDK installation directory needs to set in meson, shown in the 127command below. 128 129For additional instructions regarding DPDK cross compilation please refer to :doc:`Cross compile DPDK for ARM64 <../linux_gsg/cross_build_dpdk_for_arm64>`. 130 131.. code-block:: console 132 133 meson -Dlib_musdk_dir=/path/to/musdk build ninja -C build 134 135 136Usage Example 137------------- 138 139MVPP2 PMD requires extra out of tree kernel modules to function properly. 140`musdk_cma` sources are part of the MUSDK. Please consult 141``doc/musdk_get_started.txt`` for the detailed build instructions. 142For `mvpp2x_sysfs` please consult ``Documentation/pp22_sysfs.txt`` for the 143detailed build instructions. 144 145.. code-block:: console 146 147 insmod musdk_cma.ko 148 insmod mvpp2x_sysfs.ko 149 150Additionally interfaces used by DPDK application need to be put up: 151 152.. code-block:: console 153 154 ip link set eth0 up 155 ip link set eth2 up 156 157In order to run testpmd example application following command can be used: 158 159.. code-block:: console 160 161 ./dpdk-testpmd --vdev=eth_mvpp2,iface=eth0,iface=eth2 -c 7 -- \ 162 --burst=128 --txd=2048 --rxd=1024 --rxq=2 --txq=2 --nb-cores=2 \ 163 -i -a --rss-udp 164 165.. _extstats: 166 167Extended stats 168-------------- 169 170MVPP2 PMD supports the following extended statistics: 171 172 - ``rx_bytes``: number of RX bytes 173 - ``rx_packets``: number of RX packets 174 - ``rx_unicast_packets``: number of RX unicast packets 175 - ``rx_errors``: number of RX MAC errors 176 - ``rx_fullq_dropped``: number of RX packets dropped due to full RX queue 177 - ``rx_bm_dropped``: number of RX packets dropped due to no available buffers in the HW pool 178 - ``rx_early_dropped``: number of RX packets that were early dropped 179 - ``rx_fifo_dropped``: number of RX packets dropped due to RX fifo overrun 180 - ``rx_cls_dropped``: number of RX packets dropped by classifier 181 - ``tx_bytes``: number of TX bytes 182 - ``tx_packets``: number of TX packets 183 - ``tx_unicast_packets``: number of TX unicast packets 184 - ``tx_errors``: number of TX MAC errors 185 186 187.. _qossupport: 188 189QoS Configuration 190----------------- 191 192QoS configuration is done through external configuration file. Path to the 193file must be given as `cfg` in driver's vdev parameter list. 194 195Configuration syntax 196~~~~~~~~~~~~~~~~~~~~ 197 198.. code-block:: console 199 200 [policer <policer_id>] 201 token_unit = <token_unit> 202 color = <color_mode> 203 cir = <cir> 204 ebs = <ebs> 205 cbs = <cbs> 206 207 [port <portnum> default] 208 default_tc = <default_tc> 209 mapping_priority = <mapping_priority> 210 211 rate_limit_enable = <rate_limit_enable> 212 rate_limit = <rate_limit> 213 burst_size = <burst_size> 214 215 default_policer = <policer_id> 216 217 [port <portnum> tc <traffic_class>] 218 rxq = <rx_queue_list> 219 pcp = <pcp_list> 220 dscp = <dscp_list> 221 default_color = <default_color> 222 223 [port <portnum> tc <traffic_class>] 224 rxq = <rx_queue_list> 225 pcp = <pcp_list> 226 dscp = <dscp_list> 227 228 [port <portnum> txq <txqnum>] 229 sched_mode = <sched_mode> 230 wrr_weight = <wrr_weight> 231 232 rate_limit_enable = <rate_limit_enable> 233 rate_limit = <rate_limit> 234 burst_size = <burst_size> 235 236Where: 237 238- ``<portnum>``: DPDK Port number (0..n). 239 240- ``<default_tc>``: Default traffic class (e.g. 0) 241 242- ``<mapping_priority>``: QoS priority for mapping (`ip`, `vlan`, `ip/vlan` or `vlan/ip`). 243 244- ``<traffic_class>``: Traffic Class to be configured. 245 246- ``<rx_queue_list>``: List of DPDK RX queues (e.g. 0 1 3-4) 247 248- ``<pcp_list>``: List of PCP values to handle in particular TC (e.g. 0 1 3-4 7). 249 250- ``<dscp_list>``: List of DSCP values to handle in particular TC (e.g. 0-12 32-48 63). 251 252- ``<default_policer>``: Id of the policer configuration section to be used as default. 253 254- ``<policer_id>``: Id of the policer configuration section (0..31). 255 256- ``<token_unit>``: Policer token unit (`bytes` or `packets`). 257 258- ``<color_mode>``: Policer color mode (`aware` or `blind`). 259 260- ``<cir>``: Committed information rate in unit of kilo bits per second (data rate) or packets per second. 261 262- ``<cbs>``: Committed burst size in unit of kilo bytes or number of packets. 263 264- ``<ebs>``: Excess burst size in unit of kilo bytes or number of packets. 265 266- ``<default_color>``: Default color for specific tc. 267 268- ``<rate_limit_enable>``: Enables per port or per txq rate limiting (`0`/`1` to disable/enable). 269 270- ``<rate_limit>``: Committed information rate, in kilo bits per second. 271 272- ``<burst_size>``: Committed burst size, in kilo bytes. 273 274- ``<sched_mode>``: Egress scheduler mode (`wrr` or `sp`). 275 276- ``<wrr_weight>``: Txq weight. 277 278Setting PCP/DSCP values for the default TC is not required. All PCP/DSCP 279values not assigned explicitly to particular TC will be handled by the 280default TC. 281 282Configuration file example 283^^^^^^^^^^^^^^^^^^^^^^^^^^ 284 285.. code-block:: console 286 287 [policer 0] 288 token_unit = bytes 289 color = blind 290 cir = 100000 291 ebs = 64 292 cbs = 64 293 294 [port 0 default] 295 default_tc = 0 296 mapping_priority = ip 297 298 rate_limit_enable = 1 299 rate_limit = 1000 300 burst_size = 2000 301 302 [port 0 tc 0] 303 rxq = 0 1 304 305 [port 0 txq 0] 306 sched_mode = wrr 307 wrr_weight = 10 308 309 [port 0 txq 1] 310 sched_mode = wrr 311 wrr_weight = 100 312 313 [port 0 txq 2] 314 sched_mode = sp 315 316 [port 0 tc 1] 317 rxq = 2 318 pcp = 5 6 7 319 dscp = 26-38 320 321 [port 1 default] 322 default_tc = 0 323 mapping_priority = vlan/ip 324 325 default_policer = 0 326 327 [port 1 tc 0] 328 rxq = 0 329 dscp = 10 330 331 [port 1 tc 1] 332 rxq = 1 333 dscp = 11-20 334 335 [port 1 tc 2] 336 rxq = 2 337 dscp = 30 338 339 [port 1 txq 0] 340 rate_limit_enable = 1 341 rate_limit = 10000 342 burst_size = 2000 343 344Usage example 345^^^^^^^^^^^^^ 346 347.. code-block:: console 348 349 ./dpdk-testpmd --vdev=eth_mvpp2,iface=eth0,iface=eth2,cfg=/home/user/mrvl.conf \ 350 -c 7 -- -i -a --disable-hw-vlan-strip --rxq=3 --txq=3 351 352.. _flowapi: 353 354Flow API 355-------- 356 357PPv2 offers packet classification capabilities via classifier engine which 358can be configured via generic flow API offered by DPDK. 359 360The :ref:`flow_isolated_mode` is supported. 361 362For an additional description please refer to DPDK :doc:`../prog_guide/rte_flow`. 363 364Supported flow actions 365~~~~~~~~~~~~~~~~~~~~~~ 366 367Following flow action items are supported by the driver: 368 369* DROP 370* QUEUE 371 372Supported flow items 373~~~~~~~~~~~~~~~~~~~~ 374 375Following flow items and their respective fields are supported by the driver: 376 377* ETH 378 379 * source MAC 380 * destination MAC 381 * ethertype 382 383* VLAN 384 385 * PCP 386 * VID 387 388* IPV4 389 390 * DSCP 391 * protocol 392 * source address 393 * destination address 394 395* IPV6 396 397 * flow label 398 * next header 399 * source address 400 * destination address 401 402* UDP 403 404 * source port 405 * destination port 406 407* TCP 408 409 * source port 410 * destination port 411 412Classifier match engine 413~~~~~~~~~~~~~~~~~~~~~~~ 414 415Classifier has an internal match engine which can be configured to 416operate in either exact or maskable mode. 417 418Mode is selected upon creation of the first unique flow rule as follows: 419 420* maskable, if key size is up to 8 bytes. 421* exact, otherwise, i.e for keys bigger than 8 bytes. 422 423Where the key size equals the number of bytes of all fields specified 424in the flow items. 425 426.. table:: Examples of key size calculation 427 428 +----------------------------------------------------------------------------+-------------------+-------------+ 429 | Flow pattern | Key size in bytes | Used engine | 430 +============================================================================+===================+=============+ 431 | ETH (destination MAC) / VLAN (VID) | 6 + 2 = 8 | Maskable | 432 +----------------------------------------------------------------------------+-------------------+-------------+ 433 | VLAN (VID) / IPV4 (source address) | 2 + 4 = 6 | Maskable | 434 +----------------------------------------------------------------------------+-------------------+-------------+ 435 | TCP (source port, destination port) | 2 + 2 = 4 | Maskable | 436 +----------------------------------------------------------------------------+-------------------+-------------+ 437 | VLAN (priority) / IPV4 (source address) | 1 + 4 = 5 | Maskable | 438 +----------------------------------------------------------------------------+-------------------+-------------+ 439 | IPV4 (destination address) / UDP (source port, destination port) | 6 + 2 + 2 = 10 | Exact | 440 +----------------------------------------------------------------------------+-------------------+-------------+ 441 | VLAN (VID) / IPV6 (flow label, destination address) | 2 + 3 + 16 = 21 | Exact | 442 +----------------------------------------------------------------------------+-------------------+-------------+ 443 | IPV4 (DSCP, source address, destination address) | 1 + 4 + 4 = 9 | Exact | 444 +----------------------------------------------------------------------------+-------------------+-------------+ 445 | IPV6 (flow label, source address, destination address) | 3 + 16 + 16 = 35 | Exact | 446 +----------------------------------------------------------------------------+-------------------+-------------+ 447 448From the user perspective maskable mode means that masks specified 449via flow rules are respected. In case of exact match mode, masks 450which do not provide exact matching (all bits masked) are ignored. 451 452If the flow matches more than one classifier rule the first 453(with the lowest index) matched takes precedence. 454 455Flow rules usage example 456~~~~~~~~~~~~~~~~~~~~~~~~ 457 458Before proceeding run testpmd user application: 459 460.. code-block:: console 461 462 ./dpdk-testpmd --vdev=eth_mvpp2,iface=eth0,iface=eth2 -c 3 -- -i --p 3 -a --disable-hw-vlan-strip 463 464Example #1 465^^^^^^^^^^ 466 467.. code-block:: console 468 469 testpmd> flow create 0 ingress pattern eth src is 10:11:12:13:14:15 / end actions drop / end 470 471In this case key size is 6 bytes thus maskable type is selected. Testpmd 472will set mask to ff:ff:ff:ff:ff:ff i.e traffic explicitly matching 473above rule will be dropped. 474 475Example #2 476^^^^^^^^^^ 477 478.. code-block:: console 479 480 testpmd> flow create 0 ingress pattern ipv4 src spec 10.10.10.0 src mask 255.255.255.0 / tcp src spec 0x10 src mask 0x10 / end action drop / end 481 482In this case key size is 8 bytes thus maskable type is selected. 483Flows which have IPv4 source addresses ranging from 10.10.10.0 to 10.10.10.255 484and tcp source port set to 16 will be dropped. 485 486Example #3 487^^^^^^^^^^ 488 489.. code-block:: console 490 491 testpmd> flow create 0 ingress pattern vlan vid spec 0x10 vid mask 0x10 / ipv4 src spec 10.10.1.1 src mask 255.255.0.0 dst spec 11.11.11.1 dst mask 255.255.255.0 / end actions drop / end 492 493In this case key size is 10 bytes thus exact type is selected. 494Even though each item has partial mask set, masks will be ignored. 495As a result only flows with VID set to 16 and IPv4 source and destination 496addresses set to 10.10.1.1 and 11.11.11.1 respectively will be dropped. 497 498Limitations 499~~~~~~~~~~~ 500 501Following limitations need to be taken into account while creating flow rules: 502 503* For IPv4 exact match type the key size must be up to 12 bytes. 504* For IPv6 exact match type the key size must be up to 36 bytes. 505* Following fields cannot be partially masked (all masks are treated as 506 if they were exact): 507 508 * ETH: ethertype 509 * VLAN: PCP, VID 510 * IPv4: protocol 511 * IPv6: next header 512 * TCP/UDP: source port, destination port 513 514* Only one classifier table can be created thus all rules in the table 515 have to match table format. Table format is set during creation of 516 the first unique flow rule. 517* Up to 5 fields can be specified per flow rule. 518* Up to 20 flow rules can be added. 519 520For additional information about classifier please consult 521``doc/musdk_cls_user_guide.txt``. 522 523.. _mtrapi: 524 525Traffic metering and policing 526----------------------------- 527 528MVPP2 PMD supports DPDK traffic metering and policing that allows the following: 529 5301. Meter ingress traffic. 5312. Do policing. 5323. Gather statistics. 533 534For an additional description please refer to DPDK :doc:`Traffic Metering and Policing API <../prog_guide/traffic_metering_and_policing>`. 535 536The policer objects defined by this feature can work with the default policer defined via config file as described in :ref:`QoS Support <qossupport>`. 537 538Limitations 539~~~~~~~~~~~ 540 541The following capabilities are not supported: 542 543- MTR object meter DSCP table update 544- MTR object policer action update 545- MTR object enabled statistics 546 547Usage example 548~~~~~~~~~~~~~ 549 5501. Run testpmd user app: 551 552 .. code-block:: console 553 554 ./dpdk-testpmd --vdev=eth_mvpp2,iface=eth0,iface=eth2 -c 6 -- -i -p 3 -a --txd 1024 --rxd 1024 555 5562. Create meter profile: 557 558 .. code-block:: console 559 560 testpmd> add port meter profile 0 0 srtcm_rfc2697 2000 256 256 561 5623. Create meter: 563 564 .. code-block:: console 565 566 testpmd> create port meter 0 0 0 yes d d d 0 1 0 567 5684. Create flow rule witch meter attached: 569 570 .. code-block:: console 571 572 testpmd> flow create 0 ingress pattern ipv4 src is 10.10.10.1 / end actions meter mtr_id 0 / end 573 574For a detailed usage description please refer to "Traffic Metering and Policing" section in DPDK :doc:`Testpmd Runtime Functions <../testpmd_app_ug/testpmd_funcs>`. 575 576 577 578.. _tmapi: 579 580Traffic Management API 581---------------------- 582 583MVPP2 PMD supports generic DPDK Traffic Management API which allows to 584configure the following features: 585 5861. Hierarchical scheduling 5872. Traffic shaping 5883. Congestion management 5894. Packet marking 590 591Internally TM is represented by a hierarchy (tree) of nodes. 592Node which has a parent is called a leaf whereas node without 593parent is called a non-leaf (root). 594MVPP2 PMD supports two level hierarchy where level 0 represents ports and level 1 represents tx queues of a given port. 595 596.. figure:: img/mvpp2_tm.* 597 598Nodes hold following types of settings: 599 600- for egress scheduler configuration: weight 601- for egress rate limiter: private shaper 602- bitmask indicating which statistics counters will be read 603 604Hierarchy is always constructed from the top, i.e first a root node is added 605then some number of leaf nodes. Number of leaf nodes cannot exceed number 606of configured tx queues. 607 608After hierarchy is complete it can be committed. 609 610 611For an additional description please refer to DPDK :doc:`Traffic Management API <../prog_guide/traffic_management>`. 612 613Limitations 614~~~~~~~~~~~ 615 616The following capabilities are not supported: 617 618- Traffic manager WRED profile and WRED context 619- Traffic manager shared shaper update 620- Traffic manager packet marking 621- Maximum number of levels in hierarchy is 2 622- Currently dynamic change of a hierarchy is not supported 623 624Usage example 625~~~~~~~~~~~~~ 626 627For a detailed usage description please refer to "Traffic Management" section in DPDK :doc:`Testpmd Runtime Functions <../testpmd_app_ug/testpmd_funcs>`. 628 6291. Run testpmd as follows: 630 631 .. code-block:: console 632 633 ./dpdk-testpmd --vdev=net_mrvl,iface=eth0,iface=eth2,cfg=./qos_config -c 7 -- \ 634 -i -p 3 --disable-hw-vlan-strip --rxq 3 --txq 3 --txd 1024 --rxd 1024 635 6362. Stop all ports: 637 638 .. code-block:: console 639 640 testpmd> port stop all 641 6423. Add shaper profile: 643 644 .. code-block:: console 645 646 testpmd> add port tm node shaper profile 0 0 900000 70000 0 647 648 Parameters have following meaning:: 649 650 0 - Id of a port. 651 0 - Id of a new shaper profile. 652 900000 - Shaper rate in bytes/s. 653 70000 - Bucket size in bytes. 654 0 - Packet length adjustment - ignored. 655 6564. Add non-leaf node for port 0: 657 658 .. code-block:: console 659 660 testpmd> add port tm nonleaf node 0 3 -1 0 0 0 0 0 1 3 0 661 662 Parameters have following meaning:: 663 664 0 - Id of a port 665 3 - Id of a new node. 666 -1 - Indicate that root does not have a parent. 667 0 - Priority of the node. 668 0 - Weight of the node. 669 0 - Id of a level. Since this is a root 0 is passed. 670 0 - Id of the shaper profile. 671 0 - Number of SP priorities. 672 3 - Enable statistics for both number of transmitted packets and bytes. 673 0 - Number of shared shapers. 674 6755. Add leaf node for tx queue 0: 676 677 .. code-block:: console 678 679 testpmd> add port tm leaf node 0 0 3 0 30 1 -1 0 0 1 0 680 681 Parameters have following meaning:: 682 683 0 - Id of a port. 684 0 - Id of a new node. 685 3 - Id of the parent node. 686 0 - Priority of a node. 687 30 - WRR weight. 688 1 - Id of a level. Since this is a leaf node 1 is passed. 689 -1 - Id of a shaper. -1 indicates that shaper is not attached. 690 0 - Congestion management is not supported. 691 0 - Congestion management is not supported. 692 1 - Enable statistics counter for number of transmitted packets. 693 0 - Number of shared shapers. 694 6956. Add leaf node for tx queue 1: 696 697 .. code-block:: console 698 699 testpmd> add port tm leaf node 0 1 3 0 60 1 -1 0 0 1 0 700 701 Parameters have following meaning:: 702 703 0 - Id of a port. 704 1 - Id of a new node. 705 3 - Id of the parent node. 706 0 - Priority of a node. 707 60 - WRR weight. 708 1 - Id of a level. Since this is a leaf node 1 is passed. 709 -1 - Id of a shaper. -1 indicates that shaper is not attached. 710 0 - Congestion management is not supported. 711 0 - Congestion management is not supported. 712 1 - Enable statistics counter for number of transmitted packets. 713 0 - Number of shared shapers. 714 7157. Add leaf node for tx queue 2: 716 717 .. code-block:: console 718 719 testpmd> add port tm leaf node 0 2 3 0 99 1 -1 0 0 1 0 720 721 Parameters have following meaning:: 722 723 0 - Id of a port. 724 2 - Id of a new node. 725 3 - Id of the parent node. 726 0 - Priority of a node. 727 99 - WRR weight. 728 1 - Id of a level. Since this is a leaf node 1 is passed. 729 -1 - Id of a shaper. -1 indicates that shaper is not attached. 730 0 - Congestion management is not supported. 731 0 - Congestion management is not supported. 732 1 - Enable statistics counter for number of transmitted packets. 733 0 - Number of shared shapers. 734 7358. Commit hierarchy: 736 737 .. code-block:: console 738 739 testpmd> port tm hierarchy commit 0 no 740 741 Parameters have following meaning:: 742 743 0 - Id of a port. 744 no - Do not flush TM hierarchy if commit fails. 745 7469. Start all ports 747 748 .. code-block:: console 749 750 testpmd> port start all 751 752 753 75410. Enable forwarding 755 756 .. code-block:: console 757 758 testpmd> start 759