1llvm-mca - LLVM Machine Code Analyzer 2===================================== 3 4.. program:: llvm-mca 5 6SYNOPSIS 7-------- 8 9:program:`llvm-mca` [*options*] [input] 10 11DESCRIPTION 12----------- 13 14:program:`llvm-mca` is a performance analysis tool that uses information 15available in LLVM (e.g. scheduling models) to statically measure the performance 16of machine code in a specific CPU. 17 18Performance is measured in terms of throughput as well as processor resource 19consumption. The tool currently works for processors with an out-of-order 20backend, for which there is a scheduling model available in LLVM. 21 22The main goal of this tool is not just to predict the performance of the code 23when run on the target, but also help with diagnosing potential performance 24issues. 25 26Given an assembly code sequence, :program:`llvm-mca` estimates the Instructions 27Per Cycle (IPC), as well as hardware resource pressure. The analysis and 28reporting style were inspired by the IACA tool from Intel. 29 30For example, you can compile code with clang, output assembly, and pipe it 31directly into :program:`llvm-mca` for analysis: 32 33.. code-block:: bash 34 35 $ clang foo.c -O2 -target x86_64-unknown-unknown -S -o - | llvm-mca -mcpu=btver2 36 37Or for Intel syntax: 38 39.. code-block:: bash 40 41 $ clang foo.c -O2 -target x86_64-unknown-unknown -mllvm -x86-asm-syntax=intel -S -o - | llvm-mca -mcpu=btver2 42 43(:program:`llvm-mca` detects Intel syntax by the presence of an `.intel_syntax` 44directive at the beginning of the input. By default its output syntax matches 45that of its input.) 46 47Scheduling models are not just used to compute instruction latencies and 48throughput, but also to understand what processor resources are available 49and how to simulate them. 50 51By design, the quality of the analysis conducted by :program:`llvm-mca` is 52inevitably affected by the quality of the scheduling models in LLVM. 53 54If you see that the performance report is not accurate for a processor, 55please `file a bug <https://bugs.llvm.org/enter_bug.cgi?product=libraries>`_ 56against the appropriate backend. 57 58OPTIONS 59------- 60 61If ``input`` is "``-``" or omitted, :program:`llvm-mca` reads from standard 62input. Otherwise, it will read from the specified filename. 63 64If the :option:`-o` option is omitted, then :program:`llvm-mca` will send its output 65to standard output if the input is from standard input. If the :option:`-o` 66option specifies "``-``", then the output will also be sent to standard output. 67 68 69.. option:: -help 70 71 Print a summary of command line options. 72 73.. option:: -o <filename> 74 75 Use ``<filename>`` as the output filename. See the summary above for more 76 details. 77 78.. option:: -mtriple=<target triple> 79 80 Specify a target triple string. 81 82.. option:: -march=<arch> 83 84 Specify the architecture for which to analyze the code. It defaults to the 85 host default target. 86 87.. option:: -mcpu=<cpuname> 88 89 Specify the processor for which to analyze the code. By default, the cpu name 90 is autodetected from the host. 91 92.. option:: -output-asm-variant=<variant id> 93 94 Specify the output assembly variant for the report generated by the tool. 95 On x86, possible values are [0, 1]. A value of 0 (vic. 1) for this flag enables 96 the AT&T (vic. Intel) assembly format for the code printed out by the tool in 97 the analysis report. 98 99.. option:: -print-imm-hex 100 101 Prefer hex format for numeric literals in the output assembly printed as part 102 of the report. 103 104.. option:: -dispatch=<width> 105 106 Specify a different dispatch width for the processor. The dispatch width 107 defaults to field 'IssueWidth' in the processor scheduling model. If width is 108 zero, then the default dispatch width is used. 109 110.. option:: -register-file-size=<size> 111 112 Specify the size of the register file. When specified, this flag limits how 113 many physical registers are available for register renaming purposes. A value 114 of zero for this flag means "unlimited number of physical registers". 115 116.. option:: -iterations=<number of iterations> 117 118 Specify the number of iterations to run. If this flag is set to 0, then the 119 tool sets the number of iterations to a default value (i.e. 100). 120 121.. option:: -noalias=<bool> 122 123 If set, the tool assumes that loads and stores don't alias. This is the 124 default behavior. 125 126.. option:: -lqueue=<load queue size> 127 128 Specify the size of the load queue in the load/store unit emulated by the tool. 129 By default, the tool assumes an unbound number of entries in the load queue. 130 A value of zero for this flag is ignored, and the default load queue size is 131 used instead. 132 133.. option:: -squeue=<store queue size> 134 135 Specify the size of the store queue in the load/store unit emulated by the 136 tool. By default, the tool assumes an unbound number of entries in the store 137 queue. A value of zero for this flag is ignored, and the default store queue 138 size is used instead. 139 140.. option:: -timeline 141 142 Enable the timeline view. 143 144.. option:: -timeline-max-iterations=<iterations> 145 146 Limit the number of iterations to print in the timeline view. By default, the 147 timeline view prints information for up to 10 iterations. 148 149.. option:: -timeline-max-cycles=<cycles> 150 151 Limit the number of cycles in the timeline view. By default, the number of 152 cycles is set to 80. 153 154.. option:: -resource-pressure 155 156 Enable the resource pressure view. This is enabled by default. 157 158.. option:: -register-file-stats 159 160 Enable register file usage statistics. 161 162.. option:: -dispatch-stats 163 164 Enable extra dispatch statistics. This view collects and analyzes instruction 165 dispatch events, as well as static/dynamic dispatch stall events. This view 166 is disabled by default. 167 168.. option:: -scheduler-stats 169 170 Enable extra scheduler statistics. This view collects and analyzes instruction 171 issue events. This view is disabled by default. 172 173.. option:: -retire-stats 174 175 Enable extra retire control unit statistics. This view is disabled by default. 176 177.. option:: -instruction-info 178 179 Enable the instruction info view. This is enabled by default. 180 181.. option:: -show-encoding 182 183 Enable the printing of instruction encodings within the instruction info view. 184 185.. option:: -all-stats 186 187 Print all hardware statistics. This enables extra statistics related to the 188 dispatch logic, the hardware schedulers, the register file(s), and the retire 189 control unit. This option is disabled by default. 190 191.. option:: -all-views 192 193 Enable all the view. 194 195.. option:: -instruction-tables 196 197 Prints resource pressure information based on the static information 198 available from the processor model. This differs from the resource pressure 199 view because it doesn't require that the code is simulated. It instead prints 200 the theoretical uniform distribution of resource pressure for every 201 instruction in sequence. 202 203.. option:: -bottleneck-analysis 204 205 Print information about bottlenecks that affect the throughput. This analysis 206 can be expensive, and it is disabled by default. Bottlenecks are highlighted 207 in the summary view. 208 209.. option:: -json 210 211 Print the requested views in JSON format. The instructions and the processor 212 resources are printed as members of special top level JSON objects. The 213 individual views refer to them by index. 214 215 216EXIT STATUS 217----------- 218 219:program:`llvm-mca` returns 0 on success. Otherwise, an error message is printed 220to standard error, and the tool returns 1. 221 222USING MARKERS TO ANALYZE SPECIFIC CODE BLOCKS 223--------------------------------------------- 224:program:`llvm-mca` allows for the optional usage of special code comments to 225mark regions of the assembly code to be analyzed. A comment starting with 226substring ``LLVM-MCA-BEGIN`` marks the beginning of a code region. A comment 227starting with substring ``LLVM-MCA-END`` marks the end of a code region. For 228example: 229 230.. code-block:: none 231 232 # LLVM-MCA-BEGIN 233 ... 234 # LLVM-MCA-END 235 236If no user-defined region is specified, then :program:`llvm-mca` assumes a 237default region which contains every instruction in the input file. Every region 238is analyzed in isolation, and the final performance report is the union of all 239the reports generated for every code region. 240 241Code regions can have names. For example: 242 243.. code-block:: none 244 245 # LLVM-MCA-BEGIN A simple example 246 add %eax, %eax 247 # LLVM-MCA-END 248 249The code from the example above defines a region named "A simple example" with a 250single instruction in it. Note how the region name doesn't have to be repeated 251in the ``LLVM-MCA-END`` directive. In the absence of overlapping regions, 252an anonymous ``LLVM-MCA-END`` directive always ends the currently active user 253defined region. 254 255Example of nesting regions: 256 257.. code-block:: none 258 259 # LLVM-MCA-BEGIN foo 260 add %eax, %edx 261 # LLVM-MCA-BEGIN bar 262 sub %eax, %edx 263 # LLVM-MCA-END bar 264 # LLVM-MCA-END foo 265 266Example of overlapping regions: 267 268.. code-block:: none 269 270 # LLVM-MCA-BEGIN foo 271 add %eax, %edx 272 # LLVM-MCA-BEGIN bar 273 sub %eax, %edx 274 # LLVM-MCA-END foo 275 add %eax, %edx 276 # LLVM-MCA-END bar 277 278Note that multiple anonymous regions cannot overlap. Also, overlapping regions 279cannot have the same name. 280 281There is no support for marking regions from high-level source code, like C or 282C++. As a workaround, inline assembly directives may be used: 283 284.. code-block:: c++ 285 286 int foo(int a, int b) { 287 __asm volatile("# LLVM-MCA-BEGIN foo"); 288 a += 42; 289 __asm volatile("# LLVM-MCA-END"); 290 a *= b; 291 return a; 292 } 293 294However, this interferes with optimizations like loop vectorization and may have 295an impact on the code generated. This is because the ``__asm`` statements are 296seen as real code having important side effects, which limits how the code 297around them can be transformed. If users want to make use of inline assembly 298to emit markers, then the recommendation is to always verify that the output 299assembly is equivalent to the assembly generated in the absence of markers. 300The `Clang options to emit optimization reports <https://clang.llvm.org/docs/UsersManual.html#options-to-emit-optimization-reports>`_ 301can also help in detecting missed optimizations. 302 303HOW LLVM-MCA WORKS 304------------------ 305 306:program:`llvm-mca` takes assembly code as input. The assembly code is parsed 307into a sequence of MCInst with the help of the existing LLVM target assembly 308parsers. The parsed sequence of MCInst is then analyzed by a ``Pipeline`` module 309to generate a performance report. 310 311The Pipeline module simulates the execution of the machine code sequence in a 312loop of iterations (default is 100). During this process, the pipeline collects 313a number of execution related statistics. At the end of this process, the 314pipeline generates and prints a report from the collected statistics. 315 316Here is an example of a performance report generated by the tool for a 317dot-product of two packed float vectors of four elements. The analysis is 318conducted for target x86, cpu btver2. The following result can be produced via 319the following command using the example located at 320``test/tools/llvm-mca/X86/BtVer2/dot-product.s``: 321 322.. code-block:: bash 323 324 $ llvm-mca -mtriple=x86_64-unknown-unknown -mcpu=btver2 -iterations=300 dot-product.s 325 326.. code-block:: none 327 328 Iterations: 300 329 Instructions: 900 330 Total Cycles: 610 331 Total uOps: 900 332 333 Dispatch Width: 2 334 uOps Per Cycle: 1.48 335 IPC: 1.48 336 Block RThroughput: 2.0 337 338 339 Instruction Info: 340 [1]: #uOps 341 [2]: Latency 342 [3]: RThroughput 343 [4]: MayLoad 344 [5]: MayStore 345 [6]: HasSideEffects (U) 346 347 [1] [2] [3] [4] [5] [6] Instructions: 348 1 2 1.00 vmulps %xmm0, %xmm1, %xmm2 349 1 3 1.00 vhaddps %xmm2, %xmm2, %xmm3 350 1 3 1.00 vhaddps %xmm3, %xmm3, %xmm4 351 352 353 Resources: 354 [0] - JALU0 355 [1] - JALU1 356 [2] - JDiv 357 [3] - JFPA 358 [4] - JFPM 359 [5] - JFPU0 360 [6] - JFPU1 361 [7] - JLAGU 362 [8] - JMul 363 [9] - JSAGU 364 [10] - JSTC 365 [11] - JVALU0 366 [12] - JVALU1 367 [13] - JVIMUL 368 369 370 Resource pressure per iteration: 371 [0] [1] [2] [3] [4] [5] [6] [7] [8] [9] [10] [11] [12] [13] 372 - - - 2.00 1.00 2.00 1.00 - - - - - - - 373 374 Resource pressure by instruction: 375 [0] [1] [2] [3] [4] [5] [6] [7] [8] [9] [10] [11] [12] [13] Instructions: 376 - - - - 1.00 - 1.00 - - - - - - - vmulps %xmm0, %xmm1, %xmm2 377 - - - 1.00 - 1.00 - - - - - - - - vhaddps %xmm2, %xmm2, %xmm3 378 - - - 1.00 - 1.00 - - - - - - - - vhaddps %xmm3, %xmm3, %xmm4 379 380According to this report, the dot-product kernel has been executed 300 times, 381for a total of 900 simulated instructions. The total number of simulated micro 382opcodes (uOps) is also 900. 383 384The report is structured in three main sections. The first section collects a 385few performance numbers; the goal of this section is to give a very quick 386overview of the performance throughput. Important performance indicators are 387**IPC**, **uOps Per Cycle**, and **Block RThroughput** (Block Reciprocal 388Throughput). 389 390Field *DispatchWidth* is the maximum number of micro opcodes that are dispatched 391to the out-of-order backend every simulated cycle. 392 393IPC is computed dividing the total number of simulated instructions by the total 394number of cycles. 395 396Field *Block RThroughput* is the reciprocal of the block throughput. Block 397throughput is a theoretical quantity computed as the maximum number of blocks 398(i.e. iterations) that can be executed per simulated clock cycle in the absence 399of loop carried dependencies. Block throughput is superiorly limited by the 400dispatch rate, and the availability of hardware resources. 401 402In the absence of loop-carried data dependencies, the observed IPC tends to a 403theoretical maximum which can be computed by dividing the number of instructions 404of a single iteration by the `Block RThroughput`. 405 406Field 'uOps Per Cycle' is computed dividing the total number of simulated micro 407opcodes by the total number of cycles. A delta between Dispatch Width and this 408field is an indicator of a performance issue. In the absence of loop-carried 409data dependencies, the observed 'uOps Per Cycle' should tend to a theoretical 410maximum throughput which can be computed by dividing the number of uOps of a 411single iteration by the `Block RThroughput`. 412 413Field *uOps Per Cycle* is bounded from above by the dispatch width. That is 414because the dispatch width limits the maximum size of a dispatch group. Both IPC 415and 'uOps Per Cycle' are limited by the amount of hardware parallelism. The 416availability of hardware resources affects the resource pressure distribution, 417and it limits the number of instructions that can be executed in parallel every 418cycle. A delta between Dispatch Width and the theoretical maximum uOps per 419Cycle (computed by dividing the number of uOps of a single iteration by the 420`Block RThroughput`) is an indicator of a performance bottleneck caused by the 421lack of hardware resources. 422In general, the lower the Block RThroughput, the better. 423 424In this example, ``uOps per iteration/Block RThroughput`` is 1.50. Since there 425are no loop-carried dependencies, the observed `uOps Per Cycle` is expected to 426approach 1.50 when the number of iterations tends to infinity. The delta between 427the Dispatch Width (2.00), and the theoretical maximum uOp throughput (1.50) is 428an indicator of a performance bottleneck caused by the lack of hardware 429resources, and the *Resource pressure view* can help to identify the problematic 430resource usage. 431 432The second section of the report is the `instruction info view`. It shows the 433latency and reciprocal throughput of every instruction in the sequence. It also 434reports extra information related to the number of micro opcodes, and opcode 435properties (i.e., 'MayLoad', 'MayStore', and 'HasSideEffects'). 436 437Field *RThroughput* is the reciprocal of the instruction throughput. Throughput 438is computed as the maximum number of instructions of a same type that can be 439executed per clock cycle in the absence of operand dependencies. In this 440example, the reciprocal throughput of a vector float multiply is 1 441cycles/instruction. That is because the FP multiplier JFPM is only available 442from pipeline JFPU1. 443 444Instruction encodings are displayed within the instruction info view when flag 445`-show-encoding` is specified. 446 447Below is an example of `-show-encoding` output for the dot-product kernel: 448 449.. code-block:: none 450 451 Instruction Info: 452 [1]: #uOps 453 [2]: Latency 454 [3]: RThroughput 455 [4]: MayLoad 456 [5]: MayStore 457 [6]: HasSideEffects (U) 458 [7]: Encoding Size 459 460 [1] [2] [3] [4] [5] [6] [7] Encodings: Instructions: 461 1 2 1.00 4 c5 f0 59 d0 vmulps %xmm0, %xmm1, %xmm2 462 1 4 1.00 4 c5 eb 7c da vhaddps %xmm2, %xmm2, %xmm3 463 1 4 1.00 4 c5 e3 7c e3 vhaddps %xmm3, %xmm3, %xmm4 464 465The `Encoding Size` column shows the size in bytes of instructions. The 466`Encodings` column shows the actual instruction encodings (byte sequences in 467hex). 468 469The third section is the *Resource pressure view*. This view reports 470the average number of resource cycles consumed every iteration by instructions 471for every processor resource unit available on the target. Information is 472structured in two tables. The first table reports the number of resource cycles 473spent on average every iteration. The second table correlates the resource 474cycles to the machine instruction in the sequence. For example, every iteration 475of the instruction vmulps always executes on resource unit [6] 476(JFPU1 - floating point pipeline #1), consuming an average of 1 resource cycle 477per iteration. Note that on AMD Jaguar, vector floating-point multiply can 478only be issued to pipeline JFPU1, while horizontal floating-point additions can 479only be issued to pipeline JFPU0. 480 481The resource pressure view helps with identifying bottlenecks caused by high 482usage of specific hardware resources. Situations with resource pressure mainly 483concentrated on a few resources should, in general, be avoided. Ideally, 484pressure should be uniformly distributed between multiple resources. 485 486Timeline View 487^^^^^^^^^^^^^ 488The timeline view produces a detailed report of each instruction's state 489transitions through an instruction pipeline. This view is enabled by the 490command line option ``-timeline``. As instructions transition through the 491various stages of the pipeline, their states are depicted in the view report. 492These states are represented by the following characters: 493 494* D : Instruction dispatched. 495* e : Instruction executing. 496* E : Instruction executed. 497* R : Instruction retired. 498* = : Instruction already dispatched, waiting to be executed. 499* \- : Instruction executed, waiting to be retired. 500 501Below is the timeline view for a subset of the dot-product example located in 502``test/tools/llvm-mca/X86/BtVer2/dot-product.s`` and processed by 503:program:`llvm-mca` using the following command: 504 505.. code-block:: bash 506 507 $ llvm-mca -mtriple=x86_64-unknown-unknown -mcpu=btver2 -iterations=3 -timeline dot-product.s 508 509.. code-block:: none 510 511 Timeline view: 512 012345 513 Index 0123456789 514 515 [0,0] DeeER. . . vmulps %xmm0, %xmm1, %xmm2 516 [0,1] D==eeeER . . vhaddps %xmm2, %xmm2, %xmm3 517 [0,2] .D====eeeER . vhaddps %xmm3, %xmm3, %xmm4 518 [1,0] .DeeE-----R . vmulps %xmm0, %xmm1, %xmm2 519 [1,1] . D=eeeE---R . vhaddps %xmm2, %xmm2, %xmm3 520 [1,2] . D====eeeER . vhaddps %xmm3, %xmm3, %xmm4 521 [2,0] . DeeE-----R . vmulps %xmm0, %xmm1, %xmm2 522 [2,1] . D====eeeER . vhaddps %xmm2, %xmm2, %xmm3 523 [2,2] . D======eeeER vhaddps %xmm3, %xmm3, %xmm4 524 525 526 Average Wait times (based on the timeline view): 527 [0]: Executions 528 [1]: Average time spent waiting in a scheduler's queue 529 [2]: Average time spent waiting in a scheduler's queue while ready 530 [3]: Average time elapsed from WB until retire stage 531 532 [0] [1] [2] [3] 533 0. 3 1.0 1.0 3.3 vmulps %xmm0, %xmm1, %xmm2 534 1. 3 3.3 0.7 1.0 vhaddps %xmm2, %xmm2, %xmm3 535 2. 3 5.7 0.0 0.0 vhaddps %xmm3, %xmm3, %xmm4 536 3 3.3 0.5 1.4 <total> 537 538The timeline view is interesting because it shows instruction state changes 539during execution. It also gives an idea of how the tool processes instructions 540executed on the target, and how their timing information might be calculated. 541 542The timeline view is structured in two tables. The first table shows 543instructions changing state over time (measured in cycles); the second table 544(named *Average Wait times*) reports useful timing statistics, which should 545help diagnose performance bottlenecks caused by long data dependencies and 546sub-optimal usage of hardware resources. 547 548An instruction in the timeline view is identified by a pair of indices, where 549the first index identifies an iteration, and the second index is the 550instruction index (i.e., where it appears in the code sequence). Since this 551example was generated using 3 iterations: ``-iterations=3``, the iteration 552indices range from 0-2 inclusively. 553 554Excluding the first and last column, the remaining columns are in cycles. 555Cycles are numbered sequentially starting from 0. 556 557From the example output above, we know the following: 558 559* Instruction [1,0] was dispatched at cycle 1. 560* Instruction [1,0] started executing at cycle 2. 561* Instruction [1,0] reached the write back stage at cycle 4. 562* Instruction [1,0] was retired at cycle 10. 563 564Instruction [1,0] (i.e., vmulps from iteration #1) does not have to wait in the 565scheduler's queue for the operands to become available. By the time vmulps is 566dispatched, operands are already available, and pipeline JFPU1 is ready to 567serve another instruction. So the instruction can be immediately issued on the 568JFPU1 pipeline. That is demonstrated by the fact that the instruction only 569spent 1cy in the scheduler's queue. 570 571There is a gap of 5 cycles between the write-back stage and the retire event. 572That is because instructions must retire in program order, so [1,0] has to wait 573for [0,2] to be retired first (i.e., it has to wait until cycle 10). 574 575In the example, all instructions are in a RAW (Read After Write) dependency 576chain. Register %xmm2 written by vmulps is immediately used by the first 577vhaddps, and register %xmm3 written by the first vhaddps is used by the second 578vhaddps. Long data dependencies negatively impact the ILP (Instruction Level 579Parallelism). 580 581In the dot-product example, there are anti-dependencies introduced by 582instructions from different iterations. However, those dependencies can be 583removed at register renaming stage (at the cost of allocating register aliases, 584and therefore consuming physical registers). 585 586Table *Average Wait times* helps diagnose performance issues that are caused by 587the presence of long latency instructions and potentially long data dependencies 588which may limit the ILP. Last row, ``<total>``, shows a global average over all 589instructions measured. Note that :program:`llvm-mca`, by default, assumes at 590least 1cy between the dispatch event and the issue event. 591 592When the performance is limited by data dependencies and/or long latency 593instructions, the number of cycles spent while in the *ready* state is expected 594to be very small when compared with the total number of cycles spent in the 595scheduler's queue. The difference between the two counters is a good indicator 596of how large of an impact data dependencies had on the execution of the 597instructions. When performance is mostly limited by the lack of hardware 598resources, the delta between the two counters is small. However, the number of 599cycles spent in the queue tends to be larger (i.e., more than 1-3cy), 600especially when compared to other low latency instructions. 601 602Bottleneck Analysis 603^^^^^^^^^^^^^^^^^^^ 604The ``-bottleneck-analysis`` command line option enables the analysis of 605performance bottlenecks. 606 607This analysis is potentially expensive. It attempts to correlate increases in 608backend pressure (caused by pipeline resource pressure and data dependencies) to 609dynamic dispatch stalls. 610 611Below is an example of ``-bottleneck-analysis`` output generated by 612:program:`llvm-mca` for 500 iterations of the dot-product example on btver2. 613 614.. code-block:: none 615 616 617 Cycles with backend pressure increase [ 48.07% ] 618 Throughput Bottlenecks: 619 Resource Pressure [ 47.77% ] 620 - JFPA [ 47.77% ] 621 - JFPU0 [ 47.77% ] 622 Data Dependencies: [ 0.30% ] 623 - Register Dependencies [ 0.30% ] 624 - Memory Dependencies [ 0.00% ] 625 626 Critical sequence based on the simulation: 627 628 Instruction Dependency Information 629 +----< 2. vhaddps %xmm3, %xmm3, %xmm4 630 | 631 | < loop carried > 632 | 633 | 0. vmulps %xmm0, %xmm1, %xmm2 634 +----> 1. vhaddps %xmm2, %xmm2, %xmm3 ## RESOURCE interference: JFPA [ probability: 74% ] 635 +----> 2. vhaddps %xmm3, %xmm3, %xmm4 ## REGISTER dependency: %xmm3 636 | 637 | < loop carried > 638 | 639 +----> 1. vhaddps %xmm2, %xmm2, %xmm3 ## RESOURCE interference: JFPA [ probability: 74% ] 640 641 642According to the analysis, throughput is limited by resource pressure and not by 643data dependencies. The analysis observed increases in backend pressure during 64448.07% of the simulated run. Almost all those pressure increase events were 645caused by contention on processor resources JFPA/JFPU0. 646 647The `critical sequence` is the most expensive sequence of instructions according 648to the simulation. It is annotated to provide extra information about critical 649register dependencies and resource interferences between instructions. 650 651Instructions from the critical sequence are expected to significantly impact 652performance. By construction, the accuracy of this analysis is strongly 653dependent on the simulation and (as always) by the quality of the processor 654model in llvm. 655 656 657Extra Statistics to Further Diagnose Performance Issues 658^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 659The ``-all-stats`` command line option enables extra statistics and performance 660counters for the dispatch logic, the reorder buffer, the retire control unit, 661and the register file. 662 663Below is an example of ``-all-stats`` output generated by :program:`llvm-mca` 664for 300 iterations of the dot-product example discussed in the previous 665sections. 666 667.. code-block:: none 668 669 Dynamic Dispatch Stall Cycles: 670 RAT - Register unavailable: 0 671 RCU - Retire tokens unavailable: 0 672 SCHEDQ - Scheduler full: 272 (44.6%) 673 LQ - Load queue full: 0 674 SQ - Store queue full: 0 675 GROUP - Static restrictions on the dispatch group: 0 676 677 678 Dispatch Logic - number of cycles where we saw N micro opcodes dispatched: 679 [# dispatched], [# cycles] 680 0, 24 (3.9%) 681 1, 272 (44.6%) 682 2, 314 (51.5%) 683 684 685 Schedulers - number of cycles where we saw N micro opcodes issued: 686 [# issued], [# cycles] 687 0, 7 (1.1%) 688 1, 306 (50.2%) 689 2, 297 (48.7%) 690 691 Scheduler's queue usage: 692 [1] Resource name. 693 [2] Average number of used buffer entries. 694 [3] Maximum number of used buffer entries. 695 [4] Total number of buffer entries. 696 697 [1] [2] [3] [4] 698 JALU01 0 0 20 699 JFPU01 17 18 18 700 JLSAGU 0 0 12 701 702 703 Retire Control Unit - number of cycles where we saw N instructions retired: 704 [# retired], [# cycles] 705 0, 109 (17.9%) 706 1, 102 (16.7%) 707 2, 399 (65.4%) 708 709 Total ROB Entries: 64 710 Max Used ROB Entries: 35 ( 54.7% ) 711 Average Used ROB Entries per cy: 32 ( 50.0% ) 712 713 714 Register File statistics: 715 Total number of mappings created: 900 716 Max number of mappings used: 35 717 718 * Register File #1 -- JFpuPRF: 719 Number of physical registers: 72 720 Total number of mappings created: 900 721 Max number of mappings used: 35 722 723 * Register File #2 -- JIntegerPRF: 724 Number of physical registers: 64 725 Total number of mappings created: 0 726 Max number of mappings used: 0 727 728If we look at the *Dynamic Dispatch Stall Cycles* table, we see the counter for 729SCHEDQ reports 272 cycles. This counter is incremented every time the dispatch 730logic is unable to dispatch a full group because the scheduler's queue is full. 731 732Looking at the *Dispatch Logic* table, we see that the pipeline was only able to 733dispatch two micro opcodes 51.5% of the time. The dispatch group was limited to 734one micro opcode 44.6% of the cycles, which corresponds to 272 cycles. The 735dispatch statistics are displayed by either using the command option 736``-all-stats`` or ``-dispatch-stats``. 737 738The next table, *Schedulers*, presents a histogram displaying a count, 739representing the number of micro opcodes issued on some number of cycles. In 740this case, of the 610 simulated cycles, single opcodes were issued 306 times 741(50.2%) and there were 7 cycles where no opcodes were issued. 742 743The *Scheduler's queue usage* table shows that the average and maximum number of 744buffer entries (i.e., scheduler queue entries) used at runtime. Resource JFPU01 745reached its maximum (18 of 18 queue entries). Note that AMD Jaguar implements 746three schedulers: 747 748* JALU01 - A scheduler for ALU instructions. 749* JFPU01 - A scheduler floating point operations. 750* JLSAGU - A scheduler for address generation. 751 752The dot-product is a kernel of three floating point instructions (a vector 753multiply followed by two horizontal adds). That explains why only the floating 754point scheduler appears to be used. 755 756A full scheduler queue is either caused by data dependency chains or by a 757sub-optimal usage of hardware resources. Sometimes, resource pressure can be 758mitigated by rewriting the kernel using different instructions that consume 759different scheduler resources. Schedulers with a small queue are less resilient 760to bottlenecks caused by the presence of long data dependencies. The scheduler 761statistics are displayed by using the command option ``-all-stats`` or 762``-scheduler-stats``. 763 764The next table, *Retire Control Unit*, presents a histogram displaying a count, 765representing the number of instructions retired on some number of cycles. In 766this case, of the 610 simulated cycles, two instructions were retired during the 767same cycle 399 times (65.4%) and there were 109 cycles where no instructions 768were retired. The retire statistics are displayed by using the command option 769``-all-stats`` or ``-retire-stats``. 770 771The last table presented is *Register File statistics*. Each physical register 772file (PRF) used by the pipeline is presented in this table. In the case of AMD 773Jaguar, there are two register files, one for floating-point registers (JFpuPRF) 774and one for integer registers (JIntegerPRF). The table shows that of the 900 775instructions processed, there were 900 mappings created. Since this dot-product 776example utilized only floating point registers, the JFPuPRF was responsible for 777creating the 900 mappings. However, we see that the pipeline only used a 778maximum of 35 of 72 available register slots at any given time. We can conclude 779that the floating point PRF was the only register file used for the example, and 780that it was never resource constrained. The register file statistics are 781displayed by using the command option ``-all-stats`` or 782``-register-file-stats``. 783 784In this example, we can conclude that the IPC is mostly limited by data 785dependencies, and not by resource pressure. 786 787Instruction Flow 788^^^^^^^^^^^^^^^^ 789This section describes the instruction flow through the default pipeline of 790:program:`llvm-mca`, as well as the functional units involved in the process. 791 792The default pipeline implements the following sequence of stages used to 793process instructions. 794 795* Dispatch (Instruction is dispatched to the schedulers). 796* Issue (Instruction is issued to the processor pipelines). 797* Write Back (Instruction is executed, and results are written back). 798* Retire (Instruction is retired; writes are architecturally committed). 799 800The default pipeline only models the out-of-order portion of a processor. 801Therefore, the instruction fetch and decode stages are not modeled. Performance 802bottlenecks in the frontend are not diagnosed. :program:`llvm-mca` assumes that 803instructions have all been decoded and placed into a queue before the simulation 804start. Also, :program:`llvm-mca` does not model branch prediction. 805 806Instruction Dispatch 807"""""""""""""""""""" 808During the dispatch stage, instructions are picked in program order from a 809queue of already decoded instructions, and dispatched in groups to the 810simulated hardware schedulers. 811 812The size of a dispatch group depends on the availability of the simulated 813hardware resources. The processor dispatch width defaults to the value 814of the ``IssueWidth`` in LLVM's scheduling model. 815 816An instruction can be dispatched if: 817 818* The size of the dispatch group is smaller than processor's dispatch width. 819* There are enough entries in the reorder buffer. 820* There are enough physical registers to do register renaming. 821* The schedulers are not full. 822 823Scheduling models can optionally specify which register files are available on 824the processor. :program:`llvm-mca` uses that information to initialize register 825file descriptors. Users can limit the number of physical registers that are 826globally available for register renaming by using the command option 827``-register-file-size``. A value of zero for this option means *unbounded*. By 828knowing how many registers are available for renaming, the tool can predict 829dispatch stalls caused by the lack of physical registers. 830 831The number of reorder buffer entries consumed by an instruction depends on the 832number of micro-opcodes specified for that instruction by the target scheduling 833model. The reorder buffer is responsible for tracking the progress of 834instructions that are "in-flight", and retiring them in program order. The 835number of entries in the reorder buffer defaults to the value specified by field 836`MicroOpBufferSize` in the target scheduling model. 837 838Instructions that are dispatched to the schedulers consume scheduler buffer 839entries. :program:`llvm-mca` queries the scheduling model to determine the set 840of buffered resources consumed by an instruction. Buffered resources are 841treated like scheduler resources. 842 843Instruction Issue 844""""""""""""""""" 845Each processor scheduler implements a buffer of instructions. An instruction 846has to wait in the scheduler's buffer until input register operands become 847available. Only at that point, does the instruction becomes eligible for 848execution and may be issued (potentially out-of-order) for execution. 849Instruction latencies are computed by :program:`llvm-mca` with the help of the 850scheduling model. 851 852:program:`llvm-mca`'s scheduler is designed to simulate multiple processor 853schedulers. The scheduler is responsible for tracking data dependencies, and 854dynamically selecting which processor resources are consumed by instructions. 855It delegates the management of processor resource units and resource groups to a 856resource manager. The resource manager is responsible for selecting resource 857units that are consumed by instructions. For example, if an instruction 858consumes 1cy of a resource group, the resource manager selects one of the 859available units from the group; by default, the resource manager uses a 860round-robin selector to guarantee that resource usage is uniformly distributed 861between all units of a group. 862 863:program:`llvm-mca`'s scheduler internally groups instructions into three sets: 864 865* WaitSet: a set of instructions whose operands are not ready. 866* ReadySet: a set of instructions ready to execute. 867* IssuedSet: a set of instructions executing. 868 869Depending on the operands availability, instructions that are dispatched to the 870scheduler are either placed into the WaitSet or into the ReadySet. 871 872Every cycle, the scheduler checks if instructions can be moved from the WaitSet 873to the ReadySet, and if instructions from the ReadySet can be issued to the 874underlying pipelines. The algorithm prioritizes older instructions over younger 875instructions. 876 877Write-Back and Retire Stage 878""""""""""""""""""""""""""" 879Issued instructions are moved from the ReadySet to the IssuedSet. There, 880instructions wait until they reach the write-back stage. At that point, they 881get removed from the queue and the retire control unit is notified. 882 883When instructions are executed, the retire control unit flags the instruction as 884"ready to retire." 885 886Instructions are retired in program order. The register file is notified of the 887retirement so that it can free the physical registers that were allocated for 888the instruction during the register renaming stage. 889 890Load/Store Unit and Memory Consistency Model 891"""""""""""""""""""""""""""""""""""""""""""" 892To simulate an out-of-order execution of memory operations, :program:`llvm-mca` 893utilizes a simulated load/store unit (LSUnit) to simulate the speculative 894execution of loads and stores. 895 896Each load (or store) consumes an entry in the load (or store) queue. Users can 897specify flags ``-lqueue`` and ``-squeue`` to limit the number of entries in the 898load and store queues respectively. The queues are unbounded by default. 899 900The LSUnit implements a relaxed consistency model for memory loads and stores. 901The rules are: 902 9031. A younger load is allowed to pass an older load only if there are no 904 intervening stores or barriers between the two loads. 9052. A younger load is allowed to pass an older store provided that the load does 906 not alias with the store. 9073. A younger store is not allowed to pass an older store. 9084. A younger store is not allowed to pass an older load. 909 910By default, the LSUnit optimistically assumes that loads do not alias 911(`-noalias=true`) store operations. Under this assumption, younger loads are 912always allowed to pass older stores. Essentially, the LSUnit does not attempt 913to run any alias analysis to predict when loads and stores do not alias with 914each other. 915 916Note that, in the case of write-combining memory, rule 3 could be relaxed to 917allow reordering of non-aliasing store operations. That being said, at the 918moment, there is no way to further relax the memory model (``-noalias`` is the 919only option). Essentially, there is no option to specify a different memory 920type (e.g., write-back, write-combining, write-through; etc.) and consequently 921to weaken, or strengthen, the memory model. 922 923Other limitations are: 924 925* The LSUnit does not know when store-to-load forwarding may occur. 926* The LSUnit does not know anything about cache hierarchy and memory types. 927* The LSUnit does not know how to identify serializing operations and memory 928 fences. 929 930The LSUnit does not attempt to predict if a load or store hits or misses the L1 931cache. It only knows if an instruction "MayLoad" and/or "MayStore." For 932loads, the scheduling model provides an "optimistic" load-to-use latency (which 933usually matches the load-to-use latency for when there is a hit in the L1D). 934 935:program:`llvm-mca` does not know about serializing operations or memory-barrier 936like instructions. The LSUnit conservatively assumes that an instruction which 937has both "MayLoad" and unmodeled side effects behaves like a "soft" 938load-barrier. That means, it serializes loads without forcing a flush of the 939load queue. Similarly, instructions that "MayStore" and have unmodeled side 940effects are treated like store barriers. A full memory barrier is a "MayLoad" 941and "MayStore" instruction with unmodeled side effects. This is inaccurate, but 942it is the best that we can do at the moment with the current information 943available in LLVM. 944 945A load/store barrier consumes one entry of the load/store queue. A load/store 946barrier enforces ordering of loads/stores. A younger load cannot pass a load 947barrier. Also, a younger store cannot pass a store barrier. A younger load 948has to wait for the memory/load barrier to execute. A load/store barrier is 949"executed" when it becomes the oldest entry in the load/store queue(s). That 950also means, by construction, all of the older loads/stores have been executed. 951 952In conclusion, the full set of load/store consistency rules are: 953 954#. A store may not pass a previous store. 955#. A store may not pass a previous load (regardless of ``-noalias``). 956#. A store has to wait until an older store barrier is fully executed. 957#. A load may pass a previous load. 958#. A load may not pass a previous store unless ``-noalias`` is set. 959#. A load has to wait until an older load barrier is fully executed. 960