1llvm-mca - LLVM Machine Code Analyzer 2===================================== 3 4.. program:: llvm-mca 5 6SYNOPSIS 7-------- 8 9:program:`llvm-mca` [*options*] [input] 10 11DESCRIPTION 12----------- 13 14:program:`llvm-mca` is a performance analysis tool that uses information 15available in LLVM (e.g. scheduling models) to statically measure the performance 16of machine code in a specific CPU. 17 18Performance is measured in terms of throughput as well as processor resource 19consumption. The tool currently works for processors with a backend for which 20there is a scheduling model available in LLVM. 21 22The main goal of this tool is not just to predict the performance of the code 23when run on the target, but also help with diagnosing potential performance 24issues. 25 26Given an assembly code sequence, :program:`llvm-mca` estimates the Instructions 27Per Cycle (IPC), as well as hardware resource pressure. The analysis and 28reporting style were inspired by the IACA tool from Intel. 29 30For example, you can compile code with clang, output assembly, and pipe it 31directly into :program:`llvm-mca` for analysis: 32 33.. code-block:: bash 34 35 $ clang foo.c -O2 -target x86_64-unknown-unknown -S -o - | llvm-mca -mcpu=btver2 36 37Or for Intel syntax: 38 39.. code-block:: bash 40 41 $ clang foo.c -O2 -target x86_64-unknown-unknown -mllvm -x86-asm-syntax=intel -S -o - | llvm-mca -mcpu=btver2 42 43(:program:`llvm-mca` detects Intel syntax by the presence of an `.intel_syntax` 44directive at the beginning of the input. By default its output syntax matches 45that of its input.) 46 47Scheduling models are not just used to compute instruction latencies and 48throughput, but also to understand what processor resources are available 49and how to simulate them. 50 51By design, the quality of the analysis conducted by :program:`llvm-mca` is 52inevitably affected by the quality of the scheduling models in LLVM. 53 54If you see that the performance report is not accurate for a processor, 55please `file a bug <https://bugs.llvm.org/enter_bug.cgi?product=libraries>`_ 56against the appropriate backend. 57 58OPTIONS 59------- 60 61If ``input`` is "``-``" or omitted, :program:`llvm-mca` reads from standard 62input. Otherwise, it will read from the specified filename. 63 64If the :option:`-o` option is omitted, then :program:`llvm-mca` will send its output 65to standard output if the input is from standard input. If the :option:`-o` 66option specifies "``-``", then the output will also be sent to standard output. 67 68 69.. option:: -help 70 71 Print a summary of command line options. 72 73.. option:: -o <filename> 74 75 Use ``<filename>`` as the output filename. See the summary above for more 76 details. 77 78.. option:: -mtriple=<target triple> 79 80 Specify a target triple string. 81 82.. option:: -march=<arch> 83 84 Specify the architecture for which to analyze the code. It defaults to the 85 host default target. 86 87.. option:: -mcpu=<cpuname> 88 89 Specify the processor for which to analyze the code. By default, the cpu name 90 is autodetected from the host. 91 92.. option:: -output-asm-variant=<variant id> 93 94 Specify the output assembly variant for the report generated by the tool. 95 On x86, possible values are [0, 1]. A value of 0 (vic. 1) for this flag enables 96 the AT&T (vic. Intel) assembly format for the code printed out by the tool in 97 the analysis report. 98 99.. option:: -print-imm-hex 100 101 Prefer hex format for numeric literals in the output assembly printed as part 102 of the report. 103 104.. option:: -dispatch=<width> 105 106 Specify a different dispatch width for the processor. The dispatch width 107 defaults to field 'IssueWidth' in the processor scheduling model. If width is 108 zero, then the default dispatch width is used. 109 110.. option:: -register-file-size=<size> 111 112 Specify the size of the register file. When specified, this flag limits how 113 many physical registers are available for register renaming purposes. A value 114 of zero for this flag means "unlimited number of physical registers". 115 116.. option:: -iterations=<number of iterations> 117 118 Specify the number of iterations to run. If this flag is set to 0, then the 119 tool sets the number of iterations to a default value (i.e. 100). 120 121.. option:: -noalias=<bool> 122 123 If set, the tool assumes that loads and stores don't alias. This is the 124 default behavior. 125 126.. option:: -lqueue=<load queue size> 127 128 Specify the size of the load queue in the load/store unit emulated by the tool. 129 By default, the tool assumes an unbound number of entries in the load queue. 130 A value of zero for this flag is ignored, and the default load queue size is 131 used instead. 132 133.. option:: -squeue=<store queue size> 134 135 Specify the size of the store queue in the load/store unit emulated by the 136 tool. By default, the tool assumes an unbound number of entries in the store 137 queue. A value of zero for this flag is ignored, and the default store queue 138 size is used instead. 139 140.. option:: -timeline 141 142 Enable the timeline view. 143 144.. option:: -timeline-max-iterations=<iterations> 145 146 Limit the number of iterations to print in the timeline view. By default, the 147 timeline view prints information for up to 10 iterations. 148 149.. option:: -timeline-max-cycles=<cycles> 150 151 Limit the number of cycles in the timeline view, or use 0 for no limit. By 152 default, the number of cycles is set to 80. 153 154.. option:: -resource-pressure 155 156 Enable the resource pressure view. This is enabled by default. 157 158.. option:: -register-file-stats 159 160 Enable register file usage statistics. 161 162.. option:: -dispatch-stats 163 164 Enable extra dispatch statistics. This view collects and analyzes instruction 165 dispatch events, as well as static/dynamic dispatch stall events. This view 166 is disabled by default. 167 168.. option:: -scheduler-stats 169 170 Enable extra scheduler statistics. This view collects and analyzes instruction 171 issue events. This view is disabled by default. 172 173.. option:: -retire-stats 174 175 Enable extra retire control unit statistics. This view is disabled by default. 176 177.. option:: -instruction-info 178 179 Enable the instruction info view. This is enabled by default. 180 181.. option:: -show-encoding 182 183 Enable the printing of instruction encodings within the instruction info view. 184 185.. option:: -all-stats 186 187 Print all hardware statistics. This enables extra statistics related to the 188 dispatch logic, the hardware schedulers, the register file(s), and the retire 189 control unit. This option is disabled by default. 190 191.. option:: -all-views 192 193 Enable all the view. 194 195.. option:: -instruction-tables 196 197 Prints resource pressure information based on the static information 198 available from the processor model. This differs from the resource pressure 199 view because it doesn't require that the code is simulated. It instead prints 200 the theoretical uniform distribution of resource pressure for every 201 instruction in sequence. 202 203.. option:: -bottleneck-analysis 204 205 Print information about bottlenecks that affect the throughput. This analysis 206 can be expensive, and it is disabled by default. Bottlenecks are highlighted 207 in the summary view. Bottleneck analysis is currently not supported for 208 processors with an in-order backend. 209 210.. option:: -json 211 212 Print the requested views in JSON format. The instructions and the processor 213 resources are printed as members of special top level JSON objects. The 214 individual views refer to them by index. 215 216.. option:: -disable-cb 217 218 Force usage of the generic CustomBehaviour class rather than using the target 219 specific class. The generic class never detects any custom hazards. 220 221 222EXIT STATUS 223----------- 224 225:program:`llvm-mca` returns 0 on success. Otherwise, an error message is printed 226to standard error, and the tool returns 1. 227 228USING MARKERS TO ANALYZE SPECIFIC CODE BLOCKS 229--------------------------------------------- 230:program:`llvm-mca` allows for the optional usage of special code comments to 231mark regions of the assembly code to be analyzed. A comment starting with 232substring ``LLVM-MCA-BEGIN`` marks the beginning of a code region. A comment 233starting with substring ``LLVM-MCA-END`` marks the end of a code region. For 234example: 235 236.. code-block:: none 237 238 # LLVM-MCA-BEGIN 239 ... 240 # LLVM-MCA-END 241 242If no user-defined region is specified, then :program:`llvm-mca` assumes a 243default region which contains every instruction in the input file. Every region 244is analyzed in isolation, and the final performance report is the union of all 245the reports generated for every code region. 246 247Code regions can have names. For example: 248 249.. code-block:: none 250 251 # LLVM-MCA-BEGIN A simple example 252 add %eax, %eax 253 # LLVM-MCA-END 254 255The code from the example above defines a region named "A simple example" with a 256single instruction in it. Note how the region name doesn't have to be repeated 257in the ``LLVM-MCA-END`` directive. In the absence of overlapping regions, 258an anonymous ``LLVM-MCA-END`` directive always ends the currently active user 259defined region. 260 261Example of nesting regions: 262 263.. code-block:: none 264 265 # LLVM-MCA-BEGIN foo 266 add %eax, %edx 267 # LLVM-MCA-BEGIN bar 268 sub %eax, %edx 269 # LLVM-MCA-END bar 270 # LLVM-MCA-END foo 271 272Example of overlapping regions: 273 274.. code-block:: none 275 276 # LLVM-MCA-BEGIN foo 277 add %eax, %edx 278 # LLVM-MCA-BEGIN bar 279 sub %eax, %edx 280 # LLVM-MCA-END foo 281 add %eax, %edx 282 # LLVM-MCA-END bar 283 284Note that multiple anonymous regions cannot overlap. Also, overlapping regions 285cannot have the same name. 286 287There is no support for marking regions from high-level source code, like C or 288C++. As a workaround, inline assembly directives may be used: 289 290.. code-block:: c++ 291 292 int foo(int a, int b) { 293 __asm volatile("# LLVM-MCA-BEGIN foo"); 294 a += 42; 295 __asm volatile("# LLVM-MCA-END"); 296 a *= b; 297 return a; 298 } 299 300However, this interferes with optimizations like loop vectorization and may have 301an impact on the code generated. This is because the ``__asm`` statements are 302seen as real code having important side effects, which limits how the code 303around them can be transformed. If users want to make use of inline assembly 304to emit markers, then the recommendation is to always verify that the output 305assembly is equivalent to the assembly generated in the absence of markers. 306The `Clang options to emit optimization reports <https://clang.llvm.org/docs/UsersManual.html#options-to-emit-optimization-reports>`_ 307can also help in detecting missed optimizations. 308 309HOW LLVM-MCA WORKS 310------------------ 311 312:program:`llvm-mca` takes assembly code as input. The assembly code is parsed 313into a sequence of MCInst with the help of the existing LLVM target assembly 314parsers. The parsed sequence of MCInst is then analyzed by a ``Pipeline`` module 315to generate a performance report. 316 317The Pipeline module simulates the execution of the machine code sequence in a 318loop of iterations (default is 100). During this process, the pipeline collects 319a number of execution related statistics. At the end of this process, the 320pipeline generates and prints a report from the collected statistics. 321 322Here is an example of a performance report generated by the tool for a 323dot-product of two packed float vectors of four elements. The analysis is 324conducted for target x86, cpu btver2. The following result can be produced via 325the following command using the example located at 326``test/tools/llvm-mca/X86/BtVer2/dot-product.s``: 327 328.. code-block:: bash 329 330 $ llvm-mca -mtriple=x86_64-unknown-unknown -mcpu=btver2 -iterations=300 dot-product.s 331 332.. code-block:: none 333 334 Iterations: 300 335 Instructions: 900 336 Total Cycles: 610 337 Total uOps: 900 338 339 Dispatch Width: 2 340 uOps Per Cycle: 1.48 341 IPC: 1.48 342 Block RThroughput: 2.0 343 344 345 Instruction Info: 346 [1]: #uOps 347 [2]: Latency 348 [3]: RThroughput 349 [4]: MayLoad 350 [5]: MayStore 351 [6]: HasSideEffects (U) 352 353 [1] [2] [3] [4] [5] [6] Instructions: 354 1 2 1.00 vmulps %xmm0, %xmm1, %xmm2 355 1 3 1.00 vhaddps %xmm2, %xmm2, %xmm3 356 1 3 1.00 vhaddps %xmm3, %xmm3, %xmm4 357 358 359 Resources: 360 [0] - JALU0 361 [1] - JALU1 362 [2] - JDiv 363 [3] - JFPA 364 [4] - JFPM 365 [5] - JFPU0 366 [6] - JFPU1 367 [7] - JLAGU 368 [8] - JMul 369 [9] - JSAGU 370 [10] - JSTC 371 [11] - JVALU0 372 [12] - JVALU1 373 [13] - JVIMUL 374 375 376 Resource pressure per iteration: 377 [0] [1] [2] [3] [4] [5] [6] [7] [8] [9] [10] [11] [12] [13] 378 - - - 2.00 1.00 2.00 1.00 - - - - - - - 379 380 Resource pressure by instruction: 381 [0] [1] [2] [3] [4] [5] [6] [7] [8] [9] [10] [11] [12] [13] Instructions: 382 - - - - 1.00 - 1.00 - - - - - - - vmulps %xmm0, %xmm1, %xmm2 383 - - - 1.00 - 1.00 - - - - - - - - vhaddps %xmm2, %xmm2, %xmm3 384 - - - 1.00 - 1.00 - - - - - - - - vhaddps %xmm3, %xmm3, %xmm4 385 386According to this report, the dot-product kernel has been executed 300 times, 387for a total of 900 simulated instructions. The total number of simulated micro 388opcodes (uOps) is also 900. 389 390The report is structured in three main sections. The first section collects a 391few performance numbers; the goal of this section is to give a very quick 392overview of the performance throughput. Important performance indicators are 393**IPC**, **uOps Per Cycle**, and **Block RThroughput** (Block Reciprocal 394Throughput). 395 396Field *DispatchWidth* is the maximum number of micro opcodes that are dispatched 397to the out-of-order backend every simulated cycle. For processors with an 398in-order backend, *DispatchWidth* is the maximum number of micro opcodes issued 399to the backend every simulated cycle. 400 401IPC is computed dividing the total number of simulated instructions by the total 402number of cycles. 403 404Field *Block RThroughput* is the reciprocal of the block throughput. Block 405throughput is a theoretical quantity computed as the maximum number of blocks 406(i.e. iterations) that can be executed per simulated clock cycle in the absence 407of loop carried dependencies. Block throughput is superiorly limited by the 408dispatch rate, and the availability of hardware resources. 409 410In the absence of loop-carried data dependencies, the observed IPC tends to a 411theoretical maximum which can be computed by dividing the number of instructions 412of a single iteration by the `Block RThroughput`. 413 414Field 'uOps Per Cycle' is computed dividing the total number of simulated micro 415opcodes by the total number of cycles. A delta between Dispatch Width and this 416field is an indicator of a performance issue. In the absence of loop-carried 417data dependencies, the observed 'uOps Per Cycle' should tend to a theoretical 418maximum throughput which can be computed by dividing the number of uOps of a 419single iteration by the `Block RThroughput`. 420 421Field *uOps Per Cycle* is bounded from above by the dispatch width. That is 422because the dispatch width limits the maximum size of a dispatch group. Both IPC 423and 'uOps Per Cycle' are limited by the amount of hardware parallelism. The 424availability of hardware resources affects the resource pressure distribution, 425and it limits the number of instructions that can be executed in parallel every 426cycle. A delta between Dispatch Width and the theoretical maximum uOps per 427Cycle (computed by dividing the number of uOps of a single iteration by the 428`Block RThroughput`) is an indicator of a performance bottleneck caused by the 429lack of hardware resources. 430In general, the lower the Block RThroughput, the better. 431 432In this example, ``uOps per iteration/Block RThroughput`` is 1.50. Since there 433are no loop-carried dependencies, the observed `uOps Per Cycle` is expected to 434approach 1.50 when the number of iterations tends to infinity. The delta between 435the Dispatch Width (2.00), and the theoretical maximum uOp throughput (1.50) is 436an indicator of a performance bottleneck caused by the lack of hardware 437resources, and the *Resource pressure view* can help to identify the problematic 438resource usage. 439 440The second section of the report is the `instruction info view`. It shows the 441latency and reciprocal throughput of every instruction in the sequence. It also 442reports extra information related to the number of micro opcodes, and opcode 443properties (i.e., 'MayLoad', 'MayStore', and 'HasSideEffects'). 444 445Field *RThroughput* is the reciprocal of the instruction throughput. Throughput 446is computed as the maximum number of instructions of a same type that can be 447executed per clock cycle in the absence of operand dependencies. In this 448example, the reciprocal throughput of a vector float multiply is 1 449cycles/instruction. That is because the FP multiplier JFPM is only available 450from pipeline JFPU1. 451 452Instruction encodings are displayed within the instruction info view when flag 453`-show-encoding` is specified. 454 455Below is an example of `-show-encoding` output for the dot-product kernel: 456 457.. code-block:: none 458 459 Instruction Info: 460 [1]: #uOps 461 [2]: Latency 462 [3]: RThroughput 463 [4]: MayLoad 464 [5]: MayStore 465 [6]: HasSideEffects (U) 466 [7]: Encoding Size 467 468 [1] [2] [3] [4] [5] [6] [7] Encodings: Instructions: 469 1 2 1.00 4 c5 f0 59 d0 vmulps %xmm0, %xmm1, %xmm2 470 1 4 1.00 4 c5 eb 7c da vhaddps %xmm2, %xmm2, %xmm3 471 1 4 1.00 4 c5 e3 7c e3 vhaddps %xmm3, %xmm3, %xmm4 472 473The `Encoding Size` column shows the size in bytes of instructions. The 474`Encodings` column shows the actual instruction encodings (byte sequences in 475hex). 476 477The third section is the *Resource pressure view*. This view reports 478the average number of resource cycles consumed every iteration by instructions 479for every processor resource unit available on the target. Information is 480structured in two tables. The first table reports the number of resource cycles 481spent on average every iteration. The second table correlates the resource 482cycles to the machine instruction in the sequence. For example, every iteration 483of the instruction vmulps always executes on resource unit [6] 484(JFPU1 - floating point pipeline #1), consuming an average of 1 resource cycle 485per iteration. Note that on AMD Jaguar, vector floating-point multiply can 486only be issued to pipeline JFPU1, while horizontal floating-point additions can 487only be issued to pipeline JFPU0. 488 489The resource pressure view helps with identifying bottlenecks caused by high 490usage of specific hardware resources. Situations with resource pressure mainly 491concentrated on a few resources should, in general, be avoided. Ideally, 492pressure should be uniformly distributed between multiple resources. 493 494Timeline View 495^^^^^^^^^^^^^ 496The timeline view produces a detailed report of each instruction's state 497transitions through an instruction pipeline. This view is enabled by the 498command line option ``-timeline``. As instructions transition through the 499various stages of the pipeline, their states are depicted in the view report. 500These states are represented by the following characters: 501 502* D : Instruction dispatched. 503* e : Instruction executing. 504* E : Instruction executed. 505* R : Instruction retired. 506* = : Instruction already dispatched, waiting to be executed. 507* \- : Instruction executed, waiting to be retired. 508 509Below is the timeline view for a subset of the dot-product example located in 510``test/tools/llvm-mca/X86/BtVer2/dot-product.s`` and processed by 511:program:`llvm-mca` using the following command: 512 513.. code-block:: bash 514 515 $ llvm-mca -mtriple=x86_64-unknown-unknown -mcpu=btver2 -iterations=3 -timeline dot-product.s 516 517.. code-block:: none 518 519 Timeline view: 520 012345 521 Index 0123456789 522 523 [0,0] DeeER. . . vmulps %xmm0, %xmm1, %xmm2 524 [0,1] D==eeeER . . vhaddps %xmm2, %xmm2, %xmm3 525 [0,2] .D====eeeER . vhaddps %xmm3, %xmm3, %xmm4 526 [1,0] .DeeE-----R . vmulps %xmm0, %xmm1, %xmm2 527 [1,1] . D=eeeE---R . vhaddps %xmm2, %xmm2, %xmm3 528 [1,2] . D====eeeER . vhaddps %xmm3, %xmm3, %xmm4 529 [2,0] . DeeE-----R . vmulps %xmm0, %xmm1, %xmm2 530 [2,1] . D====eeeER . vhaddps %xmm2, %xmm2, %xmm3 531 [2,2] . D======eeeER vhaddps %xmm3, %xmm3, %xmm4 532 533 534 Average Wait times (based on the timeline view): 535 [0]: Executions 536 [1]: Average time spent waiting in a scheduler's queue 537 [2]: Average time spent waiting in a scheduler's queue while ready 538 [3]: Average time elapsed from WB until retire stage 539 540 [0] [1] [2] [3] 541 0. 3 1.0 1.0 3.3 vmulps %xmm0, %xmm1, %xmm2 542 1. 3 3.3 0.7 1.0 vhaddps %xmm2, %xmm2, %xmm3 543 2. 3 5.7 0.0 0.0 vhaddps %xmm3, %xmm3, %xmm4 544 3 3.3 0.5 1.4 <total> 545 546The timeline view is interesting because it shows instruction state changes 547during execution. It also gives an idea of how the tool processes instructions 548executed on the target, and how their timing information might be calculated. 549 550The timeline view is structured in two tables. The first table shows 551instructions changing state over time (measured in cycles); the second table 552(named *Average Wait times*) reports useful timing statistics, which should 553help diagnose performance bottlenecks caused by long data dependencies and 554sub-optimal usage of hardware resources. 555 556An instruction in the timeline view is identified by a pair of indices, where 557the first index identifies an iteration, and the second index is the 558instruction index (i.e., where it appears in the code sequence). Since this 559example was generated using 3 iterations: ``-iterations=3``, the iteration 560indices range from 0-2 inclusively. 561 562Excluding the first and last column, the remaining columns are in cycles. 563Cycles are numbered sequentially starting from 0. 564 565From the example output above, we know the following: 566 567* Instruction [1,0] was dispatched at cycle 1. 568* Instruction [1,0] started executing at cycle 2. 569* Instruction [1,0] reached the write back stage at cycle 4. 570* Instruction [1,0] was retired at cycle 10. 571 572Instruction [1,0] (i.e., vmulps from iteration #1) does not have to wait in the 573scheduler's queue for the operands to become available. By the time vmulps is 574dispatched, operands are already available, and pipeline JFPU1 is ready to 575serve another instruction. So the instruction can be immediately issued on the 576JFPU1 pipeline. That is demonstrated by the fact that the instruction only 577spent 1cy in the scheduler's queue. 578 579There is a gap of 5 cycles between the write-back stage and the retire event. 580That is because instructions must retire in program order, so [1,0] has to wait 581for [0,2] to be retired first (i.e., it has to wait until cycle 10). 582 583In the example, all instructions are in a RAW (Read After Write) dependency 584chain. Register %xmm2 written by vmulps is immediately used by the first 585vhaddps, and register %xmm3 written by the first vhaddps is used by the second 586vhaddps. Long data dependencies negatively impact the ILP (Instruction Level 587Parallelism). 588 589In the dot-product example, there are anti-dependencies introduced by 590instructions from different iterations. However, those dependencies can be 591removed at register renaming stage (at the cost of allocating register aliases, 592and therefore consuming physical registers). 593 594Table *Average Wait times* helps diagnose performance issues that are caused by 595the presence of long latency instructions and potentially long data dependencies 596which may limit the ILP. Last row, ``<total>``, shows a global average over all 597instructions measured. Note that :program:`llvm-mca`, by default, assumes at 598least 1cy between the dispatch event and the issue event. 599 600When the performance is limited by data dependencies and/or long latency 601instructions, the number of cycles spent while in the *ready* state is expected 602to be very small when compared with the total number of cycles spent in the 603scheduler's queue. The difference between the two counters is a good indicator 604of how large of an impact data dependencies had on the execution of the 605instructions. When performance is mostly limited by the lack of hardware 606resources, the delta between the two counters is small. However, the number of 607cycles spent in the queue tends to be larger (i.e., more than 1-3cy), 608especially when compared to other low latency instructions. 609 610Bottleneck Analysis 611^^^^^^^^^^^^^^^^^^^ 612The ``-bottleneck-analysis`` command line option enables the analysis of 613performance bottlenecks. 614 615This analysis is potentially expensive. It attempts to correlate increases in 616backend pressure (caused by pipeline resource pressure and data dependencies) to 617dynamic dispatch stalls. 618 619Below is an example of ``-bottleneck-analysis`` output generated by 620:program:`llvm-mca` for 500 iterations of the dot-product example on btver2. 621 622.. code-block:: none 623 624 625 Cycles with backend pressure increase [ 48.07% ] 626 Throughput Bottlenecks: 627 Resource Pressure [ 47.77% ] 628 - JFPA [ 47.77% ] 629 - JFPU0 [ 47.77% ] 630 Data Dependencies: [ 0.30% ] 631 - Register Dependencies [ 0.30% ] 632 - Memory Dependencies [ 0.00% ] 633 634 Critical sequence based on the simulation: 635 636 Instruction Dependency Information 637 +----< 2. vhaddps %xmm3, %xmm3, %xmm4 638 | 639 | < loop carried > 640 | 641 | 0. vmulps %xmm0, %xmm1, %xmm2 642 +----> 1. vhaddps %xmm2, %xmm2, %xmm3 ## RESOURCE interference: JFPA [ probability: 74% ] 643 +----> 2. vhaddps %xmm3, %xmm3, %xmm4 ## REGISTER dependency: %xmm3 644 | 645 | < loop carried > 646 | 647 +----> 1. vhaddps %xmm2, %xmm2, %xmm3 ## RESOURCE interference: JFPA [ probability: 74% ] 648 649 650According to the analysis, throughput is limited by resource pressure and not by 651data dependencies. The analysis observed increases in backend pressure during 65248.07% of the simulated run. Almost all those pressure increase events were 653caused by contention on processor resources JFPA/JFPU0. 654 655The `critical sequence` is the most expensive sequence of instructions according 656to the simulation. It is annotated to provide extra information about critical 657register dependencies and resource interferences between instructions. 658 659Instructions from the critical sequence are expected to significantly impact 660performance. By construction, the accuracy of this analysis is strongly 661dependent on the simulation and (as always) by the quality of the processor 662model in llvm. 663 664Bottleneck analysis is currently not supported for processors with an in-order 665backend. 666 667Extra Statistics to Further Diagnose Performance Issues 668^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 669The ``-all-stats`` command line option enables extra statistics and performance 670counters for the dispatch logic, the reorder buffer, the retire control unit, 671and the register file. 672 673Below is an example of ``-all-stats`` output generated by :program:`llvm-mca` 674for 300 iterations of the dot-product example discussed in the previous 675sections. 676 677.. code-block:: none 678 679 Dynamic Dispatch Stall Cycles: 680 RAT - Register unavailable: 0 681 RCU - Retire tokens unavailable: 0 682 SCHEDQ - Scheduler full: 272 (44.6%) 683 LQ - Load queue full: 0 684 SQ - Store queue full: 0 685 GROUP - Static restrictions on the dispatch group: 0 686 687 688 Dispatch Logic - number of cycles where we saw N micro opcodes dispatched: 689 [# dispatched], [# cycles] 690 0, 24 (3.9%) 691 1, 272 (44.6%) 692 2, 314 (51.5%) 693 694 695 Schedulers - number of cycles where we saw N micro opcodes issued: 696 [# issued], [# cycles] 697 0, 7 (1.1%) 698 1, 306 (50.2%) 699 2, 297 (48.7%) 700 701 Scheduler's queue usage: 702 [1] Resource name. 703 [2] Average number of used buffer entries. 704 [3] Maximum number of used buffer entries. 705 [4] Total number of buffer entries. 706 707 [1] [2] [3] [4] 708 JALU01 0 0 20 709 JFPU01 17 18 18 710 JLSAGU 0 0 12 711 712 713 Retire Control Unit - number of cycles where we saw N instructions retired: 714 [# retired], [# cycles] 715 0, 109 (17.9%) 716 1, 102 (16.7%) 717 2, 399 (65.4%) 718 719 Total ROB Entries: 64 720 Max Used ROB Entries: 35 ( 54.7% ) 721 Average Used ROB Entries per cy: 32 ( 50.0% ) 722 723 724 Register File statistics: 725 Total number of mappings created: 900 726 Max number of mappings used: 35 727 728 * Register File #1 -- JFpuPRF: 729 Number of physical registers: 72 730 Total number of mappings created: 900 731 Max number of mappings used: 35 732 733 * Register File #2 -- JIntegerPRF: 734 Number of physical registers: 64 735 Total number of mappings created: 0 736 Max number of mappings used: 0 737 738If we look at the *Dynamic Dispatch Stall Cycles* table, we see the counter for 739SCHEDQ reports 272 cycles. This counter is incremented every time the dispatch 740logic is unable to dispatch a full group because the scheduler's queue is full. 741 742Looking at the *Dispatch Logic* table, we see that the pipeline was only able to 743dispatch two micro opcodes 51.5% of the time. The dispatch group was limited to 744one micro opcode 44.6% of the cycles, which corresponds to 272 cycles. The 745dispatch statistics are displayed by either using the command option 746``-all-stats`` or ``-dispatch-stats``. 747 748The next table, *Schedulers*, presents a histogram displaying a count, 749representing the number of micro opcodes issued on some number of cycles. In 750this case, of the 610 simulated cycles, single opcodes were issued 306 times 751(50.2%) and there were 7 cycles where no opcodes were issued. 752 753The *Scheduler's queue usage* table shows that the average and maximum number of 754buffer entries (i.e., scheduler queue entries) used at runtime. Resource JFPU01 755reached its maximum (18 of 18 queue entries). Note that AMD Jaguar implements 756three schedulers: 757 758* JALU01 - A scheduler for ALU instructions. 759* JFPU01 - A scheduler floating point operations. 760* JLSAGU - A scheduler for address generation. 761 762The dot-product is a kernel of three floating point instructions (a vector 763multiply followed by two horizontal adds). That explains why only the floating 764point scheduler appears to be used. 765 766A full scheduler queue is either caused by data dependency chains or by a 767sub-optimal usage of hardware resources. Sometimes, resource pressure can be 768mitigated by rewriting the kernel using different instructions that consume 769different scheduler resources. Schedulers with a small queue are less resilient 770to bottlenecks caused by the presence of long data dependencies. The scheduler 771statistics are displayed by using the command option ``-all-stats`` or 772``-scheduler-stats``. 773 774The next table, *Retire Control Unit*, presents a histogram displaying a count, 775representing the number of instructions retired on some number of cycles. In 776this case, of the 610 simulated cycles, two instructions were retired during the 777same cycle 399 times (65.4%) and there were 109 cycles where no instructions 778were retired. The retire statistics are displayed by using the command option 779``-all-stats`` or ``-retire-stats``. 780 781The last table presented is *Register File statistics*. Each physical register 782file (PRF) used by the pipeline is presented in this table. In the case of AMD 783Jaguar, there are two register files, one for floating-point registers (JFpuPRF) 784and one for integer registers (JIntegerPRF). The table shows that of the 900 785instructions processed, there were 900 mappings created. Since this dot-product 786example utilized only floating point registers, the JFPuPRF was responsible for 787creating the 900 mappings. However, we see that the pipeline only used a 788maximum of 35 of 72 available register slots at any given time. We can conclude 789that the floating point PRF was the only register file used for the example, and 790that it was never resource constrained. The register file statistics are 791displayed by using the command option ``-all-stats`` or 792``-register-file-stats``. 793 794In this example, we can conclude that the IPC is mostly limited by data 795dependencies, and not by resource pressure. 796 797Instruction Flow 798^^^^^^^^^^^^^^^^ 799This section describes the instruction flow through the default pipeline of 800:program:`llvm-mca`, as well as the functional units involved in the process. 801 802The default pipeline implements the following sequence of stages used to 803process instructions. 804 805* Dispatch (Instruction is dispatched to the schedulers). 806* Issue (Instruction is issued to the processor pipelines). 807* Write Back (Instruction is executed, and results are written back). 808* Retire (Instruction is retired; writes are architecturally committed). 809 810The in-order pipeline implements the following sequence of stages: 811* InOrderIssue (Instruction is issued to the processor pipelines). 812* Retire (Instruction is retired; writes are architecturally committed). 813 814:program:`llvm-mca` assumes that instructions have all been decoded and placed 815into a queue before the simulation start. Therefore, the instruction fetch and 816decode stages are not modeled. Performance bottlenecks in the frontend are not 817diagnosed. Also, :program:`llvm-mca` does not model branch prediction. 818 819Instruction Dispatch 820"""""""""""""""""""" 821During the dispatch stage, instructions are picked in program order from a 822queue of already decoded instructions, and dispatched in groups to the 823simulated hardware schedulers. 824 825The size of a dispatch group depends on the availability of the simulated 826hardware resources. The processor dispatch width defaults to the value 827of the ``IssueWidth`` in LLVM's scheduling model. 828 829An instruction can be dispatched if: 830 831* The size of the dispatch group is smaller than processor's dispatch width. 832* There are enough entries in the reorder buffer. 833* There are enough physical registers to do register renaming. 834* The schedulers are not full. 835 836Scheduling models can optionally specify which register files are available on 837the processor. :program:`llvm-mca` uses that information to initialize register 838file descriptors. Users can limit the number of physical registers that are 839globally available for register renaming by using the command option 840``-register-file-size``. A value of zero for this option means *unbounded*. By 841knowing how many registers are available for renaming, the tool can predict 842dispatch stalls caused by the lack of physical registers. 843 844The number of reorder buffer entries consumed by an instruction depends on the 845number of micro-opcodes specified for that instruction by the target scheduling 846model. The reorder buffer is responsible for tracking the progress of 847instructions that are "in-flight", and retiring them in program order. The 848number of entries in the reorder buffer defaults to the value specified by field 849`MicroOpBufferSize` in the target scheduling model. 850 851Instructions that are dispatched to the schedulers consume scheduler buffer 852entries. :program:`llvm-mca` queries the scheduling model to determine the set 853of buffered resources consumed by an instruction. Buffered resources are 854treated like scheduler resources. 855 856Instruction Issue 857""""""""""""""""" 858Each processor scheduler implements a buffer of instructions. An instruction 859has to wait in the scheduler's buffer until input register operands become 860available. Only at that point, does the instruction becomes eligible for 861execution and may be issued (potentially out-of-order) for execution. 862Instruction latencies are computed by :program:`llvm-mca` with the help of the 863scheduling model. 864 865:program:`llvm-mca`'s scheduler is designed to simulate multiple processor 866schedulers. The scheduler is responsible for tracking data dependencies, and 867dynamically selecting which processor resources are consumed by instructions. 868It delegates the management of processor resource units and resource groups to a 869resource manager. The resource manager is responsible for selecting resource 870units that are consumed by instructions. For example, if an instruction 871consumes 1cy of a resource group, the resource manager selects one of the 872available units from the group; by default, the resource manager uses a 873round-robin selector to guarantee that resource usage is uniformly distributed 874between all units of a group. 875 876:program:`llvm-mca`'s scheduler internally groups instructions into three sets: 877 878* WaitSet: a set of instructions whose operands are not ready. 879* ReadySet: a set of instructions ready to execute. 880* IssuedSet: a set of instructions executing. 881 882Depending on the operands availability, instructions that are dispatched to the 883scheduler are either placed into the WaitSet or into the ReadySet. 884 885Every cycle, the scheduler checks if instructions can be moved from the WaitSet 886to the ReadySet, and if instructions from the ReadySet can be issued to the 887underlying pipelines. The algorithm prioritizes older instructions over younger 888instructions. 889 890Write-Back and Retire Stage 891""""""""""""""""""""""""""" 892Issued instructions are moved from the ReadySet to the IssuedSet. There, 893instructions wait until they reach the write-back stage. At that point, they 894get removed from the queue and the retire control unit is notified. 895 896When instructions are executed, the retire control unit flags the instruction as 897"ready to retire." 898 899Instructions are retired in program order. The register file is notified of the 900retirement so that it can free the physical registers that were allocated for 901the instruction during the register renaming stage. 902 903Load/Store Unit and Memory Consistency Model 904"""""""""""""""""""""""""""""""""""""""""""" 905To simulate an out-of-order execution of memory operations, :program:`llvm-mca` 906utilizes a simulated load/store unit (LSUnit) to simulate the speculative 907execution of loads and stores. 908 909Each load (or store) consumes an entry in the load (or store) queue. Users can 910specify flags ``-lqueue`` and ``-squeue`` to limit the number of entries in the 911load and store queues respectively. The queues are unbounded by default. 912 913The LSUnit implements a relaxed consistency model for memory loads and stores. 914The rules are: 915 9161. A younger load is allowed to pass an older load only if there are no 917 intervening stores or barriers between the two loads. 9182. A younger load is allowed to pass an older store provided that the load does 919 not alias with the store. 9203. A younger store is not allowed to pass an older store. 9214. A younger store is not allowed to pass an older load. 922 923By default, the LSUnit optimistically assumes that loads do not alias 924(`-noalias=true`) store operations. Under this assumption, younger loads are 925always allowed to pass older stores. Essentially, the LSUnit does not attempt 926to run any alias analysis to predict when loads and stores do not alias with 927each other. 928 929Note that, in the case of write-combining memory, rule 3 could be relaxed to 930allow reordering of non-aliasing store operations. That being said, at the 931moment, there is no way to further relax the memory model (``-noalias`` is the 932only option). Essentially, there is no option to specify a different memory 933type (e.g., write-back, write-combining, write-through; etc.) and consequently 934to weaken, or strengthen, the memory model. 935 936Other limitations are: 937 938* The LSUnit does not know when store-to-load forwarding may occur. 939* The LSUnit does not know anything about cache hierarchy and memory types. 940* The LSUnit does not know how to identify serializing operations and memory 941 fences. 942 943The LSUnit does not attempt to predict if a load or store hits or misses the L1 944cache. It only knows if an instruction "MayLoad" and/or "MayStore." For 945loads, the scheduling model provides an "optimistic" load-to-use latency (which 946usually matches the load-to-use latency for when there is a hit in the L1D). 947 948:program:`llvm-mca` does not know about serializing operations or memory-barrier 949like instructions. The LSUnit conservatively assumes that an instruction which 950has both "MayLoad" and unmodeled side effects behaves like a "soft" 951load-barrier. That means, it serializes loads without forcing a flush of the 952load queue. Similarly, instructions that "MayStore" and have unmodeled side 953effects are treated like store barriers. A full memory barrier is a "MayLoad" 954and "MayStore" instruction with unmodeled side effects. This is inaccurate, but 955it is the best that we can do at the moment with the current information 956available in LLVM. 957 958A load/store barrier consumes one entry of the load/store queue. A load/store 959barrier enforces ordering of loads/stores. A younger load cannot pass a load 960barrier. Also, a younger store cannot pass a store barrier. A younger load 961has to wait for the memory/load barrier to execute. A load/store barrier is 962"executed" when it becomes the oldest entry in the load/store queue(s). That 963also means, by construction, all of the older loads/stores have been executed. 964 965In conclusion, the full set of load/store consistency rules are: 966 967#. A store may not pass a previous store. 968#. A store may not pass a previous load (regardless of ``-noalias``). 969#. A store has to wait until an older store barrier is fully executed. 970#. A load may pass a previous load. 971#. A load may not pass a previous store unless ``-noalias`` is set. 972#. A load has to wait until an older load barrier is fully executed. 973 974In-order Issue and Execute 975"""""""""""""""""""""""""""""""""""" 976In-order processors are modelled as a single ``InOrderIssueStage`` stage. It 977bypasses Dispatch, Scheduler and Load/Store unit. Instructions are issued as 978soon as their operand registers are available and resource requirements are 979met. Multiple instructions can be issued in one cycle according to the value of 980the ``IssueWidth`` parameter in LLVM's scheduling model. 981 982Once issued, an instruction is moved to ``IssuedInst`` set until it is ready to 983retire. :program:`llvm-mca` ensures that writes are committed in-order. However, 984an instruction is allowed to commit writes and retire out-of-order if 985``RetireOOO`` property is true for at least one of its writes. 986 987Custom Behaviour 988"""""""""""""""""""""""""""""""""""" 989Due to certain instructions not being expressed perfectly within their 990scheduling model, :program:`llvm-ma` isn't always able to simulate them 991perfectly. Modifying the scheduling model isn't always a viable 992option though (maybe because the instruction is modeled incorrectly on 993purpose or the instruction's behaviour is quite complex). The 994CustomBehaviour class can be used in these cases to enforce proper 995instruction modeling (often by customizing data dependencies and detecting 996hazards that :program:`llvm-ma` has no way of knowing about). 997 998:program:`llvm-mca` comes with one generic and multiple target specific 999CustomBehaviour classes. The generic class will be used if the ``-disable-cb`` 1000flag is used or if a target specific CustomBehaviour class doesn't exist for 1001that target. (The generic class does nothing.) Currently, the CustomBehaviour 1002class is only a part of the in-order pipeline, but there are plans to add it 1003to the out-of-order pipeline in the future. 1004 1005CustomBehaviour's main method is `checkCustomHazard()` which uses the 1006current instruction and a list of all instructions still executing within 1007the pipeline to determine if the current instruction should be dispatched. 1008As output, the method returns an integer representing the number of cycles 1009that the current instruction must stall for (this can be an underestimate 1010if you don't know the exact number and a value of 0 represents no stall). 1011 1012If you'd like to add a CustomBehaviour class for a target that doesn't 1013already have one, refer to an existing implementation to see how to set it 1014up. Remember to look at (and add to) `/llvm-mca/lib/CMakeLists.txt`. 1015