1llvm-mca - LLVM Machine Code Analyzer 2===================================== 3 4.. program:: llvm-mca 5 6SYNOPSIS 7-------- 8 9:program:`llvm-mca` [*options*] [input] 10 11DESCRIPTION 12----------- 13 14:program:`llvm-mca` is a performance analysis tool that uses information 15available in LLVM (e.g. scheduling models) to statically measure the performance 16of machine code in a specific CPU. 17 18Performance is measured in terms of throughput as well as processor resource 19consumption. The tool currently works for processors with an out-of-order 20backend, for which there is a scheduling model available in LLVM. 21 22The main goal of this tool is not just to predict the performance of the code 23when run on the target, but also help with diagnosing potential performance 24issues. 25 26Given an assembly code sequence, :program:`llvm-mca` estimates the Instructions 27Per Cycle (IPC), as well as hardware resource pressure. The analysis and 28reporting style were inspired by the IACA tool from Intel. 29 30For example, you can compile code with clang, output assembly, and pipe it 31directly into :program:`llvm-mca` for analysis: 32 33.. code-block:: bash 34 35 $ clang foo.c -O2 -target x86_64-unknown-unknown -S -o - | llvm-mca -mcpu=btver2 36 37Or for Intel syntax: 38 39.. code-block:: bash 40 41 $ clang foo.c -O2 -target x86_64-unknown-unknown -mllvm -x86-asm-syntax=intel -S -o - | llvm-mca -mcpu=btver2 42 43Scheduling models are not just used to compute instruction latencies and 44throughput, but also to understand what processor resources are available 45and how to simulate them. 46 47By design, the quality of the analysis conducted by :program:`llvm-mca` is 48inevitably affected by the quality of the scheduling models in LLVM. 49 50If you see that the performance report is not accurate for a processor, 51please `file a bug <https://bugs.llvm.org/enter_bug.cgi?product=libraries>`_ 52against the appropriate backend. 53 54OPTIONS 55------- 56 57If ``input`` is "``-``" or omitted, :program:`llvm-mca` reads from standard 58input. Otherwise, it will read from the specified filename. 59 60If the :option:`-o` option is omitted, then :program:`llvm-mca` will send its output 61to standard output if the input is from standard input. If the :option:`-o` 62option specifies "``-``", then the output will also be sent to standard output. 63 64 65.. option:: -help 66 67 Print a summary of command line options. 68 69.. option:: -o <filename> 70 71 Use ``<filename>`` as the output filename. See the summary above for more 72 details. 73 74.. option:: -mtriple=<target triple> 75 76 Specify a target triple string. 77 78.. option:: -march=<arch> 79 80 Specify the architecture for which to analyze the code. It defaults to the 81 host default target. 82 83.. option:: -mcpu=<cpuname> 84 85 Specify the processor for which to analyze the code. By default, the cpu name 86 is autodetected from the host. 87 88.. option:: -output-asm-variant=<variant id> 89 90 Specify the output assembly variant for the report generated by the tool. 91 On x86, possible values are [0, 1]. A value of 0 (vic. 1) for this flag enables 92 the AT&T (vic. Intel) assembly format for the code printed out by the tool in 93 the analysis report. 94 95.. option:: -print-imm-hex 96 97 Prefer hex format for numeric literals in the output assembly printed as part 98 of the report. 99 100.. option:: -dispatch=<width> 101 102 Specify a different dispatch width for the processor. The dispatch width 103 defaults to field 'IssueWidth' in the processor scheduling model. If width is 104 zero, then the default dispatch width is used. 105 106.. option:: -register-file-size=<size> 107 108 Specify the size of the register file. When specified, this flag limits how 109 many physical registers are available for register renaming purposes. A value 110 of zero for this flag means "unlimited number of physical registers". 111 112.. option:: -iterations=<number of iterations> 113 114 Specify the number of iterations to run. If this flag is set to 0, then the 115 tool sets the number of iterations to a default value (i.e. 100). 116 117.. option:: -noalias=<bool> 118 119 If set, the tool assumes that loads and stores don't alias. This is the 120 default behavior. 121 122.. option:: -lqueue=<load queue size> 123 124 Specify the size of the load queue in the load/store unit emulated by the tool. 125 By default, the tool assumes an unbound number of entries in the load queue. 126 A value of zero for this flag is ignored, and the default load queue size is 127 used instead. 128 129.. option:: -squeue=<store queue size> 130 131 Specify the size of the store queue in the load/store unit emulated by the 132 tool. By default, the tool assumes an unbound number of entries in the store 133 queue. A value of zero for this flag is ignored, and the default store queue 134 size is used instead. 135 136.. option:: -timeline 137 138 Enable the timeline view. 139 140.. option:: -timeline-max-iterations=<iterations> 141 142 Limit the number of iterations to print in the timeline view. By default, the 143 timeline view prints information for up to 10 iterations. 144 145.. option:: -timeline-max-cycles=<cycles> 146 147 Limit the number of cycles in the timeline view. By default, the number of 148 cycles is set to 80. 149 150.. option:: -resource-pressure 151 152 Enable the resource pressure view. This is enabled by default. 153 154.. option:: -register-file-stats 155 156 Enable register file usage statistics. 157 158.. option:: -dispatch-stats 159 160 Enable extra dispatch statistics. This view collects and analyzes instruction 161 dispatch events, as well as static/dynamic dispatch stall events. This view 162 is disabled by default. 163 164.. option:: -scheduler-stats 165 166 Enable extra scheduler statistics. This view collects and analyzes instruction 167 issue events. This view is disabled by default. 168 169.. option:: -retire-stats 170 171 Enable extra retire control unit statistics. This view is disabled by default. 172 173.. option:: -instruction-info 174 175 Enable the instruction info view. This is enabled by default. 176 177.. option:: -all-stats 178 179 Print all hardware statistics. This enables extra statistics related to the 180 dispatch logic, the hardware schedulers, the register file(s), and the retire 181 control unit. This option is disabled by default. 182 183.. option:: -all-views 184 185 Enable all the view. 186 187.. option:: -instruction-tables 188 189 Prints resource pressure information based on the static information 190 available from the processor model. This differs from the resource pressure 191 view because it doesn't require that the code is simulated. It instead prints 192 the theoretical uniform distribution of resource pressure for every 193 instruction in sequence. 194 195.. option:: -bottleneck-analysis 196 197 Print information about bottlenecks that affect the throughput. This analysis 198 can be expensive, and it is disabled by default. Bottlenecks are highlighted 199 in the summary view. 200 201 202EXIT STATUS 203----------- 204 205:program:`llvm-mca` returns 0 on success. Otherwise, an error message is printed 206to standard error, and the tool returns 1. 207 208USING MARKERS TO ANALYZE SPECIFIC CODE BLOCKS 209--------------------------------------------- 210:program:`llvm-mca` allows for the optional usage of special code comments to 211mark regions of the assembly code to be analyzed. A comment starting with 212substring ``LLVM-MCA-BEGIN`` marks the beginning of a code region. A comment 213starting with substring ``LLVM-MCA-END`` marks the end of a code region. For 214example: 215 216.. code-block:: none 217 218 # LLVM-MCA-BEGIN 219 ... 220 # LLVM-MCA-END 221 222If no user-defined region is specified, then :program:`llvm-mca` assumes a 223default region which contains every instruction in the input file. Every region 224is analyzed in isolation, and the final performance report is the union of all 225the reports generated for every code region. 226 227Code regions can have names. For example: 228 229.. code-block:: none 230 231 # LLVM-MCA-BEGIN A simple example 232 add %eax, %eax 233 # LLVM-MCA-END 234 235The code from the example above defines a region named "A simple example" with a 236single instruction in it. Note how the region name doesn't have to be repeated 237in the ``LLVM-MCA-END`` directive. In the absence of overlapping regions, 238an anonymous ``LLVM-MCA-END`` directive always ends the currently active user 239defined region. 240 241Example of nesting regions: 242 243.. code-block:: none 244 245 # LLVM-MCA-BEGIN foo 246 add %eax, %edx 247 # LLVM-MCA-BEGIN bar 248 sub %eax, %edx 249 # LLVM-MCA-END bar 250 # LLVM-MCA-END foo 251 252Example of overlapping regions: 253 254.. code-block:: none 255 256 # LLVM-MCA-BEGIN foo 257 add %eax, %edx 258 # LLVM-MCA-BEGIN bar 259 sub %eax, %edx 260 # LLVM-MCA-END foo 261 add %eax, %edx 262 # LLVM-MCA-END bar 263 264Note that multiple anonymous regions cannot overlap. Also, overlapping regions 265cannot have the same name. 266 267There is no support for marking regions from high-level source code, like C or 268C++. As a workaround, inline assembly directives may be used: 269 270.. code-block:: c++ 271 272 int foo(int a, int b) { 273 __asm volatile("# LLVM-MCA-BEGIN foo"); 274 a += 42; 275 __asm volatile("# LLVM-MCA-END"); 276 a *= b; 277 return a; 278 } 279 280However, this interferes with optimizations like loop vectorization and may have 281an impact on the code generated. This is because the ``__asm`` statements are 282seen as real code having important side effects, which limits how the code 283around them can be transformed. If users want to make use of inline assembly 284to emit markers, then the recommendation is to always verify that the output 285assembly is equivalent to the assembly generated in the absence of markers. 286The `Clang options to emit optimization reports <https://clang.llvm.org/docs/UsersManual.html#options-to-emit-optimization-reports>`_ 287can also help in detecting missed optimizations. 288 289HOW LLVM-MCA WORKS 290------------------ 291 292:program:`llvm-mca` takes assembly code as input. The assembly code is parsed 293into a sequence of MCInst with the help of the existing LLVM target assembly 294parsers. The parsed sequence of MCInst is then analyzed by a ``Pipeline`` module 295to generate a performance report. 296 297The Pipeline module simulates the execution of the machine code sequence in a 298loop of iterations (default is 100). During this process, the pipeline collects 299a number of execution related statistics. At the end of this process, the 300pipeline generates and prints a report from the collected statistics. 301 302Here is an example of a performance report generated by the tool for a 303dot-product of two packed float vectors of four elements. The analysis is 304conducted for target x86, cpu btver2. The following result can be produced via 305the following command using the example located at 306``test/tools/llvm-mca/X86/BtVer2/dot-product.s``: 307 308.. code-block:: bash 309 310 $ llvm-mca -mtriple=x86_64-unknown-unknown -mcpu=btver2 -iterations=300 dot-product.s 311 312.. code-block:: none 313 314 Iterations: 300 315 Instructions: 900 316 Total Cycles: 610 317 Total uOps: 900 318 319 Dispatch Width: 2 320 uOps Per Cycle: 1.48 321 IPC: 1.48 322 Block RThroughput: 2.0 323 324 325 Instruction Info: 326 [1]: #uOps 327 [2]: Latency 328 [3]: RThroughput 329 [4]: MayLoad 330 [5]: MayStore 331 [6]: HasSideEffects (U) 332 333 [1] [2] [3] [4] [5] [6] Instructions: 334 1 2 1.00 vmulps %xmm0, %xmm1, %xmm2 335 1 3 1.00 vhaddps %xmm2, %xmm2, %xmm3 336 1 3 1.00 vhaddps %xmm3, %xmm3, %xmm4 337 338 339 Resources: 340 [0] - JALU0 341 [1] - JALU1 342 [2] - JDiv 343 [3] - JFPA 344 [4] - JFPM 345 [5] - JFPU0 346 [6] - JFPU1 347 [7] - JLAGU 348 [8] - JMul 349 [9] - JSAGU 350 [10] - JSTC 351 [11] - JVALU0 352 [12] - JVALU1 353 [13] - JVIMUL 354 355 356 Resource pressure per iteration: 357 [0] [1] [2] [3] [4] [5] [6] [7] [8] [9] [10] [11] [12] [13] 358 - - - 2.00 1.00 2.00 1.00 - - - - - - - 359 360 Resource pressure by instruction: 361 [0] [1] [2] [3] [4] [5] [6] [7] [8] [9] [10] [11] [12] [13] Instructions: 362 - - - - 1.00 - 1.00 - - - - - - - vmulps %xmm0, %xmm1, %xmm2 363 - - - 1.00 - 1.00 - - - - - - - - vhaddps %xmm2, %xmm2, %xmm3 364 - - - 1.00 - 1.00 - - - - - - - - vhaddps %xmm3, %xmm3, %xmm4 365 366According to this report, the dot-product kernel has been executed 300 times, 367for a total of 900 simulated instructions. The total number of simulated micro 368opcodes (uOps) is also 900. 369 370The report is structured in three main sections. The first section collects a 371few performance numbers; the goal of this section is to give a very quick 372overview of the performance throughput. Important performance indicators are 373**IPC**, **uOps Per Cycle**, and **Block RThroughput** (Block Reciprocal 374Throughput). 375 376Field *DispatchWidth* is the maximum number of micro opcodes that are dispatched 377to the out-of-order backend every simulated cycle. 378 379IPC is computed dividing the total number of simulated instructions by the total 380number of cycles. 381 382Field *Block RThroughput* is the reciprocal of the block throughput. Block 383throuhgput is a theoretical quantity computed as the maximum number of blocks 384(i.e. iterations) that can be executed per simulated clock cycle in the absence 385of loop carried dependencies. Block throughput is is superiorly 386limited by the dispatch rate, and the availability of hardware resources. 387 388In the absence of loop-carried data dependencies, the observed IPC tends to a 389theoretical maximum which can be computed by dividing the number of instructions 390of a single iteration by the `Block RThroughput`. 391 392Field 'uOps Per Cycle' is computed dividing the total number of simulated micro 393opcodes by the total number of cycles. A delta between Dispatch Width and this 394field is an indicator of a performance issue. In the absence of loop-carried 395data dependencies, the observed 'uOps Per Cycle' should tend to a theoretical 396maximum throughput which can be computed by dividing the number of uOps of a 397single iteration by the `Block RThroughput`. 398 399Field *uOps Per Cycle* is bounded from above by the dispatch width. That is 400because the dispatch width limits the maximum size of a dispatch group. Both IPC 401and 'uOps Per Cycle' are limited by the amount of hardware parallelism. The 402availability of hardware resources affects the resource pressure distribution, 403and it limits the number of instructions that can be executed in parallel every 404cycle. A delta between Dispatch Width and the theoretical maximum uOps per 405Cycle (computed by dividing the number of uOps of a single iteration by the 406`Block RThroughput`) is an indicator of a performance bottleneck caused by the 407lack of hardware resources. 408In general, the lower the Block RThroughput, the better. 409 410In this example, ``uOps per iteration/Block RThroughput`` is 1.50. Since there 411are no loop-carried dependencies, the observed `uOps Per Cycle` is expected to 412approach 1.50 when the number of iterations tends to infinity. The delta between 413the Dispatch Width (2.00), and the theoretical maximum uOp throughput (1.50) is 414an indicator of a performance bottleneck caused by the lack of hardware 415resources, and the *Resource pressure view* can help to identify the problematic 416resource usage. 417 418The second section of the report shows the latency and reciprocal 419throughput of every instruction in the sequence. That section also reports 420extra information related to the number of micro opcodes, and opcode properties 421(i.e., 'MayLoad', 'MayStore', and 'HasSideEffects'). 422 423Field *RThroughput* is the reciprocal of the instruction throughput. Throughput 424is computed as the maximum number of instructions of a same type that can be 425executed per clock cycle in the absence of operand dependencies. In this 426example, the reciprocal throughput of a vector float multiply is 1 427cycles/instruction. That is because the FP multiplier JFPM is only available 428from pipeline JFPU1. 429 430The third section is the *Resource pressure view*. This view reports 431the average number of resource cycles consumed every iteration by instructions 432for every processor resource unit available on the target. Information is 433structured in two tables. The first table reports the number of resource cycles 434spent on average every iteration. The second table correlates the resource 435cycles to the machine instruction in the sequence. For example, every iteration 436of the instruction vmulps always executes on resource unit [6] 437(JFPU1 - floating point pipeline #1), consuming an average of 1 resource cycle 438per iteration. Note that on AMD Jaguar, vector floating-point multiply can 439only be issued to pipeline JFPU1, while horizontal floating-point additions can 440only be issued to pipeline JFPU0. 441 442The resource pressure view helps with identifying bottlenecks caused by high 443usage of specific hardware resources. Situations with resource pressure mainly 444concentrated on a few resources should, in general, be avoided. Ideally, 445pressure should be uniformly distributed between multiple resources. 446 447Timeline View 448^^^^^^^^^^^^^ 449The timeline view produces a detailed report of each instruction's state 450transitions through an instruction pipeline. This view is enabled by the 451command line option ``-timeline``. As instructions transition through the 452various stages of the pipeline, their states are depicted in the view report. 453These states are represented by the following characters: 454 455* D : Instruction dispatched. 456* e : Instruction executing. 457* E : Instruction executed. 458* R : Instruction retired. 459* = : Instruction already dispatched, waiting to be executed. 460* \- : Instruction executed, waiting to be retired. 461 462Below is the timeline view for a subset of the dot-product example located in 463``test/tools/llvm-mca/X86/BtVer2/dot-product.s`` and processed by 464:program:`llvm-mca` using the following command: 465 466.. code-block:: bash 467 468 $ llvm-mca -mtriple=x86_64-unknown-unknown -mcpu=btver2 -iterations=3 -timeline dot-product.s 469 470.. code-block:: none 471 472 Timeline view: 473 012345 474 Index 0123456789 475 476 [0,0] DeeER. . . vmulps %xmm0, %xmm1, %xmm2 477 [0,1] D==eeeER . . vhaddps %xmm2, %xmm2, %xmm3 478 [0,2] .D====eeeER . vhaddps %xmm3, %xmm3, %xmm4 479 [1,0] .DeeE-----R . vmulps %xmm0, %xmm1, %xmm2 480 [1,1] . D=eeeE---R . vhaddps %xmm2, %xmm2, %xmm3 481 [1,2] . D====eeeER . vhaddps %xmm3, %xmm3, %xmm4 482 [2,0] . DeeE-----R . vmulps %xmm0, %xmm1, %xmm2 483 [2,1] . D====eeeER . vhaddps %xmm2, %xmm2, %xmm3 484 [2,2] . D======eeeER vhaddps %xmm3, %xmm3, %xmm4 485 486 487 Average Wait times (based on the timeline view): 488 [0]: Executions 489 [1]: Average time spent waiting in a scheduler's queue 490 [2]: Average time spent waiting in a scheduler's queue while ready 491 [3]: Average time elapsed from WB until retire stage 492 493 [0] [1] [2] [3] 494 0. 3 1.0 1.0 3.3 vmulps %xmm0, %xmm1, %xmm2 495 1. 3 3.3 0.7 1.0 vhaddps %xmm2, %xmm2, %xmm3 496 2. 3 5.7 0.0 0.0 vhaddps %xmm3, %xmm3, %xmm4 497 498The timeline view is interesting because it shows instruction state changes 499during execution. It also gives an idea of how the tool processes instructions 500executed on the target, and how their timing information might be calculated. 501 502The timeline view is structured in two tables. The first table shows 503instructions changing state over time (measured in cycles); the second table 504(named *Average Wait times*) reports useful timing statistics, which should 505help diagnose performance bottlenecks caused by long data dependencies and 506sub-optimal usage of hardware resources. 507 508An instruction in the timeline view is identified by a pair of indices, where 509the first index identifies an iteration, and the second index is the 510instruction index (i.e., where it appears in the code sequence). Since this 511example was generated using 3 iterations: ``-iterations=3``, the iteration 512indices range from 0-2 inclusively. 513 514Excluding the first and last column, the remaining columns are in cycles. 515Cycles are numbered sequentially starting from 0. 516 517From the example output above, we know the following: 518 519* Instruction [1,0] was dispatched at cycle 1. 520* Instruction [1,0] started executing at cycle 2. 521* Instruction [1,0] reached the write back stage at cycle 4. 522* Instruction [1,0] was retired at cycle 10. 523 524Instruction [1,0] (i.e., vmulps from iteration #1) does not have to wait in the 525scheduler's queue for the operands to become available. By the time vmulps is 526dispatched, operands are already available, and pipeline JFPU1 is ready to 527serve another instruction. So the instruction can be immediately issued on the 528JFPU1 pipeline. That is demonstrated by the fact that the instruction only 529spent 1cy in the scheduler's queue. 530 531There is a gap of 5 cycles between the write-back stage and the retire event. 532That is because instructions must retire in program order, so [1,0] has to wait 533for [0,2] to be retired first (i.e., it has to wait until cycle 10). 534 535In the example, all instructions are in a RAW (Read After Write) dependency 536chain. Register %xmm2 written by vmulps is immediately used by the first 537vhaddps, and register %xmm3 written by the first vhaddps is used by the second 538vhaddps. Long data dependencies negatively impact the ILP (Instruction Level 539Parallelism). 540 541In the dot-product example, there are anti-dependencies introduced by 542instructions from different iterations. However, those dependencies can be 543removed at register renaming stage (at the cost of allocating register aliases, 544and therefore consuming physical registers). 545 546Table *Average Wait times* helps diagnose performance issues that are caused by 547the presence of long latency instructions and potentially long data dependencies 548which may limit the ILP. Note that :program:`llvm-mca`, by default, assumes at 549least 1cy between the dispatch event and the issue event. 550 551When the performance is limited by data dependencies and/or long latency 552instructions, the number of cycles spent while in the *ready* state is expected 553to be very small when compared with the total number of cycles spent in the 554scheduler's queue. The difference between the two counters is a good indicator 555of how large of an impact data dependencies had on the execution of the 556instructions. When performance is mostly limited by the lack of hardware 557resources, the delta between the two counters is small. However, the number of 558cycles spent in the queue tends to be larger (i.e., more than 1-3cy), 559especially when compared to other low latency instructions. 560 561Bottleneck Analysis 562^^^^^^^^^^^^^^^^^^^ 563The ``-bottleneck-analysis`` command line option enables the analysis of 564performance bottlenecks. 565 566This analysis is potentially expensive. It attempts to correlate increases in 567backend pressure (caused by pipeline resource pressure and data dependencies) to 568dynamic dispatch stalls. 569 570Below is an example of ``-bottleneck-analysis`` output generated by 571:program:`llvm-mca` for 500 iterations of the dot-product example on btver2. 572 573.. code-block:: none 574 575 576 Cycles with backend pressure increase [ 48.07% ] 577 Throughput Bottlenecks: 578 Resource Pressure [ 47.77% ] 579 - JFPA [ 47.77% ] 580 - JFPU0 [ 47.77% ] 581 Data Dependencies: [ 0.30% ] 582 - Register Dependencies [ 0.30% ] 583 - Memory Dependencies [ 0.00% ] 584 585 Critical sequence based on the simulation: 586 587 Instruction Dependency Information 588 +----< 2. vhaddps %xmm3, %xmm3, %xmm4 589 | 590 | < loop carried > 591 | 592 | 0. vmulps %xmm0, %xmm1, %xmm2 593 +----> 1. vhaddps %xmm2, %xmm2, %xmm3 ## RESOURCE interference: JFPA [ probability: 74% ] 594 +----> 2. vhaddps %xmm3, %xmm3, %xmm4 ## REGISTER dependency: %xmm3 595 | 596 | < loop carried > 597 | 598 +----> 1. vhaddps %xmm2, %xmm2, %xmm3 ## RESOURCE interference: JFPA [ probability: 74% ] 599 600 601According to the analysis, throughput is limited by resource pressure and not by 602data dependencies. The analysis observed increases in backend pressure during 60348.07% of the simulated run. Almost all those pressure increase events were 604caused by contention on processor resources JFPA/JFPU0. 605 606The `critical sequence` is the most expensive sequence of instructions according 607to the simulation. It is annotated to provide extra information about critical 608register dependencies and resource interferences between instructions. 609 610Instructions from the critical sequence are expected to significantly impact 611performance. By construction, the accuracy of this analysis is strongly 612dependent on the simulation and (as always) by the quality of the processor 613model in llvm. 614 615 616Extra Statistics to Further Diagnose Performance Issues 617^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 618The ``-all-stats`` command line option enables extra statistics and performance 619counters for the dispatch logic, the reorder buffer, the retire control unit, 620and the register file. 621 622Below is an example of ``-all-stats`` output generated by :program:`llvm-mca` 623for 300 iterations of the dot-product example discussed in the previous 624sections. 625 626.. code-block:: none 627 628 Dynamic Dispatch Stall Cycles: 629 RAT - Register unavailable: 0 630 RCU - Retire tokens unavailable: 0 631 SCHEDQ - Scheduler full: 272 (44.6%) 632 LQ - Load queue full: 0 633 SQ - Store queue full: 0 634 GROUP - Static restrictions on the dispatch group: 0 635 636 637 Dispatch Logic - number of cycles where we saw N micro opcodes dispatched: 638 [# dispatched], [# cycles] 639 0, 24 (3.9%) 640 1, 272 (44.6%) 641 2, 314 (51.5%) 642 643 644 Schedulers - number of cycles where we saw N micro opcodes issued: 645 [# issued], [# cycles] 646 0, 7 (1.1%) 647 1, 306 (50.2%) 648 2, 297 (48.7%) 649 650 Scheduler's queue usage: 651 [1] Resource name. 652 [2] Average number of used buffer entries. 653 [3] Maximum number of used buffer entries. 654 [4] Total number of buffer entries. 655 656 [1] [2] [3] [4] 657 JALU01 0 0 20 658 JFPU01 17 18 18 659 JLSAGU 0 0 12 660 661 662 Retire Control Unit - number of cycles where we saw N instructions retired: 663 [# retired], [# cycles] 664 0, 109 (17.9%) 665 1, 102 (16.7%) 666 2, 399 (65.4%) 667 668 Total ROB Entries: 64 669 Max Used ROB Entries: 35 ( 54.7% ) 670 Average Used ROB Entries per cy: 32 ( 50.0% ) 671 672 673 Register File statistics: 674 Total number of mappings created: 900 675 Max number of mappings used: 35 676 677 * Register File #1 -- JFpuPRF: 678 Number of physical registers: 72 679 Total number of mappings created: 900 680 Max number of mappings used: 35 681 682 * Register File #2 -- JIntegerPRF: 683 Number of physical registers: 64 684 Total number of mappings created: 0 685 Max number of mappings used: 0 686 687If we look at the *Dynamic Dispatch Stall Cycles* table, we see the counter for 688SCHEDQ reports 272 cycles. This counter is incremented every time the dispatch 689logic is unable to dispatch a full group because the scheduler's queue is full. 690 691Looking at the *Dispatch Logic* table, we see that the pipeline was only able to 692dispatch two micro opcodes 51.5% of the time. The dispatch group was limited to 693one micro opcode 44.6% of the cycles, which corresponds to 272 cycles. The 694dispatch statistics are displayed by either using the command option 695``-all-stats`` or ``-dispatch-stats``. 696 697The next table, *Schedulers*, presents a histogram displaying a count, 698representing the number of micro opcodes issued on some number of cycles. In 699this case, of the 610 simulated cycles, single opcodes were issued 306 times 700(50.2%) and there were 7 cycles where no opcodes were issued. 701 702The *Scheduler's queue usage* table shows that the average and maximum number of 703buffer entries (i.e., scheduler queue entries) used at runtime. Resource JFPU01 704reached its maximum (18 of 18 queue entries). Note that AMD Jaguar implements 705three schedulers: 706 707* JALU01 - A scheduler for ALU instructions. 708* JFPU01 - A scheduler floating point operations. 709* JLSAGU - A scheduler for address generation. 710 711The dot-product is a kernel of three floating point instructions (a vector 712multiply followed by two horizontal adds). That explains why only the floating 713point scheduler appears to be used. 714 715A full scheduler queue is either caused by data dependency chains or by a 716sub-optimal usage of hardware resources. Sometimes, resource pressure can be 717mitigated by rewriting the kernel using different instructions that consume 718different scheduler resources. Schedulers with a small queue are less resilient 719to bottlenecks caused by the presence of long data dependencies. The scheduler 720statistics are displayed by using the command option ``-all-stats`` or 721``-scheduler-stats``. 722 723The next table, *Retire Control Unit*, presents a histogram displaying a count, 724representing the number of instructions retired on some number of cycles. In 725this case, of the 610 simulated cycles, two instructions were retired during the 726same cycle 399 times (65.4%) and there were 109 cycles where no instructions 727were retired. The retire statistics are displayed by using the command option 728``-all-stats`` or ``-retire-stats``. 729 730The last table presented is *Register File statistics*. Each physical register 731file (PRF) used by the pipeline is presented in this table. In the case of AMD 732Jaguar, there are two register files, one for floating-point registers (JFpuPRF) 733and one for integer registers (JIntegerPRF). The table shows that of the 900 734instructions processed, there were 900 mappings created. Since this dot-product 735example utilized only floating point registers, the JFPuPRF was responsible for 736creating the 900 mappings. However, we see that the pipeline only used a 737maximum of 35 of 72 available register slots at any given time. We can conclude 738that the floating point PRF was the only register file used for the example, and 739that it was never resource constrained. The register file statistics are 740displayed by using the command option ``-all-stats`` or 741``-register-file-stats``. 742 743In this example, we can conclude that the IPC is mostly limited by data 744dependencies, and not by resource pressure. 745 746Instruction Flow 747^^^^^^^^^^^^^^^^ 748This section describes the instruction flow through the default pipeline of 749:program:`llvm-mca`, as well as the functional units involved in the process. 750 751The default pipeline implements the following sequence of stages used to 752process instructions. 753 754* Dispatch (Instruction is dispatched to the schedulers). 755* Issue (Instruction is issued to the processor pipelines). 756* Write Back (Instruction is executed, and results are written back). 757* Retire (Instruction is retired; writes are architecturally committed). 758 759The default pipeline only models the out-of-order portion of a processor. 760Therefore, the instruction fetch and decode stages are not modeled. Performance 761bottlenecks in the frontend are not diagnosed. :program:`llvm-mca` assumes that 762instructions have all been decoded and placed into a queue before the simulation 763start. Also, :program:`llvm-mca` does not model branch prediction. 764 765Instruction Dispatch 766"""""""""""""""""""" 767During the dispatch stage, instructions are picked in program order from a 768queue of already decoded instructions, and dispatched in groups to the 769simulated hardware schedulers. 770 771The size of a dispatch group depends on the availability of the simulated 772hardware resources. The processor dispatch width defaults to the value 773of the ``IssueWidth`` in LLVM's scheduling model. 774 775An instruction can be dispatched if: 776 777* The size of the dispatch group is smaller than processor's dispatch width. 778* There are enough entries in the reorder buffer. 779* There are enough physical registers to do register renaming. 780* The schedulers are not full. 781 782Scheduling models can optionally specify which register files are available on 783the processor. :program:`llvm-mca` uses that information to initialize register 784file descriptors. Users can limit the number of physical registers that are 785globally available for register renaming by using the command option 786``-register-file-size``. A value of zero for this option means *unbounded*. By 787knowing how many registers are available for renaming, the tool can predict 788dispatch stalls caused by the lack of physical registers. 789 790The number of reorder buffer entries consumed by an instruction depends on the 791number of micro-opcodes specified for that instruction by the target scheduling 792model. The reorder buffer is responsible for tracking the progress of 793instructions that are "in-flight", and retiring them in program order. The 794number of entries in the reorder buffer defaults to the value specified by field 795`MicroOpBufferSize` in the target scheduling model. 796 797Instructions that are dispatched to the schedulers consume scheduler buffer 798entries. :program:`llvm-mca` queries the scheduling model to determine the set 799of buffered resources consumed by an instruction. Buffered resources are 800treated like scheduler resources. 801 802Instruction Issue 803""""""""""""""""" 804Each processor scheduler implements a buffer of instructions. An instruction 805has to wait in the scheduler's buffer until input register operands become 806available. Only at that point, does the instruction becomes eligible for 807execution and may be issued (potentially out-of-order) for execution. 808Instruction latencies are computed by :program:`llvm-mca` with the help of the 809scheduling model. 810 811:program:`llvm-mca`'s scheduler is designed to simulate multiple processor 812schedulers. The scheduler is responsible for tracking data dependencies, and 813dynamically selecting which processor resources are consumed by instructions. 814It delegates the management of processor resource units and resource groups to a 815resource manager. The resource manager is responsible for selecting resource 816units that are consumed by instructions. For example, if an instruction 817consumes 1cy of a resource group, the resource manager selects one of the 818available units from the group; by default, the resource manager uses a 819round-robin selector to guarantee that resource usage is uniformly distributed 820between all units of a group. 821 822:program:`llvm-mca`'s scheduler internally groups instructions into three sets: 823 824* WaitSet: a set of instructions whose operands are not ready. 825* ReadySet: a set of instructions ready to execute. 826* IssuedSet: a set of instructions executing. 827 828Depending on the operands availability, instructions that are dispatched to the 829scheduler are either placed into the WaitSet or into the ReadySet. 830 831Every cycle, the scheduler checks if instructions can be moved from the WaitSet 832to the ReadySet, and if instructions from the ReadySet can be issued to the 833underlying pipelines. The algorithm prioritizes older instructions over younger 834instructions. 835 836Write-Back and Retire Stage 837""""""""""""""""""""""""""" 838Issued instructions are moved from the ReadySet to the IssuedSet. There, 839instructions wait until they reach the write-back stage. At that point, they 840get removed from the queue and the retire control unit is notified. 841 842When instructions are executed, the retire control unit flags the instruction as 843"ready to retire." 844 845Instructions are retired in program order. The register file is notified of the 846retirement so that it can free the physical registers that were allocated for 847the instruction during the register renaming stage. 848 849Load/Store Unit and Memory Consistency Model 850"""""""""""""""""""""""""""""""""""""""""""" 851To simulate an out-of-order execution of memory operations, :program:`llvm-mca` 852utilizes a simulated load/store unit (LSUnit) to simulate the speculative 853execution of loads and stores. 854 855Each load (or store) consumes an entry in the load (or store) queue. Users can 856specify flags ``-lqueue`` and ``-squeue`` to limit the number of entries in the 857load and store queues respectively. The queues are unbounded by default. 858 859The LSUnit implements a relaxed consistency model for memory loads and stores. 860The rules are: 861 8621. A younger load is allowed to pass an older load only if there are no 863 intervening stores or barriers between the two loads. 8642. A younger load is allowed to pass an older store provided that the load does 865 not alias with the store. 8663. A younger store is not allowed to pass an older store. 8674. A younger store is not allowed to pass an older load. 868 869By default, the LSUnit optimistically assumes that loads do not alias 870(`-noalias=true`) store operations. Under this assumption, younger loads are 871always allowed to pass older stores. Essentially, the LSUnit does not attempt 872to run any alias analysis to predict when loads and stores do not alias with 873each other. 874 875Note that, in the case of write-combining memory, rule 3 could be relaxed to 876allow reordering of non-aliasing store operations. That being said, at the 877moment, there is no way to further relax the memory model (``-noalias`` is the 878only option). Essentially, there is no option to specify a different memory 879type (e.g., write-back, write-combining, write-through; etc.) and consequently 880to weaken, or strengthen, the memory model. 881 882Other limitations are: 883 884* The LSUnit does not know when store-to-load forwarding may occur. 885* The LSUnit does not know anything about cache hierarchy and memory types. 886* The LSUnit does not know how to identify serializing operations and memory 887 fences. 888 889The LSUnit does not attempt to predict if a load or store hits or misses the L1 890cache. It only knows if an instruction "MayLoad" and/or "MayStore." For 891loads, the scheduling model provides an "optimistic" load-to-use latency (which 892usually matches the load-to-use latency for when there is a hit in the L1D). 893 894:program:`llvm-mca` does not know about serializing operations or memory-barrier 895like instructions. The LSUnit conservatively assumes that an instruction which 896has both "MayLoad" and unmodeled side effects behaves like a "soft" 897load-barrier. That means, it serializes loads without forcing a flush of the 898load queue. Similarly, instructions that "MayStore" and have unmodeled side 899effects are treated like store barriers. A full memory barrier is a "MayLoad" 900and "MayStore" instruction with unmodeled side effects. This is inaccurate, but 901it is the best that we can do at the moment with the current information 902available in LLVM. 903 904A load/store barrier consumes one entry of the load/store queue. A load/store 905barrier enforces ordering of loads/stores. A younger load cannot pass a load 906barrier. Also, a younger store cannot pass a store barrier. A younger load 907has to wait for the memory/load barrier to execute. A load/store barrier is 908"executed" when it becomes the oldest entry in the load/store queue(s). That 909also means, by construction, all of the older loads/stores have been executed. 910 911In conclusion, the full set of load/store consistency rules are: 912 913#. A store may not pass a previous store. 914#. A store may not pass a previous load (regardless of ``-noalias``). 915#. A store has to wait until an older store barrier is fully executed. 916#. A load may pass a previous load. 917#. A load may not pass a previous store unless ``-noalias`` is set. 918#. A load has to wait until an older load barrier is fully executed. 919