1llvm-mca - LLVM Machine Code Analyzer 2===================================== 3 4SYNOPSIS 5-------- 6 7:program:`llvm-mca` [*options*] [input] 8 9DESCRIPTION 10----------- 11 12:program:`llvm-mca` is a performance analysis tool that uses information 13available in LLVM (e.g. scheduling models) to statically measure the performance 14of machine code in a specific CPU. 15 16Performance is measured in terms of throughput as well as processor resource 17consumption. The tool currently works for processors with an out-of-order 18backend, for which there is a scheduling model available in LLVM. 19 20The main goal of this tool is not just to predict the performance of the code 21when run on the target, but also help with diagnosing potential performance 22issues. 23 24Given an assembly code sequence, :program:`llvm-mca` estimates the Instructions 25Per Cycle (IPC), as well as hardware resource pressure. The analysis and 26reporting style were inspired by the IACA tool from Intel. 27 28For example, you can compile code with clang, output assembly, and pipe it 29directly into :program:`llvm-mca` for analysis: 30 31.. code-block:: bash 32 33 $ clang foo.c -O2 -target x86_64-unknown-unknown -S -o - | llvm-mca -mcpu=btver2 34 35Or for Intel syntax: 36 37.. code-block:: bash 38 39 $ clang foo.c -O2 -target x86_64-unknown-unknown -mllvm -x86-asm-syntax=intel -S -o - | llvm-mca -mcpu=btver2 40 41Scheduling models are not just used to compute instruction latencies and 42throughput, but also to understand what processor resources are available 43and how to simulate them. 44 45By design, the quality of the analysis conducted by :program:`llvm-mca` is 46inevitably affected by the quality of the scheduling models in LLVM. 47 48If you see that the performance report is not accurate for a processor, 49please `file a bug <https://bugs.llvm.org/enter_bug.cgi?product=libraries>`_ 50against the appropriate backend. 51 52OPTIONS 53------- 54 55If ``input`` is "``-``" or omitted, :program:`llvm-mca` reads from standard 56input. Otherwise, it will read from the specified filename. 57 58If the :option:`-o` option is omitted, then :program:`llvm-mca` will send its output 59to standard output if the input is from standard input. If the :option:`-o` 60option specifies "``-``", then the output will also be sent to standard output. 61 62 63.. option:: -help 64 65 Print a summary of command line options. 66 67.. option:: -mtriple=<target triple> 68 69 Specify a target triple string. 70 71.. option:: -march=<arch> 72 73 Specify the architecture for which to analyze the code. It defaults to the 74 host default target. 75 76.. option:: -mcpu=<cpuname> 77 78 Specify the processor for which to analyze the code. By default, the cpu name 79 is autodetected from the host. 80 81.. option:: -output-asm-variant=<variant id> 82 83 Specify the output assembly variant for the report generated by the tool. 84 On x86, possible values are [0, 1]. A value of 0 (vic. 1) for this flag enables 85 the AT&T (vic. Intel) assembly format for the code printed out by the tool in 86 the analysis report. 87 88.. option:: -dispatch=<width> 89 90 Specify a different dispatch width for the processor. The dispatch width 91 defaults to field 'IssueWidth' in the processor scheduling model. If width is 92 zero, then the default dispatch width is used. 93 94.. option:: -register-file-size=<size> 95 96 Specify the size of the register file. When specified, this flag limits how 97 many physical registers are available for register renaming purposes. A value 98 of zero for this flag means "unlimited number of physical registers". 99 100.. option:: -iterations=<number of iterations> 101 102 Specify the number of iterations to run. If this flag is set to 0, then the 103 tool sets the number of iterations to a default value (i.e. 100). 104 105.. option:: -noalias=<bool> 106 107 If set, the tool assumes that loads and stores don't alias. This is the 108 default behavior. 109 110.. option:: -lqueue=<load queue size> 111 112 Specify the size of the load queue in the load/store unit emulated by the tool. 113 By default, the tool assumes an unbound number of entries in the load queue. 114 A value of zero for this flag is ignored, and the default load queue size is 115 used instead. 116 117.. option:: -squeue=<store queue size> 118 119 Specify the size of the store queue in the load/store unit emulated by the 120 tool. By default, the tool assumes an unbound number of entries in the store 121 queue. A value of zero for this flag is ignored, and the default store queue 122 size is used instead. 123 124.. option:: -timeline 125 126 Enable the timeline view. 127 128.. option:: -timeline-max-iterations=<iterations> 129 130 Limit the number of iterations to print in the timeline view. By default, the 131 timeline view prints information for up to 10 iterations. 132 133.. option:: -timeline-max-cycles=<cycles> 134 135 Limit the number of cycles in the timeline view. By default, the number of 136 cycles is set to 80. 137 138.. option:: -resource-pressure 139 140 Enable the resource pressure view. This is enabled by default. 141 142.. option:: -register-file-stats 143 144 Enable register file usage statistics. 145 146.. option:: -dispatch-stats 147 148 Enable extra dispatch statistics. This view collects and analyzes instruction 149 dispatch events, as well as static/dynamic dispatch stall events. This view 150 is disabled by default. 151 152.. option:: -scheduler-stats 153 154 Enable extra scheduler statistics. This view collects and analyzes instruction 155 issue events. This view is disabled by default. 156 157.. option:: -retire-stats 158 159 Enable extra retire control unit statistics. This view is disabled by default. 160 161.. option:: -instruction-info 162 163 Enable the instruction info view. This is enabled by default. 164 165.. option:: -all-stats 166 167 Print all hardware statistics. This enables extra statistics related to the 168 dispatch logic, the hardware schedulers, the register file(s), and the retire 169 control unit. This option is disabled by default. 170 171.. option:: -all-views 172 173 Enable all the view. 174 175.. option:: -instruction-tables 176 177 Prints resource pressure information based on the static information 178 available from the processor model. This differs from the resource pressure 179 view because it doesn't require that the code is simulated. It instead prints 180 the theoretical uniform distribution of resource pressure for every 181 instruction in sequence. 182 183.. option:: -bottleneck-analysis 184 185 Print information about bottlenecks that affect the throughput. This analysis 186 can be expensive, and it is disabled by default. Bottlenecks are highlighted 187 in the summary view. 188 189 190EXIT STATUS 191----------- 192 193:program:`llvm-mca` returns 0 on success. Otherwise, an error message is printed 194to standard error, and the tool returns 1. 195 196USING MARKERS TO ANALYZE SPECIFIC CODE BLOCKS 197--------------------------------------------- 198:program:`llvm-mca` allows for the optional usage of special code comments to 199mark regions of the assembly code to be analyzed. A comment starting with 200substring ``LLVM-MCA-BEGIN`` marks the beginning of a code region. A comment 201starting with substring ``LLVM-MCA-END`` marks the end of a code region. For 202example: 203 204.. code-block:: none 205 206 # LLVM-MCA-BEGIN 207 ... 208 # LLVM-MCA-END 209 210If no user-defined region is specified, then :program:`llvm-mca` assumes a 211default region which contains every instruction in the input file. Every region 212is analyzed in isolation, and the final performance report is the union of all 213the reports generated for every code region. 214 215Code regions can have names. For example: 216 217.. code-block:: none 218 219 # LLVM-MCA-BEGIN A simple example 220 add %eax, %eax 221 # LLVM-MCA-END 222 223The code from the example above defines a region named "A simple example" with a 224single instruction in it. Note how the region name doesn't have to be repeated 225in the ``LLVM-MCA-END`` directive. In the absence of overlapping regions, 226an anonymous ``LLVM-MCA-END`` directive always ends the currently active user 227defined region. 228 229Example of nesting regions: 230 231.. code-block:: none 232 233 # LLVM-MCA-BEGIN foo 234 add %eax, %edx 235 # LLVM-MCA-BEGIN bar 236 sub %eax, %edx 237 # LLVM-MCA-END bar 238 # LLVM-MCA-END foo 239 240Example of overlapping regions: 241 242.. code-block:: none 243 244 # LLVM-MCA-BEGIN foo 245 add %eax, %edx 246 # LLVM-MCA-BEGIN bar 247 sub %eax, %edx 248 # LLVM-MCA-END foo 249 add %eax, %edx 250 # LLVM-MCA-END bar 251 252Note that multiple anonymous regions cannot overlap. Also, overlapping regions 253cannot have the same name. 254 255There is no support for marking regions from high-level source code, like C or 256C++. As a workaround, inline assembly directives may be used: 257 258.. code-block:: c++ 259 260 int foo(int a, int b) { 261 __asm volatile("# LLVM-MCA-BEGIN foo"); 262 a += 42; 263 __asm volatile("# LLVM-MCA-END"); 264 a *= b; 265 return a; 266 } 267 268However, this interferes with optimizations like loop vectorization and may have 269an impact on the code generated. This is because the ``__asm`` statements are 270seen as real code having important side effects, which limits how the code 271around them can be transformed. If users want to make use of inline assembly 272to emit markers, then the recommendation is to always verify that the output 273assembly is equivalent to the assembly generated in the absence of markers. 274The `Clang options to emit optimization reports <https://clang.llvm.org/docs/UsersManual.html#options-to-emit-optimization-reports>`_ 275can also help in detecting missed optimizations. 276 277HOW LLVM-MCA WORKS 278------------------ 279 280:program:`llvm-mca` takes assembly code as input. The assembly code is parsed 281into a sequence of MCInst with the help of the existing LLVM target assembly 282parsers. The parsed sequence of MCInst is then analyzed by a ``Pipeline`` module 283to generate a performance report. 284 285The Pipeline module simulates the execution of the machine code sequence in a 286loop of iterations (default is 100). During this process, the pipeline collects 287a number of execution related statistics. At the end of this process, the 288pipeline generates and prints a report from the collected statistics. 289 290Here is an example of a performance report generated by the tool for a 291dot-product of two packed float vectors of four elements. The analysis is 292conducted for target x86, cpu btver2. The following result can be produced via 293the following command using the example located at 294``test/tools/llvm-mca/X86/BtVer2/dot-product.s``: 295 296.. code-block:: bash 297 298 $ llvm-mca -mtriple=x86_64-unknown-unknown -mcpu=btver2 -iterations=300 dot-product.s 299 300.. code-block:: none 301 302 Iterations: 300 303 Instructions: 900 304 Total Cycles: 610 305 Total uOps: 900 306 307 Dispatch Width: 2 308 uOps Per Cycle: 1.48 309 IPC: 1.48 310 Block RThroughput: 2.0 311 312 313 Instruction Info: 314 [1]: #uOps 315 [2]: Latency 316 [3]: RThroughput 317 [4]: MayLoad 318 [5]: MayStore 319 [6]: HasSideEffects (U) 320 321 [1] [2] [3] [4] [5] [6] Instructions: 322 1 2 1.00 vmulps %xmm0, %xmm1, %xmm2 323 1 3 1.00 vhaddps %xmm2, %xmm2, %xmm3 324 1 3 1.00 vhaddps %xmm3, %xmm3, %xmm4 325 326 327 Resources: 328 [0] - JALU0 329 [1] - JALU1 330 [2] - JDiv 331 [3] - JFPA 332 [4] - JFPM 333 [5] - JFPU0 334 [6] - JFPU1 335 [7] - JLAGU 336 [8] - JMul 337 [9] - JSAGU 338 [10] - JSTC 339 [11] - JVALU0 340 [12] - JVALU1 341 [13] - JVIMUL 342 343 344 Resource pressure per iteration: 345 [0] [1] [2] [3] [4] [5] [6] [7] [8] [9] [10] [11] [12] [13] 346 - - - 2.00 1.00 2.00 1.00 - - - - - - - 347 348 Resource pressure by instruction: 349 [0] [1] [2] [3] [4] [5] [6] [7] [8] [9] [10] [11] [12] [13] Instructions: 350 - - - - 1.00 - 1.00 - - - - - - - vmulps %xmm0, %xmm1, %xmm2 351 - - - 1.00 - 1.00 - - - - - - - - vhaddps %xmm2, %xmm2, %xmm3 352 - - - 1.00 - 1.00 - - - - - - - - vhaddps %xmm3, %xmm3, %xmm4 353 354According to this report, the dot-product kernel has been executed 300 times, 355for a total of 900 simulated instructions. The total number of simulated micro 356opcodes (uOps) is also 900. 357 358The report is structured in three main sections. The first section collects a 359few performance numbers; the goal of this section is to give a very quick 360overview of the performance throughput. Important performance indicators are 361**IPC**, **uOps Per Cycle**, and **Block RThroughput** (Block Reciprocal 362Throughput). 363 364IPC is computed dividing the total number of simulated instructions by the total 365number of cycles. In the absence of loop-carried data dependencies, the 366observed IPC tends to a theoretical maximum which can be computed by dividing 367the number of instructions of a single iteration by the *Block RThroughput*. 368 369Field 'uOps Per Cycle' is computed dividing the total number of simulated micro 370opcodes by the total number of cycles. A delta between Dispatch Width and this 371field is an indicator of a performance issue. In the absence of loop-carried 372data dependencies, the observed 'uOps Per Cycle' should tend to a theoretical 373maximum throughput which can be computed by dividing the number of uOps of a 374single iteration by the *Block RThroughput*. 375 376Field *uOps Per Cycle* is bounded from above by the dispatch width. That is 377because the dispatch width limits the maximum size of a dispatch group. Both IPC 378and 'uOps Per Cycle' are limited by the amount of hardware parallelism. The 379availability of hardware resources affects the resource pressure distribution, 380and it limits the number of instructions that can be executed in parallel every 381cycle. A delta between Dispatch Width and the theoretical maximum uOps per 382Cycle (computed by dividing the number of uOps of a single iteration by the 383*Block RTrhoughput*) is an indicator of a performance bottleneck caused by the 384lack of hardware resources. 385In general, the lower the Block RThroughput, the better. 386 387In this example, ``uOps per iteration/Block RThroughput`` is 1.50. Since there 388are no loop-carried dependencies, the observed *uOps Per Cycle* is expected to 389approach 1.50 when the number of iterations tends to infinity. The delta between 390the Dispatch Width (2.00), and the theoretical maximum uOp throughput (1.50) is 391an indicator of a performance bottleneck caused by the lack of hardware 392resources, and the *Resource pressure view* can help to identify the problematic 393resource usage. 394 395The second section of the report shows the latency and reciprocal 396throughput of every instruction in the sequence. That section also reports 397extra information related to the number of micro opcodes, and opcode properties 398(i.e., 'MayLoad', 'MayStore', and 'HasSideEffects'). 399 400The third section is the *Resource pressure view*. This view reports 401the average number of resource cycles consumed every iteration by instructions 402for every processor resource unit available on the target. Information is 403structured in two tables. The first table reports the number of resource cycles 404spent on average every iteration. The second table correlates the resource 405cycles to the machine instruction in the sequence. For example, every iteration 406of the instruction vmulps always executes on resource unit [6] 407(JFPU1 - floating point pipeline #1), consuming an average of 1 resource cycle 408per iteration. Note that on AMD Jaguar, vector floating-point multiply can 409only be issued to pipeline JFPU1, while horizontal floating-point additions can 410only be issued to pipeline JFPU0. 411 412The resource pressure view helps with identifying bottlenecks caused by high 413usage of specific hardware resources. Situations with resource pressure mainly 414concentrated on a few resources should, in general, be avoided. Ideally, 415pressure should be uniformly distributed between multiple resources. 416 417Timeline View 418^^^^^^^^^^^^^ 419The timeline view produces a detailed report of each instruction's state 420transitions through an instruction pipeline. This view is enabled by the 421command line option ``-timeline``. As instructions transition through the 422various stages of the pipeline, their states are depicted in the view report. 423These states are represented by the following characters: 424 425* D : Instruction dispatched. 426* e : Instruction executing. 427* E : Instruction executed. 428* R : Instruction retired. 429* = : Instruction already dispatched, waiting to be executed. 430* \- : Instruction executed, waiting to be retired. 431 432Below is the timeline view for a subset of the dot-product example located in 433``test/tools/llvm-mca/X86/BtVer2/dot-product.s`` and processed by 434:program:`llvm-mca` using the following command: 435 436.. code-block:: bash 437 438 $ llvm-mca -mtriple=x86_64-unknown-unknown -mcpu=btver2 -iterations=3 -timeline dot-product.s 439 440.. code-block:: none 441 442 Timeline view: 443 012345 444 Index 0123456789 445 446 [0,0] DeeER. . . vmulps %xmm0, %xmm1, %xmm2 447 [0,1] D==eeeER . . vhaddps %xmm2, %xmm2, %xmm3 448 [0,2] .D====eeeER . vhaddps %xmm3, %xmm3, %xmm4 449 [1,0] .DeeE-----R . vmulps %xmm0, %xmm1, %xmm2 450 [1,1] . D=eeeE---R . vhaddps %xmm2, %xmm2, %xmm3 451 [1,2] . D====eeeER . vhaddps %xmm3, %xmm3, %xmm4 452 [2,0] . DeeE-----R . vmulps %xmm0, %xmm1, %xmm2 453 [2,1] . D====eeeER . vhaddps %xmm2, %xmm2, %xmm3 454 [2,2] . D======eeeER vhaddps %xmm3, %xmm3, %xmm4 455 456 457 Average Wait times (based on the timeline view): 458 [0]: Executions 459 [1]: Average time spent waiting in a scheduler's queue 460 [2]: Average time spent waiting in a scheduler's queue while ready 461 [3]: Average time elapsed from WB until retire stage 462 463 [0] [1] [2] [3] 464 0. 3 1.0 1.0 3.3 vmulps %xmm0, %xmm1, %xmm2 465 1. 3 3.3 0.7 1.0 vhaddps %xmm2, %xmm2, %xmm3 466 2. 3 5.7 0.0 0.0 vhaddps %xmm3, %xmm3, %xmm4 467 468The timeline view is interesting because it shows instruction state changes 469during execution. It also gives an idea of how the tool processes instructions 470executed on the target, and how their timing information might be calculated. 471 472The timeline view is structured in two tables. The first table shows 473instructions changing state over time (measured in cycles); the second table 474(named *Average Wait times*) reports useful timing statistics, which should 475help diagnose performance bottlenecks caused by long data dependencies and 476sub-optimal usage of hardware resources. 477 478An instruction in the timeline view is identified by a pair of indices, where 479the first index identifies an iteration, and the second index is the 480instruction index (i.e., where it appears in the code sequence). Since this 481example was generated using 3 iterations: ``-iterations=3``, the iteration 482indices range from 0-2 inclusively. 483 484Excluding the first and last column, the remaining columns are in cycles. 485Cycles are numbered sequentially starting from 0. 486 487From the example output above, we know the following: 488 489* Instruction [1,0] was dispatched at cycle 1. 490* Instruction [1,0] started executing at cycle 2. 491* Instruction [1,0] reached the write back stage at cycle 4. 492* Instruction [1,0] was retired at cycle 10. 493 494Instruction [1,0] (i.e., vmulps from iteration #1) does not have to wait in the 495scheduler's queue for the operands to become available. By the time vmulps is 496dispatched, operands are already available, and pipeline JFPU1 is ready to 497serve another instruction. So the instruction can be immediately issued on the 498JFPU1 pipeline. That is demonstrated by the fact that the instruction only 499spent 1cy in the scheduler's queue. 500 501There is a gap of 5 cycles between the write-back stage and the retire event. 502That is because instructions must retire in program order, so [1,0] has to wait 503for [0,2] to be retired first (i.e., it has to wait until cycle 10). 504 505In the example, all instructions are in a RAW (Read After Write) dependency 506chain. Register %xmm2 written by vmulps is immediately used by the first 507vhaddps, and register %xmm3 written by the first vhaddps is used by the second 508vhaddps. Long data dependencies negatively impact the ILP (Instruction Level 509Parallelism). 510 511In the dot-product example, there are anti-dependencies introduced by 512instructions from different iterations. However, those dependencies can be 513removed at register renaming stage (at the cost of allocating register aliases, 514and therefore consuming physical registers). 515 516Table *Average Wait times* helps diagnose performance issues that are caused by 517the presence of long latency instructions and potentially long data dependencies 518which may limit the ILP. Note that :program:`llvm-mca`, by default, assumes at 519least 1cy between the dispatch event and the issue event. 520 521When the performance is limited by data dependencies and/or long latency 522instructions, the number of cycles spent while in the *ready* state is expected 523to be very small when compared with the total number of cycles spent in the 524scheduler's queue. The difference between the two counters is a good indicator 525of how large of an impact data dependencies had on the execution of the 526instructions. When performance is mostly limited by the lack of hardware 527resources, the delta between the two counters is small. However, the number of 528cycles spent in the queue tends to be larger (i.e., more than 1-3cy), 529especially when compared to other low latency instructions. 530 531Extra Statistics to Further Diagnose Performance Issues 532^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 533The ``-all-stats`` command line option enables extra statistics and performance 534counters for the dispatch logic, the reorder buffer, the retire control unit, 535and the register file. 536 537Below is an example of ``-all-stats`` output generated by :program:`llvm-mca` 538for 300 iterations of the dot-product example discussed in the previous 539sections. 540 541.. code-block:: none 542 543 Dynamic Dispatch Stall Cycles: 544 RAT - Register unavailable: 0 545 RCU - Retire tokens unavailable: 0 546 SCHEDQ - Scheduler full: 272 (44.6%) 547 LQ - Load queue full: 0 548 SQ - Store queue full: 0 549 GROUP - Static restrictions on the dispatch group: 0 550 551 552 Dispatch Logic - number of cycles where we saw N micro opcodes dispatched: 553 [# dispatched], [# cycles] 554 0, 24 (3.9%) 555 1, 272 (44.6%) 556 2, 314 (51.5%) 557 558 559 Schedulers - number of cycles where we saw N micro opcodes issued: 560 [# issued], [# cycles] 561 0, 7 (1.1%) 562 1, 306 (50.2%) 563 2, 297 (48.7%) 564 565 Scheduler's queue usage: 566 [1] Resource name. 567 [2] Average number of used buffer entries. 568 [3] Maximum number of used buffer entries. 569 [4] Total number of buffer entries. 570 571 [1] [2] [3] [4] 572 JALU01 0 0 20 573 JFPU01 17 18 18 574 JLSAGU 0 0 12 575 576 577 Retire Control Unit - number of cycles where we saw N instructions retired: 578 [# retired], [# cycles] 579 0, 109 (17.9%) 580 1, 102 (16.7%) 581 2, 399 (65.4%) 582 583 Total ROB Entries: 64 584 Max Used ROB Entries: 35 ( 54.7% ) 585 Average Used ROB Entries per cy: 32 ( 50.0% ) 586 587 588 Register File statistics: 589 Total number of mappings created: 900 590 Max number of mappings used: 35 591 592 * Register File #1 -- JFpuPRF: 593 Number of physical registers: 72 594 Total number of mappings created: 900 595 Max number of mappings used: 35 596 597 * Register File #2 -- JIntegerPRF: 598 Number of physical registers: 64 599 Total number of mappings created: 0 600 Max number of mappings used: 0 601 602If we look at the *Dynamic Dispatch Stall Cycles* table, we see the counter for 603SCHEDQ reports 272 cycles. This counter is incremented every time the dispatch 604logic is unable to dispatch a full group because the scheduler's queue is full. 605 606Looking at the *Dispatch Logic* table, we see that the pipeline was only able to 607dispatch two micro opcodes 51.5% of the time. The dispatch group was limited to 608one micro opcode 44.6% of the cycles, which corresponds to 272 cycles. The 609dispatch statistics are displayed by either using the command option 610``-all-stats`` or ``-dispatch-stats``. 611 612The next table, *Schedulers*, presents a histogram displaying a count, 613representing the number of micro opcodes issued on some number of cycles. In 614this case, of the 610 simulated cycles, single opcodes were issued 306 times 615(50.2%) and there were 7 cycles where no opcodes were issued. 616 617The *Scheduler's queue usage* table shows that the average and maximum number of 618buffer entries (i.e., scheduler queue entries) used at runtime. Resource JFPU01 619reached its maximum (18 of 18 queue entries). Note that AMD Jaguar implements 620three schedulers: 621 622* JALU01 - A scheduler for ALU instructions. 623* JFPU01 - A scheduler floating point operations. 624* JLSAGU - A scheduler for address generation. 625 626The dot-product is a kernel of three floating point instructions (a vector 627multiply followed by two horizontal adds). That explains why only the floating 628point scheduler appears to be used. 629 630A full scheduler queue is either caused by data dependency chains or by a 631sub-optimal usage of hardware resources. Sometimes, resource pressure can be 632mitigated by rewriting the kernel using different instructions that consume 633different scheduler resources. Schedulers with a small queue are less resilient 634to bottlenecks caused by the presence of long data dependencies. The scheduler 635statistics are displayed by using the command option ``-all-stats`` or 636``-scheduler-stats``. 637 638The next table, *Retire Control Unit*, presents a histogram displaying a count, 639representing the number of instructions retired on some number of cycles. In 640this case, of the 610 simulated cycles, two instructions were retired during the 641same cycle 399 times (65.4%) and there were 109 cycles where no instructions 642were retired. The retire statistics are displayed by using the command option 643``-all-stats`` or ``-retire-stats``. 644 645The last table presented is *Register File statistics*. Each physical register 646file (PRF) used by the pipeline is presented in this table. In the case of AMD 647Jaguar, there are two register files, one for floating-point registers (JFpuPRF) 648and one for integer registers (JIntegerPRF). The table shows that of the 900 649instructions processed, there were 900 mappings created. Since this dot-product 650example utilized only floating point registers, the JFPuPRF was responsible for 651creating the 900 mappings. However, we see that the pipeline only used a 652maximum of 35 of 72 available register slots at any given time. We can conclude 653that the floating point PRF was the only register file used for the example, and 654that it was never resource constrained. The register file statistics are 655displayed by using the command option ``-all-stats`` or 656``-register-file-stats``. 657 658In this example, we can conclude that the IPC is mostly limited by data 659dependencies, and not by resource pressure. 660 661Instruction Flow 662^^^^^^^^^^^^^^^^ 663This section describes the instruction flow through the default pipeline of 664:program:`llvm-mca`, as well as the functional units involved in the process. 665 666The default pipeline implements the following sequence of stages used to 667process instructions. 668 669* Dispatch (Instruction is dispatched to the schedulers). 670* Issue (Instruction is issued to the processor pipelines). 671* Write Back (Instruction is executed, and results are written back). 672* Retire (Instruction is retired; writes are architecturally committed). 673 674The default pipeline only models the out-of-order portion of a processor. 675Therefore, the instruction fetch and decode stages are not modeled. Performance 676bottlenecks in the frontend are not diagnosed. :program:`llvm-mca` assumes that 677instructions have all been decoded and placed into a queue before the simulation 678start. Also, :program:`llvm-mca` does not model branch prediction. 679 680Instruction Dispatch 681"""""""""""""""""""" 682During the dispatch stage, instructions are picked in program order from a 683queue of already decoded instructions, and dispatched in groups to the 684simulated hardware schedulers. 685 686The size of a dispatch group depends on the availability of the simulated 687hardware resources. The processor dispatch width defaults to the value 688of the ``IssueWidth`` in LLVM's scheduling model. 689 690An instruction can be dispatched if: 691 692* The size of the dispatch group is smaller than processor's dispatch width. 693* There are enough entries in the reorder buffer. 694* There are enough physical registers to do register renaming. 695* The schedulers are not full. 696 697Scheduling models can optionally specify which register files are available on 698the processor. :program:`llvm-mca` uses that information to initialize register 699file descriptors. Users can limit the number of physical registers that are 700globally available for register renaming by using the command option 701``-register-file-size``. A value of zero for this option means *unbounded*. By 702knowing how many registers are available for renaming, the tool can predict 703dispatch stalls caused by the lack of physical registers. 704 705The number of reorder buffer entries consumed by an instruction depends on the 706number of micro-opcodes specified for that instruction by the target scheduling 707model. The reorder buffer is responsible for tracking the progress of 708instructions that are "in-flight", and retiring them in program order. The 709number of entries in the reorder buffer defaults to the value specified by field 710`MicroOpBufferSize` in the target scheduling model. 711 712Instructions that are dispatched to the schedulers consume scheduler buffer 713entries. :program:`llvm-mca` queries the scheduling model to determine the set 714of buffered resources consumed by an instruction. Buffered resources are 715treated like scheduler resources. 716 717Instruction Issue 718""""""""""""""""" 719Each processor scheduler implements a buffer of instructions. An instruction 720has to wait in the scheduler's buffer until input register operands become 721available. Only at that point, does the instruction becomes eligible for 722execution and may be issued (potentially out-of-order) for execution. 723Instruction latencies are computed by :program:`llvm-mca` with the help of the 724scheduling model. 725 726:program:`llvm-mca`'s scheduler is designed to simulate multiple processor 727schedulers. The scheduler is responsible for tracking data dependencies, and 728dynamically selecting which processor resources are consumed by instructions. 729It delegates the management of processor resource units and resource groups to a 730resource manager. The resource manager is responsible for selecting resource 731units that are consumed by instructions. For example, if an instruction 732consumes 1cy of a resource group, the resource manager selects one of the 733available units from the group; by default, the resource manager uses a 734round-robin selector to guarantee that resource usage is uniformly distributed 735between all units of a group. 736 737:program:`llvm-mca`'s scheduler internally groups instructions into three sets: 738 739* WaitSet: a set of instructions whose operands are not ready. 740* ReadySet: a set of instructions ready to execute. 741* IssuedSet: a set of instructions executing. 742 743Depending on the operands availability, instructions that are dispatched to the 744scheduler are either placed into the WaitSet or into the ReadySet. 745 746Every cycle, the scheduler checks if instructions can be moved from the WaitSet 747to the ReadySet, and if instructions from the ReadySet can be issued to the 748underlying pipelines. The algorithm prioritizes older instructions over younger 749instructions. 750 751Write-Back and Retire Stage 752""""""""""""""""""""""""""" 753Issued instructions are moved from the ReadySet to the IssuedSet. There, 754instructions wait until they reach the write-back stage. At that point, they 755get removed from the queue and the retire control unit is notified. 756 757When instructions are executed, the retire control unit flags the instruction as 758"ready to retire." 759 760Instructions are retired in program order. The register file is notified of the 761retirement so that it can free the physical registers that were allocated for 762the instruction during the register renaming stage. 763 764Load/Store Unit and Memory Consistency Model 765"""""""""""""""""""""""""""""""""""""""""""" 766To simulate an out-of-order execution of memory operations, :program:`llvm-mca` 767utilizes a simulated load/store unit (LSUnit) to simulate the speculative 768execution of loads and stores. 769 770Each load (or store) consumes an entry in the load (or store) queue. Users can 771specify flags ``-lqueue`` and ``-squeue`` to limit the number of entries in the 772load and store queues respectively. The queues are unbounded by default. 773 774The LSUnit implements a relaxed consistency model for memory loads and stores. 775The rules are: 776 7771. A younger load is allowed to pass an older load only if there are no 778 intervening stores or barriers between the two loads. 7792. A younger load is allowed to pass an older store provided that the load does 780 not alias with the store. 7813. A younger store is not allowed to pass an older store. 7824. A younger store is not allowed to pass an older load. 783 784By default, the LSUnit optimistically assumes that loads do not alias 785(`-noalias=true`) store operations. Under this assumption, younger loads are 786always allowed to pass older stores. Essentially, the LSUnit does not attempt 787to run any alias analysis to predict when loads and stores do not alias with 788each other. 789 790Note that, in the case of write-combining memory, rule 3 could be relaxed to 791allow reordering of non-aliasing store operations. That being said, at the 792moment, there is no way to further relax the memory model (``-noalias`` is the 793only option). Essentially, there is no option to specify a different memory 794type (e.g., write-back, write-combining, write-through; etc.) and consequently 795to weaken, or strengthen, the memory model. 796 797Other limitations are: 798 799* The LSUnit does not know when store-to-load forwarding may occur. 800* The LSUnit does not know anything about cache hierarchy and memory types. 801* The LSUnit does not know how to identify serializing operations and memory 802 fences. 803 804The LSUnit does not attempt to predict if a load or store hits or misses the L1 805cache. It only knows if an instruction "MayLoad" and/or "MayStore." For 806loads, the scheduling model provides an "optimistic" load-to-use latency (which 807usually matches the load-to-use latency for when there is a hit in the L1D). 808 809:program:`llvm-mca` does not know about serializing operations or memory-barrier 810like instructions. The LSUnit conservatively assumes that an instruction which 811has both "MayLoad" and unmodeled side effects behaves like a "soft" 812load-barrier. That means, it serializes loads without forcing a flush of the 813load queue. Similarly, instructions that "MayStore" and have unmodeled side 814effects are treated like store barriers. A full memory barrier is a "MayLoad" 815and "MayStore" instruction with unmodeled side effects. This is inaccurate, but 816it is the best that we can do at the moment with the current information 817available in LLVM. 818 819A load/store barrier consumes one entry of the load/store queue. A load/store 820barrier enforces ordering of loads/stores. A younger load cannot pass a load 821barrier. Also, a younger store cannot pass a store barrier. A younger load 822has to wait for the memory/load barrier to execute. A load/store barrier is 823"executed" when it becomes the oldest entry in the load/store queue(s). That 824also means, by construction, all of the older loads/stores have been executed. 825 826In conclusion, the full set of load/store consistency rules are: 827 828#. A store may not pass a previous store. 829#. A store may not pass a previous load (regardless of ``-noalias``). 830#. A store has to wait until an older store barrier is fully executed. 831#. A load may pass a previous load. 832#. A load may not pass a previous store unless ``-noalias`` is set. 833#. A load has to wait until an older load barrier is fully executed. 834