1================================ 2Source Level Debugging with LLVM 3================================ 4 5.. contents:: 6 :local: 7 8Introduction 9============ 10 11This document is the central repository for all information pertaining to debug 12information in LLVM. It describes the :ref:`actual format that the LLVM debug 13information takes <format>`, which is useful for those interested in creating 14front-ends or dealing directly with the information. Further, this document 15provides specific examples of what debug information for C/C++ looks like. 16 17Philosophy behind LLVM debugging information 18-------------------------------------------- 19 20The idea of the LLVM debugging information is to capture how the important 21pieces of the source-language's Abstract Syntax Tree map onto LLVM code. 22Several design aspects have shaped the solution that appears here. The 23important ones are: 24 25* Debugging information should have very little impact on the rest of the 26 compiler. No transformations, analyses, or code generators should need to 27 be modified because of debugging information. 28 29* LLVM optimizations should interact in :ref:`well-defined and easily described 30 ways <intro_debugopt>` with the debugging information. 31 32* Because LLVM is designed to support arbitrary programming languages, 33 LLVM-to-LLVM tools should not need to know anything about the semantics of 34 the source-level-language. 35 36* Source-level languages are often **widely** different from one another. 37 LLVM should not put any restrictions of the flavor of the source-language, 38 and the debugging information should work with any language. 39 40* With code generator support, it should be possible to use an LLVM compiler 41 to compile a program to native machine code and standard debugging 42 formats. This allows compatibility with traditional machine-code level 43 debuggers, like GDB or DBX. 44 45The approach used by the LLVM implementation is to use a small set of 46:ref:`intrinsic functions <format_common_intrinsics>` to define a mapping 47between LLVM program objects and the source-level objects. The description of 48the source-level program is maintained in LLVM metadata in an 49:ref:`implementation-defined format <ccxx_frontend>` (the C/C++ front-end 50currently uses working draft 7 of the `DWARF 3 standard 51<http://www.eagercon.com/dwarf/dwarf3std.htm>`_). 52 53When a program is being debugged, a debugger interacts with the user and turns 54the stored debug information into source-language specific information. As 55such, a debugger must be aware of the source-language, and is thus tied to a 56specific language or family of languages. 57 58Debug information consumers 59--------------------------- 60 61The role of debug information is to provide meta information normally stripped 62away during the compilation process. This meta information provides an LLVM 63user a relationship between generated code and the original program source 64code. 65 66Currently, there are two backend consumers of debug info: DwarfDebug and 67CodeViewDebug. DwarfDebug produces DWARF suitable for use with GDB, LLDB, and 68other DWARF-based debuggers. :ref:`CodeViewDebug <codeview>` produces CodeView, 69the Microsoft debug info format, which is usable with Microsoft debuggers such 70as Visual Studio and WinDBG. LLVM's debug information format is mostly derived 71from and inspired by DWARF, but it is feasible to translate into other target 72debug info formats such as STABS. 73 74It would also be reasonable to use debug information to feed profiling tools 75for analysis of generated code, or, tools for reconstructing the original 76source from generated code. 77 78.. _intro_debugopt: 79 80Debug information and optimizations 81----------------------------------- 82 83An extremely high priority of LLVM debugging information is to make it interact 84well with optimizations and analysis. In particular, the LLVM debug 85information provides the following guarantees: 86 87* LLVM debug information **always provides information to accurately read 88 the source-level state of the program**, regardless of which LLVM 89 optimizations have been run. :doc:`HowToUpdateDebugInfo` specifies how debug 90 info should be updated in various kinds of code transformations to avoid 91 breaking this guarantee, and how to preserve as much useful debug info as 92 possible. Note that some optimizations may impact the ability to modify the 93 current state of the program with a debugger, such as setting program 94 variables, or calling functions that have been deleted. 95 96* As desired, LLVM optimizations can be upgraded to be aware of debugging 97 information, allowing them to update the debugging information as they 98 perform aggressive optimizations. This means that, with effort, the LLVM 99 optimizers could optimize debug code just as well as non-debug code. 100 101* LLVM debug information does not prevent optimizations from 102 happening (for example inlining, basic block reordering/merging/cleanup, 103 tail duplication, etc). 104 105* LLVM debug information is automatically optimized along with the rest of 106 the program, using existing facilities. For example, duplicate 107 information is automatically merged by the linker, and unused information 108 is automatically removed. 109 110Basically, the debug information allows you to compile a program with 111"``-O0 -g``" and get full debug information, allowing you to arbitrarily modify 112the program as it executes from a debugger. Compiling a program with 113"``-O3 -g``" gives you full debug information that is always available and 114accurate for reading (e.g., you get accurate stack traces despite tail call 115elimination and inlining), but you might lose the ability to modify the program 116and call functions which were optimized out of the program, or inlined away 117completely. 118 119The :doc:`LLVM test-suite <TestSuiteMakefileGuide>` provides a framework to 120test the optimizer's handling of debugging information. It can be run like 121this: 122 123.. code-block:: bash 124 125 % cd llvm/projects/test-suite/MultiSource/Benchmarks # or some other level 126 % make TEST=dbgopt 127 128This will test impact of debugging information on optimization passes. If 129debugging information influences optimization passes then it will be reported 130as a failure. See :doc:`TestingGuide` for more information on LLVM test 131infrastructure and how to run various tests. 132 133.. _format: 134 135Debugging information format 136============================ 137 138LLVM debugging information has been carefully designed to make it possible for 139the optimizer to optimize the program and debugging information without 140necessarily having to know anything about debugging information. In 141particular, the use of metadata avoids duplicated debugging information from 142the beginning, and the global dead code elimination pass automatically deletes 143debugging information for a function if it decides to delete the function. 144 145To do this, most of the debugging information (descriptors for types, 146variables, functions, source files, etc) is inserted by the language front-end 147in the form of LLVM metadata. 148 149Debug information is designed to be agnostic about the target debugger and 150debugging information representation (e.g. DWARF/Stabs/etc). It uses a generic 151pass to decode the information that represents variables, types, functions, 152namespaces, etc: this allows for arbitrary source-language semantics and 153type-systems to be used, as long as there is a module written for the target 154debugger to interpret the information. 155 156To provide basic functionality, the LLVM debugger does have to make some 157assumptions about the source-level language being debugged, though it keeps 158these to a minimum. The only common features that the LLVM debugger assumes 159exist are `source files <LangRef.html#difile>`_, and `program objects 160<LangRef.html#diglobalvariable>`_. These abstract objects are used by a 161debugger to form stack traces, show information about local variables, etc. 162 163This section of the documentation first describes the representation aspects 164common to any source-language. :ref:`ccxx_frontend` describes the data layout 165conventions used by the C and C++ front-ends. 166 167Debug information descriptors are `specialized metadata nodes 168<LangRef.html#specialized-metadata>`_, first-class subclasses of ``Metadata``. 169 170.. _format_common_intrinsics: 171 172Debugger intrinsic functions 173---------------------------- 174 175LLVM uses several intrinsic functions (name prefixed with "``llvm.dbg``") to 176track source local variables through optimization and code generation. 177 178``llvm.dbg.addr`` 179^^^^^^^^^^^^^^^^^^^^ 180 181.. code-block:: llvm 182 183 void @llvm.dbg.addr(metadata, metadata, metadata) 184 185This intrinsic provides information about a local element (e.g., variable). 186The first argument is metadata holding the address of variable, typically a 187static alloca in the function entry block. The second argument is a 188`local variable <LangRef.html#dilocalvariable>`_ containing a description of 189the variable. The third argument is a `complex expression 190<LangRef.html#diexpression>`_. An `llvm.dbg.addr` intrinsic describes the 191*address* of a source variable. 192 193.. code-block:: text 194 195 %i.addr = alloca i32, align 4 196 call void @llvm.dbg.addr(metadata i32* %i.addr, metadata !1, 197 metadata !DIExpression()), !dbg !2 198 !1 = !DILocalVariable(name: "i", ...) ; int i 199 !2 = !DILocation(...) 200 ... 201 %buffer = alloca [256 x i8], align 8 202 ; The address of i is buffer+64. 203 call void @llvm.dbg.addr(metadata [256 x i8]* %buffer, metadata !3, 204 metadata !DIExpression(DW_OP_plus, 64)), !dbg !4 205 !3 = !DILocalVariable(name: "i", ...) ; int i 206 !4 = !DILocation(...) 207 208A frontend should generate exactly one call to ``llvm.dbg.addr`` at the point 209of declaration of a source variable. Optimization passes that fully promote the 210variable from memory to SSA values will replace this call with possibly 211multiple calls to `llvm.dbg.value`. Passes that delete stores are effectively 212partial promotion, and they will insert a mix of calls to ``llvm.dbg.value`` 213and ``llvm.dbg.addr`` to track the source variable value when it is available. 214After optimization, there may be multiple calls to ``llvm.dbg.addr`` describing 215the program points where the variables lives in memory. All calls for the same 216concrete source variable must agree on the memory location. 217 218 219``llvm.dbg.declare`` 220^^^^^^^^^^^^^^^^^^^^ 221 222.. code-block:: llvm 223 224 void @llvm.dbg.declare(metadata, metadata, metadata) 225 226This intrinsic is identical to `llvm.dbg.addr`, except that there can only be 227one call to `llvm.dbg.declare` for a given concrete `local variable 228<LangRef.html#dilocalvariable>`_. It is not control-dependent, meaning that if 229a call to `llvm.dbg.declare` exists and has a valid location argument, that 230address is considered to be the true home of the variable across its entire 231lifetime. This makes it hard for optimizations to preserve accurate debug info 232in the presence of ``llvm.dbg.declare``, so we are transitioning away from it, 233and we plan to deprecate it in future LLVM releases. 234 235 236``llvm.dbg.value`` 237^^^^^^^^^^^^^^^^^^ 238 239.. code-block:: llvm 240 241 void @llvm.dbg.value(metadata, metadata, metadata) 242 243This intrinsic provides information when a user source variable is set to a new 244value. The first argument is the new value (wrapped as metadata). The second 245argument is a `local variable <LangRef.html#dilocalvariable>`_ containing a 246description of the variable. The third argument is a `complex expression 247<LangRef.html#diexpression>`_. 248 249An `llvm.dbg.value` intrinsic describes the *value* of a source variable 250directly, not its address. Note that the value operand of this intrinsic may 251be indirect (i.e, a pointer to the source variable), provided that interpreting 252the complex expression derives the direct value. 253 254Object lifetimes and scoping 255============================ 256 257In many languages, the local variables in functions can have their lifetimes or 258scopes limited to a subset of a function. In the C family of languages, for 259example, variables are only live (readable and writable) within the source 260block that they are defined in. In functional languages, values are only 261readable after they have been defined. Though this is a very obvious concept, 262it is non-trivial to model in LLVM, because it has no notion of scoping in this 263sense, and does not want to be tied to a language's scoping rules. 264 265In order to handle this, the LLVM debug format uses the metadata attached to 266llvm instructions to encode line number and scoping information. Consider the 267following C fragment, for example: 268 269.. code-block:: c 270 271 1. void foo() { 272 2. int X = 21; 273 3. int Y = 22; 274 4. { 275 5. int Z = 23; 276 6. Z = X; 277 7. } 278 8. X = Y; 279 9. } 280 281.. FIXME: Update the following example to use llvm.dbg.addr once that is the 282 default in clang. 283 284Compiled to LLVM, this function would be represented like this: 285 286.. code-block:: text 287 288 ; Function Attrs: nounwind ssp uwtable 289 define void @foo() #0 !dbg !4 { 290 entry: 291 %X = alloca i32, align 4 292 %Y = alloca i32, align 4 293 %Z = alloca i32, align 4 294 call void @llvm.dbg.declare(metadata i32* %X, metadata !11, metadata !DIExpression()), !dbg !13 295 store i32 21, i32* %X, align 4, !dbg !13 296 call void @llvm.dbg.declare(metadata i32* %Y, metadata !14, metadata !DIExpression()), !dbg !15 297 store i32 22, i32* %Y, align 4, !dbg !15 298 call void @llvm.dbg.declare(metadata i32* %Z, metadata !16, metadata !DIExpression()), !dbg !18 299 store i32 23, i32* %Z, align 4, !dbg !18 300 %0 = load i32, i32* %X, align 4, !dbg !19 301 store i32 %0, i32* %Z, align 4, !dbg !20 302 %1 = load i32, i32* %Y, align 4, !dbg !21 303 store i32 %1, i32* %X, align 4, !dbg !22 304 ret void, !dbg !23 305 } 306 307 ; Function Attrs: nounwind readnone 308 declare void @llvm.dbg.declare(metadata, metadata, metadata) #1 309 310 attributes #0 = { nounwind ssp uwtable "less-precise-fpmad"="false" "frame-pointer"="all" "no-infs-fp-math"="false" "no-nans-fp-math"="false" "stack-protector-buffer-size"="8" "unsafe-fp-math"="false" "use-soft-float"="false" } 311 attributes #1 = { nounwind readnone } 312 313 !llvm.dbg.cu = !{!0} 314 !llvm.module.flags = !{!7, !8, !9} 315 !llvm.ident = !{!10} 316 317 !0 = !DICompileUnit(language: DW_LANG_C99, file: !1, producer: "clang version 3.7.0 (trunk 231150) (llvm/trunk 231154)", isOptimized: false, runtimeVersion: 0, emissionKind: FullDebug, enums: !2, retainedTypes: !2, subprograms: !3, globals: !2, imports: !2) 318 !1 = !DIFile(filename: "/dev/stdin", directory: "/Users/dexonsmith/data/llvm/debug-info") 319 !2 = !{} 320 !3 = !{!4} 321 !4 = distinct !DISubprogram(name: "foo", scope: !1, file: !1, line: 1, type: !5, isLocal: false, isDefinition: true, scopeLine: 1, isOptimized: false, variables: !2) 322 !5 = !DISubroutineType(types: !6) 323 !6 = !{null} 324 !7 = !{i32 2, !"Dwarf Version", i32 2} 325 !8 = !{i32 2, !"Debug Info Version", i32 3} 326 !9 = !{i32 1, !"PIC Level", i32 2} 327 !10 = !{!"clang version 3.7.0 (trunk 231150) (llvm/trunk 231154)"} 328 !11 = !DILocalVariable(name: "X", scope: !4, file: !1, line: 2, type: !12) 329 !12 = !DIBasicType(name: "int", size: 32, align: 32, encoding: DW_ATE_signed) 330 !13 = !DILocation(line: 2, column: 9, scope: !4) 331 !14 = !DILocalVariable(name: "Y", scope: !4, file: !1, line: 3, type: !12) 332 !15 = !DILocation(line: 3, column: 9, scope: !4) 333 !16 = !DILocalVariable(name: "Z", scope: !17, file: !1, line: 5, type: !12) 334 !17 = distinct !DILexicalBlock(scope: !4, file: !1, line: 4, column: 5) 335 !18 = !DILocation(line: 5, column: 11, scope: !17) 336 !19 = !DILocation(line: 6, column: 11, scope: !17) 337 !20 = !DILocation(line: 6, column: 9, scope: !17) 338 !21 = !DILocation(line: 8, column: 9, scope: !4) 339 !22 = !DILocation(line: 8, column: 7, scope: !4) 340 !23 = !DILocation(line: 9, column: 3, scope: !4) 341 342 343This example illustrates a few important details about LLVM debugging 344information. In particular, it shows how the ``llvm.dbg.declare`` intrinsic and 345location information, which are attached to an instruction, are applied 346together to allow a debugger to analyze the relationship between statements, 347variable definitions, and the code used to implement the function. 348 349.. code-block:: llvm 350 351 call void @llvm.dbg.declare(metadata i32* %X, metadata !11, metadata !DIExpression()), !dbg !13 352 ; [debug line = 2:7] [debug variable = X] 353 354The first intrinsic ``%llvm.dbg.declare`` encodes debugging information for the 355variable ``X``. The metadata ``!dbg !13`` attached to the intrinsic provides 356scope information for the variable ``X``. 357 358.. code-block:: text 359 360 !13 = !DILocation(line: 2, column: 9, scope: !4) 361 !4 = distinct !DISubprogram(name: "foo", scope: !1, file: !1, line: 1, type: !5, 362 isLocal: false, isDefinition: true, scopeLine: 1, 363 isOptimized: false, variables: !2) 364 365Here ``!13`` is metadata providing `location information 366<LangRef.html#dilocation>`_. In this example, scope is encoded by ``!4``, a 367`subprogram descriptor <LangRef.html#disubprogram>`_. This way the location 368information attached to the intrinsics indicates that the variable ``X`` is 369declared at line number 2 at a function level scope in function ``foo``. 370 371Now lets take another example. 372 373.. code-block:: llvm 374 375 call void @llvm.dbg.declare(metadata i32* %Z, metadata !16, metadata !DIExpression()), !dbg !18 376 ; [debug line = 5:9] [debug variable = Z] 377 378The third intrinsic ``%llvm.dbg.declare`` encodes debugging information for 379variable ``Z``. The metadata ``!dbg !18`` attached to the intrinsic provides 380scope information for the variable ``Z``. 381 382.. code-block:: text 383 384 !17 = distinct !DILexicalBlock(scope: !4, file: !1, line: 4, column: 5) 385 !18 = !DILocation(line: 5, column: 11, scope: !17) 386 387Here ``!18`` indicates that ``Z`` is declared at line number 5 and column 388number 11 inside of lexical scope ``!17``. The lexical scope itself resides 389inside of subprogram ``!4`` described above. 390 391The scope information attached with each instruction provides a straightforward 392way to find instructions covered by a scope. 393 394Object lifetime in optimized code 395================================= 396 397In the example above, every variable assignment uniquely corresponds to a 398memory store to the variable's position on the stack. However in heavily 399optimized code LLVM promotes most variables into SSA values, which can 400eventually be placed in physical registers or memory locations. To track SSA 401values through compilation, when objects are promoted to SSA values an 402``llvm.dbg.value`` intrinsic is created for each assignment, recording the 403variable's new location. Compared with the ``llvm.dbg.declare`` intrinsic: 404 405* A dbg.value terminates the effect of any preceding dbg.values for (any 406 overlapping fragments of) the specified variable. 407* The dbg.value's position in the IR defines where in the instruction stream 408 the variable's value changes. 409* Operands can be constants, indicating the variable is assigned a 410 constant value. 411 412Care must be taken to update ``llvm.dbg.value`` intrinsics when optimization 413passes alter or move instructions and blocks -- the developer could observe such 414changes reflected in the value of variables when debugging the program. For any 415execution of the optimized program, the set of variable values presented to the 416developer by the debugger should not show a state that would never have existed 417in the execution of the unoptimized program, given the same input. Doing so 418risks misleading the developer by reporting a state that does not exist, 419damaging their understanding of the optimized program and undermining their 420trust in the debugger. 421 422Sometimes perfectly preserving variable locations is not possible, often when a 423redundant calculation is optimized out. In such cases, a ``llvm.dbg.value`` 424with operand ``undef`` should be used, to terminate earlier variable locations 425and let the debugger present ``optimized out`` to the developer. Withholding 426these potentially stale variable values from the developer diminishes the 427amount of available debug information, but increases the reliability of the 428remaining information. 429 430To illustrate some potential issues, consider the following example: 431 432.. code-block:: llvm 433 434 define i32 @foo(i32 %bar, i1 %cond) { 435 entry: 436 call @llvm.dbg.value(metadata i32 0, metadata !1, metadata !2) 437 br i1 %cond, label %truebr, label %falsebr 438 truebr: 439 %tval = add i32 %bar, 1 440 call @llvm.dbg.value(metadata i32 %tval, metadata !1, metadata !2) 441 %g1 = call i32 @gazonk() 442 br label %exit 443 falsebr: 444 %fval = add i32 %bar, 2 445 call @llvm.dbg.value(metadata i32 %fval, metadata !1, metadata !2) 446 %g2 = call i32 @gazonk() 447 br label %exit 448 exit: 449 %merge = phi [ %tval, %truebr ], [ %fval, %falsebr ] 450 %g = phi [ %g1, %truebr ], [ %g2, %falsebr ] 451 call @llvm.dbg.value(metadata i32 %merge, metadata !1, metadata !2) 452 call @llvm.dbg.value(metadata i32 %g, metadata !3, metadata !2) 453 %plusten = add i32 %merge, 10 454 %toret = add i32 %plusten, %g 455 call @llvm.dbg.value(metadata i32 %toret, metadata !1, metadata !2) 456 ret i32 %toret 457 } 458 459Containing two source-level variables in ``!1`` and ``!3``. The function could, 460perhaps, be optimized into the following code: 461 462.. code-block:: llvm 463 464 define i32 @foo(i32 %bar, i1 %cond) { 465 entry: 466 %g = call i32 @gazonk() 467 %addoper = select i1 %cond, i32 11, i32 12 468 %plusten = add i32 %bar, %addoper 469 %toret = add i32 %plusten, %g 470 ret i32 %toret 471 } 472 473What ``llvm.dbg.value`` intrinsics should be placed to represent the original variable 474locations in this code? Unfortunately the second, third and fourth 475dbg.values for ``!1`` in the source function have had their operands 476(%tval, %fval, %merge) optimized out. Assuming we cannot recover them, we 477might consider this placement of dbg.values: 478 479.. code-block:: llvm 480 481 define i32 @foo(i32 %bar, i1 %cond) { 482 entry: 483 call @llvm.dbg.value(metadata i32 0, metadata !1, metadata !2) 484 %g = call i32 @gazonk() 485 call @llvm.dbg.value(metadata i32 %g, metadata !3, metadata !2) 486 %addoper = select i1 %cond, i32 11, i32 12 487 %plusten = add i32 %bar, %addoper 488 %toret = add i32 %plusten, %g 489 call @llvm.dbg.value(metadata i32 %toret, metadata !1, metadata !2) 490 ret i32 %toret 491 } 492 493However, this will cause ``!3`` to have the return value of ``@gazonk()`` at 494the same time as ``!1`` has the constant value zero -- a pair of assignments 495that never occurred in the unoptimized program. To avoid this, we must terminate 496the range that ``!1`` has the constant value assignment by inserting an undef 497dbg.value before the dbg.value for ``!3``: 498 499.. code-block:: llvm 500 501 define i32 @foo(i32 %bar, i1 %cond) { 502 entry: 503 call @llvm.dbg.value(metadata i32 0, metadata !1, metadata !2) 504 %g = call i32 @gazonk() 505 call @llvm.dbg.value(metadata i32 undef, metadata !1, metadata !2) 506 call @llvm.dbg.value(metadata i32 %g, metadata !3, metadata !2) 507 %addoper = select i1 %cond, i32 11, i32 12 508 %plusten = add i32 %bar, %addoper 509 %toret = add i32 %plusten, %g 510 call @llvm.dbg.value(metadata i32 %toret, metadata !1, metadata !2) 511 ret i32 %toret 512 } 513 514In general, if any dbg.value has its operand optimized out and cannot be 515recovered, then an undef dbg.value is necessary to terminate earlier variable 516locations. Additional undef dbg.values may be necessary when the debugger can 517observe re-ordering of assignments. 518 519How variable location metadata is transformed during CodeGen 520============================================================ 521 522LLVM preserves debug information throughout mid-level and backend passes, 523ultimately producing a mapping between source-level information and 524instruction ranges. This 525is relatively straightforwards for line number information, as mapping 526instructions to line numbers is a simple association. For variable locations 527however the story is more complex. As each ``llvm.dbg.value`` intrinsic 528represents a source-level assignment of a value to a source variable, the 529variable location intrinsics effectively embed a small imperative program 530within the LLVM IR. By the end of CodeGen, this becomes a mapping from each 531variable to their machine locations over ranges of instructions. 532From IR to object emission, the major transformations which affect variable 533location fidelity are: 534 5351. Instruction Selection 5362. Register allocation 5373. Block layout 538 539each of which are discussed below. In addition, instruction scheduling can 540significantly change the ordering of the program, and occurs in a number of 541different passes. 542 543Some variable locations are not transformed during CodeGen. Stack locations 544specified by ``llvm.dbg.declare`` are valid and unchanging for the entire 545duration of the function, and are recorded in a simple MachineFunction table. 546Location changes in the prologue and epilogue of a function are also ignored: 547frame setup and destruction may take several instructions, require a 548disproportionate amount of debugging information in the output binary to 549describe, and should be stepped over by debuggers anyway. 550 551Variable locations in Instruction Selection and MIR 552--------------------------------------------------- 553 554Instruction selection creates a MIR function from an IR function, and just as 555it transforms ``intermediate`` instructions into machine instructions, so must 556``intermediate`` variable locations become machine variable locations. 557Within IR, variable locations are always identified by a Value, but in MIR 558there can be different types of variable locations. In addition, some IR 559locations become unavailable, for example if the operation of multiple IR 560instructions are combined into one machine instruction (such as 561multiply-and-accumulate) then intermediate Values are lost. To track variable 562locations through instruction selection, they are first separated into 563locations that do not depend on code generation (constants, stack locations, 564allocated virtual registers) and those that do. For those that do, debug 565metadata is attached to SDNodes in SelectionDAGs. After instruction selection 566has occurred and a MIR function is created, if the SDNode associated with debug 567metadata is allocated a virtual register, that virtual register is used as the 568variable location. If the SDNode is folded into a machine instruction or 569otherwise transformed into a non-register, the variable location becomes 570unavailable. 571 572Locations that are unavailable are treated as if they have been optimized out: 573in IR the location would be assigned ``undef`` by a debug intrinsic, and in MIR 574the equivalent location is used. 575 576After MIR locations are assigned to each variable, machine pseudo-instructions 577corresponding to each ``llvm.dbg.value`` and ``llvm.dbg.addr`` intrinsic are 578inserted. There are two forms of this type of instruction. 579 580The first form, ``DBG_VALUE``, appears thus: 581 582.. code-block:: text 583 584 DBG_VALUE %1, $noreg, !123, !DIExpression() 585 586And has the following operands: 587 * The first operand can record the variable location as a register, 588 a frame index, an immediate, or the base address register if the original 589 debug intrinsic referred to memory. ``$noreg`` indicates the variable 590 location is undefined, equivalent to an ``undef`` dbg.value operand. 591 * The type of the second operand indicates whether the variable location is 592 directly referred to by the DBG_VALUE, or whether it is indirect. The 593 ``$noreg`` register signifies the former, an immediate operand (0) the 594 latter. 595 * Operand 3 is the Variable field of the original debug intrinsic. 596 * Operand 4 is the Expression field of the original debug intrinsic. 597 598The second form, ``DBG_VALUE_LIST``, appears thus: 599 600.. code-block:: text 601 602 DBG_VALUE_LIST !123, !DIExpression(DW_OP_LLVM_arg, 0, DW_OP_LLVM_arg, 1, DW_OP_plus), %1, %2 603 604And has the following operands: 605 * The first operand is the Variable field of the original debug intrinsic. 606 * The second operand is the Expression field of the original debug intrinsic. 607 * Any number of operands, from the 3rd onwards, record a sequence of variable 608 location operands, which may take any of the same values as the first 609 operand of the ``DBG_VALUE`` instruction above. These variable location 610 operands are inserted into the final DWARF Expression in positions indicated 611 by the DW_OP_LLVM_arg operator in the `DIExpression 612 <LangRef.html#diexpression>`. 613 614The position at which the DBG_VALUEs are inserted should correspond to the 615positions of their matching ``llvm.dbg.value`` intrinsics in the IR block. As 616with optimization, LLVM aims to preserve the order in which variable 617assignments occurred in the source program. However SelectionDAG performs some 618instruction scheduling, which can reorder assignments (discussed below). 619Function parameter locations are moved to the beginning of the function if 620they're not already, to ensure they're immediately available on function entry. 621 622To demonstrate variable locations during instruction selection, consider 623the following example: 624 625.. code-block:: llvm 626 627 define i32 @foo(i32* %addr) { 628 entry: 629 call void @llvm.dbg.value(metadata i32 0, metadata !3, metadata !DIExpression()), !dbg !5 630 br label %bb1, !dbg !5 631 632 bb1: ; preds = %bb1, %entry 633 %bar.0 = phi i32 [ 0, %entry ], [ %add, %bb1 ] 634 call void @llvm.dbg.value(metadata i32 %bar.0, metadata !3, metadata !DIExpression()), !dbg !5 635 %addr1 = getelementptr i32, i32 *%addr, i32 1, !dbg !5 636 call void @llvm.dbg.value(metadata i32 *%addr1, metadata !3, metadata !DIExpression()), !dbg !5 637 %loaded1 = load i32, i32* %addr1, !dbg !5 638 %addr2 = getelementptr i32, i32 *%addr, i32 %bar.0, !dbg !5 639 call void @llvm.dbg.value(metadata i32 *%addr2, metadata !3, metadata !DIExpression()), !dbg !5 640 %loaded2 = load i32, i32* %addr2, !dbg !5 641 %add = add i32 %bar.0, 1, !dbg !5 642 call void @llvm.dbg.value(metadata i32 %add, metadata !3, metadata !DIExpression()), !dbg !5 643 %added = add i32 %loaded1, %loaded2 644 %cond = icmp ult i32 %added, %bar.0, !dbg !5 645 br i1 %cond, label %bb1, label %bb2, !dbg !5 646 647 bb2: ; preds = %bb1 648 ret i32 0, !dbg !5 649 } 650 651If one compiles this IR with ``llc -o - -start-after=codegen-prepare -stop-after=expand-isel-pseudos -mtriple=x86_64--``, the following MIR is produced: 652 653.. code-block:: text 654 655 bb.0.entry: 656 successors: %bb.1(0x80000000) 657 liveins: $rdi 658 659 %2:gr64 = COPY $rdi 660 %3:gr32 = MOV32r0 implicit-def dead $eflags 661 DBG_VALUE 0, $noreg, !3, !DIExpression(), debug-location !5 662 663 bb.1.bb1: 664 successors: %bb.1(0x7c000000), %bb.2(0x04000000) 665 666 %0:gr32 = PHI %3, %bb.0, %1, %bb.1 667 DBG_VALUE %0, $noreg, !3, !DIExpression(), debug-location !5 668 DBG_VALUE %2, $noreg, !3, !DIExpression(DW_OP_plus_uconst, 4, DW_OP_stack_value), debug-location !5 669 %4:gr32 = MOV32rm %2, 1, $noreg, 4, $noreg, debug-location !5 :: (load 4 from %ir.addr1) 670 %5:gr64_nosp = MOVSX64rr32 %0, debug-location !5 671 DBG_VALUE $noreg, $noreg, !3, !DIExpression(), debug-location !5 672 %1:gr32 = INC32r %0, implicit-def dead $eflags, debug-location !5 673 DBG_VALUE %1, $noreg, !3, !DIExpression(), debug-location !5 674 %6:gr32 = ADD32rm %4, %2, 4, killed %5, 0, $noreg, implicit-def dead $eflags :: (load 4 from %ir.addr2) 675 %7:gr32 = SUB32rr %6, %0, implicit-def $eflags, debug-location !5 676 JB_1 %bb.1, implicit $eflags, debug-location !5 677 JMP_1 %bb.2, debug-location !5 678 679 bb.2.bb2: 680 %8:gr32 = MOV32r0 implicit-def dead $eflags 681 $eax = COPY %8, debug-location !5 682 RET 0, $eax, debug-location !5 683 684Observe first that there is a DBG_VALUE instruction for every ``llvm.dbg.value`` 685intrinsic in the source IR, ensuring no source level assignments go missing. 686Then consider the different ways in which variable locations have been recorded: 687 688* For the first dbg.value an immediate operand is used to record a zero value. 689* The dbg.value of the PHI instruction leads to a DBG_VALUE of virtual register 690 ``%0``. 691* The first GEP has its effect folded into the first load instruction 692 (as a 4-byte offset), but the variable location is salvaged by folding 693 the GEPs effect into the DIExpression. 694* The second GEP is also folded into the corresponding load. However, it is 695 insufficiently simple to be salvaged, and is emitted as a ``$noreg`` 696 DBG_VALUE, indicating that the variable takes on an undefined location. 697* The final dbg.value has its Value placed in virtual register ``%1``. 698 699Instruction Scheduling 700---------------------- 701 702A number of passes can reschedule instructions, notably instruction selection 703and the pre-and-post RA machine schedulers. Instruction scheduling can 704significantly change the nature of the program -- in the (very unlikely) worst 705case the instruction sequence could be completely reversed. In such 706circumstances LLVM follows the principle applied to optimizations, that it is 707better for the debugger not to display any state than a misleading state. 708Thus, whenever instructions are advanced in order of execution, any 709corresponding DBG_VALUE is kept in its original position, and if an instruction 710is delayed then the variable is given an undefined location for the duration 711of the delay. To illustrate, consider this pseudo-MIR: 712 713.. code-block:: text 714 715 %1:gr32 = MOV32rm %0, 1, $noreg, 4, $noreg, debug-location !5 :: (load 4 from %ir.addr1) 716 DBG_VALUE %1, $noreg, !1, !2 717 %4:gr32 = ADD32rr %3, %2, implicit-def dead $eflags 718 DBG_VALUE %4, $noreg, !3, !4 719 %7:gr32 = SUB32rr %6, %5, implicit-def dead $eflags 720 DBG_VALUE %7, $noreg, !5, !6 721 722Imagine that the SUB32rr were moved forward to give us the following MIR: 723 724.. code-block:: text 725 726 %7:gr32 = SUB32rr %6, %5, implicit-def dead $eflags 727 %1:gr32 = MOV32rm %0, 1, $noreg, 4, $noreg, debug-location !5 :: (load 4 from %ir.addr1) 728 DBG_VALUE %1, $noreg, !1, !2 729 %4:gr32 = ADD32rr %3, %2, implicit-def dead $eflags 730 DBG_VALUE %4, $noreg, !3, !4 731 DBG_VALUE %7, $noreg, !5, !6 732 733In this circumstance LLVM would leave the MIR as shown above. Were we to move 734the DBG_VALUE of virtual register %7 upwards with the SUB32rr, we would re-order 735assignments and introduce a new state of the program. Whereas with the solution 736above, the debugger will see one fewer combination of variable values, because 737``!3`` and ``!5`` will change value at the same time. This is preferred over 738misrepresenting the original program. 739 740In comparison, if one sunk the MOV32rm, LLVM would produce the following: 741 742.. code-block:: text 743 744 DBG_VALUE $noreg, $noreg, !1, !2 745 %4:gr32 = ADD32rr %3, %2, implicit-def dead $eflags 746 DBG_VALUE %4, $noreg, !3, !4 747 %7:gr32 = SUB32rr %6, %5, implicit-def dead $eflags 748 DBG_VALUE %7, $noreg, !5, !6 749 %1:gr32 = MOV32rm %0, 1, $noreg, 4, $noreg, debug-location !5 :: (load 4 from %ir.addr1) 750 DBG_VALUE %1, $noreg, !1, !2 751 752Here, to avoid presenting a state in which the first assignment to ``!1`` 753disappears, the DBG_VALUE at the top of the block assigns the variable the 754undefined location, until its value is available at the end of the block where 755an additional DBG_VALUE is added. Were any other DBG_VALUE for ``!1`` to occur 756in the instructions that the MOV32rm was sunk past, the DBG_VALUE for ``%1`` 757would be dropped and the debugger would never observe it in the variable. This 758accurately reflects that the value is not available during the corresponding 759portion of the original program. 760 761Variable locations during Register Allocation 762--------------------------------------------- 763 764To avoid debug instructions interfering with the register allocator, the 765LiveDebugVariables pass extracts variable locations from a MIR function and 766deletes the corresponding DBG_VALUE instructions. Some localized copy 767propagation is performed within blocks. After register allocation, the 768VirtRegRewriter pass re-inserts DBG_VALUE instructions in their original 769positions, translating virtual register references into their physical 770machine locations. To avoid encoding incorrect variable locations, in this 771pass any DBG_VALUE of a virtual register that is not live, is replaced by 772the undefined location. 773 774LiveDebugValues expansion of variable locations 775----------------------------------------------- 776 777After all optimizations have run and shortly before emission, the 778LiveDebugValues pass runs to achieve two aims: 779 780* To propagate the location of variables through copies and register spills, 781* For every block, to record every valid variable location in that block. 782 783After this pass the DBG_VALUE instruction changes meaning: rather than 784corresponding to a source-level assignment where the variable may change value, 785it asserts the location of a variable in a block, and loses effect outside the 786block. Propagating variable locations through copies and spills is 787straightforwards: determining the variable location in every basic block 788requires the consideration of control flow. Consider the following IR, which 789presents several difficulties: 790 791.. code-block:: text 792 793 define dso_local i32 @foo(i1 %cond, i32 %input) !dbg !12 { 794 entry: 795 br i1 %cond, label %truebr, label %falsebr 796 797 bb1: 798 %value = phi i32 [ %value1, %truebr ], [ %value2, %falsebr ] 799 br label %exit, !dbg !26 800 801 truebr: 802 call void @llvm.dbg.value(metadata i32 %input, metadata !30, metadata !DIExpression()), !dbg !23 803 call void @llvm.dbg.value(metadata i32 1, metadata !22, metadata !DIExpression()), !dbg !23 804 %value1 = add i32 %input, 1 805 br label %bb1 806 807 falsebr: 808 call void @llvm.dbg.value(metadata i32 %input, metadata !30, metadata !DIExpression()), !dbg !23 809 call void @llvm.dbg.value(metadata i32 2, metadata !22, metadata !DIExpression()), !dbg !23 810 %value = add i32 %input, 2 811 br label %bb1 812 813 exit: 814 ret i32 %value, !dbg !30 815 } 816 817Here the difficulties are: 818 819* The control flow is roughly the opposite of basic block order 820* The value of the ``!22`` variable merges into ``%bb1``, but there is no PHI 821 node 822 823As mentioned above, the ``llvm.dbg.value`` intrinsics essentially form an 824imperative program embedded in the IR, with each intrinsic defining a variable 825location. This *could* be converted to an SSA form by mem2reg, in the same way 826that it uses use-def chains to identify control flow merges and insert phi 827nodes for IR Values. However, because debug variable locations are defined for 828every machine instruction, in effect every IR instruction uses every variable 829location, which would lead to a large number of debugging intrinsics being 830generated. 831 832Examining the example above, variable ``!30`` is assigned ``%input`` on both 833conditional paths through the function, while ``!22`` is assigned differing 834constant values on either path. Where control flow merges in ``%bb1`` we would 835want ``!30`` to keep its location (``%input``), but ``!22`` to become undefined 836as we cannot determine at runtime what value it should have in %bb1 without 837inserting a PHI node. mem2reg does not insert the PHI node to avoid changing 838codegen when debugging is enabled, and does not insert the other dbg.values 839to avoid adding very large numbers of intrinsics. 840 841Instead, LiveDebugValues determines variable locations when control 842flow merges. A dataflow analysis is used to propagate locations between blocks: 843when control flow merges, if a variable has the same location in all 844predecessors then that location is propagated into the successor. If the 845predecessor locations disagree, the location becomes undefined. 846 847Once LiveDebugValues has run, every block should have all valid variable 848locations described by DBG_VALUE instructions within the block. Very little 849effort is then required by supporting classes (such as 850DbgEntityHistoryCalculator) to build a map of each instruction to every 851valid variable location, without the need to consider control flow. From 852the example above, it is otherwise difficult to determine that the location 853of variable ``!30`` should flow "up" into block ``%bb1``, but that the location 854of variable ``!22`` should not flow "down" into the ``%exit`` block. 855 856.. _ccxx_frontend: 857 858C/C++ front-end specific debug information 859========================================== 860 861The C and C++ front-ends represent information about the program in a 862format that is effectively identical to `DWARF <http://www.dwarfstd.org/>`_ 863in terms of information content. This allows code generators to 864trivially support native debuggers by generating standard dwarf 865information, and contains enough information for non-dwarf targets to 866translate it as needed. 867 868This section describes the forms used to represent C and C++ programs. Other 869languages could pattern themselves after this (which itself is tuned to 870representing programs in the same way that DWARF does), or they could choose 871to provide completely different forms if they don't fit into the DWARF model. 872As support for debugging information gets added to the various LLVM 873source-language front-ends, the information used should be documented here. 874 875The following sections provide examples of a few C/C++ constructs and 876the debug information that would best describe those constructs. The 877canonical references are the ``DINode`` classes defined in 878``include/llvm/IR/DebugInfoMetadata.h`` and the implementations of the 879helper functions in ``lib/IR/DIBuilder.cpp``. 880 881C/C++ source file information 882----------------------------- 883 884``llvm::Instruction`` provides easy access to metadata attached with an 885instruction. One can extract line number information encoded in LLVM IR using 886``Instruction::getDebugLoc()`` and ``DILocation::getLine()``. 887 888.. code-block:: c++ 889 890 if (DILocation *Loc = I->getDebugLoc()) { // Here I is an LLVM instruction 891 unsigned Line = Loc->getLine(); 892 StringRef File = Loc->getFilename(); 893 StringRef Dir = Loc->getDirectory(); 894 bool ImplicitCode = Loc->isImplicitCode(); 895 } 896 897When the flag ImplicitCode is true then it means that the Instruction has been 898added by the front-end but doesn't correspond to source code written by the user. For example 899 900.. code-block:: c++ 901 902 if (MyBoolean) { 903 MyObject MO; 904 ... 905 } 906 907At the end of the scope the MyObject's destructor is called but it isn't written 908explicitly. This information is useful to avoid to have counters on brackets when 909making code coverage. 910 911C/C++ global variable information 912--------------------------------- 913 914Given an integer global variable declared as follows: 915 916.. code-block:: c 917 918 _Alignas(8) int MyGlobal = 100; 919 920a C/C++ front-end would generate the following descriptors: 921 922.. code-block:: text 923 924 ;; 925 ;; Define the global itself. 926 ;; 927 @MyGlobal = global i32 100, align 8, !dbg !0 928 929 ;; 930 ;; List of debug info of globals 931 ;; 932 !llvm.dbg.cu = !{!1} 933 934 ;; Some unrelated metadata. 935 !llvm.module.flags = !{!6, !7} 936 !llvm.ident = !{!8} 937 938 ;; Define the global variable itself 939 !0 = distinct !DIGlobalVariable(name: "MyGlobal", scope: !1, file: !2, line: 1, type: !5, isLocal: false, isDefinition: true, align: 64) 940 941 ;; Define the compile unit. 942 !1 = distinct !DICompileUnit(language: DW_LANG_C99, file: !2, 943 producer: "clang version 4.0.0", 944 isOptimized: false, runtimeVersion: 0, emissionKind: FullDebug, 945 enums: !3, globals: !4) 946 947 ;; 948 ;; Define the file 949 ;; 950 !2 = !DIFile(filename: "/dev/stdin", 951 directory: "/Users/dexonsmith/data/llvm/debug-info") 952 953 ;; An empty array. 954 !3 = !{} 955 956 ;; The Array of Global Variables 957 !4 = !{!0} 958 959 ;; 960 ;; Define the type 961 ;; 962 !5 = !DIBasicType(name: "int", size: 32, encoding: DW_ATE_signed) 963 964 ;; Dwarf version to output. 965 !6 = !{i32 2, !"Dwarf Version", i32 4} 966 967 ;; Debug info schema version. 968 !7 = !{i32 2, !"Debug Info Version", i32 3} 969 970 ;; Compiler identification 971 !8 = !{!"clang version 4.0.0"} 972 973 974The align value in DIGlobalVariable description specifies variable alignment in 975case it was forced by C11 _Alignas(), C++11 alignas() keywords or compiler 976attribute __attribute__((aligned ())). In other case (when this field is missing) 977alignment is considered default. This is used when producing DWARF output 978for DW_AT_alignment value. 979 980C/C++ function information 981-------------------------- 982 983Given a function declared as follows: 984 985.. code-block:: c 986 987 int main(int argc, char *argv[]) { 988 return 0; 989 } 990 991a C/C++ front-end would generate the following descriptors: 992 993.. code-block:: text 994 995 ;; 996 ;; Define the anchor for subprograms. 997 ;; 998 !4 = !DISubprogram(name: "main", scope: !1, file: !1, line: 1, type: !5, 999 isLocal: false, isDefinition: true, scopeLine: 1, 1000 flags: DIFlagPrototyped, isOptimized: false, 1001 variables: !2) 1002 1003 ;; 1004 ;; Define the subprogram itself. 1005 ;; 1006 define i32 @main(i32 %argc, i8** %argv) !dbg !4 { 1007 ... 1008 } 1009 1010C++ specific debug information 1011============================== 1012 1013C++ special member functions information 1014---------------------------------------- 1015 1016DWARF v5 introduces attributes defined to enhance debugging information of C++ programs. LLVM can generate (or omit) these appropriate DWARF attributes. In C++ a special member function Ctors, Dtors, Copy/Move Ctors, assignment operators can be declared with C++11 keyword deleted. This is represented in LLVM using spFlags value DISPFlagDeleted. 1017 1018Given a class declaration with copy constructor declared as deleted: 1019 1020.. code-block:: c 1021 1022 class foo { 1023 public: 1024 foo(const foo&) = deleted; 1025 }; 1026 1027A C++ frontend would generate following: 1028 1029.. code-block:: text 1030 1031 !17 = !DISubprogram(name: "foo", scope: !11, file: !1, line: 5, type: !18, scopeLine: 5, flags: DIFlagPublic | DIFlagPrototyped, spFlags: DISPFlagDeleted) 1032 1033and this will produce an additional DWARF attribute as: 1034 1035.. code-block:: text 1036 1037 DW_TAG_subprogram [7] * 1038 DW_AT_name [DW_FORM_strx1] (indexed (00000006) string = "foo") 1039 DW_AT_decl_line [DW_FORM_data1] (5) 1040 ... 1041 DW_AT_deleted [DW_FORM_flag_present] (true) 1042 1043Fortran specific debug information 1044================================== 1045 1046Fortran function information 1047---------------------------- 1048 1049There are a few DWARF attributes defined to support client debugging of Fortran programs. LLVM can generate (or omit) the appropriate DWARF attributes for the prefix-specs of ELEMENTAL, PURE, IMPURE, RECURSIVE, and NON_RECURSIVE. This is done by using the spFlags values: DISPFlagElemental, DISPFlagPure, and DISPFlagRecursive. 1050 1051.. code-block:: fortran 1052 1053 elemental function elem_func(a) 1054 1055a Fortran front-end would generate the following descriptors: 1056 1057.. code-block:: text 1058 1059 !11 = distinct !DISubprogram(name: "subroutine2", scope: !1, file: !1, 1060 line: 5, type: !8, scopeLine: 6, 1061 spFlags: DISPFlagDefinition | DISPFlagElemental, unit: !0, 1062 retainedNodes: !2) 1063 1064and this will materialize an additional DWARF attribute as: 1065 1066.. code-block:: text 1067 1068 DW_TAG_subprogram [3] 1069 DW_AT_low_pc [DW_FORM_addr] (0x0000000000000010 ".text") 1070 DW_AT_high_pc [DW_FORM_data4] (0x00000001) 1071 ... 1072 DW_AT_elemental [DW_FORM_flag_present] (true) 1073 1074There are a few DWARF tags defined to represent Fortran specific constructs i.e DW_TAG_string_type for representing Fortran character(n). In LLVM this is represented as DIStringType. 1075 1076.. code-block:: fortran 1077 1078 character(len=*), intent(in) :: string 1079 1080a Fortran front-end would generate the following descriptors: 1081 1082.. code-block:: text 1083 1084 !DILocalVariable(name: "string", arg: 1, scope: !10, file: !3, line: 4, type: !15) 1085 !DIStringType(name: "character(*)!2", stringLength: !16, stringLengthExpression: !DIExpression(), size: 32) 1086 1087and this will materialize in DWARF tags as: 1088 1089.. code-block:: text 1090 1091 DW_TAG_string_type 1092 DW_AT_name ("character(*)!2") 1093 DW_AT_string_length (0x00000064) 1094 0x00000064: DW_TAG_variable 1095 DW_AT_location (DW_OP_fbreg +16) 1096 DW_AT_type (0x00000083 "integer*8") 1097 ... 1098 DW_AT_artificial (true) 1099 1100Debugging information format 1101============================ 1102 1103Debugging Information Extension for Objective C Properties 1104---------------------------------------------------------- 1105 1106Introduction 1107^^^^^^^^^^^^ 1108 1109Objective C provides a simpler way to declare and define accessor methods using 1110declared properties. The language provides features to declare a property and 1111to let compiler synthesize accessor methods. 1112 1113The debugger lets developer inspect Objective C interfaces and their instance 1114variables and class variables. However, the debugger does not know anything 1115about the properties defined in Objective C interfaces. The debugger consumes 1116information generated by compiler in DWARF format. The format does not support 1117encoding of Objective C properties. This proposal describes DWARF extensions to 1118encode Objective C properties, which the debugger can use to let developers 1119inspect Objective C properties. 1120 1121Proposal 1122^^^^^^^^ 1123 1124Objective C properties exist separately from class members. A property can be 1125defined only by "setter" and "getter" selectors, and be calculated anew on each 1126access. Or a property can just be a direct access to some declared ivar. 1127Finally it can have an ivar "automatically synthesized" for it by the compiler, 1128in which case the property can be referred to in user code directly using the 1129standard C dereference syntax as well as through the property "dot" syntax, but 1130there is no entry in the ``@interface`` declaration corresponding to this ivar. 1131 1132To facilitate debugging, these properties we will add a new DWARF TAG into the 1133``DW_TAG_structure_type`` definition for the class to hold the description of a 1134given property, and a set of DWARF attributes that provide said description. 1135The property tag will also contain the name and declared type of the property. 1136 1137If there is a related ivar, there will also be a DWARF property attribute placed 1138in the ``DW_TAG_member`` DIE for that ivar referring back to the property TAG 1139for that property. And in the case where the compiler synthesizes the ivar 1140directly, the compiler is expected to generate a ``DW_TAG_member`` for that 1141ivar (with the ``DW_AT_artificial`` set to 1), whose name will be the name used 1142to access this ivar directly in code, and with the property attribute pointing 1143back to the property it is backing. 1144 1145The following examples will serve as illustration for our discussion: 1146 1147.. code-block:: objc 1148 1149 @interface I1 { 1150 int n2; 1151 } 1152 1153 @property int p1; 1154 @property int p2; 1155 @end 1156 1157 @implementation I1 1158 @synthesize p1; 1159 @synthesize p2 = n2; 1160 @end 1161 1162This produces the following DWARF (this is a "pseudo dwarfdump" output): 1163 1164.. code-block:: none 1165 1166 0x00000100: TAG_structure_type [7] * 1167 AT_APPLE_runtime_class( 0x10 ) 1168 AT_name( "I1" ) 1169 AT_decl_file( "Objc_Property.m" ) 1170 AT_decl_line( 3 ) 1171 1172 0x00000110 TAG_APPLE_property 1173 AT_name ( "p1" ) 1174 AT_type ( {0x00000150} ( int ) ) 1175 1176 0x00000120: TAG_APPLE_property 1177 AT_name ( "p2" ) 1178 AT_type ( {0x00000150} ( int ) ) 1179 1180 0x00000130: TAG_member [8] 1181 AT_name( "_p1" ) 1182 AT_APPLE_property ( {0x00000110} "p1" ) 1183 AT_type( {0x00000150} ( int ) ) 1184 AT_artificial ( 0x1 ) 1185 1186 0x00000140: TAG_member [8] 1187 AT_name( "n2" ) 1188 AT_APPLE_property ( {0x00000120} "p2" ) 1189 AT_type( {0x00000150} ( int ) ) 1190 1191 0x00000150: AT_type( ( int ) ) 1192 1193Note, the current convention is that the name of the ivar for an 1194auto-synthesized property is the name of the property from which it derives 1195with an underscore prepended, as is shown in the example. But we actually 1196don't need to know this convention, since we are given the name of the ivar 1197directly. 1198 1199Also, it is common practice in ObjC to have different property declarations in 1200the @interface and @implementation - e.g. to provide a read-only property in 1201the interface, and a read-write interface in the implementation. In that case, 1202the compiler should emit whichever property declaration will be in force in the 1203current translation unit. 1204 1205Developers can decorate a property with attributes which are encoded using 1206``DW_AT_APPLE_property_attribute``. 1207 1208.. code-block:: objc 1209 1210 @property (readonly, nonatomic) int pr; 1211 1212.. code-block:: none 1213 1214 TAG_APPLE_property [8] 1215 AT_name( "pr" ) 1216 AT_type ( {0x00000147} (int) ) 1217 AT_APPLE_property_attribute (DW_APPLE_PROPERTY_readonly, DW_APPLE_PROPERTY_nonatomic) 1218 1219The setter and getter method names are attached to the property using 1220``DW_AT_APPLE_property_setter`` and ``DW_AT_APPLE_property_getter`` attributes. 1221 1222.. code-block:: objc 1223 1224 @interface I1 1225 @property (setter=myOwnP3Setter:) int p3; 1226 -(void)myOwnP3Setter:(int)a; 1227 @end 1228 1229 @implementation I1 1230 @synthesize p3; 1231 -(void)myOwnP3Setter:(int)a{ } 1232 @end 1233 1234The DWARF for this would be: 1235 1236.. code-block:: none 1237 1238 0x000003bd: TAG_structure_type [7] * 1239 AT_APPLE_runtime_class( 0x10 ) 1240 AT_name( "I1" ) 1241 AT_decl_file( "Objc_Property.m" ) 1242 AT_decl_line( 3 ) 1243 1244 0x000003cd TAG_APPLE_property 1245 AT_name ( "p3" ) 1246 AT_APPLE_property_setter ( "myOwnP3Setter:" ) 1247 AT_type( {0x00000147} ( int ) ) 1248 1249 0x000003f3: TAG_member [8] 1250 AT_name( "_p3" ) 1251 AT_type ( {0x00000147} ( int ) ) 1252 AT_APPLE_property ( {0x000003cd} ) 1253 AT_artificial ( 0x1 ) 1254 1255New DWARF Tags 1256^^^^^^^^^^^^^^ 1257 1258+-----------------------+--------+ 1259| TAG | Value | 1260+=======================+========+ 1261| DW_TAG_APPLE_property | 0x4200 | 1262+-----------------------+--------+ 1263 1264New DWARF Attributes 1265^^^^^^^^^^^^^^^^^^^^ 1266 1267+--------------------------------+--------+-----------+ 1268| Attribute | Value | Classes | 1269+================================+========+===========+ 1270| DW_AT_APPLE_property | 0x3fed | Reference | 1271+--------------------------------+--------+-----------+ 1272| DW_AT_APPLE_property_getter | 0x3fe9 | String | 1273+--------------------------------+--------+-----------+ 1274| DW_AT_APPLE_property_setter | 0x3fea | String | 1275+--------------------------------+--------+-----------+ 1276| DW_AT_APPLE_property_attribute | 0x3feb | Constant | 1277+--------------------------------+--------+-----------+ 1278 1279New DWARF Constants 1280^^^^^^^^^^^^^^^^^^^ 1281 1282+--------------------------------------+-------+ 1283| Name | Value | 1284+======================================+=======+ 1285| DW_APPLE_PROPERTY_readonly | 0x01 | 1286+--------------------------------------+-------+ 1287| DW_APPLE_PROPERTY_getter | 0x02 | 1288+--------------------------------------+-------+ 1289| DW_APPLE_PROPERTY_assign | 0x04 | 1290+--------------------------------------+-------+ 1291| DW_APPLE_PROPERTY_readwrite | 0x08 | 1292+--------------------------------------+-------+ 1293| DW_APPLE_PROPERTY_retain | 0x10 | 1294+--------------------------------------+-------+ 1295| DW_APPLE_PROPERTY_copy | 0x20 | 1296+--------------------------------------+-------+ 1297| DW_APPLE_PROPERTY_nonatomic | 0x40 | 1298+--------------------------------------+-------+ 1299| DW_APPLE_PROPERTY_setter | 0x80 | 1300+--------------------------------------+-------+ 1301| DW_APPLE_PROPERTY_atomic | 0x100 | 1302+--------------------------------------+-------+ 1303| DW_APPLE_PROPERTY_weak | 0x200 | 1304+--------------------------------------+-------+ 1305| DW_APPLE_PROPERTY_strong | 0x400 | 1306+--------------------------------------+-------+ 1307| DW_APPLE_PROPERTY_unsafe_unretained | 0x800 | 1308+--------------------------------------+-------+ 1309| DW_APPLE_PROPERTY_nullability | 0x1000| 1310+--------------------------------------+-------+ 1311| DW_APPLE_PROPERTY_null_resettable | 0x2000| 1312+--------------------------------------+-------+ 1313| DW_APPLE_PROPERTY_class | 0x4000| 1314+--------------------------------------+-------+ 1315 1316Name Accelerator Tables 1317----------------------- 1318 1319Introduction 1320^^^^^^^^^^^^ 1321 1322The "``.debug_pubnames``" and "``.debug_pubtypes``" formats are not what a 1323debugger needs. The "``pub``" in the section name indicates that the entries 1324in the table are publicly visible names only. This means no static or hidden 1325functions show up in the "``.debug_pubnames``". No static variables or private 1326class variables are in the "``.debug_pubtypes``". Many compilers add different 1327things to these tables, so we can't rely upon the contents between gcc, icc, or 1328clang. 1329 1330The typical query given by users tends not to match up with the contents of 1331these tables. For example, the DWARF spec states that "In the case of the name 1332of a function member or static data member of a C++ structure, class or union, 1333the name presented in the "``.debug_pubnames``" section is not the simple name 1334given by the ``DW_AT_name attribute`` of the referenced debugging information 1335entry, but rather the fully qualified name of the data or function member." 1336So the only names in these tables for complex C++ entries is a fully 1337qualified name. Debugger users tend not to enter their search strings as 1338"``a::b::c(int,const Foo&) const``", but rather as "``c``", "``b::c``" , or 1339"``a::b::c``". So the name entered in the name table must be demangled in 1340order to chop it up appropriately and additional names must be manually entered 1341into the table to make it effective as a name lookup table for debuggers to 1342use. 1343 1344All debuggers currently ignore the "``.debug_pubnames``" table as a result of 1345its inconsistent and useless public-only name content making it a waste of 1346space in the object file. These tables, when they are written to disk, are not 1347sorted in any way, leaving every debugger to do its own parsing and sorting. 1348These tables also include an inlined copy of the string values in the table 1349itself making the tables much larger than they need to be on disk, especially 1350for large C++ programs. 1351 1352Can't we just fix the sections by adding all of the names we need to this 1353table? No, because that is not what the tables are defined to contain and we 1354won't know the difference between the old bad tables and the new good tables. 1355At best we could make our own renamed sections that contain all of the data we 1356need. 1357 1358These tables are also insufficient for what a debugger like LLDB needs. LLDB 1359uses clang for its expression parsing where LLDB acts as a PCH. LLDB is then 1360often asked to look for type "``foo``" or namespace "``bar``", or list items in 1361namespace "``baz``". Namespaces are not included in the pubnames or pubtypes 1362tables. Since clang asks a lot of questions when it is parsing an expression, 1363we need to be very fast when looking up names, as it happens a lot. Having new 1364accelerator tables that are optimized for very quick lookups will benefit this 1365type of debugging experience greatly. 1366 1367We would like to generate name lookup tables that can be mapped into memory 1368from disk, and used as is, with little or no up-front parsing. We would also 1369be able to control the exact content of these different tables so they contain 1370exactly what we need. The Name Accelerator Tables were designed to fix these 1371issues. In order to solve these issues we need to: 1372 1373* Have a format that can be mapped into memory from disk and used as is 1374* Lookups should be very fast 1375* Extensible table format so these tables can be made by many producers 1376* Contain all of the names needed for typical lookups out of the box 1377* Strict rules for the contents of tables 1378 1379Table size is important and the accelerator table format should allow the reuse 1380of strings from common string tables so the strings for the names are not 1381duplicated. We also want to make sure the table is ready to be used as-is by 1382simply mapping the table into memory with minimal header parsing. 1383 1384The name lookups need to be fast and optimized for the kinds of lookups that 1385debuggers tend to do. Optimally we would like to touch as few parts of the 1386mapped table as possible when doing a name lookup and be able to quickly find 1387the name entry we are looking for, or discover there are no matches. In the 1388case of debuggers we optimized for lookups that fail most of the time. 1389 1390Each table that is defined should have strict rules on exactly what is in the 1391accelerator tables and documented so clients can rely on the content. 1392 1393Hash Tables 1394^^^^^^^^^^^ 1395 1396Standard Hash Tables 1397"""""""""""""""""""" 1398 1399Typical hash tables have a header, buckets, and each bucket points to the 1400bucket contents: 1401 1402.. code-block:: none 1403 1404 .------------. 1405 | HEADER | 1406 |------------| 1407 | BUCKETS | 1408 |------------| 1409 | DATA | 1410 `------------' 1411 1412The BUCKETS are an array of offsets to DATA for each hash: 1413 1414.. code-block:: none 1415 1416 .------------. 1417 | 0x00001000 | BUCKETS[0] 1418 | 0x00002000 | BUCKETS[1] 1419 | 0x00002200 | BUCKETS[2] 1420 | 0x000034f0 | BUCKETS[3] 1421 | | ... 1422 | 0xXXXXXXXX | BUCKETS[n_buckets] 1423 '------------' 1424 1425So for ``bucket[3]`` in the example above, we have an offset into the table 14260x000034f0 which points to a chain of entries for the bucket. Each bucket must 1427contain a next pointer, full 32 bit hash value, the string itself, and the data 1428for the current string value. 1429 1430.. code-block:: none 1431 1432 .------------. 1433 0x000034f0: | 0x00003500 | next pointer 1434 | 0x12345678 | 32 bit hash 1435 | "erase" | string value 1436 | data[n] | HashData for this bucket 1437 |------------| 1438 0x00003500: | 0x00003550 | next pointer 1439 | 0x29273623 | 32 bit hash 1440 | "dump" | string value 1441 | data[n] | HashData for this bucket 1442 |------------| 1443 0x00003550: | 0x00000000 | next pointer 1444 | 0x82638293 | 32 bit hash 1445 | "main" | string value 1446 | data[n] | HashData for this bucket 1447 `------------' 1448 1449The problem with this layout for debuggers is that we need to optimize for the 1450negative lookup case where the symbol we're searching for is not present. So 1451if we were to lookup "``printf``" in the table above, we would make a 32-bit 1452hash for "``printf``", it might match ``bucket[3]``. We would need to go to 1453the offset 0x000034f0 and start looking to see if our 32 bit hash matches. To 1454do so, we need to read the next pointer, then read the hash, compare it, and 1455skip to the next bucket. Each time we are skipping many bytes in memory and 1456touching new pages just to do the compare on the full 32 bit hash. All of 1457these accesses then tell us that we didn't have a match. 1458 1459Name Hash Tables 1460"""""""""""""""" 1461 1462To solve the issues mentioned above we have structured the hash tables a bit 1463differently: a header, buckets, an array of all unique 32 bit hash values, 1464followed by an array of hash value data offsets, one for each hash value, then 1465the data for all hash values: 1466 1467.. code-block:: none 1468 1469 .-------------. 1470 | HEADER | 1471 |-------------| 1472 | BUCKETS | 1473 |-------------| 1474 | HASHES | 1475 |-------------| 1476 | OFFSETS | 1477 |-------------| 1478 | DATA | 1479 `-------------' 1480 1481The ``BUCKETS`` in the name tables are an index into the ``HASHES`` array. By 1482making all of the full 32 bit hash values contiguous in memory, we allow 1483ourselves to efficiently check for a match while touching as little memory as 1484possible. Most often checking the 32 bit hash values is as far as the lookup 1485goes. If it does match, it usually is a match with no collisions. So for a 1486table with "``n_buckets``" buckets, and "``n_hashes``" unique 32 bit hash 1487values, we can clarify the contents of the ``BUCKETS``, ``HASHES`` and 1488``OFFSETS`` as: 1489 1490.. code-block:: none 1491 1492 .-------------------------. 1493 | HEADER.magic | uint32_t 1494 | HEADER.version | uint16_t 1495 | HEADER.hash_function | uint16_t 1496 | HEADER.bucket_count | uint32_t 1497 | HEADER.hashes_count | uint32_t 1498 | HEADER.header_data_len | uint32_t 1499 | HEADER_DATA | HeaderData 1500 |-------------------------| 1501 | BUCKETS | uint32_t[n_buckets] // 32 bit hash indexes 1502 |-------------------------| 1503 | HASHES | uint32_t[n_hashes] // 32 bit hash values 1504 |-------------------------| 1505 | OFFSETS | uint32_t[n_hashes] // 32 bit offsets to hash value data 1506 |-------------------------| 1507 | ALL HASH DATA | 1508 `-------------------------' 1509 1510So taking the exact same data from the standard hash example above we end up 1511with: 1512 1513.. code-block:: none 1514 1515 .------------. 1516 | HEADER | 1517 |------------| 1518 | 0 | BUCKETS[0] 1519 | 2 | BUCKETS[1] 1520 | 5 | BUCKETS[2] 1521 | 6 | BUCKETS[3] 1522 | | ... 1523 | ... | BUCKETS[n_buckets] 1524 |------------| 1525 | 0x........ | HASHES[0] 1526 | 0x........ | HASHES[1] 1527 | 0x........ | HASHES[2] 1528 | 0x........ | HASHES[3] 1529 | 0x........ | HASHES[4] 1530 | 0x........ | HASHES[5] 1531 | 0x12345678 | HASHES[6] hash for BUCKETS[3] 1532 | 0x29273623 | HASHES[7] hash for BUCKETS[3] 1533 | 0x82638293 | HASHES[8] hash for BUCKETS[3] 1534 | 0x........ | HASHES[9] 1535 | 0x........ | HASHES[10] 1536 | 0x........ | HASHES[11] 1537 | 0x........ | HASHES[12] 1538 | 0x........ | HASHES[13] 1539 | 0x........ | HASHES[n_hashes] 1540 |------------| 1541 | 0x........ | OFFSETS[0] 1542 | 0x........ | OFFSETS[1] 1543 | 0x........ | OFFSETS[2] 1544 | 0x........ | OFFSETS[3] 1545 | 0x........ | OFFSETS[4] 1546 | 0x........ | OFFSETS[5] 1547 | 0x000034f0 | OFFSETS[6] offset for BUCKETS[3] 1548 | 0x00003500 | OFFSETS[7] offset for BUCKETS[3] 1549 | 0x00003550 | OFFSETS[8] offset for BUCKETS[3] 1550 | 0x........ | OFFSETS[9] 1551 | 0x........ | OFFSETS[10] 1552 | 0x........ | OFFSETS[11] 1553 | 0x........ | OFFSETS[12] 1554 | 0x........ | OFFSETS[13] 1555 | 0x........ | OFFSETS[n_hashes] 1556 |------------| 1557 | | 1558 | | 1559 | | 1560 | | 1561 | | 1562 |------------| 1563 0x000034f0: | 0x00001203 | .debug_str ("erase") 1564 | 0x00000004 | A 32 bit array count - number of HashData with name "erase" 1565 | 0x........ | HashData[0] 1566 | 0x........ | HashData[1] 1567 | 0x........ | HashData[2] 1568 | 0x........ | HashData[3] 1569 | 0x00000000 | String offset into .debug_str (terminate data for hash) 1570 |------------| 1571 0x00003500: | 0x00001203 | String offset into .debug_str ("collision") 1572 | 0x00000002 | A 32 bit array count - number of HashData with name "collision" 1573 | 0x........ | HashData[0] 1574 | 0x........ | HashData[1] 1575 | 0x00001203 | String offset into .debug_str ("dump") 1576 | 0x00000003 | A 32 bit array count - number of HashData with name "dump" 1577 | 0x........ | HashData[0] 1578 | 0x........ | HashData[1] 1579 | 0x........ | HashData[2] 1580 | 0x00000000 | String offset into .debug_str (terminate data for hash) 1581 |------------| 1582 0x00003550: | 0x00001203 | String offset into .debug_str ("main") 1583 | 0x00000009 | A 32 bit array count - number of HashData with name "main" 1584 | 0x........ | HashData[0] 1585 | 0x........ | HashData[1] 1586 | 0x........ | HashData[2] 1587 | 0x........ | HashData[3] 1588 | 0x........ | HashData[4] 1589 | 0x........ | HashData[5] 1590 | 0x........ | HashData[6] 1591 | 0x........ | HashData[7] 1592 | 0x........ | HashData[8] 1593 | 0x00000000 | String offset into .debug_str (terminate data for hash) 1594 `------------' 1595 1596So we still have all of the same data, we just organize it more efficiently for 1597debugger lookup. If we repeat the same "``printf``" lookup from above, we 1598would hash "``printf``" and find it matches ``BUCKETS[3]`` by taking the 32 bit 1599hash value and modulo it by ``n_buckets``. ``BUCKETS[3]`` contains "6" which 1600is the index into the ``HASHES`` table. We would then compare any consecutive 160132 bit hashes values in the ``HASHES`` array as long as the hashes would be in 1602``BUCKETS[3]``. We do this by verifying that each subsequent hash value modulo 1603``n_buckets`` is still 3. In the case of a failed lookup we would access the 1604memory for ``BUCKETS[3]``, and then compare a few consecutive 32 bit hashes 1605before we know that we have no match. We don't end up marching through 1606multiple words of memory and we really keep the number of processor data cache 1607lines being accessed as small as possible. 1608 1609The string hash that is used for these lookup tables is the Daniel J. 1610Bernstein hash which is also used in the ELF ``GNU_HASH`` sections. It is a 1611very good hash for all kinds of names in programs with very few hash 1612collisions. 1613 1614Empty buckets are designated by using an invalid hash index of ``UINT32_MAX``. 1615 1616Details 1617^^^^^^^ 1618 1619These name hash tables are designed to be generic where specializations of the 1620table get to define additional data that goes into the header ("``HeaderData``"), 1621how the string value is stored ("``KeyType``") and the content of the data for each 1622hash value. 1623 1624Header Layout 1625""""""""""""" 1626 1627The header has a fixed part, and the specialized part. The exact format of the 1628header is: 1629 1630.. code-block:: c 1631 1632 struct Header 1633 { 1634 uint32_t magic; // 'HASH' magic value to allow endian detection 1635 uint16_t version; // Version number 1636 uint16_t hash_function; // The hash function enumeration that was used 1637 uint32_t bucket_count; // The number of buckets in this hash table 1638 uint32_t hashes_count; // The total number of unique hash values and hash data offsets in this table 1639 uint32_t header_data_len; // The bytes to skip to get to the hash indexes (buckets) for correct alignment 1640 // Specifically the length of the following HeaderData field - this does not 1641 // include the size of the preceding fields 1642 HeaderData header_data; // Implementation specific header data 1643 }; 1644 1645The header starts with a 32 bit "``magic``" value which must be ``'HASH'`` 1646encoded as an ASCII integer. This allows the detection of the start of the 1647hash table and also allows the table's byte order to be determined so the table 1648can be correctly extracted. The "``magic``" value is followed by a 16 bit 1649``version`` number which allows the table to be revised and modified in the 1650future. The current version number is 1. ``hash_function`` is a ``uint16_t`` 1651enumeration that specifies which hash function was used to produce this table. 1652The current values for the hash function enumerations include: 1653 1654.. code-block:: c 1655 1656 enum HashFunctionType 1657 { 1658 eHashFunctionDJB = 0u, // Daniel J Bernstein hash function 1659 }; 1660 1661``bucket_count`` is a 32 bit unsigned integer that represents how many buckets 1662are in the ``BUCKETS`` array. ``hashes_count`` is the number of unique 32 bit 1663hash values that are in the ``HASHES`` array, and is the same number of offsets 1664are contained in the ``OFFSETS`` array. ``header_data_len`` specifies the size 1665in bytes of the ``HeaderData`` that is filled in by specialized versions of 1666this table. 1667 1668Fixed Lookup 1669"""""""""""" 1670 1671The header is followed by the buckets, hashes, offsets, and hash value data. 1672 1673.. code-block:: c 1674 1675 struct FixedTable 1676 { 1677 uint32_t buckets[Header.bucket_count]; // An array of hash indexes into the "hashes[]" array below 1678 uint32_t hashes [Header.hashes_count]; // Every unique 32 bit hash for the entire table is in this table 1679 uint32_t offsets[Header.hashes_count]; // An offset that corresponds to each item in the "hashes[]" array above 1680 }; 1681 1682``buckets`` is an array of 32 bit indexes into the ``hashes`` array. The 1683``hashes`` array contains all of the 32 bit hash values for all names in the 1684hash table. Each hash in the ``hashes`` table has an offset in the ``offsets`` 1685array that points to the data for the hash value. 1686 1687This table setup makes it very easy to repurpose these tables to contain 1688different data, while keeping the lookup mechanism the same for all tables. 1689This layout also makes it possible to save the table to disk and map it in 1690later and do very efficient name lookups with little or no parsing. 1691 1692DWARF lookup tables can be implemented in a variety of ways and can store a lot 1693of information for each name. We want to make the DWARF tables extensible and 1694able to store the data efficiently so we have used some of the DWARF features 1695that enable efficient data storage to define exactly what kind of data we store 1696for each name. 1697 1698The ``HeaderData`` contains a definition of the contents of each HashData chunk. 1699We might want to store an offset to all of the debug information entries (DIEs) 1700for each name. To keep things extensible, we create a list of items, or 1701Atoms, that are contained in the data for each name. First comes the type of 1702the data in each atom: 1703 1704.. code-block:: c 1705 1706 enum AtomType 1707 { 1708 eAtomTypeNULL = 0u, 1709 eAtomTypeDIEOffset = 1u, // DIE offset, check form for encoding 1710 eAtomTypeCUOffset = 2u, // DIE offset of the compiler unit header that contains the item in question 1711 eAtomTypeTag = 3u, // DW_TAG_xxx value, should be encoded as DW_FORM_data1 (if no tags exceed 255) or DW_FORM_data2 1712 eAtomTypeNameFlags = 4u, // Flags from enum NameFlags 1713 eAtomTypeTypeFlags = 5u, // Flags from enum TypeFlags 1714 }; 1715 1716The enumeration values and their meanings are: 1717 1718.. code-block:: none 1719 1720 eAtomTypeNULL - a termination atom that specifies the end of the atom list 1721 eAtomTypeDIEOffset - an offset into the .debug_info section for the DWARF DIE for this name 1722 eAtomTypeCUOffset - an offset into the .debug_info section for the CU that contains the DIE 1723 eAtomTypeDIETag - The DW_TAG_XXX enumeration value so you don't have to parse the DWARF to see what it is 1724 eAtomTypeNameFlags - Flags for functions and global variables (isFunction, isInlined, isExternal...) 1725 eAtomTypeTypeFlags - Flags for types (isCXXClass, isObjCClass, ...) 1726 1727Then we allow each atom type to define the atom type and how the data for each 1728atom type data is encoded: 1729 1730.. code-block:: c 1731 1732 struct Atom 1733 { 1734 uint16_t type; // AtomType enum value 1735 uint16_t form; // DWARF DW_FORM_XXX defines 1736 }; 1737 1738The ``form`` type above is from the DWARF specification and defines the exact 1739encoding of the data for the Atom type. See the DWARF specification for the 1740``DW_FORM_`` definitions. 1741 1742.. code-block:: c 1743 1744 struct HeaderData 1745 { 1746 uint32_t die_offset_base; 1747 uint32_t atom_count; 1748 Atoms atoms[atom_count0]; 1749 }; 1750 1751``HeaderData`` defines the base DIE offset that should be added to any atoms 1752that are encoded using the ``DW_FORM_ref1``, ``DW_FORM_ref2``, 1753``DW_FORM_ref4``, ``DW_FORM_ref8`` or ``DW_FORM_ref_udata``. It also defines 1754what is contained in each ``HashData`` object -- ``Atom.form`` tells us how large 1755each field will be in the ``HashData`` and the ``Atom.type`` tells us how this data 1756should be interpreted. 1757 1758For the current implementations of the "``.apple_names``" (all functions + 1759globals), the "``.apple_types``" (names of all types that are defined), and 1760the "``.apple_namespaces``" (all namespaces), we currently set the ``Atom`` 1761array to be: 1762 1763.. code-block:: c 1764 1765 HeaderData.atom_count = 1; 1766 HeaderData.atoms[0].type = eAtomTypeDIEOffset; 1767 HeaderData.atoms[0].form = DW_FORM_data4; 1768 1769This defines the contents to be the DIE offset (eAtomTypeDIEOffset) that is 1770encoded as a 32 bit value (DW_FORM_data4). This allows a single name to have 1771multiple matching DIEs in a single file, which could come up with an inlined 1772function for instance. Future tables could include more information about the 1773DIE such as flags indicating if the DIE is a function, method, block, 1774or inlined. 1775 1776The KeyType for the DWARF table is a 32 bit string table offset into the 1777".debug_str" table. The ".debug_str" is the string table for the DWARF which 1778may already contain copies of all of the strings. This helps make sure, with 1779help from the compiler, that we reuse the strings between all of the DWARF 1780sections and keeps the hash table size down. Another benefit to having the 1781compiler generate all strings as DW_FORM_strp in the debug info, is that 1782DWARF parsing can be made much faster. 1783 1784After a lookup is made, we get an offset into the hash data. The hash data 1785needs to be able to deal with 32 bit hash collisions, so the chunk of data 1786at the offset in the hash data consists of a triple: 1787 1788.. code-block:: c 1789 1790 uint32_t str_offset 1791 uint32_t hash_data_count 1792 HashData[hash_data_count] 1793 1794If "str_offset" is zero, then the bucket contents are done. 99.9% of the 1795hash data chunks contain a single item (no 32 bit hash collision): 1796 1797.. code-block:: none 1798 1799 .------------. 1800 | 0x00001023 | uint32_t KeyType (.debug_str[0x0001023] => "main") 1801 | 0x00000004 | uint32_t HashData count 1802 | 0x........ | uint32_t HashData[0] DIE offset 1803 | 0x........ | uint32_t HashData[1] DIE offset 1804 | 0x........ | uint32_t HashData[2] DIE offset 1805 | 0x........ | uint32_t HashData[3] DIE offset 1806 | 0x00000000 | uint32_t KeyType (end of hash chain) 1807 `------------' 1808 1809If there are collisions, you will have multiple valid string offsets: 1810 1811.. code-block:: none 1812 1813 .------------. 1814 | 0x00001023 | uint32_t KeyType (.debug_str[0x0001023] => "main") 1815 | 0x00000004 | uint32_t HashData count 1816 | 0x........ | uint32_t HashData[0] DIE offset 1817 | 0x........ | uint32_t HashData[1] DIE offset 1818 | 0x........ | uint32_t HashData[2] DIE offset 1819 | 0x........ | uint32_t HashData[3] DIE offset 1820 | 0x00002023 | uint32_t KeyType (.debug_str[0x0002023] => "print") 1821 | 0x00000002 | uint32_t HashData count 1822 | 0x........ | uint32_t HashData[0] DIE offset 1823 | 0x........ | uint32_t HashData[1] DIE offset 1824 | 0x00000000 | uint32_t KeyType (end of hash chain) 1825 `------------' 1826 1827Current testing with real world C++ binaries has shown that there is around 1 182832 bit hash collision per 100,000 name entries. 1829 1830Contents 1831^^^^^^^^ 1832 1833As we said, we want to strictly define exactly what is included in the 1834different tables. For DWARF, we have 3 tables: "``.apple_names``", 1835"``.apple_types``", and "``.apple_namespaces``". 1836 1837"``.apple_names``" sections should contain an entry for each DWARF DIE whose 1838``DW_TAG`` is a ``DW_TAG_label``, ``DW_TAG_inlined_subroutine``, or 1839``DW_TAG_subprogram`` that has address attributes: ``DW_AT_low_pc``, 1840``DW_AT_high_pc``, ``DW_AT_ranges`` or ``DW_AT_entry_pc``. It also contains 1841``DW_TAG_variable`` DIEs that have a ``DW_OP_addr`` in the location (global and 1842static variables). All global and static variables should be included, 1843including those scoped within functions and classes. For example using the 1844following code: 1845 1846.. code-block:: c 1847 1848 static int var = 0; 1849 1850 void f () 1851 { 1852 static int var = 0; 1853 } 1854 1855Both of the static ``var`` variables would be included in the table. All 1856functions should emit both their full names and their basenames. For C or C++, 1857the full name is the mangled name (if available) which is usually in the 1858``DW_AT_MIPS_linkage_name`` attribute, and the ``DW_AT_name`` contains the 1859function basename. If global or static variables have a mangled name in a 1860``DW_AT_MIPS_linkage_name`` attribute, this should be emitted along with the 1861simple name found in the ``DW_AT_name`` attribute. 1862 1863"``.apple_types``" sections should contain an entry for each DWARF DIE whose 1864tag is one of: 1865 1866* DW_TAG_array_type 1867* DW_TAG_class_type 1868* DW_TAG_enumeration_type 1869* DW_TAG_pointer_type 1870* DW_TAG_reference_type 1871* DW_TAG_string_type 1872* DW_TAG_structure_type 1873* DW_TAG_subroutine_type 1874* DW_TAG_typedef 1875* DW_TAG_union_type 1876* DW_TAG_ptr_to_member_type 1877* DW_TAG_set_type 1878* DW_TAG_subrange_type 1879* DW_TAG_base_type 1880* DW_TAG_const_type 1881* DW_TAG_file_type 1882* DW_TAG_namelist 1883* DW_TAG_packed_type 1884* DW_TAG_volatile_type 1885* DW_TAG_restrict_type 1886* DW_TAG_atomic_type 1887* DW_TAG_interface_type 1888* DW_TAG_unspecified_type 1889* DW_TAG_shared_type 1890 1891Only entries with a ``DW_AT_name`` attribute are included, and the entry must 1892not be a forward declaration (``DW_AT_declaration`` attribute with a non-zero 1893value). For example, using the following code: 1894 1895.. code-block:: c 1896 1897 int main () 1898 { 1899 int *b = 0; 1900 return *b; 1901 } 1902 1903We get a few type DIEs: 1904 1905.. code-block:: none 1906 1907 0x00000067: TAG_base_type [5] 1908 AT_encoding( DW_ATE_signed ) 1909 AT_name( "int" ) 1910 AT_byte_size( 0x04 ) 1911 1912 0x0000006e: TAG_pointer_type [6] 1913 AT_type( {0x00000067} ( int ) ) 1914 AT_byte_size( 0x08 ) 1915 1916The DW_TAG_pointer_type is not included because it does not have a ``DW_AT_name``. 1917 1918"``.apple_namespaces``" section should contain all ``DW_TAG_namespace`` DIEs. 1919If we run into a namespace that has no name this is an anonymous namespace, and 1920the name should be output as "``(anonymous namespace)``" (without the quotes). 1921Why? This matches the output of the ``abi::cxa_demangle()`` that is in the 1922standard C++ library that demangles mangled names. 1923 1924 1925Language Extensions and File Format Changes 1926^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 1927 1928Objective-C Extensions 1929"""""""""""""""""""""" 1930 1931"``.apple_objc``" section should contain all ``DW_TAG_subprogram`` DIEs for an 1932Objective-C class. The name used in the hash table is the name of the 1933Objective-C class itself. If the Objective-C class has a category, then an 1934entry is made for both the class name without the category, and for the class 1935name with the category. So if we have a DIE at offset 0x1234 with a name of 1936method "``-[NSString(my_additions) stringWithSpecialString:]``", we would add 1937an entry for "``NSString``" that points to DIE 0x1234, and an entry for 1938"``NSString(my_additions)``" that points to 0x1234. This allows us to quickly 1939track down all Objective-C methods for an Objective-C class when doing 1940expressions. It is needed because of the dynamic nature of Objective-C where 1941anyone can add methods to a class. The DWARF for Objective-C methods is also 1942emitted differently from C++ classes where the methods are not usually 1943contained in the class definition, they are scattered about across one or more 1944compile units. Categories can also be defined in different shared libraries. 1945So we need to be able to quickly find all of the methods and class functions 1946given the Objective-C class name, or quickly find all methods and class 1947functions for a class + category name. This table does not contain any 1948selector names, it just maps Objective-C class names (or class names + 1949category) to all of the methods and class functions. The selectors are added 1950as function basenames in the "``.debug_names``" section. 1951 1952In the "``.apple_names``" section for Objective-C functions, the full name is 1953the entire function name with the brackets ("``-[NSString 1954stringWithCString:]``") and the basename is the selector only 1955("``stringWithCString:``"). 1956 1957Mach-O Changes 1958"""""""""""""" 1959 1960The sections names for the apple hash tables are for non-mach-o files. For 1961mach-o files, the sections should be contained in the ``__DWARF`` segment with 1962names as follows: 1963 1964* "``.apple_names``" -> "``__apple_names``" 1965* "``.apple_types``" -> "``__apple_types``" 1966* "``.apple_namespaces``" -> "``__apple_namespac``" (16 character limit) 1967* "``.apple_objc``" -> "``__apple_objc``" 1968 1969.. _codeview: 1970 1971CodeView Debug Info Format 1972========================== 1973 1974LLVM supports emitting CodeView, the Microsoft debug info format, and this 1975section describes the design and implementation of that support. 1976 1977Format Background 1978----------------- 1979 1980CodeView as a format is clearly oriented around C++ debugging, and in C++, the 1981majority of debug information tends to be type information. Therefore, the 1982overriding design constraint of CodeView is the separation of type information 1983from other "symbol" information so that type information can be efficiently 1984merged across translation units. Both type information and symbol information is 1985generally stored as a sequence of records, where each record begins with a 198616-bit record size and a 16-bit record kind. 1987 1988Type information is usually stored in the ``.debug$T`` section of the object 1989file. All other debug info, such as line info, string table, symbol info, and 1990inlinee info, is stored in one or more ``.debug$S`` sections. There may only be 1991one ``.debug$T`` section per object file, since all other debug info refers to 1992it. If a PDB (enabled by the ``/Zi`` MSVC option) was used during compilation, 1993the ``.debug$T`` section will contain only an ``LF_TYPESERVER2`` record pointing 1994to the PDB. When using PDBs, symbol information appears to remain in the object 1995file ``.debug$S`` sections. 1996 1997Type records are referred to by their index, which is the number of records in 1998the stream before a given record plus ``0x1000``. Many common basic types, such 1999as the basic integral types and unqualified pointers to them, are represented 2000using type indices less than ``0x1000``. Such basic types are built in to 2001CodeView consumers and do not require type records. 2002 2003Each type record may only contain type indices that are less than its own type 2004index. This ensures that the graph of type stream references is acyclic. While 2005the source-level type graph may contain cycles through pointer types (consider a 2006linked list struct), these cycles are removed from the type stream by always 2007referring to the forward declaration record of user-defined record types. Only 2008"symbol" records in the ``.debug$S`` streams may refer to complete, 2009non-forward-declaration type records. 2010 2011Working with CodeView 2012--------------------- 2013 2014These are instructions for some common tasks for developers working to improve 2015LLVM's CodeView support. Most of them revolve around using the CodeView dumper 2016embedded in ``llvm-readobj``. 2017 2018* Testing MSVC's output:: 2019 2020 $ cl -c -Z7 foo.cpp # Use /Z7 to keep types in the object file 2021 $ llvm-readobj --codeview foo.obj 2022 2023* Getting LLVM IR debug info out of Clang:: 2024 2025 $ clang -g -gcodeview --target=x86_64-windows-msvc foo.cpp -S -emit-llvm 2026 2027 Use this to generate LLVM IR for LLVM test cases. 2028 2029* Generate and dump CodeView from LLVM IR metadata:: 2030 2031 $ llc foo.ll -filetype=obj -o foo.obj 2032 $ llvm-readobj --codeview foo.obj > foo.txt 2033 2034 Use this pattern in lit test cases and FileCheck the output of llvm-readobj 2035 2036Improving LLVM's CodeView support is a process of finding interesting type 2037records, constructing a C++ test case that makes MSVC emit those records, 2038dumping the records, understanding them, and then generating equivalent records 2039in LLVM's backend. 2040