[lldb/DWARF] Add support for type units in dwp files

all that was needed was to teach lldb's DWARF context about the
debug_tu_index section.

[lldb/DWARF] Re-enable basic dwp support

Summary:
This patch removes the bitrotted SymbolFileDWARF(Dwo)Dwp classes, and
replaces them with dwp support implemented directly inside
SymbolFileDWARFDwo,

[lldb/DWARF] Re-enable basic dwp support

Summary:
This patch removes the bitrotted SymbolFileDWARF(Dwo)Dwp classes, and
replaces them with dwp support implemented directly inside
SymbolFileDWARFDwo, in a manner mirroring the implementation in llvm.
This patch does:
- add support for the .debug_cu_index section to our DWARFContext
- adds a llvm::DWARFUnitIndex argument to the DWARFUnit constructors.
This argument is used to look up the offsets of the debug_info and
debug_abbrev contributions in the sections of the dwp file.
- makes sure the creation of the DebugInfo object as well as the initial
discovery of DWARFUnits is thread-safe, as we can now call this
concurrently when doing parallel indexing.

This patch does not:
- use the DWARFUnitIndex to search for other kinds of contributions
(debug_loc, debug_ranges, etc.). This means that units which reference
these sections will not work correctly. These will be handled by
follow-up patches, but even the present level of support is sufficient
to enable basic functionality.
- Make the llvm::DWARFContext thread-safe. Right now, it just avoids this
problem by ensuring everything is initialized ahead of time. However,
this is something we will run into more often as we try to use more of
llvm, and so I plan to start looking into our options here.

Reviewers: JDevlieghere, aprantl, clayborg

Subscribers: mgorny, mgrang, lldb-commits

Tags: #lldb

Differential Revision: https://reviews.llvm.org/D73783

show more ...

[lldb][NFC] Fix all formatting errors in .cpp file headers

Summary:
A *.cpp file header in LLDB (and in LLDB) should like this:
```
//===-- TestUtilities.cpp ----------------------------------------

[lldb][NFC] Fix all formatting errors in .cpp file headers

Summary:
A *.cpp file header in LLDB (and in LLDB) should like this:
```
//===-- TestUtilities.cpp -------------------------------------------------===//
```
However in LLDB most of our source files have arbitrary changes to this format and
these changes are spreading through LLDB as folks usually just use the existing
source files as templates for their new files (most notably the unnecessary
editor language indicator `-*- C++ -*-` is spreading and in every review
someone is pointing out that this is wrong, resulting in people pointing out that this
is done in the same way in other files).

This patch removes most of these inconsistencies including the editor language indicators,
all the different missing/additional '-' characters, files that center the file name, missing
trailing `===//` (mostly caused by clang-format breaking the line).

Reviewers: aprantl, espindola, jfb, shafik, JDevlieghere

Reviewed By: JDevlieghere

Subscribers: dexonsmith, wuzish, emaste, sdardis, nemanjai, kbarton, MaskRay, atanasyan, arphaman, jfb, abidh, jsji, JDevlieghere, usaxena95, lldb-commits

Tags: #lldb

Differential Revision: https://reviews.llvm.org/D73258

show more ...

[lldb/DWARF] Change how we construct a llvm::DWARFContext

Summary:
The goal of this patch is two-fold. First, it fixes a use-after-free in
the construction of the llvm DWARFContext. This happened be

[lldb/DWARF] Change how we construct a llvm::DWARFContext

Summary:
The goal of this patch is two-fold. First, it fixes a use-after-free in
the construction of the llvm DWARFContext. This happened because the
construction code was throwing away the lldb DataExtractors it got while
reading the sections (unlike their llvm counterparts, these are also
responsible for memory ownership). In most cases this did not matter,
because the sections are just slices of the mmapped file data. But this
isn't the case for compressed elf sections, in which case the section is
decompressed into a heap buffer. A similar thing also happen with object
files which are loaded from process memory.

The second goal is to make it explicit which sections go into the llvm
DWARFContext -- any access to the sections through both DWARF parsers
carries a risk of parsing things twice, so it's better if this is a
conscious decision. Also, this avoids loading completely irrelevant
sections (e.g. .text). At present, the only section that needs to be
present in the llvm DWARFContext is the debug_line_str. Using it through
both APIs is not a problem, as there is no parsing involved.

The first goal is achieved by loading the sections through the existing
lldb DWARFContext APIs, which already do the caching. The second by
explicitly enumerating the sections we wish to load.

Reviewers: JDevlieghere, aprantl

Subscribers: lldb-commits

Tags: #lldb

Differential Revision: https://reviews.llvm.org/D72917

show more ...

[lldb/DWARF] Move location list sections into DWARFContext

These are the last sections not managed by the DWARFContext object. I
also introduce separate SectionType enums for dwo section variants, a

[lldb/DWARF] Move location list sections into DWARFContext

These are the last sections not managed by the DWARFContext object. I
also introduce separate SectionType enums for dwo section variants, as
this is necessary for proper handling of single-file split dwarf.

show more ...

[lldb] Add boilerplate to recognize the .debug_rnglists.dwo section

[DWARFContext] Strip leading dot in section names

The LLVM context doesn't expect the leading dot in the section name.

llvm-svn: 365978

Add convenience methods to convert LLDB to LLVM data structures.

This patch adds two convenience methods named GetAsLLVM to the LLDB
counterparts of the DWARF DataExtractor and the DWARF context. Th

Add convenience methods to convert LLDB to LLVM data structures.

This patch adds two convenience methods named GetAsLLVM to the LLDB
counterparts of the DWARF DataExtractor and the DWARF context. The
DWARFContext, once created, is cached for future usage.

Differential revision: https://reviews.llvm.org/D64535

llvm-svn: 365819

show more ...

DWARF: Add support for type units+split dwarf combo

Summary:
With the last round of refactors, supporting type units in dwo files
becomes almost trivial. This patch contains a couple of small fixes,

DWARF: Add support for type units+split dwarf combo

Summary:
With the last round of refactors, supporting type units in dwo files
becomes almost trivial. This patch contains a couple of small fixes,
which taken as a whole make type units work in the split dwarf scenario
(both DWARF4 and DWARF5):
- DWARFContext: make sure we actually read the debug_types.dwo section
- DWARFUnit: set string offsets base on all units in the dwo file, not
just the main CU
- ManualDWARFIndex: index all units in the file
- SymbolFileDWARFDwo: Search for the single compile unit in the file, as
we can no longer assume it will be the first one

The last part makes it obvious that there is still some work to be done
here, namely that we do not support dwo files with multiple compile
units. That is something that should be easier after the DIERef
refactors, but it still requires more work.

Tests are added for the type units+split dwarf + dwarf4/5 scenarios, as
well as a test that checks we behave reasonably in the presence of dwo
files with multiple CUs.

Reviewers: clayborg, JDevlieghere, aprantl

Subscribers: arphaman, lldb-commits

Differential Revision: https://reviews.llvm.org/D63643

llvm-svn: 364274

show more ...

DWARF: port debug_ranges/rnglists over to DWARFContext

llvm-svn: 363400

DWARFContext: Make loading of sections thread-safe

Summary:
SymbolFileDWARF used to load debug sections in a thread-safe manner.
When we moved to DWARFContext, we dropped the thread-safe part, becau

DWARFContext: Make loading of sections thread-safe

Summary:
SymbolFileDWARF used to load debug sections in a thread-safe manner.
When we moved to DWARFContext, we dropped the thread-safe part, because
we thought it was not necessary.

It turns out this was only mostly correct.

The "mostly" part is there because this is a problem only if we use the
manual index, as that is the only source of intra-module paralelism.
Also, this only seems to occur for extremely simple files (like the ones
I've been creating for tests lately), where we've managed to start
indexing before loading the debug_str section. Then, two threads start
to load the section simultaneously and produce wrong results.

On more complex files, something seems to be loading the debug_str section
before we start indexing, as I haven't been able to reproduce this
there, but I have not investigated what it is.

I've tried to come up with a test for this, but I haven't been able to
reproduce the problem reliably. Still, while doing so, I created a way
to generate many compile units on demand. Given that most of our tests
work with only one or two compile units, it seems like this could be
useful anyway.

Reviewers: aprantl, JDevlieghere, clayborg

Subscribers: arphaman, lldb-commits

Differential Revision: https://reviews.llvm.org/D62316

llvm-svn: 361602

show more ...

DWARF: Introduce DWARFTypeUnit class

Summary:
This patch introduces the DWARFTypeUnit class, and teaches lldb to parse
type units out of both the debug_types section (DWARF v4), and from the
regular

DWARF: Introduce DWARFTypeUnit class

Summary:
This patch introduces the DWARFTypeUnit class, and teaches lldb to parse
type units out of both the debug_types section (DWARF v4), and from the
regular debug_info section (DWARF v5).

The most important piece of functionality - resolving DW_AT_signatures
to connect type forward declarations to their definitions - is not
implemented here, but even without that, a lot of functionality becomes
available. I've added tests for the commands that start to work after
this patch.

The changes in this patch were greatly inspired by D61505, which in turn took
over changes from D32167.

Reviewers: JDevlieghere, clayborg, aprantl

Subscribers: mgorny, jankratochvil, lldb-commits

Differential Revision: https://reviews.llvm.org/D62008

llvm-svn: 361360

show more ...

DWARF: Port debug_addr over to DWARFContext

llvm-svn: 361232

DWARF: Port most of other sections over to DWARFContext

This moves the sections from SymbolFileDWARF to DWARFContext, where it
was trivial to do so. A couple of sections are still left in
SymbolFile

DWARF: Port most of other sections over to DWARFContext

This moves the sections from SymbolFileDWARF to DWARFContext, where it
was trivial to do so. A couple of sections are still left in
SymbolFileDWARF. These will be handled by separate patches.

llvm-svn: 361127

show more ...

Make DWARFContext dwo-aware and port debug_info sections over

Summary:
The previous attempt and moving section handling over to DWARFContext
(D59611) failed because it did not take into account the

Make DWARFContext dwo-aware and port debug_info sections over

Summary:
The previous attempt and moving section handling over to DWARFContext
(D59611) failed because it did not take into account the dwo sections
correctly. All DWARFContexts (even those in SymbolFileDWARFDwo) used the
main module for loading the sections, but in the dwo scenario some
sections should come from the dwo file.

This patch fixes that by making the DWARFContext aware of whether it a
dwo context or a regular one. A dwo context gets two sections lists, and
it knows where to look for a particular type of a section. This isn't
fully consistent with how the llvm DWARFContext behaves, because that
one leaves it up to the user to know whether it should ask for a dwo
section or not. However, for the time being, it seems useful to have a
single entity which knows how to peice together the debug info in dwo
and non-dwo scenarios. The rough roadmap for the future is:
- port over the rest of the sections to DWARFContext
- find a way to get rid of SymbolFileDWARFDwo/Dwp/DwpDwo. This will
likely involve adding the ability for the DWARFContext to spawn
dwo sub-contexts, similarly to how it's done in llvm.
- get rid of the special handling of the "dwo" contexts by making
sure everything knows whether it should ask for the .dwo version of
the section or not (similarly to how llvm's DWARFUnits do that)

To demonstrate how the DWARFContext should behave in this new world, I
port the debug_info section (which is debug_info.dwo in the dwo file)
handling to DWARFContext. The rest of the sections will come in
subsequent patches.

Reviewers: aprantl, clayborg, JDevlieghere

Subscribers: zturner, lldb-commits

Differential Revision: https://reviews.llvm.org/D62012

llvm-svn: 361000

show more ...

DWARFContext: Return empty data extractors instead of null pointers

Summary:
There are several reasons for doing this:
- generally, there's no reason to differentiate between a section being
absen

DWARFContext: Return empty data extractors instead of null pointers

Summary:
There are several reasons for doing this:
- generally, there's no reason to differentiate between a section being
absent and it being present, but empty
- it matches more closely what llvm DWARF parser is doing (which also
doesn't differentiate the two cases)
- SymbolFileDWARF also doesn't differentiate the two cases, which makes
porting the rest of sections easier
- it fixes a bug in how the return-null-if-empty logic was implemented
(it returned nullptr only the second time we tried to get the
debug_aranges section), which meant that we hit an assert when trying
to parse an empty-but-present section

Reviewers: JDevlieghere, clayborg, aprantl

Subscribers: zturner, lldb-commits

Differential Revision: https://reviews.llvm.org/D61942

llvm-svn: 360874

show more ...

Revert "Move the rest of the sections over to DWARFContext."

This reverts commit r356682 because it breaks the DWO flavours of some
tests:
lldb-Suite :: lang/c/const_variables/TestConstVariables

Revert "Move the rest of the sections over to DWARFContext."

This reverts commit r356682 because it breaks the DWO flavours of some
tests:
lldb-Suite :: lang/c/const_variables/TestConstVariables.py
lldb-Suite :: lang/c/local_variables/TestLocalVariables.py
lldb-Suite :: lang/c/vla/TestVLA.py

llvm-svn: 356773

show more ...

Move the rest of the sections over to DWARFContext.

This is mostly mechanical, and just moves the remaining non-DWO
related sections over to DWARFContext.

Differential Revision: https://reviews.llv

Move the rest of the sections over to DWARFContext.

This is mostly mechanical, and just moves the remaining non-DWO
related sections over to DWARFContext.

Differential Revision: https://reviews.llvm.org/D59611

llvm-svn: 356682

show more ...

Introduce DWARFContext.

LLVM's DWARF parsing library has a class called DWARFContext which holds
all of the various DWARF data sections and lots of other information.
LLDB's on the other hand stores

Introduce DWARFContext.

LLVM's DWARF parsing library has a class called DWARFContext which holds
all of the various DWARF data sections and lots of other information.
LLDB's on the other hand stores all of this directly in SymbolFileDWARF
/ SymbolFileDWARFDwo and passes this interface around through the
parsing library. Obviously this is incompatible with a world where the
low level interface does not depend on the high level interface, so we
need to move towards a model similar to LLVM's - i.e. all of the context
needed for low level parsing should be in a single class, and that class
gets passed around.

This patch is a small incremental step towards achieving this. The
interface and internals deviate from LLVM's for technical reasons, but
the high level idea is the same. The goal is, eventually, to remove all
occurrences of SymbolFileDWARF from the low level parsing code.

For now I've chosen a very simple section - the .debug_aranges section
to move into DWARFContext while leaving everything else unchanged. In
the short term this is a bit confusing because now the information you
need might come from either of 2 different locations. But it's a huge
refactor to do this all at once and runs a much higher risk of breaking
things. So I think it would be wise to do this in very small pieces.

TL;DR - No functional change

Differential Revision: https://reviews.llvm.org/D59562

llvm-svn: 356612

show more ...