[lldb] use one shared ThreadPool and task groups

As a preparation for parallelizing loading of symbols (D122975),
it is necessary to use just one thread pool to avoid using
a thread pool from inside

[lldb] use one shared ThreadPool and task groups

As a preparation for parallelizing loading of symbols (D122975),
it is necessary to use just one thread pool to avoid using
a thread pool from inside a task of another thread pool.

Differential Revision: https://reviews.llvm.org/D123226

show more ...

[LLDB] Remove cases of using namespace llvm:: from header file

We have using namespace llvm::dwarf in dwarf.h header globally. Replacing that
with a using namespace within lldb_private::dwarf and mo

[LLDB] Remove cases of using namespace llvm:: from header file

We have using namespace llvm::dwarf in dwarf.h header globally. Replacing that
with a using namespace within lldb_private::dwarf and moving to a
using namespace lldb_private::dwarf in .cpp files and fully qualified names
in the few header files.

Differential Revision: https://reviews.llvm.org/D120836

show more ...

[lldb] Convert DWARF log to the new API

This also deletes some dead log statements (log initialization commented
out).

Cache the manual DWARF index out to the LLDB cache directory when the LLDB index cache is enabled.

This patch add the ability to cache the manual DWARF indexing results to disk for faster subsequent

Cache the manual DWARF index out to the LLDB cache directory when the LLDB index cache is enabled.

This patch add the ability to cache the manual DWARF indexing results to disk for faster subsequent debug sessions. Manual DWARF indexing is time consuming and causes all DWARF to be fully parsed and indexed each time you debug a binary that doesn't have an acceptable accelerator table. Acceptable accelerator tables include .debug_names in DWARF5 or Apple accelerator tables.

This patch breaks up testing by testing all of the encoding and decoding of required C++ objects in a gtest unit test, and then has a test to verify the debug info cache is generated correctly.

This patch also adds the ability to track when a symbol table or DWARF index is loaded or saved to the cache in the "statistics dump" command. This is essential to know in statistics as it can help explain why a debug session was slower or faster than expected.

Reviewed By: labath, wallace

Differential Revision: https://reviews.llvm.org/D115951

show more ...

Add new key/value pairs to the module statistics for "statistics dump".

The new key/value pairs that are added to each module's stats are:
"debugInfoByteSize": The size in bytes of debug info for ea

Add new key/value pairs to the module statistics for "statistics dump".

The new key/value pairs that are added to each module's stats are:
"debugInfoByteSize": The size in bytes of debug info for each module.
"debugInfoIndexTime": The time in seconds that it took to index the debug info.
"debugInfoParseTime": The time in seconds that debug info had to be parsed.

At the top level we add up all of the debug info size, parse time and index time with the following keys:
"totalDebugInfoByteSize": The size in bytes of all debug info in all modules.
"totalDebugInfoIndexTime": The time in seconds that it took to index all debug info if it was indexed for all modules.
"totalDebugInfoParseTime": The time in seconds that debug info was parsed for all modules.

Differential Revision: https://reviews.llvm.org/D112501

show more ...

[nfc] [lldb] Assertions for D106270 - [DWARF5] Fix offset check when using .debug_names

Skeleton vs. DWO units mismatch has been fixed in D106270. As they both
have type DWARFUnit it is a bit diffic

[nfc] [lldb] Assertions for D106270 - [DWARF5] Fix offset check when using .debug_names

Skeleton vs. DWO units mismatch has been fixed in D106270. As they both
have type DWARFUnit it is a bit difficult to debug. So it is better to
make it safe against future changes.

Reviewed By: kimanh, clayborg

Differential Revision: https://reviews.llvm.org/D107659

show more ...

[DWARF5] Fix offset check when using .debug_names

When going through the CU entries in the name index,
make sure to compare the name entry's CU
offset against the skeleton CU's offset.

Previously t

[DWARF5] Fix offset check when using .debug_names

When going through the CU entries in the name index,
make sure to compare the name entry's CU
offset against the skeleton CU's offset.

Previously there would be a mismatch, since the
wrong offset was compared, and thus no suitable
entry was found.

Reviewed By: jankratochvil

Differential Revision: https://reviews.llvm.org/D106270

show more ...

Add a progress class that can track long running operations in LLDB.

LLDB can often appear deadlocked to users that use IDEs when it is indexing DWARF, or parsing symbol tables. These long running o

Add a progress class that can track long running operations in LLDB.

LLDB can often appear deadlocked to users that use IDEs when it is indexing DWARF, or parsing symbol tables. These long running operations can make a debug session appear to be doing nothing even though a lot of work is going on inside LLDB. This patch adds a public API to allow clients to listen to debugger events that report progress and will allow UI to create an activity window or display that can show users what is going on and keep them informed of expensive operations that are going on inside LLDB.

Differential Revision: https://reviews.llvm.org/D97739

show more ...

[lldb] Abstract scoped timer logic behind LLDB_SCOPED_TIMER (NFC)

This patch introduces a LLDB_SCOPED_TIMER macro to hide the needlessly
repetitive creation of scoped timers in LLDB. It's similar to

[lldb] Abstract scoped timer logic behind LLDB_SCOPED_TIMER (NFC)

This patch introduces a LLDB_SCOPED_TIMER macro to hide the needlessly
repetitive creation of scoped timers in LLDB. It's similar to the
LLDB_LOG(F) macro.

Differential revision: https://reviews.llvm.org/D93663

show more ...

Add optimal thread strategy

Add an optimal thread strategy to execute specified amount of tasks.
This strategy should prevent us from creating too many threads if we
occasionaly have an unexpectedly

Add optimal thread strategy

Add an optimal thread strategy to execute specified amount of tasks.
This strategy should prevent us from creating too many threads if we
occasionaly have an unexpectedly small amount of tasks.

Differential Revision: https://reviews.llvm.org/D87765

show more ...

[lldb/Host] Remove TaskPool and replace its uses with llvm::ThreadPool

Remove LLDB's TaskPool and replace its uses with LLVM's ThreadPool.

Differential revision: https://reviews.llvm.org/D78337

[nfc] [lldb] DWARF callbacks: DIERef -> DWARFDIE

Pavel Labath wrote in D73206:
The internal representation of DebugNames and Apple indexes is fixed by
the relevant (pseudo-)standards, so we can't re

[nfc] [lldb] DWARF callbacks: DIERef -> DWARFDIE

Pavel Labath wrote in D73206:
The internal representation of DebugNames and Apple indexes is fixed by
the relevant (pseudo-)standards, so we can't really change it. The
question is how to efficiently (and cleanly) convert from the internal
representation to some common thing. The conversion from AppleIndex to
DIERef is trivial (which is not surprising as it was the first and the
overall design was optimized for that). With debug_names, the situation
gets more tricky. The internal representation of debug_names uses
CU-relative DIE offsets, but DIERef wants an absolute offset. That means
the index has to do more work to produce the common representation. And
it needs to do that for all results, even though a lot of the index
users are really interested only in a single entry. With the switch to
user_id_t, _all_ indexes would have to do some extra work to encode it,
only for their users to have to immediately decode it back. Having
a iterator/callback based api would allow us to minimize the impact of
that, as it would only need to happen for the entries that are really
used. And /I think/ we could make it interface returns DWARFDies
directly, and each index converts to that using the most direct approach
available.

Jan Kratochvil:
It also makes all the callers shorter as they no longer need to fetch
DWARFDIE from DIERef (and handling if not found by ReportInvalidDIERef)
but the callers are already served DWARFDIE which they need.
In some cases the DWARFDIE had to be fetched both by callee (DWARFIndex
implementation) and caller.

Differential Revision: https://reviews.llvm.org/D77970

show more ...

[DWARF] Rename a function and comment it for clarity.

Pointed out by Adrian.

[nfc] [lldb] Introduce DWARF callbacks (fixed-up)

As requested by @labath in https://reviews.llvm.org/D73206#1949516
providing DWARF index callbacks refactorization.

It is a re-landing of the patch

[nfc] [lldb] Introduce DWARF callbacks (fixed-up)

As requested by @labath in https://reviews.llvm.org/D73206#1949516
providing DWARF index callbacks refactorization.

It is a re-landing of the patch with a regression fix the previous
commit had.

Differential Revision: https://reviews.llvm.org/D77327

show more ...

Revert "[nfc] [lldb] Introduce DWARF callbacks"

This reverts commit bd47c470d13b1c57ecf37c1faf0324833d3a4542.

It broke Green Dragon, reason is unknown to me so far:
http://green.lab.llvm.org/gree

Revert "[nfc] [lldb] Introduce DWARF callbacks"

This reverts commit bd47c470d13b1c57ecf37c1faf0324833d3a4542.

It broke Green Dragon, reason is unknown to me so far:
http://green.lab.llvm.org/green/view/LLDB/job/lldb-cmake/15323/consoleFull

Differential Revision: https://reviews.llvm.org/D77327

show more ...

[nfc] [lldb] Introduce DWARF callbacks

As requested by @labath in https://reviews.llvm.org/D73206#1949516
providing DWARF index callbacks refactorization.

Differential Revision: https://reviews.llv

[nfc] [lldb] Introduce DWARF callbacks

As requested by @labath in https://reviews.llvm.org/D73206#1949516
providing DWARF index callbacks refactorization.

Differential Revision: https://reviews.llvm.org/D77327

show more ...

[DWARF] Not all the constant variables are "static".

Fixes rdar://problem/61402307

Differential Revision: https://reviews.llvm.org/D77698

[ManualDWARFIndex] Remove dead code, in preparation for moving this function.

[nfc] [lldb] Unindent code - obvious part

It is an obvious part of D77326.

It removes some needless deep indentation and some redundant statements.
It prepares the code for a more clean next patch

[nfc] [lldb] Unindent code - obvious part

It is an obvious part of D77326.

It removes some needless deep indentation and some redundant statements.
It prepares the code for a more clean next patch - DWARF index callbacks
in D77327.

show more ...

[lldb/DWARF] Don't index dwp file multiple times

Summary:
When we added support for type units in dwo files, we changed the
"manual" dwarf index to index _all_ dwarf units in the dwo file instead
of

[lldb/DWARF] Don't index dwp file multiple times

Summary:
When we added support for type units in dwo files, we changed the
"manual" dwarf index to index _all_ dwarf units in the dwo file instead
of just the split unit belonging to our skeleton unit. This was fine for
dwo files, as they contain only a single compile units and type units do
not have a split type unit which would point to them.

However, this does not work for dwp files because, these files do
contain multiple split compile units, and the current approach means
that each unit gets indexed multiple times (once for each split unit =>
n^2 complexity).

This patch teaches the manual dwarf index to treat dwp files specially.
Any type units in the dwp file added to the main list of compile units
and indexed with them in a single batch. Split compile units in dwp
files are still indexed as a part of their skeleton unit -- this is done
because we need the DW_AT_language attribute from the skeleton unit to
index them properly.

Handling of dwo files remains unchanged -- all units (type and skeleton)
are indexed when we reach the dwo file through the split unit.

Reviewers: clayborg, JDevlieghere, aprantl

Subscribers: arphaman, lldb-commits

Tags: #lldb

Differential Revision: https://reviews.llvm.org/D74964

show more ...

[lldb/DWARF] Always construct a DWARFDebugInfo object

Change the return value of SymbolFileDWARF::DebugInfo from a pointer to
a reference, and remove all null checks.

Previously, we were not constr

[lldb/DWARF] Always construct a DWARFDebugInfo object

Change the return value of SymbolFileDWARF::DebugInfo from a pointer to
a reference, and remove all null checks.

Previously, we were not constructing the DebugInfo object when the
debug_info section was empty. Now we always construct the object but
it will return an empty list of dwarf units (a thing which it already
supported).

show more ...

[lldb][NFC] Make all CompilerDeclContext parameters references instead of pointers

Summary:
All of our lookup APIs either use `CompilerDeclContext &` or `CompilerDeclContext *` semi-randomly it seem

[lldb][NFC] Make all CompilerDeclContext parameters references instead of pointers

Summary:
All of our lookup APIs either use `CompilerDeclContext &` or `CompilerDeclContext *` semi-randomly it seems.
This leads to us constantly converting between those two types (and doing nullptr checks when going from
pointer to reference). It also leads to the confusing situation where we have two possible ways to express
that we don't have a CompilerDeclContex: either a nullptr or an invalid CompilerDeclContext (aka a default
constructed CompilerDeclContext).

This moves all APIs to use references and gets rid of all the nullptr checks and conversions.

Reviewers: labath, mib, shafik

Reviewed By: labath, shafik

Subscribers: shafik, arphaman, abidh, JDevlieghere, lldb-commits

Tags: #lldb

Differential Revision: https://reviews.llvm.org/D74607

show more ...

[lldb/DWARF] Don't assume that a SymbolFileDWARFDwo contains one compile unit

Summary:
This is a preparatory patch to re-enable DWP support in lldb (we already
have code claiming to do that, but it

[lldb/DWARF] Don't assume that a SymbolFileDWARFDwo contains one compile unit

Summary:
This is a preparatory patch to re-enable DWP support in lldb (we already
have code claiming to do that, but it has been completely broken for a
while now).

The idea of the new approach is to make the SymbolFileDWARFDwo class
handle both dwo and dwo files, similar to how llvm uses one DWARFContext
to handle the two.

The first step is to remove the assumption that a SymbolFileDWARFDwo
holds just a single compile unit, i.e. the GetBaseCompileUnit method.
This requires changing the way how we reach the skeleton compile unit
(and the lldb_private::CompileUnit) from a dwo unit, which was
previously done via GetSymbolFile()->GetBaseCompileUnit() (and some
virtual dispatch).

The new approach reuses the "user data" mechanism of DWARFUnits, which
was used to link dwarf units (both skeleton and split) to their
lldb_private counterparts. Now, this is done only for non-dwo units, and
instead of that, the dwo units holds a pointer to the relevant skeleton
unit.

Reviewers: JDevlieghere, aprantl, clayborg

Reviewed By: JDevlieghere, clayborg

Subscribers: arphaman, lldb-commits

Tags: #lldb

Differential Revision: https://reviews.llvm.org/D73781

show more ...

[lldb] Move non-DWARF code: DWARFUnit -> SymbolFileDWARF

This patchset is removing non-DWARF code from DWARFUnit for better
future merge with LLVM DWARF as discussed with @labath.

Differential revi

[lldb] Move non-DWARF code: DWARFUnit -> SymbolFileDWARF

This patchset is removing non-DWARF code from DWARFUnit for better
future merge with LLVM DWARF as discussed with @labath.

Differential revision: https://reviews.llvm.org/D70646

show more ...

[lldb][NFC] Fix all formatting errors in .cpp file headers

Summary:
A *.cpp file header in LLDB (and in LLDB) should like this:
```
//===-- TestUtilities.cpp ----------------------------------------

[lldb][NFC] Fix all formatting errors in .cpp file headers

Summary:
A *.cpp file header in LLDB (and in LLDB) should like this:
```
//===-- TestUtilities.cpp -------------------------------------------------===//
```
However in LLDB most of our source files have arbitrary changes to this format and
these changes are spreading through LLDB as folks usually just use the existing
source files as templates for their new files (most notably the unnecessary
editor language indicator `-*- C++ -*-` is spreading and in every review
someone is pointing out that this is wrong, resulting in people pointing out that this
is done in the same way in other files).

This patch removes most of these inconsistencies including the editor language indicators,
all the different missing/additional '-' characters, files that center the file name, missing
trailing `===//` (mostly caused by clang-format breaking the line).

Reviewers: aprantl, espindola, jfb, shafik, JDevlieghere

Reviewed By: JDevlieghere

Subscribers: dexonsmith, wuzish, emaste, sdardis, nemanjai, kbarton, MaskRay, atanasyan, arphaman, jfb, abidh, jsji, JDevlieghere, usaxena95, lldb-commits

Tags: #lldb

Differential Revision: https://reviews.llvm.org/D73258

show more ...