[LLDB][ClangExpression] Prevent nullptr namespace map access during logging

Some codepaths lead to `namespace_map == nullptr` when we get to
`ClangASTSource::FindCompleteType`. This occurred while d

[LLDB][ClangExpression] Prevent nullptr namespace map access during logging

Some codepaths lead to `namespace_map == nullptr` when we get to
`ClangASTSource::FindCompleteType`. This occurred while debugging
an lldb session that had `settings set target.import-std-module true`.

In that case, with `LLDBLog::Expressions` logging enabled, we would
dereference a `nullptr` and crash.

This commit moves the logging until after we check for `nullptr`.

**Testing**

* Fixed the specific crash I was seeing while debugging an `lldb`
session with `import-std-module` enabled.

Differential Revision: https://reviews.llvm.org/D130561

show more ...

Use llvm::less_first and llvm::less_second (NFC)

Use the range-based overload of llvm::sort where possible

Reviewed By: MaskRay

Differential Revision: https://reviews.llvm.org/D130403

[lldb] Rename Logging.h to LLDBLog.h and clean up includes

Most of our code was including Log.h even though that is not where the
"lldb" log channel is defined (Log.h defines the generic logging
inf

[lldb] Rename Logging.h to LLDBLog.h and clean up includes

Most of our code was including Log.h even though that is not where the
"lldb" log channel is defined (Log.h defines the generic logging
infrastructure). This worked because Log.h included Logging.h, even
though it should.

After the recent refactor, it became impossible the two files include
each other in this direction (the opposite inclusion is needed), so this
patch removes the workaround that was put in place and cleans up all
files to include the right thing. It also renames the file to LLDBLog to
better reflect its purpose.

show more ...

[lldb] Convert "LLDB" log channel to the new API

Remove redundant return and continue statements (NFC)

Identified with readability-redundant-control-flow.

Revert "[lldb] Disable minimal import mode for RecordDecls that back FieldDecls"

This reverts commit 3bf96b0329be554c67282b0d7d8da6a864b9e38f.

It causes crashes as reported in PR52257 and a few oth

Revert "[lldb] Disable minimal import mode for RecordDecls that back FieldDecls"

This reverts commit 3bf96b0329be554c67282b0d7d8da6a864b9e38f.

It causes crashes as reported in PR52257 and a few other places. A reproducer is bundled with this commit to verify any fix forward. The original test is left in place, but marked XFAIL as it now produces the wrong result.

Reviewed By: teemperor

Differential Revision: https://reviews.llvm.org/D113449

show more ...

[LLDB] Add type to the output for FieldDecl when logging in ClangASTSource::layoutRecordType

I was debugging a problem and noticed that it would have been helpful to have
the type of each FieldDecl

[LLDB] Add type to the output for FieldDecl when logging in ClangASTSource::layoutRecordType

I was debugging a problem and noticed that it would have been helpful to have
the type of each FieldDecl when looking at the output from
ClangASTSource::layoutRecordType.

Differential Revision: https://reviews.llvm.org/D108257

show more ...

[LLDB] Fix off by one logging placeholders in ClangASTSource::layoutRecordType()

D72391 Added some additional information to the logging but in this case instead of using
placeholder 2 and 3 they us

[LLDB] Fix off by one logging placeholders in ClangASTSource::layoutRecordType()

D72391 Added some additional information to the logging but in this case instead of using
placeholder 2 and 3 they used 3 and 4.

show more ...

[lldb] Use a struct to pass function search options to Module::FindFunction

Rather than passing two booleans around, which is especially error prone
with them being next to each other, use a struct

[lldb] Use a struct to pass function search options to Module::FindFunction

Rather than passing two booleans around, which is especially error prone
with them being next to each other, use a struct with named fields
instead.

Differential revision: https://reviews.llvm.org/D107295

show more ...

[lldb] Disable minimal import mode for RecordDecls that back FieldDecls

Clang adds a Decl in two phases to a DeclContext. First it adds it invisible and
then it makes it visible (which will add it t

[lldb] Disable minimal import mode for RecordDecls that back FieldDecls

Clang adds a Decl in two phases to a DeclContext. First it adds it invisible and
then it makes it visible (which will add it to the lookup data structures). It's
important that we can't do lookups into the DeclContext we are currently adding
the Decl to during this process as once the Decl has been added, any lookup will
automatically build a new lookup map and add the added Decl to it. The second
step would then add the Decl a second time to the lookup which will lead to
weird errors later one. I made adding a Decl twice to a lookup an assertion
error in D84827.

In the first step Clang also does some computations on the added Decl if it's
for example a FieldDecl that is added to a RecordDecl.

One of these computations is checking if the FieldDecl is of a record type
and the record type has a deleted constexpr destructor which will delete
the constexpr destructor of the record that got the FieldDecl.

This can lead to a bug with the way we implement MinimalImport in LLDB
and the following code:

```
struct Outer {
typedef int HookToOuter;
struct NestedClass {
HookToOuter RefToOuter;
} NestedClassMember; // We are adding this.
};
```

1. We just imported `Outer` minimally so far.
2. We are now asked to add `NestedClassMember` as a FieldDecl.
3. We import `NestedClass` minimally.
4. We add `NestedClassMember` and clang does a lookup for a constexpr dtor in
`NestedClass`. `NestedClassMember` hasn't been added to the lookup.
5. The lookup into `NestedClass` will now load the members of `NestedClass`.
6. We try to import the type of `RefToOuter` which will try to import the `HookToOuter` typedef.
7. We import the typedef and while importing we check for conflicts in `Outer` via a lookup.
8. The lookup into `Outer` will cause the invisible `NestedClassMember` to be added to the lookup.
9. We continue normally until we get back to the `addDecl` call in step 2.
10. We now add `NestedClassMember` to the lookup even though we already did that in step 8.

The fix here is disabling the minimal import for RecordTypes from FieldDecls. We
actually already did this, but so far we only force the definition of the type
to be imported *after* we imported the FieldDecl. This just moves that code
*before* we import the FieldDecl so prevent the issue above.

Reviewed By: shafik, aprantl

Differential Revision: https://reviews.llvm.org/D102993

show more ...

[lldb] Move ClangModulesDeclVendor ownership to ClangPersistentVariables from Target

More decoupling of plugins and non-plugins. Target doesn't need to
manage ClangModulesDeclVendor and ClangPersist

[lldb] Move ClangModulesDeclVendor ownership to ClangPersistentVariables from Target

More decoupling of plugins and non-plugins. Target doesn't need to
manage ClangModulesDeclVendor and ClangPersistentVariables is always available
in situations where you need ClangModulesDeclVendor.

Differential Revision: https://reviews.llvm.org/D102811

show more ...

[LLDB] Fix LLDB_LOG calls to use correct formatting

It looks like a previous change switched these from LLDB_LOGF but did not update the format strings.

Differential Revision: https://reviews.llvm.

[LLDB] Fix LLDB_LOG calls to use correct formatting

It looks like a previous change switched these from LLDB_LOGF but did not update the format strings.

Differential Revision: https://reviews.llvm.org/D96550

show more ...

[lldb] Access the ModuleList through iterators where possible (NFC)

Replace uses of GetModuleAtIndexUnlocked and
GetModulePointerAtIndexUnlocked with the ModuleIterable and
ModuleIterableNoLocking w

[lldb] Access the ModuleList through iterators where possible (NFC)

Replace uses of GetModuleAtIndexUnlocked and
GetModulePointerAtIndexUnlocked with the ModuleIterable and
ModuleIterableNoLocking where applicable.

Differential revision: https://reviews.llvm.org/D94271

show more ...

[lldb] Introduce separate scratch ASTs for debug info types and types imported from C++ modules.

Right now we have one large AST for all types in LLDB. All ODR violations in
types we reconstruct are

[lldb] Introduce separate scratch ASTs for debug info types and types imported from C++ modules.

Right now we have one large AST for all types in LLDB. All ODR violations in
types we reconstruct are resolved by just letting the ASTImporter handle the
conflicts (either by merging types or somehow trying to introduce a duplicated
declaration in the AST). This works ok for the normal types we build from debug
information as most of them are just simple CXXRecordDecls or empty template
declarations.

However, with a loaded `std` C++ module we have alternative versions of pretty
much all declarations in the `std` namespace that are much more fleshed out than
the debug information declarations. They have all the information that is lost
when converting to DWARF, such as default arguments, template default arguments,
the actual uninstantiated template declarations and so on.

When we merge these C++ module types into the big scratch AST (that might
already contain debug information types) we give the ASTImporter the tricky task
of somehow creating a consistent AST out of all these declarations. Usually this
ends in a messy AST that contains a mostly broken mix of both module and debug
info declarations. The ASTImporter in LLDB is also importing types with the
MinimalImport setting, which usually means the only information we have when
merging two types is often just the name of the declaration and the information
that it contains some child declarations. This makes it pretty much impossible
to even implement a better merging logic (as the names of C++ module
declarations and debug info declarations are identical).

This patch works around this whole merging problem by separating C++ module
types from debug information types. This is done by splitting up the single
scratch AST into two: One default AST for debug information and a dedicated AST
for C++ module types.

The C++ module AST is implemented as a 'specialised AST' that lives within the
default ScratchTypeSystemClang. When we select the scratch AST we can explicitly
request that we want such a isolated sub-AST of the scratch AST. I kept the
infrastructure more general as we probably can use the same mechanism for other
features that introduce conflicting types (such as programs that are compiled
with a custom -wchar-size= option).

There are just two places where we explicitly have request the C++ module AST:
When we export persistent declarations (`$mytype`) and when we create our
persistent result variable (`$0`, `$1`, ...). There are a few formatters that
were previously assuming that there is only one scratch AST which I cleaned up
in a preparation revision here (D92757).

Reviewed By: aprantl

Differential Revision: https://reviews.llvm.org/D92759

show more ...

[lldb][NFC] Rename TypeSystemClang::GetScratch to ScratchTypeSystemClang::GetForTarget

Also add some documentation while I'm at it.

[lldb][NFC] Add a FIXME for ClangASTSource::FindExternalLexicalDecls's unused 'decls' parameter

[lldb][NFC] Remove ImportInProgress lock in ClangASTSource

Summary:

The ClangASTSource has a lock that globally disables all lookups into the
external AST source when we explicitly "guarded" copy a

[lldb][NFC] Remove ImportInProgress lock in ClangASTSource

Summary:

The ClangASTSource has a lock that globally disables all lookups into the
external AST source when we explicitly "guarded" copy a type. It's not used for
anything else, so importing declarations or importing types that are
dependencies of a declaration actually won't activate that lock. The lookups it
is supposed to prevent also don't actually happen in our test suite. The check
in `ClangExpressionDeclMap::FindExternalVisibleDecls` is never executed and the
check in the `ClangASTSource::FindExternalVisibleDeclsByName` is only ever
reached by the `Import-std-module` tests (which explicitly do a lookup into the
expression context on purpose).

This lock was added in 6abfabff6158076eccdf6fcac5a12894039de2c9 as a replacement
for a list of types we already looked up which appeared to be an optimisation
strategy. I assume back then this lock had a purpose but these days the
ASTImporter and LLDB seem to be smart enough to avoid whatever lookups this
tried to prevent.

I would say we remove it from LLDB. The main reason is that it blocks D81561
(which explicitly does a specific lookup to resolve placeholder types produced
by `-flimit-debug-info`) but it's semantics are also very confusing. The naming
implies it's a flag to indicate when we import something at the moment which is
practically never true as described above. Also the fact that it makes our
ExternalASTSource alternate between doing lookups into the debug info and
pretending it doesn't know any external decls could really break our lookup in
some weird way if Clang decides to cache a fake empty lookup result that was
generated while the lock was active.

Reviewers: labath, shafik, JDevlieghere, aprantl

Reviewed By: labath, JDevlieghere, aprantl

Subscribers: aprantl, abidh

Differential Revision: https://reviews.llvm.org/D81749

show more ...

[lldb] Split ClangASTSource::CompleteType

Move the part of the code which is responsible for finding a complete
definition of the type into a separate function (FindCompleteType). This
is split off

[lldb] Split ClangASTSource::CompleteType

Move the part of the code which is responsible for finding a complete
definition of the type into a separate function (FindCompleteType). This
is split off from D81561, as it's a generally useful cleanup.

No functional change.

show more ...

[lldb] Fix more typos in log statements in ClangASTSource

The indexes need to start at 0 but in D74951 I removed the first parameter
and didn't decrement all the indexes. This patch at least makes s

[lldb] Fix more typos in log statements in ClangASTSource

The indexes need to start at 0 but in D74951 I removed the first parameter
and didn't decrement all the indexes. This patch at least makes sure that
LLDB logging no longer crashes (but it still deadlocks).

show more ...

[lldb][NFC] Use C++11 for loop in ClangASTSource::FindExternalLexicalDecls

[lldb] Fix typo in ClangASTSource logging statement

[lldb] Fix several LLDB_LOGs with wrong indices in ClangASTSource.cpp

[lldb] Delete some commented out code in ClangASTSource.cpp

Avoid SourceManager.h include in RawCommentList.h, add missing incs

SourceManager.h includes FileManager.h, which is expensive due to
dependencies on LLVM FS headers.

Remove dead BeforeThanCompare

Avoid SourceManager.h include in RawCommentList.h, add missing incs

SourceManager.h includes FileManager.h, which is expensive due to
dependencies on LLVM FS headers.

Remove dead BeforeThanCompare specialization.

Sink ASTContext::addComment to cpp file.

This reduces the time to compile a file that does nothing but include
ASTContext.h from ~3.4s to ~2.8s for me.

Saves these includes:
219 - ../clang/include/clang/Basic/SourceManager.h
204 - ../clang/include/clang/Basic/FileSystemOptions.h
204 - ../clang/include/clang/Basic/FileManager.h
165 - ../llvm/include/llvm/Support/VirtualFileSystem.h
164 - ../llvm/include/llvm/Support/SourceMgr.h
164 - ../llvm/include/llvm/Support/SMLoc.h
161 - ../llvm/include/llvm/Support/Path.h
141 - ../llvm/include/llvm/ADT/BitVector.h
128 - ../llvm/include/llvm/Support/MemoryBuffer.h
124 - ../llvm/include/llvm/Support/FileSystem.h
124 - ../llvm/include/llvm/Support/Chrono.h
124 - .../MSVCSTL/include/stack
122 - ../llvm/include/llvm-c/Types.h
122 - ../llvm/include/llvm/Support/NativeFormatting.h
122 - ../llvm/include/llvm/Support/FormatProviders.h
122 - ../llvm/include/llvm/Support/CBindingWrapping.h
122 - .../MSVCSTL/include/xtimec.h
122 - .../MSVCSTL/include/ratio
122 - .../MSVCSTL/include/chrono
121 - ../llvm/include/llvm/Support/FormatVariadicDetails.h
118 - ../llvm/include/llvm/Support/MD5.h
109 - .../MSVCSTL/include/deque
105 - ../llvm/include/llvm/Support/Host.h
105 - ../llvm/include/llvm/Support/Endian.h

Reviewed By: aaron.ballman, hans

Differential Revision: https://reviews.llvm.org/D75279

show more ...