Revert "Adapt LLDB for D120540."

This reverts commit ca73de43744503a557b1f3709c0ff4751798702f.

That patch was just hiding the problem, instead of fixing it.

Adapt LLDB for D120540.

In https://reviews.llvm.org/D120540 the -fcxx-modules flag changed
semantics and specifying it explicitly seems to no longer be what we
want here.

Fix LLDB test broken by 499d0b96cb52c828e7fc4d58825b5e8b3f9931c5

[clang] createInvocationFromCommandLine -> createInvocation, delete former. NFC

(Followup from 40c13720a4b977d4347bbde53c52a4d0703823c2)

Differential Revision: https://reviews.llvm.org/D125012

[C++20][Modules][1/8] Track valid import state.

In C++20 modules imports must be together and at the start of the module.
Rather than growing more ad-hoc flags to test state, this keeps track of the

[C++20][Modules][1/8] Track valid import state.

In C++20 modules imports must be together and at the start of the module.
Rather than growing more ad-hoc flags to test state, this keeps track of the
phase of of a valid module TU (first decl, global module frag, module,
private module frag). If the phasing is broken (with some diagnostic) the
pattern does not conform to a valid C++20 module, and we set the state
accordingly.

We can thus issue diagnostics when imports appear in the wrong places and
decouple the C++20 modules state from other module variants (modules-ts and
clang modules). Additionally, we attempt to diagnose wrong imports before
trying to find the module where possible (the latter will generally emit an
unhelpful diagnostic about the module not being available).

Although this generally simplifies the handling of C++20 module import
diagnostics, the motivation was that, in particular, it allows detecting
invalid imports like:

import module A;

int some_decl();

import module B;

where being in a module purview is insufficient to identify them.

Differential Revision: https://reviews.llvm.org/D118893

show more ...

[Preprocessor] Reduce the memory overhead of `#define` directives (Recommit)

Recently we observed high memory pressure caused by clang during some parallel builds.
We discovered that we have several

[Preprocessor] Reduce the memory overhead of `#define` directives (Recommit)

Recently we observed high memory pressure caused by clang during some parallel builds.
We discovered that we have several projects that have a large number of #define directives
in their TUs (on the order of millions), which caused huge memory consumption in clang due
to a lot of allocations for MacroInfo. We would like to reduce the memory overhead of
clang for a single #define to reduce the memory overhead for these files, to allow us to
reduce the memory pressure on the system during highly parallel builds. This change achieves
that by removing the SmallVector in MacroInfo and instead storing the tokens in an array
allocated using the bump pointer allocator, after all tokens are lexed.

The added unit test with 1000000 #define directives illustrates the problem. Prior to this
change, on arm64 macOS, clang's PP bump pointer allocator allocated 272007616 bytes, and
used roughly 272 bytes per #define. After this change, clang's PP bump pointer allocator
allocates 120002016 bytes, and uses only roughly 120 bytes per #define.

For an example test file that we have internally with 7.8 million #define directives, this
change produces the following improvement on arm64 macOS: Persistent allocation footprint for
this test case file as it's being compiled to LLVM IR went down 22% from 5.28 GB to 4.07 GB
and the total allocations went down 14% from 8.26 GB to 7.05 GB. Furthermore, this change
reduced the total number of allocations made by the system for this clang invocation from
1454853 to 133663, an order of magnitude improvement.

The recommit fixes the LLDB build failure.

Differential Revision: https://reviews.llvm.org/D117348

show more ...

[lldb] Rename Logging.h to LLDBLog.h and clean up includes

Most of our code was including Log.h even though that is not where the
"lldb" log channel is defined (Log.h defines the generic logging
inf

[lldb] Rename Logging.h to LLDBLog.h and clean up includes

Most of our code was including Log.h even though that is not where the
"lldb" log channel is defined (Log.h defines the generic logging
infrastructure). This worked because Log.h included Logging.h, even
though it should.

After the recent refactor, it became impossible the two files include
each other in this direction (the opposite inclusion is needed), so this
patch removes the workaround that was put in place and cleans up all
files to include the right thing. It also renames the file to LLDBLog to
better reflect its purpose.

show more ...

[lldb] Convert "LLDB" log channel to the new API

[lldb] Replace default bodies of special member functions with = default;

Replace default bodies of special member functions with = default;

$ run-clang-tidy.py -header-filter='lldb' -checks='-*,mo

[lldb] Replace default bodies of special member functions with = default;

Replace default bodies of special member functions with = default;

$ run-clang-tidy.py -header-filter='lldb' -checks='-*,modernize-use-equals-default' -fix ,

https://clang.llvm.org/extra/clang-tidy/checks/modernize-use-equals-default.html

Differential revision: https://reviews.llvm.org/D104041

show more ...

[clang][PATCH][nfc] Refactor TargetInfo::adjust to pass DiagnosticsEngine to allow diagnostics on target-unsupported options

Reviewed By: aaron.ballman

Differential Revision: https://reviews.llvm.o

[clang][PATCH][nfc] Refactor TargetInfo::adjust to pass DiagnosticsEngine to allow diagnostics on target-unsupported options

Reviewed By: aaron.ballman

Differential Revision: https://reviews.llvm.org/D104729

show more ...

Revert "[clang][PATCH][nfc] Refactor TargetInfo::adjust to pass DiagnosticsEngine to allow diagnostics on target-unsupported options"

This reverts commit 2dbe1c675fe94eeb7973dcc25b049d25f4ca4fa0.
Mo

Revert "[clang][PATCH][nfc] Refactor TargetInfo::adjust to pass DiagnosticsEngine to allow diagnostics on target-unsupported options"

This reverts commit 2dbe1c675fe94eeb7973dcc25b049d25f4ca4fa0.
More buildbot failures

show more ...

[clang][PATCH][nfc] Refactor TargetInfo::adjust to pass DiagnosticsEngine to allow diagnostics on target-unsupported options

Reviewed By: aaron.ballman

Differential Revision: https://reviews.llvm.o

[clang][PATCH][nfc] Refactor TargetInfo::adjust to pass DiagnosticsEngine to allow diagnostics on target-unsupported options

Reviewed By: aaron.ballman

Differential Revision: https://reviews.llvm.org/D104729

show more ...

[lldb][NFC] Code style fixes for ClangModulesDeclVendor

Doxygenify comments and remove {} around one line if's.

[lldb][NFC] std::set -> DenseSet in ClangModulesDeclVendor

ModuleID is just an integer so let's use a DenseSet.

Revert "Lex: Migrate HeaderSearch::LoadedModuleMaps to FileEntryRef"

This reverts commit a40db5502b2515a6f2f1676b5d7a655ae0f41179.
and follow-up d636b881bb9214938973098a012fad453082c444

Somewhat sp

Revert "Lex: Migrate HeaderSearch::LoadedModuleMaps to FileEntryRef"

This reverts commit a40db5502b2515a6f2f1676b5d7a655ae0f41179.
and follow-up d636b881bb9214938973098a012fad453082c444

Somewhat speculative, likely broke check-clang on Windows:
https://reviews.llvm.org/D92975#2453482

show more ...

Adapt lldb to a40db5502b2515a6f2f1676b5d7a655ae0f41179

The bots just told me about a place in LLDB I missed in
a40db5502b2515a6f2f1676b5d7a655ae0f41179 when changing
`HeaderSearch::LoadedModuleMaps`

Adapt lldb to a40db5502b2515a6f2f1676b5d7a655ae0f41179

The bots just told me about a place in LLDB I missed in
a40db5502b2515a6f2f1676b5d7a655ae0f41179 when changing
`HeaderSearch::LoadedModuleMaps`, but I think this will fix it.

show more ...

[lldb] Add support for using variables with C++ keywords names in non-C++ expressions

LLDB is currently always activating C++ when parsing expressions as LLDB itself
is using C++ features when creat

[lldb] Add support for using variables with C++ keywords names in non-C++ expressions

LLDB is currently always activating C++ when parsing expressions as LLDB itself
is using C++ features when creating the final AST that will be codegen'd
(specifically, references to variables, namespaces and using declarations are
used).

This is causing problems for users that have variables in non-C++ programs (e.g.
plain C or Objective-C) that have names which are keywords in C++. Expressions
referencing those variables fail to parse as LLDB's Clang parser thinks those
identifiers are C++ keywords and not identifiers that may belong to a
declaration.

We can't just disable C++ in the expression parser for those situations as
replacing the functionality of the injected C++ code isn't trivial. So this
patch is just disabling most keywords that are exclusive to C++ in LLDB's Clang
parser when we are in a non-C++ expression. There are a few keywords we can't
disable for now:

* `using` as that's currently used in some situations to inject variables into the expression function.
* `__null` as that's used by LLDB to define `NULL`/`Nil`/`nil`.

Getting rid of these last two keywords is possible but is a large enough change
that this will be handled in follow up patches.

Note that this only changes the keyword status of those tokens but this patch
does not remove any C++ functionality from the expression parser. The type
system still follows C++ rules and so does the rest of the expression parser.

There is another small change that gives the hardcoded macro definitions in LLDB
a higher precedence than the macros imported from the Objective-C modules. The
reason for this is that the Objective-C modules in LLDB are actually parsed in
Objective-C++ mode and they end up providing the C++ definitions of certain
system macros (like `NULL` being defined as `nullptr`). So we have to move the
LLDB definition forward and surround the definition from the module with an
`#ifdef` to make sure that we use the correct LLDB definition that doesn't
reference C++ keywords. Or to give an example, this is how the expression source
code changes:

Before:
```
#define NULL (nullptr) // injected module definition
#ifndef NULL
#define NULL (__null) // hardcoded LLDB definition
#endif
```

After:
```
#ifndef NULL
#define NULL (__null) // hardcoded LLDB definition
#endif
#ifndef NULL
#define NULL (nullptr) // injected module definition
#endif
```

Fixes rdar://10356912

Reviewed By: shafik

Differential Revision: https://reviews.llvm.org/D82770

show more ...

[lldb] Extract reproducer providers & co into their own header.

Extract all the provider related logic from Reproducer.h and move it
into its own header ReproducerProvider.h. These classes are seein

[lldb] Extract reproducer providers & co into their own header.

Extract all the provider related logic from Reproducer.h and move it
into its own header ReproducerProvider.h. These classes are seeing most
of the development these days and this reorganization reduces
incremental compilation from ~520 to ~110 files when making changes to
the new header.

show more ...

[lldb] Use std::make_unique<> (NFC)

Update the rest of lldb to use std::make_unique<>. I used clang-tidy to
automate this, which probably missed cases that are wrapped in ifdefs.

[lldb] Replace calls to new with std::make_shared<> (NFC)

[lldb] Print full Clang diagnostics when the ClangModulesDeclVendor fails to compile a module

Summary:
When the ClangModulesDeclVendor currently fails it just prints very basic and often incomplete

[lldb] Print full Clang diagnostics when the ClangModulesDeclVendor fails to compile a module

Summary:
When the ClangModulesDeclVendor currently fails it just prints very basic and often incomplete diagnostics without any source locations:
```
(lldb) p @import Foundation
error: while importing modules:
'foo/bar.h' file not found
could not build module 'Darwin'
[...]
```
or even just
```
(lldb) p @import Foundation
error: while importing modules:
could not build module 'Darwin'
[...]
```

These diagnostics help neither the user nor us with figuring out what is the reason for the failure.

This patch wires up a full TextDiagnosticPrinter in the ClangModulesDeclVendor and makes
sure we always return the error stream to the user when we fail to compile our modules.

Fixes rdar://63216849

Reviewers: aprantl, jdoerfert

Reviewed By: aprantl

Subscribers: JDevlieghere

Differential Revision: https://reviews.llvm.org/D79947

show more ...

[lldb] Move clang-based files out of Symbol

Summary:
This change represents the move of ClangASTImporter, ClangASTMetadata,
ClangExternalASTSourceCallbacks, ClangUtil, CxxModuleHandler, and
TypeSyst

[lldb] Move clang-based files out of Symbol

Summary:
This change represents the move of ClangASTImporter, ClangASTMetadata,
ClangExternalASTSourceCallbacks, ClangUtil, CxxModuleHandler, and
TypeSystemClang from lldbSource to lldbPluginExpressionParserClang.h

This explicitly removes knowledge of clang internals from lldbSymbol,
moving towards a more generic core implementation of lldb.

Reviewers: JDevlieghere, davide, aprantl, teemperor, clayborg, labath, jingham, shafik

Subscribers: emaste, mgorny, arphaman, jfb, usaxena95, lldb-commits

Tags: #lldb

Differential Revision: https://reviews.llvm.org/D73661

show more ...

[lldb][NFC] Add safe Decl->CompilerDecl conversion function TypeSystemClang

This adds a conversion function from clang::Decl to CompilerDecl. It checks
that the TypeSystemClang in the CompilerDecl a

[lldb][NFC] Add safe Decl->CompilerDecl conversion function TypeSystemClang

This adds a conversion function from clang::Decl to CompilerDecl. It checks
that the TypeSystemClang in the CompilerDecl actually fits to the clang::Decl
AST during creation, thus preventing the creation of CompilerDecl instances with
inconsistent state.

show more ...

Make llvm::StringRef to std::string conversions explicit.

This is how it should've been and brings it more in line with
std::string_view. There should be no functional change here.

This is mostly m

Make llvm::StringRef to std::string conversions explicit.

This is how it should've been and brings it more in line with
std::string_view. There should be no functional change here.

This is mostly mechanical from a custom clang-tidy check, with a lot of
manual fixups. It uncovers a lot of minor inefficiencies.

This doesn't actually modify StringRef yet, I'll do that in a follow-up.

show more ...

[lldb][NFC] Fix all formatting errors in .cpp file headers

Summary:
A *.cpp file header in LLDB (and in LLDB) should like this:
```
//===-- TestUtilities.cpp ----------------------------------------

[lldb][NFC] Fix all formatting errors in .cpp file headers

Summary:
A *.cpp file header in LLDB (and in LLDB) should like this:
```
//===-- TestUtilities.cpp -------------------------------------------------===//
```
However in LLDB most of our source files have arbitrary changes to this format and
these changes are spreading through LLDB as folks usually just use the existing
source files as templates for their new files (most notably the unnecessary
editor language indicator `-*- C++ -*-` is spreading and in every review
someone is pointing out that this is wrong, resulting in people pointing out that this
is done in the same way in other files).

This patch removes most of these inconsistencies including the editor language indicators,
all the different missing/additional '-' characters, files that center the file name, missing
trailing `===//` (mostly caused by clang-format breaking the line).

Reviewers: aprantl, espindola, jfb, shafik, JDevlieghere

Reviewed By: JDevlieghere

Subscribers: dexonsmith, wuzish, emaste, sdardis, nemanjai, kbarton, MaskRay, atanasyan, arphaman, jfb, abidh, jsji, JDevlieghere, usaxena95, lldb-commits

Tags: #lldb

Differential Revision: https://reviews.llvm.org/D73258

show more ...