[lldb] Add image dump pcm-info command

Add `pcm-info` to the `target module dump` subcommands.

This dump command shows information about clang .pcm files. This command
effectively runs `clang -modu

[lldb] Add image dump pcm-info command

Add `pcm-info` to the `target module dump` subcommands.

This dump command shows information about clang .pcm files. This command
effectively runs `clang -module-file-info` and produces identical output.

The .pcm file format is tightly coupled to the clang version. The clang
embedded in lldb is not guaranteed to match the version of the clang executable
available on the local system.

There have been times when I've needed to view the details about a .pcm file
produced by lldb's embedded clang, but because the clang executable was a
slightly different version, the `-module-file-info` invocation failed. With
this command, users can inspect .pcm files generated by lldb too.

Differential Revision: https://reviews.llvm.org/D129456

show more ...

[Tooling/DependencyScanning & Preprocessor] Refactor dependency scanning to produce pre-lexed preprocessor directive tokens, instead of minimized sources

This is a commit with the following changes:

[Tooling/DependencyScanning & Preprocessor] Refactor dependency scanning to produce pre-lexed preprocessor directive tokens, instead of minimized sources

This is a commit with the following changes:

* Remove `ExcludedPreprocessorDirectiveSkipMapping` and related functionality

Removes `ExcludedPreprocessorDirectiveSkipMapping`; its intended benefit for fast skipping of excluded directived blocks
will be superseded by a follow-up patch in the series that will use dependency scanning lexing for the same purpose.

* Refactor dependency scanning to produce pre-lexed preprocessor directive tokens, instead of minimized sources

Replaces the "source minimization" mechanism with a mechanism that produces lexed dependency directives tokens.

* Make the special lexing for dependency scanning a first-class feature of the `Preprocessor` and `Lexer`

This is bringing the following benefits:

* Full access to the preprocessor state during dependency scanning. E.g. a component can see what includes were taken and where they were located in the actual sources.
* Improved performance for dependency scanning. Measurements with a release+thin-LTO build shows ~ -11% reduction in wall time.
* Opportunity to use dependency scanning lexing to speed-up skipping of excluded conditional blocks during normal preprocessing (as follow-up, not part of this patch).

For normal preprocessing measurements show differences are below the noise level.

Since, after this change, we don't minimize sources and pass them in place of the real sources, `DependencyScanningFilesystem` is not technically necessary, but it has valuable performance benefits for caching file `stat`s along with the results of scanning the sources. So the setup of using the `DependencyScanningFilesystem` during a dependency scan remains.

Differential Revision: https://reviews.llvm.org/D125486
Differential Revision: https://reviews.llvm.org/D125487
Differential Revision: https://reviews.llvm.org/D125488

show more ...

[Tooling/DependencyScanning] Rename refactorings towards transitioning dependency scanning to use pre-lexed preprocessor directive tokens

This is first of a series of patches for making the special

[Tooling/DependencyScanning] Rename refactorings towards transitioning dependency scanning to use pre-lexed preprocessor directive tokens

This is first of a series of patches for making the special lexing for dependency scanning a first-class feature of the `Preprocessor` and `Lexer`.
This patch only includes NFC renaming changes to make reviewing of the functionality changing parts easier.

Differential Revision: https://reviews.llvm.org/D125484

show more ...

Reimplement `__builtin_dump_struct` in Sema.

Compared to the old implementation:

* In C++, we only recurse into aggregate classes.
* Unnamed bit-fields are not printed.
* Constant evaluation is sup

Reimplement `__builtin_dump_struct` in Sema.

Compared to the old implementation:

* In C++, we only recurse into aggregate classes.
* Unnamed bit-fields are not printed.
* Constant evaluation is supported.
* Proper conversion is done when passing arguments through `...`.
* Additional arguments are supported and are injected prior to the
format string; this directly supports use with `fprintf`, for example.
* An arbitrary callable can be passed rather than only a function
pointer. In particular, in C++, a function template or overload set is
acceptable.
* All text generated by Clang is printed via `%s` rather than directly;
this avoids issues where Clang's pretty-printing output might itself
contain a `%` character.
* Fields of types that we don't know how to print are printed with a
`"*%p"` format and passed by address to the print function.
* No return value is produced.

Reviewed By: aaron.ballman, erichkeane, yihanaa

Differential Revision: https://reviews.llvm.org/D124221

show more ...

Add HLSL Language Option and Preprocessor

Bringing in HLSL as a language as well as language options for each of
the HLSL language standards.

While the HLSL language is unimplemented, this patch ad

Add HLSL Language Option and Preprocessor

Bringing in HLSL as a language as well as language options for each of
the HLSL language standards.

While the HLSL language is unimplemented, this patch adds the
HLSL-specific preprocessor defines which enables testing of the command
line options through the driver.

Reviewed By: pete, rnk

Differential Revision: https://reviews.llvm.org/D122087

show more ...

[C++20][Modules][HU 1/5] Introduce header units as a module type.

This is the first in a series of patches that introduce C++20 importable
header units.

These differ from clang header modules in th

[C++20][Modules][HU 1/5] Introduce header units as a module type.

This is the first in a series of patches that introduce C++20 importable
header units.

These differ from clang header modules in that:
(a) they are identifiable by an internal name
(b) they represent the top level source for a single header - although
that might include or import other headers.

We name importable header units with the path by which they are specified
(although that need not be the absolute path for the file).

So "foo/bar.h" would have a name "foo/bar.h". Header units are made a
separate module type so that we can deal with diagnosing places where they
are permitted but a named module is not.

Differential Revision: https://reviews.llvm.org/D121095

show more ...

Silence some "not all control paths return a value" warnings; NFC

[Modules] Add module structure output to -module-file-info.

It is useful to be able to visualise the C++20 modules content of a PCM file
both for inspection and for testing. In particular, when add

[Modules] Add module structure output to -module-file-info.

It is useful to be able to visualise the C++20 modules content of a PCM file
both for inspection and for testing. In particular, when adding more module
types to support C++20 Partitions and Header Units, we would like to be able
to confirm that the output PCM has the intended structure.

The existing scheme for dumping data is restricted to the content of the AST
file control block, which does not include structural data beyond imports.

The change here makes use of the AST unit that is set up by BeginSourceFile
to query for the information on the primary and sub-modules. We can then
inspect each of these in turn, accounting for Global, Private, Imported and
Exported modules/fragments and then showing the sub-stucture of the main
module(s).

The disadvantage of this mechanism is that it has no easy method to control
the granularity of the output. Perhaps more detailed inspection would be
better handled by a stand-alone module inspection tool.

Differential Revision: https://reviews.llvm.org/D119823

show more ...

[Templight] Don't display empty strings for names of unnamed template parameters

Patch originally by oktal3000: https://github.com/mikael-s-persson/templight/pull/40

When a template parameter is un

[Templight] Don't display empty strings for names of unnamed template parameters

Patch originally by oktal3000: https://github.com/mikael-s-persson/templight/pull/40

When a template parameter is unnamed, the name of -templight-dump might return
an empty string. This is fine, they are unnamed after all, but it might be more
user friendly to at least describe what entity is unnamed.

Differential Revision: https://reviews.llvm.org/D115521

show more ...

[clang][lex] NFC: Simplify calls to `LookupFile`

The `{HeaderSearch,Preprocessor}::LookupFile()` functions take an out-parameter `const DirectoryLookup *&`. Most callers end up creating a `const Dir

[clang][lex] NFC: Simplify calls to `LookupFile`

The `{HeaderSearch,Preprocessor}::LookupFile()` functions take an out-parameter `const DirectoryLookup *&`. Most callers end up creating a `const DirectoryLookup *` variable that's otherwise unused.

This patch changes the out-parameter from reference to a pointer, making it possible to simply pass `nullptr` to the function without the ceremony.

Reviewed By: ahoppen

Differential Revision: https://reviews.llvm.org/D117312

show more ...

[clang][NFC] Inclusive terms: replace some uses of sanity in clang

Rewording of comments to avoid using `sanity test, sanity check`.

Reviewed By: aaron.ballman, Quuxplusone

Differential Revision:

[clang][NFC] Inclusive terms: replace some uses of sanity in clang

Rewording of comments to avoid using `sanity test, sanity check`.

Reviewed By: aaron.ballman, Quuxplusone

Differential Revision: https://reviews.llvm.org/D114025

show more ...

[clang-scan-deps] Add an API for clang dependency scanner to perform
module lookup by name alone

This removes the need to create a fake source file that imports a
module.

rdar://64538073

Different

[clang-scan-deps] Add an API for clang dependency scanner to perform
module lookup by name alone

This removes the need to create a fake source file that imports a
module.

rdar://64538073

Differential Revision: https://reviews.llvm.org/D109485

show more ...

Frontend: Respect -fno-temp-file when creating a PCH

When creating a PCH file the use of a temp file will be dictated by the
presence or absence of the -fno-temp-file flag. Creating a module file
wi

Frontend: Respect -fno-temp-file when creating a PCH

When creating a PCH file the use of a temp file will be dictated by the
presence or absence of the -fno-temp-file flag. Creating a module file
will always use a temp file via the new ForceUseTemporary flag.

This fixes bug 50033.

show more ...

[clang][deps] Handle precompiled headers' AST files

The `PreprocessOnlyAction` doesn't support loading the AST file of a precompiled header. This is problematic for dependency scanning, since the `#

[clang][deps] Handle precompiled headers' AST files

The `PreprocessOnlyAction` doesn't support loading the AST file of a precompiled header. This is problematic for dependency scanning, since the `#include` manufactured for the PCH is treated as textual. This means the PCH contents get scanned with each TU, which is redundant. Moreover, dependencies of the PCH end up being considered dependency of the TU.

To handle AST file of PCH properly, this patch creates new `FrontendAction` that behaves the same way `PreprocessorOnlyAction` does, but treats the manufactured PCH `#include` as a normal compilation would (by not claiming it only uses a preprocessor and creating the default AST consumer).

The AST file is now reported as a file dependency of the TU.

Depends on D103519.

Reviewed By: Bigcheese

Differential Revision: https://reviews.llvm.org/D103524

show more ...

[clang][modules] Build inferred modules

This patch enables explicitly building inferred modules.

Effectively a cherry-pick of https://github.com/apple/llvm-project/pull/699 authored by @Bigcheese w

[clang][modules] Build inferred modules

This patch enables explicitly building inferred modules.

Effectively a cherry-pick of https://github.com/apple/llvm-project/pull/699 authored by @Bigcheese with libclang and dependency scanner changes omitted.

Contains the following changes:

1. [Clang] Fix the header paths in clang::Module for inferred modules.
* The UmbrellaAsWritten and NameAsWritten fields in clang::Module are a lie for framework modules. For those they actually are the path to the header or umbrella relative to the clang::Module::Directory.
* The exception to this case is for inferred modules. Here it actually is the name as written, because we print out the module and read it back in when implicitly building modules. This causes a problem when explicitly building an inferred module, as we skip the printing out step.
* In order to fix this issue this patch adds a new field for the path we want to use in getInputBufferForModule. It also makes NameAsWritten actually be the name written in the module map file (or that would be, in the case of an inferred module).

2. [Clang] Allow explicitly building an inferred module.
* Building the actual module still fails, but make sure it fails for the right reason.

Split from D100934.

Reviewed By: dexonsmith

Differential Revision: https://reviews.llvm.org/D102491

show more ...

[SystemZ][z/OS][Windows] Add new OF_TextWithCRLF flag and use this flag instead of OF_Text

Problem:
On SystemZ we need to open text files in text mode. On Windows, files opened in text mode adds a C

[SystemZ][z/OS][Windows] Add new OF_TextWithCRLF flag and use this flag instead of OF_Text

Problem:
On SystemZ we need to open text files in text mode. On Windows, files opened in text mode adds a CRLF '\r\n' which may not be desirable.

Solution:
This patch adds two new flags

- OF_CRLF which indicates that CRLF translation is used.
- OF_TextWithCRLF = OF_Text | OF_CRLF indicates that the file is text and uses CRLF translation.

Developers should now use either the OF_Text or OF_TextWithCRLF for text files and OF_None for binary files. If the developer doesn't want carriage returns on Windows, they should use OF_Text, if they do want carriage returns on Windows, they should use OF_TextWithCRLF.

So this is the behaviour per platform with my patch:

z/OS:
OF_None: open in binary mode
OF_Text : open in text mode
OF_TextWithCRLF: open in text mode

Windows:
OF_None: open file with no carriage return
OF_Text: open file with no carriage return
OF_TextWithCRLF: open file with carriage return

The Major change is in llvm/lib/Support/Windows/Path.inc to only set text mode if the OF_CRLF is set.
```
if (Flags & OF_CRLF)
CrtOpenFlags |= _O_TEXT;
```

These following files are the ones that still use OF_Text which I left unchanged. I modified all these except raw_ostream.cpp in recent patches so I know these were previously in Binary mode on Windows.
./llvm/lib/Support/raw_ostream.cpp
./llvm/lib/TableGen/Main.cpp
./llvm/tools/dsymutil/DwarfLinkerForBinary.cpp
./llvm/unittests/Support/Path.cpp
./clang/lib/StaticAnalyzer/Core/HTMLDiagnostics.cpp
./clang/lib/Frontend/CompilerInstance.cpp
./clang/lib/Driver/Driver.cpp
./clang/lib/Driver/ToolChains/Clang.cpp

Reviewed By: MaskRay

Differential Revision: https://reviews.llvm.org/D99426

show more ...

[OpenCL] Added distinct file extension for C++ for OpenCL.

Files compiled with C++ for OpenCL mode can now have a distinct
file extension - clcpp, then clang driver picks the compilation
mode automa

[OpenCL] Added distinct file extension for C++ for OpenCL.

Files compiled with C++ for OpenCL mode can now have a distinct
file extension - clcpp, then clang driver picks the compilation
mode automatically (-x clcpp) without the use of -cl-std=clc++.

Differential Revision: https://reviews.llvm.org/D96771

show more ...

[SystemZ][z/OS] Distinguish between text and binary files on z/OS

This patch consists of the initial changes to help distinguish between text and binary content correctly on z/OS. I would like to ge

[SystemZ][z/OS] Distinguish between text and binary files on z/OS

This patch consists of the initial changes to help distinguish between text and binary content correctly on z/OS. I would like to get feedback from Windows users on setting OF_None for all ToolOutputFiles. This seems to have been done as an optimization to prevent CRLF translation on Windows in the past.

Reviewed By: zibi

Differential Revision: https://reviews.llvm.org/D97785

show more ...

Revert "Module: Use FileEntryRef and DirectoryEntryRef in Umbrella, Header, and DirectoryName, NFC"

This (mostly) reverts 32c501dd88b62787d3a5ffda7aabcf4650dbe3cd. Hit a
case where this causes a be

Revert "Module: Use FileEntryRef and DirectoryEntryRef in Umbrella, Header, and DirectoryName, NFC"

This (mostly) reverts 32c501dd88b62787d3a5ffda7aabcf4650dbe3cd. Hit a
case where this causes a behaviour change, perhaps the same root cause
that triggered the revert of a40db5502b2515a6f2f1676b5d7a655ae0f41179 in
7799ef7121aa7d59f4bd95cdf70035de724ead6f.

(The API changes in DirectoryEntry.h have NOT been reverted as a number
of subsequent commits depend on those.)

https://reviews.llvm.org/D90497#2582166

show more ...

Frontend: Fix layering between create{,Default}OutputFile, NFC

Fix layering between `CompilerInstance::createDefaultOutputFile` and the
two versions of `createOutputFile`.

- Add missing configurati

Frontend: Fix layering between create{,Default}OutputFile, NFC

Fix layering between `CompilerInstance::createDefaultOutputFile` and the
two versions of `createOutputFile`.

- Add missing configuration flags to `createDefaultOutputFile` so that
GeneratePCHAction and GenerateModuleFromModuleMapAction can use it.
They previously promised that temporary files were turned on; now
`createDefaultOutputFile` handles that logic.
- Lift the logic handling `InFile` and `Extension` to
`createDefaultOutputFile`, since it's only the callers of that
function that are using it.
- Rename the deeper of the two `createOutputFile`s to
`createOutputFileImpl` and make it private to `CompilerInstance` (to
prove that no one else is using it).
- Sink the logic for adding to `CompilerInstance::OutputFiles` down to
`createOutputFileImpl`, allowing two "optional" (but always used)
`std::string*` out parameters to be removed.
- Instead of passing a `std::error_code` out parameter into
`createOutputFileImpl`, have it return `Expected<>`.
- As a drive-by, inline `CompilerInstance::addOutputFile` into its only
caller, `createOutputFileImpl`.

Clean layering makes it easier for a future commit to extract
`createOutputFileImpl` out of `CompilerInstance`.

Differential Revision: https://reviews.llvm.org/D93248

show more ...

[ASTReader] Allow controlling separately whether validation should be disabled for a PCH vs a module file

This addresses an issue with how the PCH preable works, specifically:

1. When using a PCH/p

[ASTReader] Allow controlling separately whether validation should be disabled for a PCH vs a module file

This addresses an issue with how the PCH preable works, specifically:

1. When using a PCH/preamble the module hash changes and a different cache directory is used
2. When the preamble is used, PCH & PCM validation is disabled.

Due to combination of #1 and #2, reparsing with preamble enabled can end up loading a stale module file before a header change and using it without updating it because validation is disabled and it doesn’t check that the header has changed and the module file is out-of-date.

rdar://72611253

Differential Revision: https://reviews.llvm.org/D95159

show more ...

Module: Use FileEntryRef and DirectoryEntryRef in Umbrella, Header, and DirectoryName, NFC

Push `FileEntryRef` and `DirectoryEntryRef` further, using it them
`Module::Umbrella`, `Module::Header::Ent

Module: Use FileEntryRef and DirectoryEntryRef in Umbrella, Header, and DirectoryName, NFC

Push `FileEntryRef` and `DirectoryEntryRef` further, using it them
`Module::Umbrella`, `Module::Header::Entry`, and
`Module::DirectoryName::Entry`.

- Add `DirectoryEntryRef::operator const DirectoryEntry *` and
`OptionalDirectoryEntryRefDegradesToDirectoryEntryPtr`, to get the
same "degrades to `DirectoryEntry*` behaviour `FileEntryRef` enjoys
(this avoids a bunch of churn in various clang tools).
- Fix the `DirectoryEntryRef` constructor from `MapEntry` to take it by
`const&`.

Note that we cannot get rid of the `...AsWritten` names leveraging the
new classes, since these need to be as written in the `ModuleMap` file
and the module directory path is preprended for the lookup in the
`FileManager`.

Differential Revision: https://reviews.llvm.org/D90497

show more ...

[Frontend] Add flag to allow PCM generation despite compiler errors

As with precompiled headers, it's useful for indexers to be able to
continue through compiler errors in dependent modules.

Resolv

[Frontend] Add flag to allow PCM generation despite compiler errors

As with precompiled headers, it's useful for indexers to be able to
continue through compiler errors in dependent modules.

Resolves rdar://69816264

Reviewed By: akyrtzi

Differential Revision: https://reviews.llvm.org/D91580

show more ...

Suppress printing template arguments that match default template
arguments of types by default.

This somewhat improves the worst-case printing of types like
std::string, std::vector, etc., where man

Suppress printing template arguments that match default template
arguments of types by default.

This somewhat improves the worst-case printing of types like
std::string, std::vector, etc., where many irrelevant default arguments
can be included in the type as printed if we've lost the type sugar.

show more ...

clang/Frontend: Use MemoryBufferRef in FrontendInputFile (and remove SourceManager::getBuffer)

In order to drop the final callers to `SourceManager::getBuffer`, change
`FrontendInputFile` to use `Op

clang/Frontend: Use MemoryBufferRef in FrontendInputFile (and remove SourceManager::getBuffer)

In order to drop the final callers to `SourceManager::getBuffer`, change
`FrontendInputFile` to use `Optional<MemoryBufferRef>`. Also updated
the "unowned" version of `SourceManager::createFileID` to take a
`MemoryBufferRef` (it now calls `MemoryBuffer::getMemBuffer`, which
creates a `MemoryBuffer` that does not own the buffer data).

Differential Revision: https://reviews.llvm.org/D89427

show more ...