[clang] Require strict matches for defines for PCH in GCC style directories

When clang includes a PCH, it tolerates some amount of differences
between the defines used when creating and when includi

[clang] Require strict matches for defines for PCH in GCC style directories

When clang includes a PCH, it tolerates some amount of differences
between the defines used when creating and when including the PCH
- this seems to be intentionally allowed in
c379c072405f39bca1d3552408fc0427328e8b6d (and later extended in
b63687519610a73dd565be1fec28332211b4df5b).

When using a PCH (or when picking a PCH out of a directory containing
multiple candidates) Clang used to accept the header if there were
defines on the command line when creating the PCH that are missing
when using the PCH, or vice versa, defines only set when using the
PCH.

The only cases where Clang explicitly rejected the use of a PCH
is if there was an explicit conflict between the options, e.g.
-DFOO=1 vs -DFOO=2, or -DFOO vs -UFOO.

The latter commit added a FIXME that we really should check whether
mismatched defines actually were used somewhere in the PCH, so that
the define would affect the outcome. This FIXME has stood unaddressed
since 2012.

This differs from GCC, which rejects PCH files if the defines differ
at all.

When explicitly including a single PCH file, the relaxed policy
of allowing minor differences is harmless for correct use cases
(but may fail to diagnose mismtaches), and potentially allow using
PCHs in wider cases (where the user intentionally know that the
differences in defines are harmless for the PCH).

However, for GCC style PCH directories, with a directory containing
multiple PCH variants and the compiler should pick the correct match
out of them, Clang's relaxed logic was problematic. The directory
could contain two otherwise identical PCHs, but one built with -DFOO
and one without. When attempting to include a PCH and iterating over
the candidates in the directory, Clang would essentially pick the
first one out of the two, even if there existed a better, exact
match in the directory.

Keep the relaxed checking when specificlly including one named
PCH file, but require strict matches when trying to pick the right
candidate out of a GCC style directory with alternatives.

This fixes https://github.com/lhmouse/mcfgthread/issues/63.

Differential Revision: https://reviews.llvm.org/D126676

(cherry picked from commit c843c921a1a385bb805b2338d980436c94f83f19)

show more ...

Use the range-based overload of llvm::sort where possible

Reviewed By: MaskRay

Differential Revision: https://reviews.llvm.org/D130403

[clang] Cleanup ASTContext before output files in crash recovery for modules

When we recover from a crash in a module compilation thread, we need to
ensure any output streams owned by the ASTConsume

[clang] Cleanup ASTContext before output files in crash recovery for modules

When we recover from a crash in a module compilation thread, we need to
ensure any output streams owned by the ASTConsumer (e.g. in
RawPCHContainerGenerator) are deleted before we call clearOutputFiles().
This has the same theoretical issues with proxy streams that Duncan
discusses in the commit 2d133867833fe8eb. In practice, this was observed
as a use-after-free crash on a downstream branch that uses such a proxy
stream in this code path. Add an assertion so it won't regress.

Differential Revision: https://reviews.llvm.org/D129220

rdar://96525032

show more ...

[HLSL] Add ExternalSemaSource & vector alias

HLSL vector types are ext_vector types, but they are also exposed via a
template syntax `vector<T, #>`. This is morally equavalent to the code:

```c++
t

[HLSL] Add ExternalSemaSource & vector alias

HLSL vector types are ext_vector types, but they are also exposed via a
template syntax `vector<T, #>`. This is morally equavalent to the code:

```c++
template <typename T, int Size>
using vector = T __attribute__((ext_vector_type(Size)))
```

The problem is that templates aren't supported before HLSL 2021, and
type aliases still aren't supported in HLSL.

To resolve this (and other issues where HLSL can't represent its own
types), we rely on an external AST & Sema source being registered for
HLSL code.

This patch adds the HLSLExternalSemaSource and registers the vector
type alias.

Depends on D127802

Differential Revision: https://reviews.llvm.org/D128012

show more ...

Prefer `getCurrentFileOrBufferName` in `FrontendAction::EndSourceFile`

`getCurrentFile` here causes an assertion on some condition.
`getCurrentFileOrBufferName` is preferrable instead.

llvm#55950

Prefer `getCurrentFileOrBufferName` in `FrontendAction::EndSourceFile`

`getCurrentFile` here causes an assertion on some condition.
`getCurrentFileOrBufferName` is preferrable instead.

llvm#55950

Differential Revision: https://reviews.llvm.org/D127509

show more ...

Revert "wip"

This reverts commit 0dd243fa8a4ec98d6cabbad16e6b485a093c6dea.

I accidentally pushed this! Oops!

wip

Use llvm::less_first (NFC)

Revert "[llvm][clang][bolt][NFC] Use llvm::less_first() when applicable"

This reverts commit 3988bd13988aad72ec979beb2361e8738584926b.

Did not build on this bot:
https://lab.llvm.org/buildbot#build

Revert "[llvm][clang][bolt][NFC] Use llvm::less_first() when applicable"

This reverts commit 3988bd13988aad72ec979beb2361e8738584926b.

Did not build on this bot:
https://lab.llvm.org/buildbot#builders/215/builds/6372

/usr/include/c++/9/bits/predefined_ops.h:177:11: error: no match for call to
‘(llvm::less_first) (std::pair<long unsigned int, llvm::bolt::BinaryBasicBlock*>&, const std::pair<long unsigned int, std::nullptr_t>&)’
177 | { return bool(_M_comp(*__it, __val)); }

show more ...

[llvm][clang][bolt][NFC] Use llvm::less_first() when applicable

One could reuse this functor instead of rolling out your own version.
There were a couple other cases where the code was similar, but

[llvm][clang][bolt][NFC] Use llvm::less_first() when applicable

One could reuse this functor instead of rolling out your own version.
There were a couple other cases where the code was similar, but not
quite the same, such as it might have an assertion in the lambda or other
constructs. Thus, I've not touched any of those, as it might change the
behavior in some way.

As per https://discourse.llvm.org/t/submitting-simple-nfc-patches/62640/3?u=steakhal
Chris Lattner
> LLVM intentionally has a “yes, you can apply common sense judgement to
> things” policy when it comes to code review. If you are doing mechanical
> patches (e.g. adopting less_first) that apply to the entire monorepo,
> then you don’t need everyone in the monorepo to sign off on it. Having
> some +1 validation from someone is useful, but you don’t need everyone
> whose code you touch to weigh in.

Differential Revision: https://reviews.llvm.org/D126068

show more ...

[clang] NFCI: Use DirectoryEntryRef in FrontendAction::BeginSourceFile()

This patch removes use of the deprecated `DirectoryEntry::getName()` from `FrontendAction::BeginSourceFile()`.

Reviewed By:

[clang] NFCI: Use DirectoryEntryRef in FrontendAction::BeginSourceFile()

This patch removes use of the deprecated `DirectoryEntry::getName()` from `FrontendAction::BeginSourceFile()`.

Reviewed By: bnbarham

Differential Revision: https://reviews.llvm.org/D123853

show more ...

[C++20][Modules][HU 4/5] Handle pre-processed header units.

We wish to support emitting a pre-processed output for an importable
header unit, that can be consumed to produce the same header units as

[C++20][Modules][HU 4/5] Handle pre-processed header units.

We wish to support emitting a pre-processed output for an importable
header unit, that can be consumed to produce the same header units as
the original source.

This means that ee need to find the original filename used to produce
the re-preprocessed output, so that it can be assigned as the module
name. This is peeked from the first line of the pre-processed source
when the action sets up the files.

Differential Revision: https://reviews.llvm.org/D121098

show more ...

[C++20][Modules][HU 2/5] Support searching Header Units in user or system search paths.

This is support for the user-facing options to create importable header units
from headers in the user or syst

[C++20][Modules][HU 2/5] Support searching Header Units in user or system search paths.

This is support for the user-facing options to create importable header units
from headers in the user or system search paths (or to be given an absolute path).

This means that an incomplete header path will be passed by the driver and the
lookup carried out using the search paths present when the front end is run.

To support this, we introduce file fypes for c++-{user,system,header-unit}-header.
These terms are the same as the ones used by GCC, to minimise the differences for
tooling (and users).

The preprocessor checks for headers before issuing a warning for
"#pragma once" in a header build. We ensure that the importable header units
are recognised as headers in order to avoid such warnings.

Differential Revision: https://reviews.llvm.org/D121096

show more ...

[clang] Improve laziness of resolving module map headers.

clang has support for lazy headers in module maps - if size and/or
modtime and provided in the cppmap file, headers are only resolved when
a

[clang] Improve laziness of resolving module map headers.

clang has support for lazy headers in module maps - if size and/or
modtime and provided in the cppmap file, headers are only resolved when
an include directive for a file with that size/modtime is encoutered.

Before this change, the lazy resolution was all-or-nothing per module.
That means as soon as even one file in that module potentially matched
an include, all lazy files in that module were resolved. With this
change, only files with matching size/modtime will be resolved.

The goal is to avoid unnecessary stat() calls on non-included files,
which is especially valuable on networked file systems, with higher
latency.

Differential Revision: https://reviews.llvm.org/D120569

show more ...

[clang][modules] Infer framework modules in explicit builds

This patch enables inferring framework modules in explicit builds in all contexts. Until now, inferring framework modules only worked with

[clang][modules] Infer framework modules in explicit builds

This patch enables inferring framework modules in explicit builds in all contexts. Until now, inferring framework modules only worked with `-fimplicit-module-maps` due to this block of code:

```
// HeaderSearch::loadFrameworkModule
case LMM_InvalidModuleMap:
// Try to infer a module map from the framework directory.
if (HSOpts->ImplicitModuleMaps)
ModMap.inferFrameworkModule(Dir, IsSystem, /*Parent=*/nullptr);
break;
```

Reviewed By: Bigcheese

Differential Revision: https://reviews.llvm.org/D113880

show more ...

[clang] Don't clear AST if we have consumers running after the main action

Downstream users may have Clang plugins. By default these plugins run
after the main action if they are specified on the co

[clang] Don't clear AST if we have consumers running after the main action

Downstream users may have Clang plugins. By default these plugins run
after the main action if they are specified on the command line.

Since these plugins are ASTConsumers, presumably they inspect the AST.
So we shouldn't clear it if any plugins run after the main action.

Reviewed By: dblaikie, hans

Differential Revision: https://reviews.llvm.org/D112190

show more ...

[clang] Add plugin ActionType to run command line plugin before main action

Currently we have a way to run a plugin if specified on the command line
after the main action, and ways to unconditionall

[clang] Add plugin ActionType to run command line plugin before main action

Currently we have a way to run a plugin if specified on the command line
after the main action, and ways to unconditionally run the plugin before
or after the main action, but no way to run a plugin if specified on the
command line before the main action.

This introduces the missing option.

This is helpful because -clear-ast-before-backend clears the AST before
codegen, while some plugins may want access to the AST.

Reviewed By: dblaikie

Differential Revision: https://reviews.llvm.org/D112096

show more ...

[clang][modules] NFC: Propagate import `SourceLocation` into `HeaderSearch::lookupModule`

This patch propagates the import `SourceLocation` into `HeaderSearch::lookupModule`. This enables remarks on

[clang][modules] NFC: Propagate import `SourceLocation` into `HeaderSearch::lookupModule`

This patch propagates the import `SourceLocation` into `HeaderSearch::lookupModule`. This enables remarks on search path usage (implemented in D102923) to point to the source code that initiated header search.

Reviewed By: dexonsmith

Differential Revision: https://reviews.llvm.org/D111557

show more ...

[NFC] `goto fail` has failed us in the past...

This patch replaces reliance on `goto failure` pattern with
`llvm::scope_exit`.

Reviewed By: bkramer

Differential Revision: https://reviews.llvm.org/

[NFC] `goto fail` has failed us in the past...

This patch replaces reliance on `goto failure` pattern with
`llvm::scope_exit`.

Reviewed By: bkramer

Differential Revision: https://reviews.llvm.org/D109865

show more ...

[clang][modules] Build inferred modules

This patch enables explicitly building inferred modules.

Effectively a cherry-pick of https://github.com/apple/llvm-project/pull/699 authored by @Bigcheese w

[clang][modules] Build inferred modules

This patch enables explicitly building inferred modules.

Effectively a cherry-pick of https://github.com/apple/llvm-project/pull/699 authored by @Bigcheese with libclang and dependency scanner changes omitted.

Contains the following changes:

1. [Clang] Fix the header paths in clang::Module for inferred modules.
* The UmbrellaAsWritten and NameAsWritten fields in clang::Module are a lie for framework modules. For those they actually are the path to the header or umbrella relative to the clang::Module::Directory.
* The exception to this case is for inferred modules. Here it actually is the name as written, because we print out the module and read it back in when implicitly building modules. This causes a problem when explicitly building an inferred module, as we skip the printing out step.
* In order to fix this issue this patch adds a new field for the path we want to use in getInputBufferForModule. It also makes NameAsWritten actually be the name written in the module map file (or that would be, in the case of an inferred module).

2. [Clang] Allow explicitly building an inferred module.
* Building the actual module still fails, but make sure it fails for the right reason.

Split from D100934.

Reviewed By: dexonsmith

Differential Revision: https://reviews.llvm.org/D102491

show more ...

[clang-repl] Recommit "Land initial infrastructure for incremental parsing"

Original commit message:

In http://lists.llvm.org/pipermail/llvm-dev/2020-July/143257.html we have
mentioned our plan

[clang-repl] Recommit "Land initial infrastructure for incremental parsing"

Original commit message:

In http://lists.llvm.org/pipermail/llvm-dev/2020-July/143257.html we have
mentioned our plans to make some of the incremental compilation facilities
available in llvm mainline.

This patch proposes a minimal version of a repl, clang-repl, which enables
interpreter-like interaction for C++. For instance:

./bin/clang-repl
clang-repl> int i = 42;
clang-repl> extern "C" int printf(const char*,...);
clang-repl> auto r1 = printf("i=%d\n", i);
i=42
clang-repl> quit

The patch allows very limited functionality, for example, it crashes on invalid
C++. The design of the proposed patch follows closely the design of cling. The
idea is to gather feedback and gradually evolve both clang-repl and cling to
what the community agrees upon.

The IncrementalParser class is responsible for driving the clang parser and
codegen and allows the compiler infrastructure to process more than one input.
Every input adds to the “ever-growing” translation unit. That model is enabled
by an IncrementalAction which prevents teardown when HandleTranslationUnit.

The IncrementalExecutor class hides some of the underlying implementation
details of the concrete JIT infrastructure. It exposes the minimal set of
functionality required by our incremental compiler/interpreter.

The Transaction class keeps track of the AST and the LLVM IR for each
incremental input. That tracking information will be later used to implement
error recovery.

The Interpreter class orchestrates the IncrementalParser and the
IncrementalExecutor to model interpreter-like behavior. It provides the public
API which can be used (in future) when using the interpreter library.

Differential revision: https://reviews.llvm.org/D96033

show more ...

Revert "[clang-repl] Land initial infrastructure for incremental parsing"

This reverts commit 44a4000181e1a25027e87f2ae4e71cb876a7a275.

We are seeing build failures due to missing dependency to lib

Revert "[clang-repl] Land initial infrastructure for incremental parsing"

This reverts commit 44a4000181e1a25027e87f2ae4e71cb876a7a275.

We are seeing build failures due to missing dependency to libSupport and
CMake Error at tools/clang/tools/clang-repl/cmake_install.cmake
file INSTALL cannot find

show more ...

[clang-repl] Land initial infrastructure for incremental parsing

In http://lists.llvm.org/pipermail/llvm-dev/2020-July/143257.html we have
mentioned our plans to make some of the incremental compila

[clang-repl] Land initial infrastructure for incremental parsing

In http://lists.llvm.org/pipermail/llvm-dev/2020-July/143257.html we have
mentioned our plans to make some of the incremental compilation facilities
available in llvm mainline.

This patch proposes a minimal version of a repl, clang-repl, which enables
interpreter-like interaction for C++. For instance:

./bin/clang-repl
clang-repl> int i = 42;
clang-repl> extern "C" int printf(const char*,...);
clang-repl> auto r1 = printf("i=%d\n", i);
i=42
clang-repl> quit

The patch allows very limited functionality, for example, it crashes on invalid
C++. The design of the proposed patch follows closely the design of cling. The
idea is to gather feedback and gradually evolve both clang-repl and cling to
what the community agrees upon.

The IncrementalParser class is responsible for driving the clang parser and
codegen and allows the compiler infrastructure to process more than one input.
Every input adds to the “ever-growing” translation unit. That model is enabled
by an IncrementalAction which prevents teardown when HandleTranslationUnit.

The IncrementalExecutor class hides some of the underlying implementation
details of the concrete JIT infrastructure. It exposes the minimal set of
functionality required by our incremental compiler/interpreter.

The Transaction class keeps track of the AST and the LLVM IR for each
incremental input. That tracking information will be later used to implement
error recovery.

The Interpreter class orchestrates the IncrementalParser and the
IncrementalExecutor to model interpreter-like behavior. It provides the public
API which can be used (in future) when using the interpreter library.

Differential revision: https://reviews.llvm.org/D96033

show more ...

[ASTReader] Allow controlling separately whether validation should be disabled for a PCH vs a module file

This addresses an issue with how the PCH preable works, specifically:

1. When using a PCH/p

[ASTReader] Allow controlling separately whether validation should be disabled for a PCH vs a module file

This addresses an issue with how the PCH preable works, specifically:

1. When using a PCH/preamble the module hash changes and a different cache directory is used
2. When the preamble is used, PCH & PCM validation is disabled.

Due to combination of #1 and #2, reparsing with preamble enabled can end up loading a stale module file before a header change and using it without updating it because validation is disabled and it doesn’t check that the header has changed and the module file is out-of-date.

rdar://72611253

Differential Revision: https://reviews.llvm.org/D95159

show more ...

Revert "Lex: Migrate HeaderSearch::LoadedModuleMaps to FileEntryRef"

This reverts commit a40db5502b2515a6f2f1676b5d7a655ae0f41179.
and follow-up d636b881bb9214938973098a012fad453082c444

Somewhat sp

Revert "Lex: Migrate HeaderSearch::LoadedModuleMaps to FileEntryRef"

This reverts commit a40db5502b2515a6f2f1676b5d7a655ae0f41179.
and follow-up d636b881bb9214938973098a012fad453082c444

Somewhat speculative, likely broke check-clang on Windows:
https://reviews.llvm.org/D92975#2453482

show more ...