[Frontend] when attaching a preamble, don't generate the long predefines buffer.

We know we're going to overwrite it anyway.
It'd be a bit of work to coordinate not generating it at all, but setting

[Frontend] when attaching a preamble, don't generate the long predefines buffer.

We know we're going to overwrite it anyway.
It'd be a bit of work to coordinate not generating it at all, but setting this
flag avoids generating ~10k of the 13k string.

Differential Revision: https://reviews.llvm.org/D125180

show more ...

Frontend: Delete output streams before closing CompilerInstance outputs

Delete the output streams coming from
CompilerInstance::createOutputFile() and friends once writes are
finished. Concretely, r

Frontend: Delete output streams before closing CompilerInstance outputs

Delete the output streams coming from
CompilerInstance::createOutputFile() and friends once writes are
finished. Concretely, replacing `OS->flush()` with `OS.reset()` in:

- `ExtractAPIAction::EndSourceFileAction()`
- `PrecompiledPreambleAction::setEmittedPreamblePCH()`
- `cc1_main()'s support for `-ftime-trace`

This fixes theoretical bugs related to proxy streams, which may have
cleanups to run in their destructor. For example, a proxy that
CompilerInstance sometimes uses is `buffer_ostream`, which wraps a
`raw_ostream` lacking pwrite support and adds it. `flush()` does not
promise that output is complete; `buffer_ostream` needs to wait until
the destructor to forward anything so that it can service later calls to
`pwrite()`. If the destructor isn't called then the proxied stream
hasn't received any content.

This also protects against some logic bugs, triggering a null
dereference on a later attempt to write to the stream.

No tests, since in practice these particular code paths never use
use `buffer_ostream`; you need to be writing a binary file to a
pipe (such as stdout) to hit it, but `-extract-api` writes a text file
and the other two use computed filenames that will never (in practice)
be a pipe. This is effectively NFC, for now.

But I have some other patches in the works that add guard rails,
crashing if the stream hasn't been destructed by the time the
CompilerInstance is told to keep the output file, since in most cases
this is a problem.

Differential Revision: https://reviews.llvm.org/D124635

show more ...

[Frontend] shrink in-memory PCH buffers to fit

After building a PCH, the vector capacity is on average ~1/3 unused.
If we're going to keep it in memory for a while, reallocate to the right size.
Tak

[Frontend] shrink in-memory PCH buffers to fit

After building a PCH, the vector capacity is on average ~1/3 unused.
If we're going to keep it in memory for a while, reallocate to the right size.
Take care to do this once clang is destroyed so that we can reuse its
memory rather than requesting more.

Differential Revision: https://reviews.llvm.org/D124242

show more ...

Reland [Frontend] avoid copy of PCH data when PrecompiledPreamble stores it in memory

This reverts commit eadf35270727ca743c11b07040bbfedd415ab6dc.

The reland fixes a couple of places in clang that

Reland [Frontend] avoid copy of PCH data when PrecompiledPreamble stores it in memory

This reverts commit eadf35270727ca743c11b07040bbfedd415ab6dc.

The reland fixes a couple of places in clang that were unneccesarily
requesting a null-terminated buffer of the PCH, and hitting assertions.

show more ...

Revert "[Frontend] avoid copy of PCH data when PrecompiledPreamble stores it in memory"

This reverts commit 6e22dac2e2955db1310c63aec215fc22d8da258e.

Seems to cause bot failures e.g.
https://lab.ll

Revert "[Frontend] avoid copy of PCH data when PrecompiledPreamble stores it in memory"

This reverts commit 6e22dac2e2955db1310c63aec215fc22d8da258e.

Seems to cause bot failures e.g.
https://lab.llvm.org/buildbot/#/builders/109/builds/37071

show more ...

[Frontend] avoid copy of PCH data when PrecompiledPreamble stores it in memory

Instead of unconditionally copying the PCHBuffer into an ostream which can be
backed either by a string or a file, just

[Frontend] avoid copy of PCH data when PrecompiledPreamble stores it in memory

Instead of unconditionally copying the PCHBuffer into an ostream which can be
backed either by a string or a file, just make the PCHBuffer itself the
in-memory storage.

Differential Revision: https://reviews.llvm.org/D124180

show more ...

[Frontend] Simplify PrecompiledPreamble::PCHStorage. NFC

- Remove fiddly union, preambles are heavyweight
- Remove fiddly move constructors in TempPCHFile and PCHStorage, use unique_ptr
- Remove unn

[Frontend] Simplify PrecompiledPreamble::PCHStorage. NFC

- Remove fiddly union, preambles are heavyweight
- Remove fiddly move constructors in TempPCHFile and PCHStorage, use unique_ptr
- Remove unneccesary accessors on PCHStorage
- Remove trivial InMemoryStorage
- Move implementation details into cpp file

This is a prefactoring, followup change will change the in-memory PCHStorage to
avoid extra string copies while creating it.

Differential Revision: https://reviews.llvm.org/D124177

show more ...

[clang][lex] NFCI: Use FileEntryRef in PPCallbacks::InclusionDirective()

This patch changes type of the `File` parameter in `PPCallbacks::InclusionDirective()` from `const FileEntry *` to `Optional<

[clang][lex] NFCI: Use FileEntryRef in PPCallbacks::InclusionDirective()

This patch changes type of the `File` parameter in `PPCallbacks::InclusionDirective()` from `const FileEntry *` to `Optional<FileEntryRef>`.

With the API change in place, this patch then removes some uses of the deprecated `FileEntry::getName()` (e.g. in `DependencyGraph.cpp` and `ModuleDependencyCollector.cpp`).

Reviewed By: dexonsmith, bnbarham

Differential Revision: https://reviews.llvm.org/D123574

show more ...

[clang] Change ordering of PreableCallbacks to make sure PP can be referenced in them

Currently, BeforeExecute is called before BeginSourceFile which does not allow
using PP in the callbacks. Change

[clang] Change ordering of PreableCallbacks to make sure PP can be referenced in them

Currently, BeforeExecute is called before BeginSourceFile which does not allow
using PP in the callbacks. Change the ordering to ensure it is possible.
This is a prerequisite for D114370.

Originated from a discussion with @kadircet.

Reviewed By: sammccall

Differential Revision: https://reviews.llvm.org/D114525

show more ...

[clang][clangd] Avoid inconsistent target creation

As proposed in D97109, I tried to make target creation consistent in `clang` and `clangd` by replacing the original procedure with a single functio

[clang][clangd] Avoid inconsistent target creation

As proposed in D97109, I tried to make target creation consistent in `clang` and `clangd` by replacing the original procedure with a single function introduced in D97493.

This also helps `clangd` works with CUDA, OpenMP, etc.

Reviewed By: kadircet

Differential Revision: https://reviews.llvm.org/D98128

show more ...

Frontend: Take VFS and MainFileBuffer by reference in PrecompiledPreamble::CanReuse, NFC

Clarify that `PrecompiledPreamble::CanReuse` requires non-null arguments
for `VFS` and `MainFileBuffer`, taki

Frontend: Take VFS and MainFileBuffer by reference in PrecompiledPreamble::CanReuse, NFC

Clarify that `PrecompiledPreamble::CanReuse` requires non-null arguments
for `VFS` and `MainFileBuffer`, taking them by reference instead of by
pointer.

Differential Revision: https://reviews.llvm.org/D91297

show more ...

[ASTReader] Allow controlling separately whether validation should be disabled for a PCH vs a module file

This addresses an issue with how the PCH preable works, specifically:

1. When using a PCH/p

[ASTReader] Allow controlling separately whether validation should be disabled for a PCH vs a module file

This addresses an issue with how the PCH preable works, specifically:

1. When using a PCH/preamble the module hash changes and a different cache directory is used
2. When the preamble is used, PCH & PCM validation is disabled.

Due to combination of #1 and #2, reparsing with preamble enabled can end up loading a stale module file before a header change and using it without updating it because validation is disabled and it doesn’t check that the header has changed and the module file is out-of-date.

rdar://72611253

Differential Revision: https://reviews.llvm.org/D95159

show more ...

ADT: Stop peeking inside AlignedCharArrayUnion, NFC

Update all the users of `AlignedCharArrayUnion` to stop peeking inside
(to look at `buffer`) so that a follow-up patch can replace it with an
alia

ADT: Stop peeking inside AlignedCharArrayUnion, NFC

Update all the users of `AlignedCharArrayUnion` to stop peeking inside
(to look at `buffer`) so that a follow-up patch can replace it with an
alias to `std::aligned_union_t`.

This was reviewed as part of https://reviews.llvm.org/D92512, but I'm
splitting this bit out to commit first to reduce churn in case the
change to `AlignedCharArrayUnion` needs to be reverted for some
unexpected reason.

show more ...

Frontend: Change ComputePreambleBounds to take MemoryBufferRef, NFC

Avoid requiring an actual MemoryBuffer in ComputePreambleBounds, when
a MemoryBufferRef will do just fine.

Differential Revision:

Frontend: Change ComputePreambleBounds to take MemoryBufferRef, NFC

Avoid requiring an actual MemoryBuffer in ComputePreambleBounds, when
a MemoryBufferRef will do just fine.

Differential Revision: https://reviews.llvm.org/D90890

show more ...

clang/Basic: Replace SourceManager::getMemoryBufferForFile, NFC

Replace `SourceManager::getMemoryBufferForFile`, which returned a
dereferenceable `MemoryBuffer*` and had a `bool*Invalid` out paramet

clang/Basic: Replace SourceManager::getMemoryBufferForFile, NFC

Replace `SourceManager::getMemoryBufferForFile`, which returned a
dereferenceable `MemoryBuffer*` and had a `bool*Invalid` out parameter,
with `getMemoryBufferForFileOrNone` (returning
`Optional<MemoryBufferRef>`) and `getMemoryBufferForFileOrFake`
(returning `MemoryBufferRef`).

Differential Revision: https://reviews.llvm.org/D89429

show more ...

Correctly set CompilingPCH in PrecompilePreambleAction.

This fixes a crash bug in clangd when used with modules. ASTWriter would
end up writing references to submodules into the PCH file, but upon
r

Correctly set CompilingPCH in PrecompilePreambleAction.

This fixes a crash bug in clangd when used with modules. ASTWriter would
end up writing references to submodules into the PCH file, but upon
reading the submodules would not exists and
HeaderFileInfoTrait::ReadData would crash.

Differential Revision: https://reviews.llvm.org/D85532

show more ...

Reland [clangd] Parse std::make_unique, and emit template diagnostics at expansion.

This was originally 658af9435071 and reverted in 665dbe91f2ed.
The clang bug this triggered was fixed in 05ed3efc2

Reland [clangd] Parse std::make_unique, and emit template diagnostics at expansion.

This was originally 658af9435071 and reverted in 665dbe91f2ed.
The clang bug this triggered was fixed in 05ed3efc2ac.

show more ...

Revert "[clangd] Parse std::make_unique, and emit template diagnostics at expansion."

This reverts commit 658af9435071d5da017c1d65298bdea19ec095e1.
Breaks tests on windows: http://45.33.8.238/win/17

Revert "[clangd] Parse std::make_unique, and emit template diagnostics at expansion."

This reverts commit 658af9435071d5da017c1d65298bdea19ec095e1.
Breaks tests on windows: http://45.33.8.238/win/17229/step_9.txt

I think this is uncovering a latent bug when a late-parsed preamble is
used with an eagerly-parsed file.

show more ...

[clangd] Parse std::make_unique, and emit template diagnostics at expansion.

Summary:
Parsing std::make_unique is an exception to the usual non-parsing of function
bodies in the preamble. (A hook is

[clangd] Parse std::make_unique, and emit template diagnostics at expansion.

Summary:
Parsing std::make_unique is an exception to the usual non-parsing of function
bodies in the preamble. (A hook is added to PreambleCallbacks to allow this).
This allows us to diagnose make_unique<Foo>(wrong arg list), and opens the door
to providing signature help (by detecting where the arg list is forwarded to).
This function is trivial (checked libc++ and libstdc++) and doesn't result in
any extra templates being instantiated, so this should be cheap.

This uncovered a second issue (already visible with class templates)...

Errors produced by template instantiation have primary locations within the
template, with instantiation stack reported as notes.
For templates defined in headers, these end up reported at the #include
directive, which isn't terribly helpful as the header itself is probably fine.
This patch reports them at the instantiation site (the first location in the
instantiation stack that's in the main file). This in turn required a bit of
refactoring in Diagnostics so we can delay relocating the diagnostic until all
notes are available.

https://github.com/clangd/clangd/issues/412

Reviewers: hokein, aaron.ballman

Subscribers: ilya-biryukov, MaskRay, jkorous, arphaman, kadircet, usaxena95, cfe-commits

Tags: #clang

Differential Revision: https://reviews.llvm.org/D81351

show more ...

[Preamble] Invalidate preamble when missing headers become present.

Summary:
To avoid excessive extra stat()s, only check the possible locations of
headers that weren't found at all (leading to a co

[Preamble] Invalidate preamble when missing headers become present.

Summary:
To avoid excessive extra stat()s, only check the possible locations of
headers that weren't found at all (leading to a compile error).
For headers that *were* found, we don't check for files earlier on the
search path that could override them.

Reviewers: kadircet

Subscribers: javed.absar, jkorous, arphaman, usaxena95, cfe-commits

Tags: #clang

Differential Revision: https://reviews.llvm.org/D77942

show more ...

[clang] Const correct ComputePreambleBounds

Avoid including FileManager.h from SourceManager.h

Most clients of SourceManager.h need to do things like turning source
locations into file & line number pairs, but this doesn't require
bringing in

Avoid including FileManager.h from SourceManager.h

Most clients of SourceManager.h need to do things like turning source
locations into file & line number pairs, but this doesn't require
bringing in FileManager.h and LLVM's FS headers.

The main code change here is to sink SM::createFileID into the cpp file.
I reason that this is not performance critical because it doesn't happen
on the diagnostic path, it happens along the paths of macro expansion
(could be hot) and new includes (less hot).

Saves some includes:
309 - /usr/local/google/home/rnk/llvm-project/clang/include/clang/Basic/FileManager.h
272 - /usr/local/google/home/rnk/llvm-project/clang/include/clang/Basic/FileSystemOptions.h
271 - /usr/local/google/home/rnk/llvm-project/llvm/include/llvm/Support/VirtualFileSystem.h
267 - /usr/local/google/home/rnk/llvm-project/llvm/include/llvm/Support/FileSystem.h
266 - /usr/local/google/home/rnk/llvm-project/llvm/include/llvm/Support/Chrono.h

Differential Revision: https://reviews.llvm.org/D75406

show more ...

Make llvm::StringRef to std::string conversions explicit.

This is how it should've been and brings it more in line with
std::string_view. There should be no functional change here.

This is mostly m

Make llvm::StringRef to std::string conversions explicit.

This is how it should've been and brings it more in line with
std::string_view. There should be no functional change here.

This is mostly mechanical from a custom clang-tidy check, with a lot of
manual fixups. It uncovers a lot of minor inefficiencies.

This doesn't actually modify StringRef yet, I'll do that in a follow-up.

show more ...

[Frontend] Clean up some dead code in PrecompiledPreamble. NFC

[Clang] Migrate llvm::make_unique to std::make_unique

Now that we've moved to C++14, we no longer need the llvm::make_unique
implementation from STLExtras.h. This patch is a mechanical replacement
o

[Clang] Migrate llvm::make_unique to std::make_unique

Now that we've moved to C++14, we no longer need the llvm::make_unique
implementation from STLExtras.h. This patch is a mechanical replacement
of (hopefully) all the llvm::make_unique instances across the monorepo.

Differential revision: https://reviews.llvm.org/D66259

llvm-svn: 368942

show more ...