[clang][AST] Improve AST Reader/Writer memory footprint

Reduce memory footprint of AST Reader/Writer:
1. Adjust internal data containers' element type.
2. Switch to set for deduplication of deferred

[clang][AST] Improve AST Reader/Writer memory footprint

Reduce memory footprint of AST Reader/Writer:
1. Adjust internal data containers' element type.
2. Switch to set for deduplication of deferred diags.

Differential Revision: https://reviews.llvm.org/D101793

show more ...

Make iteration over the DeclContext::lookup_result safe.

The idiom:
```
DeclContext::lookup_result R = DeclContext::lookup(Name);
for (auto *D : R) {...}
```

is not safe when in the loop body we tr

Make iteration over the DeclContext::lookup_result safe.

The idiom:
```
DeclContext::lookup_result R = DeclContext::lookup(Name);
for (auto *D : R) {...}
```

is not safe when in the loop body we trigger deserialization from an AST file.
The deserialization can insert new declarations in the StoredDeclsList whose
underlying type is a vector. When the vector decides to reallocate its storage
the pointer we hold becomes invalid.

This patch replaces a SmallVector with an singly-linked list. The current
approach stores a SmallVector<NamedDecl*, 4> which is around 8 pointers.
The linked list is 3, 5, or 7. We do better in terms of memory usage for small
cases (and worse in terms of locality -- the linked list entries won't be near
each other, but will be near their corresponding declarations, and we were going
to fetch those memory pages anyway). For larger cases: the vector uses a
doubling strategy for reallocation, so will generally be between half-full and
full. Let's say it's 75% full on average, so there's N * 4/3 + 4 pointers' worth
of space allocated currently and will be 2N pointers with the linked list. So we
break even when there are N=6 entries and slightly lose in terms of memory usage
after that. We suspect that's still a win on average.

Thanks to @rsmith!

Differential revision: https://reviews.llvm.org/D91524

show more ...

Merge some of the PCH object support with modular codegen

I was trying to pick this up a bit when reviewing D48426 (& perhaps D69778) - in any case, looks like D48426 added a module level flag that

Merge some of the PCH object support with modular codegen

I was trying to pick this up a bit when reviewing D48426 (& perhaps D69778) - in any case, looks like D48426 added a module level flag that might not be needed.

The D48426 implementation worked by setting a module level flag, then code generating contents from the PCH a special case in ASTContext::DeclMustBeEmitted would be used to delay emitting the definition of these functions if they came from a Module with this flag.

This strategy is similar to the one initially implemented for modular codegen that was removed in D29901 in favor of the modular decls list and a bit on each decl to specify whether it's homed to a module.

One major difference between PCH object support and modular code generation, other than the specific list of decls that are homed, is the compilation model: MSVC PCH modules are built into the object file for some other source file (when compiling that source file /Yc is specified to say "this compilation is where the PCH is homed"), whereas modular code generation invokes a separate compilation for the PCH alone. So the current modular code generation test of to decide if a decl should be emitted "is the module where this decl is serialized the current main file" has to be extended (as Lubos did in D69778) to also test the command line flag -building-pch-with-obj.

Otherwise the whole thing is basically streamlined down to the modular code generation path.

This even offers one extra material improvement compared to the existing divergent implementation: Homed functions are not emitted into object files that use the pch. Instead at -O0 they are not emitted into the IR at all, and at -O1 they are emitted using available_externally (existing functionality implemented for modular code generation). The pch-codegen test has been updated to reflect this new behavior.

[If possible: I'd love it if we could not have the extra MSVC-style way of accessing dllexport-pch-homing, and just do it the modular codegen way, but I understand that it might be a limitation of existing build systems. @hans / @thakis: Do either of you know if it'd be practical to move to something more similar to .pcm handling, where the pch itself is passed to the compilation, rather than homed as a side effect of compiling some other source file?]

Reviewers: llunak, hans

Differential Revision: https://reviews.llvm.org/D83652

show more ...

recommit 1b978ddba05c [CUDA][HIP][OpenMP] Emit deferred diagnostics by a post-parsing AST travese

Differential Revision: https://reviews.llvm.org/D70172

[clang] Improve LLVM-style RTTI support in ExternalASTSource/ExternalSemaSource

Summary:
We currently have some very basic LLVM-style RTTI support in the ExternalASTSource class hierarchy
based on t

[clang] Improve LLVM-style RTTI support in ExternalASTSource/ExternalSemaSource

Summary:
We currently have some very basic LLVM-style RTTI support in the ExternalASTSource class hierarchy
based on the `SemaSource` bool( to discriminate it form the ExternalSemaSource). As ExternalASTSource
is supposed to be subclassed we should have extendable LLVM-style RTTI in this class hierarchy to make life easier
for projects building on top of Clang.

Most notably the current RTTI implementation forces LLDB to implement RTTI for its
own ExternalASTSource class (ClangExternalASTSourceCommon) by keeping a global set of
ExternalASTSources that are known to be ClangExternalASTSourceCommon. Projects
using Clang currently have to dosimilar workarounds to get RTTI support for their subclasses.

This patch turns this into full-fledged LLVM-style RTTI based on a static `ID` variable similar to
other LLVM class hierarchies. Also removes the friend declaration from ExternalASTSource to
its child class that was only used to grant access to the `SemaSource` member.

Reviewers: aprantl, dblaikie, rjmccall

Reviewed By: aprantl

Subscribers: riccibruno, labath, lhames, cfe-commits

Tags: #clang

Differential Revision: https://reviews.llvm.org/D71397

show more ...

Update the file headers across all of the LLVM projects in the monorepo
to reflect the new license.

We understand that people may be surprised that we're moving the header
entirely to discuss the ne

Update the file headers across all of the LLVM projects in the monorepo
to reflect the new license.

We understand that people may be surprised that we're moving the header
entirely to discuss the new license. We checked this carefully with the
Foundation's lawyer and we believe this is the correct approach.

Essentially, all code in the project is now made available by the LLVM
project under our new license, so you will see that the license headers
include that license only. Some of our contributors have contributed
code under our old license, and accordingly, we have retained a copy of
our old license notice in the top-level files in each project and
repository.

llvm-svn: 351636

show more ...

Remove trailing space

sed -Ei 's/[[:space:]]+$//' include/**/*.{def,h,td} lib/**/*.{cpp,h}

llvm-svn: 338291

[clang-cl] Don't emit dllexport inline functions etc. from pch files (PR37801)

With MSVC, PCH files are created along with an object file that needs to
be linked into the final library or executable

[clang-cl] Don't emit dllexport inline functions etc. from pch files (PR37801)

With MSVC, PCH files are created along with an object file that needs to
be linked into the final library or executable. That object file
contains the code generated when building the headers. In particular, it
will include definitions of inline dllexport functions, and because they
are emitted in this object file, other files using the PCH do not need
to emit them. See the bug for an example.

This patch makes clang-cl match MSVC's behaviour in this regard, causing
significant compile-time savings when building dlls using precompiled
headers.

For example, in a 64-bit optimized shared library build of Chromium with
PCH, it reduces the binary size and compile time of
stroke_opacity_custom.obj from 9315564 bytes to 3659629 bytes and 14.6
to 6.63 s. The wall-clock time of building blink_core.dll goes from
38m41s to 22m33s. ("user" time goes from 1979m to 1142m).

Differential Revision: https://reviews.llvm.org/D48426

llvm-svn: 335466

show more ...

Remove \brief commands from doxygen comments.

This is similar to the LLVM change https://reviews.llvm.org/D46290.

We've been running doxygen with the autobrief option for a couple of
years now. Thi

Remove \brief commands from doxygen comments.

This is similar to the LLVM change https://reviews.llvm.org/D46290.

We've been running doxygen with the autobrief option for a couple of
years now. This makes the \brief markers into our comments
redundant. Since they are a visual distraction and we don't want to
encourage more \brief markers in new code either, this patch removes
them all.

Patch produced by

for i in $(git grep -l '\@brief'); do perl -pi -e 's/\@brief //g' $i & done
for i in $(git grep -l '\\brief'); do perl -pi -e 's/\\brief //g' $i & done

Differential Revision: https://reviews.llvm.org/D46320

llvm-svn: 331834

show more ...

[modules] Correctly overload getModule in the MultiplexExternalSemaSource

Summary:
The MultiplexExternalSemaSource doesn't correctly overload the `getModule` function,
causing the multiplexer to not

[modules] Correctly overload getModule in the MultiplexExternalSemaSource

Summary:
The MultiplexExternalSemaSource doesn't correctly overload the `getModule` function,
causing the multiplexer to not forward this call as intended.

Reviewers: v.g.vassilev

Reviewed By: v.g.vassilev

Subscribers: cfe-commits

Differential Revision: https://reviews.llvm.org/D39416

llvm-svn: 323122

show more ...

Fix warnings. [-Wdocumentation]

llvm-svn: 315573

Modular Codegen: Support homing debug info for types in modular objects

Matching the function-homing support for modular codegen. Any type
implicitly (implicit template specializations) or explicitl

Modular Codegen: Support homing debug info for types in modular objects

Matching the function-homing support for modular codegen. Any type
implicitly (implicit template specializations) or explicitly defined in
a module is attached to that module's object file and omitted elsewhere
(only a declaration used if necessary for references).

llvm-svn: 299987

show more ...

Modular Codegen: Add/use a bit in serialized function definitions to track whether they are the subject of modular codegen

Some decls are created not where they are written, but in other module
file

Modular Codegen: Add/use a bit in serialized function definitions to track whether they are the subject of modular codegen

Some decls are created not where they are written, but in other module
files/users (implicit special members and function template implicit
specializations). To correctly identify them, use a bit next to the definition
to track the modular codegen property.

Discussed whether the module file bit could be omitted in favor of
reconstituting from the modular codegen decls list - best guess today is that
the efficiency improvement of not having to deserialize the whole list whenever
any function is queried by a module user is worth it for the small size
increase of this redundant (list + bit-on-def) representation.

Reviewers: rsmith

Differential Revision: https://reviews.llvm.org/D29901

llvm-svn: 299982

show more ...

Prototype of modules codegen

First pass at generating weak definitions of inline functions from module files
(& skipping (-O0) or emitting available_externally (optimizations)
definitions where thos

Prototype of modules codegen

First pass at generating weak definitions of inline functions from module files
(& skipping (-O0) or emitting available_externally (optimizations)
definitions where those modules are used).

External functions defined in modules are emitted into the modular
object file as well (this may turn an existing ODR violation (if that
module were imported into multiple translations) into valid/linkable
code).

Internal symbols (static functions, for example) are not correctly
supported yet. The symbol will be produced, internal, in the modular
object - unreferenceable from the users.

Reviewers: rsmith

Differential Revision: https://reviews.llvm.org/D28845

llvm-svn: 293456

show more ...

[Sema] Use unique_ptr instead of raw pointers in the late-parsed templates map.

Summary:
This is possible now that MapVector supports move-only values.

Depends on D25404.

Reviewers: timshen

Subsc

[Sema] Use unique_ptr instead of raw pointers in the late-parsed templates map.

Summary:
This is possible now that MapVector supports move-only values.

Depends on D25404.

Reviewers: timshen

Subscribers: cfe-commits

Differential Revision: https://reviews.llvm.org/D25405

llvm-svn: 283766

show more ...

Method Pool in modules: we make sure that if a module contains an entry for
a selector, the entry should be complete, containing everything introduced by
that module and all modules it imports.

Befo

Method Pool in modules: we make sure that if a module contains an entry for
a selector, the entry should be complete, containing everything introduced by
that module and all modules it imports.

Before writing out the method pool of a module, we sync up the out of date
selectors by pulling in methods for the selectors, from all modules it imports.

In ReadMethodPool, after pulling in the method pool entry for module A, this
lets us skip the modules that module A imports.

rdar://problem/25900131

llvm-svn: 268091

show more ...

Store list of undefined-but-used objects in a deterministic order to fix
non-deterministic diagnostics (and non-deterministic PCH files). Check these
when building a module rather than serializing it

Store list of undefined-but-used objects in a deterministic order to fix
non-deterministic diagnostics (and non-deterministic PCH files). Check these
when building a module rather than serializing it; it's not reasonable for a
module's use to be satisfied by a definition in the user of the module.

llvm-svn: 264466

show more ...

Roll-back r250822.

Summary: It breaks the build for the ASTMatchers

Subscribers: klimek, cfe-commits

Differential Revision: http://reviews.llvm.org/D13893

llvm-svn: 250827

Apply modernize-use-default to clang.

Summary: Replace empty bodies of default constructors and destructors with '= default'.

Reviewers: bkramer, klimek

Subscribers: klimek, alexfh, cfe-commits

D

Apply modernize-use-default to clang.

Summary: Replace empty bodies of default constructors and destructors with '= default'.

Reviewers: bkramer, klimek

Subscribers: klimek, alexfh, cfe-commits

Differential Revision: http://reviews.llvm.org/D13890

llvm-svn: 250822

show more ...

function_ref-ize ExternalASTSource::FindExternalLexicalDecl and remove its
useless return value. Switch to using it directly when completing the
redeclaration chain for an anonymous declaration, and

function_ref-ize ExternalASTSource::FindExternalLexicalDecl and remove its
useless return value. Switch to using it directly when completing the
redeclaration chain for an anonymous declaration, and reduce the set of
declarations that we load in the process to just those of the right kind.

llvm-svn: 244161

show more ...

Detect uses of mismatching forms of 'new' and 'delete'

Emit warning when operand to `delete` is allocated with `new[]` or
operand to `delete[]` is allocated with `new`.

rev 2 update:
`getNewExprFro

Detect uses of mismatching forms of 'new' and 'delete'

Emit warning when operand to `delete` is allocated with `new[]` or
operand to `delete[]` is allocated with `new`.

rev 2 update:
`getNewExprFromInitListOrExpr` should return `dyn_cast_or_null`
instead of `dyn_cast`, since `E` might be null.

Reviewers: rtrieu, jordan_rose, rsmith

Subscribers: majnemer, cfe-commits

Differential Revision: http://reviews.llvm.org/D4661

llvm-svn: 237608

show more ...

Revert "Detect uses of mismatching forms of 'new' and 'delete'"

This reverts commit 742dc9b6c9686ab52860b7da39c3a126d8a97fbc.

This is generating multiple segfaults in our internal builds.
Test case

Revert "Detect uses of mismatching forms of 'new' and 'delete'"

This reverts commit 742dc9b6c9686ab52860b7da39c3a126d8a97fbc.

This is generating multiple segfaults in our internal builds.
Test case coming up shortly.

llvm-svn: 237391

show more ...

Detect uses of mismatching forms of 'new' and 'delete'

Emit warning when operand to `delete` is allocated with `new[]` or
operand to `delete[]` is allocated with `new`.

Reviewers: rtrieu, jordan_ro

Detect uses of mismatching forms of 'new' and 'delete'

Emit warning when operand to `delete` is allocated with `new[]` or
operand to `delete[]` is allocated with `new`.

Reviewers: rtrieu, jordan_rose, rsmith

Subscribers: majnemer, cfe-commits

Differential Revision: http://reviews.llvm.org/D4661

llvm-svn: 237368

show more ...

[Modules] Preserve source order for the map of late parsed templates.

Clang was inserting these into a dense map. While it never iterated the
dense map during normal compilation, it did when emittin

[Modules] Preserve source order for the map of late parsed templates.

Clang was inserting these into a dense map. While it never iterated the
dense map during normal compilation, it did when emitting a module. Fix
this by using a standard MapVector to preserve the order in which we
encounter the late parsed templates.

I suspect this still isn't ideal, as we don't seem to remove things from
this map even when we mark the templates as no longer late parsed. But
I don't know enough about this particular extension to craft a nice,
subtle test case covering this. I've managed to get the stress test to
at least do some late parsing and demonstrate the core problem here.
This patch fixes the test and provides deterministic behavior which is
a strict improvement over the prior state.

I've cleaned up some of the code here as well to be explicit about
inserting when that is what is actually going on.

llvm-svn: 233264

show more ...

[modules] Deserialize CXXCtorInitializer list for a constructor lazily.

Previously we'd deserialize the list of mem-initializers for a constructor when
we deserialized the declaration of the constru

[modules] Deserialize CXXCtorInitializer list for a constructor lazily.

Previously we'd deserialize the list of mem-initializers for a constructor when
we deserialized the declaration of the constructor. That could trigger a
significant amount of unnecessary work (pulling in all base classes
recursively, for a start) and was causing problems for the modules buildbot due
to cyclic deserializations. We now deserialize these on demand.

This creates a certain amount of duplication with the handling of
CXXBaseSpecifiers; I'll look into reducing that next.

llvm-svn: 233052

show more ...