Fix use-after-free in ModuleManager

When removing out-of-date modules we might have left behind a VisitOrder
that contains pointers to freed ModuleFiles. This was very rarely seen,
because it only

Fix use-after-free in ModuleManager

When removing out-of-date modules we might have left behind a VisitOrder
that contains pointers to freed ModuleFiles. This was very rarely seen,
because it only happens when modules go out of date and the VisitOrder
happens to have the right size to not be recomputed.

Thanks ASan!

rdar://23181512

llvm-svn: 250963

show more ...

[modules] Remove now-dead code for lazy loading of files specified by -fmodule-file=.

llvm-svn: 244417

[modules] Remove unused ModuleManager::visitDepthFirst function.

llvm-svn: 244289

[Modules] Wrap the main ModuleManager visitor in a function_ref.

Avoids the awkward passing of an opaque void *UserData argument. No
functional change intended.

llvm-svn: 243213

Fix dumb use-after-free bug introduced in r242868.

llvm-svn: 242960

[modules] Stop performing PCM lookups for all identifiers when building with C++ modules. Instead, serialize a list of interesting identifiers and mark those ones out of date on module import. Avoidi

[modules] Stop performing PCM lookups for all identifiers when building with C++ modules. Instead, serialize a list of interesting identifiers and mark those ones out of date on module import. Avoiding the identifier lookups here gives a 20-30% speedup in builds with large numbers of modules. No functionality change intended.

llvm-svn: 242868

show more ...

[modules] Change module manager visitation order to be a bit more stable when
more modules are added: visit modules depth-first rather than breadth-first.
The visitation is still (approximately) olde

[modules] Change module manager visitation order to be a bit more stable when
more modules are added: visit modules depth-first rather than breadth-first.
The visitation is still (approximately) oldest-to-newest, and still guarantees
that a module is visited before anything it imports, so modules that are
imported by others sometimes need to jump to a later position in the visitation
order when more modules are loaded, but independent module trees don't
interfere with each other any more.

llvm-svn: 242863

show more ...

Make the clang module container format selectable from the command line.
- introduces a new cc1 option -fmodule-format=[raw,obj]
with 'raw' being the default
- supports arbitrary module container f

Make the clang module container format selectable from the command line.
- introduces a new cc1 option -fmodule-format=[raw,obj]
with 'raw' being the default
- supports arbitrary module container formats that libclang is agnostic to
- adds the format to the module hash to avoid collisions
- splits the old PCHContainerOperations into PCHContainerWriter and
a PCHContainerReader.

Thanks to Richard Smith for reviewing this patch!

llvm-svn: 242499

show more ...

Revert r240270 ("Fixed/added namespace ending comments using clang-tidy").

llvm-svn: 240353

Fixed/added namespace ending comments using clang-tidy. NFC

The patch is generated using this command:

$ tools/extra/clang-tidy/tool/run-clang-tidy.py -fix \
-checks=-*,llvm-namespace-comme

Fixed/added namespace ending comments using clang-tidy. NFC

The patch is generated using this command:

$ tools/extra/clang-tidy/tool/run-clang-tidy.py -fix \
-checks=-*,llvm-namespace-comment -header-filter='llvm/.*|clang/.*' \
work/llvm/tools/clang

To reduce churn, not touching namespaces spanning less than 10 lines.

llvm-svn: 240270

show more ...

Introduce a PCHContainerOperations interface (NFC).

A PCHContainerOperations abstract interface provides operations for
creating and unwrapping containers for serialized ASTs (precompiled
headers an

Introduce a PCHContainerOperations interface (NFC).

A PCHContainerOperations abstract interface provides operations for
creating and unwrapping containers for serialized ASTs (precompiled
headers and clang modules). The default implementation is
RawPCHContainerOperations, which uses a flat file for the output.

The main application for this interface will be an
ObjectFilePCHContainerOperations implementation that uses LLVM to
wrap the module in an ELF/Mach-O/COFF container to store debug info
alongside the AST.

rdar://problem/20091852

llvm-svn: 240225

show more ...

Allow skipping imports in the module visitor.

Skip imports when we know that we do not need to visit any imports
because we've already deserialized the redecls from a module.

llvm-svn: 237782

A couple of readASTFileSignature improvements (NFC)

* Strength reduce a std::function to a function pointer,
* Factor out checking the AST file magic number,
* Add a brief doc comment to readAStFile

A couple of readASTFileSignature improvements (NFC)

* Strength reduce a std::function to a function pointer,
* Factor out checking the AST file magic number,
* Add a brief doc comment to readAStFileSignature

Thanks to Chandler for spotting these oddities.

llvm-svn: 233050

show more ...

Make module files passed to a module build via -fmodule-file= available to
consumers of that module.

Previously, such a file would only be available if the module happened to
actually import somethi

Make module files passed to a module build via -fmodule-file= available to
consumers of that module.

Previously, such a file would only be available if the module happened to
actually import something from that module.

llvm-svn: 232583

show more ...

Revert "Wrap clang module files in a Mach-O, ELF, or COFF container."

llvm-svn: 230454

Wrap clang module files in a Mach-O, ELF, or COFF container.
This is a necessary prerequisite for debugging with modules.
The .pcm files become containers that hold the serialized AST which allows
us

Wrap clang module files in a Mach-O, ELF, or COFF container.
This is a necessary prerequisite for debugging with modules.
The .pcm files become containers that hold the serialized AST which allows
us to store debug information in the module file that can be shared by all
object files that were built importing the module.

This reapplies r230044 with a fixed configure+make build and updated
dependencies and testcase requirements. Over the last iteration this
version adds
- missing target requirements for testcases that specify an x86 triple,
- a missing clangCodeGen.a dependency to libClang.a in the make build.

rdar://problem/19104245

llvm-svn: 230423

show more ...

Revert "Wrap clang module files in a Mach-O, ELF, or COFF container."

This reverts commit r230305.
Off to fix another round of missing dependencies on various platforms.

llvm-svn: 230309

Wrap clang module files in a Mach-O, ELF, or COFF container.
This is a necessary prerequisite for debugging with modules.
The .pcm files become containers that hold the serialized AST which allows
us

Wrap clang module files in a Mach-O, ELF, or COFF container.
This is a necessary prerequisite for debugging with modules.
The .pcm files become containers that hold the serialized AST which allows
us to store debug information in the module file that can be shared by all
object files that were built importing the module.

rdar://problem/19104245

This reapplies r230044 with a fixed configure+make build and updated
dependencies. Take 3.

llvm-svn: 230305

show more ...

Revert "Wrap clang module files in a Mach-O, ELF, or COFF container."

This reverts commit 230099.

The Linux configure+make build variant still needs some work.

llvm-svn: 230103

Wrap clang module files in a Mach-O, ELF, or COFF container.
This is a necessary prerequisite for debugging with modules.
The .pcm files become containers that hold the serialized AST which allows
us

Wrap clang module files in a Mach-O, ELF, or COFF container.
This is a necessary prerequisite for debugging with modules.
The .pcm files become containers that hold the serialized AST which allows
us to store debug information in the module file that can be shared by all
object files that were built importing the module.

rdar://problem/19104245

This reapplies r230044 with a fixed configure+make build and updated
dependencies. Take 2.

llvm-svn: 230089

show more ...

Revert "Wrap clang module files in a Mach-O, ELF, or COFF container."

This reverts commit r230067.

Investigating another batch of problems found by the bots.

llvm-svn: 230073

Wrap clang module files in a Mach-O, ELF, or COFF container.
This is a necessary prerequisite for debugging with modules.
The .pcm files become containers that hold the serialized AST which allows
us

Wrap clang module files in a Mach-O, ELF, or COFF container.
This is a necessary prerequisite for debugging with modules.
The .pcm files become containers that hold the serialized AST which allows
us to store debug information in the module file that can be shared by all
object files that were built importing the module.

rdar://problem/19104245

This reapplies r230044 with a fixed configure+make build and updated
dependencies.

llvm-svn: 230067

show more ...

Revert "Wrap clang module files in a Mach-O, ELF, or COFF container."

This reverts commit r230044 while dealing with buildbot breakage.

Conflicts:
test/Modules/module_container.m

llvm-svn: 230052

Wrap clang module files in a Mach-O, ELF, or COFF container.
This is a necessary prerequisite for debugging with modules.
The .pcm files become containers that hold the serialized AST which allows
us

Wrap clang module files in a Mach-O, ELF, or COFF container.
This is a necessary prerequisite for debugging with modules.
The .pcm files become containers that hold the serialized AST which allows
us to store debug information in the module file that can be shared by all
object files that were built importing the module.

rdar://problem/19104245

llvm-svn: 230044

show more ...

[modules] When explicitly importing a module, it's fine for the imported module
to be newer than we were expecting. That happens if .pcm's get moved between
file systems during a distributed build. (

[modules] When explicitly importing a module, it's fine for the imported module
to be newer than we were expecting. That happens if .pcm's get moved between
file systems during a distributed build. (It's still not OK for them to actually
be different, though, so we still check the size and signature matches.)

llvm-svn: 222507

show more ...