[Tooling/DependencyScanning] Enable passing a `vfs::FileSystem` object to `DependencyScanningTool`

Also include a unit test to validate that the `vfs::FileSystem` object is properly used.

Different

[Tooling/DependencyScanning] Enable passing a `vfs::FileSystem` object to `DependencyScanningTool`

Also include a unit test to validate that the `vfs::FileSystem` object is properly used.

Differential Revision: https://reviews.llvm.org/D129912

show more ...

[clang][deps] Override dependency and serialized diag files for modules

When building modules, override secondary outputs (dependency file,
dependency targets, serialized diagnostic file) in additio

[clang][deps] Override dependency and serialized diag files for modules

When building modules, override secondary outputs (dependency file,
dependency targets, serialized diagnostic file) in addition to the pcm
file path. This avoids inheriting per-TU command-line options that
cause non-determinism in the results (non-deterministic command-line for
the module build, non-determinism in which TU's .diag and .d files will
contain the module outputs). In clang-scan-deps we infer whether to
generate dependency or serialized diagnostic files based on an original
command-line. In a real build system this should be modeled explicitly.

Differential Revision: https://reviews.llvm.org/D129389

show more ...

[DependencyScanningTool.cpp] Use `using namespace` instead of wrapping the `.cpp` file contents in namespaces, NFC

This makes the file consistent with the coding style of the rest of LLVM.

[clang][deps] Further canonicalize implicit modules options in dep scan

Disable or canonicalize compiler options that are not relevant in
explicit module builds, similar to what we already did for t

[clang][deps] Further canonicalize implicit modules options in dep scan

Disable or canonicalize compiler options that are not relevant in
explicit module builds, similar to what we already did for the modules
cache path. This reduces uninteresting differences between
command-lines, which is particularly useful if there is a tool that can
cache the compilations.

Differential Revision: https://reviews.llvm.org/D127883

show more ...

[clang][deps] Make order of module dependencies deterministic

This fixes the underlying module dependencies, which had a
non-deterministic order, which was also visible in the order of calls to
Depe

[clang][deps] Make order of module dependencies deterministic

This fixes the underlying module dependencies, which had a
non-deterministic order, which was also visible in the order of calls to
DependencyConsumer methods. This was not directly observable in
the clang-scan-deps utility, because it was previously seeing a sorted
order from std::map in DependencyScanningTool. However, the underlying
API previously created a likely issue for any other clients. Note: if
you only apply the change from DependencyScanningTool, you can see the
issue in clang-scan-deps, and existing tests will fail
non-deterministicaly.

Differential Revision: https://reviews.llvm.org/D127243

show more ...

[clang][deps] NFC: Inline function with single caller

[clang][deps] Remove '-fmodules-cache-path=' arguments

With explicit modules build, the '-fmodules-cache-path=' argument is unused.

This patch removes the argument to avoid warnings or errors (with

[clang][deps] Remove '-fmodules-cache-path=' arguments

With explicit modules build, the '-fmodules-cache-path=' argument is unused.

This patch removes the argument to avoid warnings or errors (with '-Werror') stemming from that.

Depends on D118915.

Reviewed By: dexonsmith

Differential Revision: https://reviews.llvm.org/D120474

show more ...

[clang][deps] Generate '-fmodule-file=' only for direct dependencies

The `clang-scan-deps` tool currently generates `-fmodule-file=` command-line arguments for the whole transitive closure of modula

[clang][deps] Generate '-fmodule-file=' only for direct dependencies

The `clang-scan-deps` tool currently generates `-fmodule-file=` command-line arguments for the whole transitive closure of modular dependencies. This is not necessary, we only need to provide the direct dependencies on the command line. Information about transitive dependencies is stored within the `.pcm` files of direct dependencies. This makes the command lines shorter, but should be a NFC otherwise (unless there are bugs in the loading mechanism for explicit modules).

Depends on D120465.

Reviewed By: Bigcheese

Differential Revision: https://reviews.llvm.org/D118915

show more ...

[clang][deps] Disable implicit module maps

Since D113473, we don't report any module map files via `-fmodule-map-file=` in explicit builds. The ultimate goal here is to make sure Clang doesn't open/

[clang][deps] Disable implicit module maps

Since D113473, we don't report any module map files via `-fmodule-map-file=` in explicit builds. The ultimate goal here is to make sure Clang doesn't open/read/parse/evaluate unnecessary module maps.

However, implicit module maps still end up reading all reachable module maps. This patch disables implicit module maps in explicit builds.

Unfortunately, we still need to report some module map files that aren't encoded in PCM files of dependencies: module maps that are necessary to correctly evaluate includes in modules marked as `[no_undeclared_includes]`.

Depends on D120464.

Reviewed By: dexonsmith

Differential Revision: https://reviews.llvm.org/D120465

show more ...

[clang][deps] Return the whole TU command line

The dependency scanner already generates canonical -cc1 command lines that can be used to compile discovered modular dependencies.

For translation uni

[clang][deps] Return the whole TU command line

The dependency scanner already generates canonical -cc1 command lines that can be used to compile discovered modular dependencies.

For translation unit command lines, the scanner only generates additional driver arguments the build system is expected to append to the original command line.

While this works most of the time, there are situations where that's not the case. For example with `-Wunused-command-line-argument`, Clang will complain about the `-fmodules-cache-path=` argument that's not being used in explicit modular builds. Combine that with `-Werror` and the build outright fails.

To prevent such failures, this patch changes the dependency scanner to return the full driver command line to compile the original translation unit. This gives us more opportunities to massage the arguments into something reasonable.

Reviewed By: Bigcheese

Differential Revision: https://reviews.llvm.org/D118986

show more ...

[clang][deps] NFC: Update documentation

In D113473, the dependency scanner stopped emitting "-fmodule-map-file=" arguments. Potential build systems are expected to not add any such arguments on thei

[clang][deps] NFC: Update documentation

In D113473, the dependency scanner stopped emitting "-fmodule-map-file=" arguments. Potential build systems are expected to not add any such arguments on their own. This commit removes mentions of such arguments to avoid confusion.

show more ...

[clang][deps] Don't emit `-fmodule-map-file=`

During explicit modules build, when all modules are provided via `-fmodule-file=<path>` and implicit modules and implicit module maps are disabled (`-fn

[clang][deps] Don't emit `-fmodule-map-file=`

During explicit modules build, when all modules are provided via `-fmodule-file=<path>` and implicit modules and implicit module maps are disabled (`-fno-implicit-modules`, `-fno-implicit-module-maps`), we don't need to load the original module map files at all. This patch stops emitting the `-fmodule-map-file=` arguments we don't need, saving some compilation time due to avoiding parsing such module maps and making the command line shorter.

Reviewed By: bnbarham

Differential Revision: https://reviews.llvm.org/D113473

show more ...

[clang][modules] Cache loads of modules imported by PCH

During explicit modular build, PCM files are typically specified via the `-fmodule-file=<path>` command-line option. Early during the compilat

[clang][modules] Cache loads of modules imported by PCH

During explicit modular build, PCM files are typically specified via the `-fmodule-file=<path>` command-line option. Early during the compilation, Clang uses the `ASTReader` to read their contents and caches the result so that the module isn't loaded implicitly later on. A listener is attached to the `ASTReader` to collect names of the modules read from the PCM files. However, if the PCM has already been loaded previously via PCH:
1. the `ASTReader` doesn't do anything for the second time,
2. the listener is not invoked at all,
3. the module load result is not cached,
4. the compilation fails when attempting to load the module implicitly later on.

This patch solves this problem by attaching the listener to the `ASTReader` for PCH reading as well.

Reviewed By: dexonsmith

Differential Revision: https://reviews.llvm.org/D111560

show more ...

[clang][deps] NFC: Remove CompilationDatabase from DependencyScanningTool API

This patch simplifies the dependency scanner API. Depends on D108980.

Reviewed By: dexonsmith

Differential Revision: h

[clang][deps] NFC: Remove CompilationDatabase from DependencyScanningTool API

This patch simplifies the dependency scanner API. Depends on D108980.

Reviewed By: dexonsmith

Differential Revision: https://reviews.llvm.org/D108981

show more ...

[clang][deps] NFC: Remove CompilationDatabase from DependencyScanningWorker API

This patch simplifies the dependency scanner API. Depends on D108979.

Reviewed By: dexonsmith

Differential Revision:

[clang][deps] NFC: Remove CompilationDatabase from DependencyScanningWorker API

This patch simplifies the dependency scanner API. Depends on D108979.

Reviewed By: dexonsmith

Differential Revision: https://reviews.llvm.org/D108980

show more ...

[clang-scan-deps] Add an API for clang dependency scanner to perform
module lookup by name alone

This removes the need to create a fake source file that imports a
module.

rdar://64538073

Different

[clang-scan-deps] Add an API for clang dependency scanner to perform
module lookup by name alone

This removes the need to create a fake source file that imports a
module.

rdar://64538073

Differential Revision: https://reviews.llvm.org/D109485

show more ...

[clang][deps] Ensure deterministic order of TU '-fmodule-file=' arguments

Translation units with multiple direct modular dependencies trigger a non-deterministic ordering in `clang-scan-deps`. This

[clang][deps] Ensure deterministic order of TU '-fmodule-file=' arguments

Translation units with multiple direct modular dependencies trigger a non-deterministic ordering in `clang-scan-deps`. This boils down to usage of `std::unordered_map`, which gets replaced by `std::map` in this patch.

Depends on D103526.

Reviewed By: dexonsmith

Differential Revision: https://reviews.llvm.org/D103807

show more ...

[clang][deps] NFC: Handle `DependencyOutputOptions` only once

There's no need to pass `DependencyOutputOptions` to each call of `handleFileDependency`, since the options don't ever change.

This pat

[clang][deps] NFC: Handle `DependencyOutputOptions` only once

There's no need to pass `DependencyOutputOptions` to each call of `handleFileDependency`, since the options don't ever change.

This patch adds new `handleDependencyOutputOpts` method to the `DependencyConsumer` interface and the dependency scanner uses it to report the options only once.

Reviewed By: dexonsmith

Differential Revision: https://reviews.llvm.org/D104104

show more ...

[clang][deps] Handle modular dependencies present in PCH

When a translation unit uses a PCH and imports the same modules as the PCH, we'd prefer to resolve to those modules instead of inventing new

[clang][deps] Handle modular dependencies present in PCH

When a translation unit uses a PCH and imports the same modules as the PCH, we'd prefer to resolve to those modules instead of inventing new modules and reporting them as modular dependencies. Since the PCH modules have already been built nudge the compiler to reuse them when deciding whether to build a new module and don't report them as regular modular dependencies.

Depends on D103524 & D103802.

Reviewed By: dexonsmith

Differential Revision: https://reviews.llvm.org/D103526

show more ...

[clang][deps] NFC: Stop assuming the TU's context hash

The context hash of modular dependencies can be different from the context hash of the original translation unit if we modify their `CompilerIn

[clang][deps] NFC: Stop assuming the TU's context hash

The context hash of modular dependencies can be different from the context hash of the original translation unit if we modify their `CompilerInvocation`s.

Stop assuming the TU's context hash everywhere.

No functionality change here, since we're still currently using the unmodified TU CompilerInvocation to compute the context hash.

Reviewed By: dexonsmith

Differential Revision: https://reviews.llvm.org/D102473

show more ...

[clang][deps] Only generate absolute paths when asked to

Add option to `clang-scan-deps` to enable/disable generation of command-line arguments with absolute paths. This is essentially a revert of D

[clang][deps] Only generate absolute paths when asked to

Add option to `clang-scan-deps` to enable/disable generation of command-line arguments with absolute paths. This is essentially a revert of D100533, but with improved naming and added test.

Reviewed By: dexonsmith

Differential Revision: https://reviews.llvm.org/D101051

show more ...

[clang][deps] NFC: Remove unused FullDependencies member

This patch removes the `FullDependencies::AdditionalNonPathCommandLine` member, as it's value-initialized and never mutated.

Reviewed By: de

[clang][deps] NFC: Remove unused FullDependencies member

This patch removes the `FullDependencies::AdditionalNonPathCommandLine` member, as it's value-initialized and never mutated.

Reviewed By: dexonsmith

Differential Revision: https://reviews.llvm.org/D100536

show more ...

[clang][deps] Simplify function discovering .pcm and .modulemap files

This patch simplifies (and renames) the `appendCommonModuleArguments` function.

It no longer tries to construct the command lin

[clang][deps] Simplify function discovering .pcm and .modulemap files

This patch simplifies (and renames) the `appendCommonModuleArguments` function.

It no longer tries to construct the command line for explicitly building modules. Instead, it only performs the DFS traversal of modular dependencies and queries the callbacks to collect paths to `.pcm` and `.modulemap` files.

This makes it more flexible and usable in two contexts:

* Generating additional command line arguments for the main TU in modular build. The `std::vector<std::string>` output parameters can be used to manually generate appropriate command line flags.
* Generate full command line for a module. The output parameters can be the corresponding parts of `CompilerInvocation`. (In a follow-up patch.)

Reviewed By: dexonsmith

Differential Revision: https://reviews.llvm.org/D100531

show more ...

[clang][deps] NFC: Extract ModuleID struct

This patch extracts the `ModuleName` and `ContextHash` members of `ClangModuleDep`, `FullDependencies` and `ModuleDeps` into a single struct `ModuleID`. Th

[clang][deps] NFC: Extract ModuleID struct

This patch extracts the `ModuleName` and `ContextHash` members of `ClangModuleDep`, `FullDependencies` and `ModuleDeps` into a single struct `ModuleID`. This makes it easier to understand how the full dependency graph works.

Reviewed By: Bigcheese, dexonsmith

Differential Revision: https://reviews.llvm.org/D98943

show more ...

Make llvm::StringRef to std::string conversions explicit.

This is how it should've been and brings it more in line with
std::string_view. There should be no functional change here.

This is mostly m

Make llvm::StringRef to std::string conversions explicit.

This is how it should've been and brings it more in line with
std::string_view. There should be no functional change here.

This is mostly mechanical from a custom clang-tidy check, with a lot of
manual fixups. It uncovers a lot of minor inefficiencies.

This doesn't actually modify StringRef yet, I'll do that in a follow-up.

show more ...