[clang][driver] Introduce new -fdriver-only flag

This patch introduces the new -fdriver-only flag which instructs Clang to only execute the driver logic without running individual jobs. In a way, th

[clang][driver] Introduce new -fdriver-only flag

This patch introduces the new -fdriver-only flag which instructs Clang to only execute the driver logic without running individual jobs. In a way, this is very similar to -###, with the following differences:
* it doesn't automatically print all jobs,
* it doesn't avoid side effects (e.g. it will generate compilation database when -MJ is specified).

This flag will be useful in testing D121997.

Reviewed By: dexonsmith, egorzhdan

Differential Revision: https://reviews.llvm.org/D127408

show more ...

[clang] createInvocationFromCommandLine -> createInvocation, delete former. NFC

(Followup from 40c13720a4b977d4347bbde53c52a4d0703823c2)

Differential Revision: https://reviews.llvm.org/D125012

[Driver] Make "upgrade" of -include to include-pch optional; disable in clangd

If clang is passed "-include foo.h", it will rewrite to "-include-pch foo.h.pch"
before passing it to cc1, if foo.h.pch

[Driver] Make "upgrade" of -include to include-pch optional; disable in clangd

If clang is passed "-include foo.h", it will rewrite to "-include-pch foo.h.pch"
before passing it to cc1, if foo.h.pch exists.

Existence is checked, but validity is not. This is probably a reasonable
assumption for the compiler itself, but not for clang-based tools where the
actual compiler may be a different version of clang, or even GCC.
In the end, we lose our -include, we gain a -include-pch that can't be used,
and the file often fails to parse.

I would like to turn this off for all non-clang invocations (i.e.
createInvocationFromCommandLine), but we have explicit tests of this behavior
for libclang and I can't work out the implications of changing it.

Instead this patch:
- makes it optional in the driver, default on (no change)
- makes it optional in createInvocationFromCommandLine, default on (no change)
- changes driver to do IO through the VFS so it can be tested
- tests the option
- turns the option off in clangd where the problem was reported

Subsequent patches should make libclang opt in explicitly and flip the default
for all other tools. It's probably also time to extract an options struct
for createInvocationFromCommandLine.

Fixes https://github.com/clangd/clangd/issues/856
Fixes https://github.com/clangd/vscode-clangd/issues/324

Differential Revision: https://reviews.llvm.org/D124970

show more ...

[Frontend] give createInvocationFromCommandLine an options struct

It's accumulating way too many optional params (see D124970)

While here, improve the name and the documentation.

Differential Revi

[Frontend] give createInvocationFromCommandLine an options struct

It's accumulating way too many optional params (see D124970)

While here, improve the name and the documentation.

Differential Revision: https://reviews.llvm.org/D124971

show more ...

[clangd] Bail-out when an empty compile flag is encountered

Fixes https://github.com/clangd/clangd/issues/865

Differential Revision: https://reviews.llvm.org/D109894

[clangd] Avoid "expected one compiler job" by picking the first eligible job.

This happens in createInvocationWithCommandLine but only clangd currently passes
ShouldRecoverOnErorrs (sic).

One cause

[clangd] Avoid "expected one compiler job" by picking the first eligible job.

This happens in createInvocationWithCommandLine but only clangd currently passes
ShouldRecoverOnErorrs (sic).

One cause of this (with correct command) is several -arch arguments for mac
multi-arch support.

Fixes https://github.com/clangd/clangd/issues/827

Differential Revision: https://reviews.llvm.org/D107632

show more ...

[clangd] Use CommandMangler in TestTU

This makes testing setup look closer to production.

Differential Revision: https://reviews.llvm.org/D106535

[flang][driver] Add the new flang compiler and frontend drivers

Summary:

This is the first patch implementing the new Flang driver as outlined in [1],
[2] & [3]. It creates Flang driver (`flang-new

[flang][driver] Add the new flang compiler and frontend drivers

Summary:

This is the first patch implementing the new Flang driver as outlined in [1],
[2] & [3]. It creates Flang driver (`flang-new`) and Flang frontend driver
(`flang-new -fc1`). These will be renamed as `flang` and `flang -fc1` once the
current Flang throwaway driver, `flang`, can be replaced with `flang-new`.

Currently only 2 options are supported: `-help` and `--version`.

`flang-new` is implemented in terms of libclangDriver, defaulting the driver
mode to `FlangMode` (added to libclangDriver in [4]). This ensures that the
driver runs in Flang mode regardless of the name of the binary inferred from
argv[0].

The design of the new Flang compiler and frontend drivers is inspired by it
counterparts in Clang [3]. Currently, the new Flang compiler and frontend
drivers re-use Clang libraries: clangBasic, clangDriver and clangFrontend.

To identify Flang options, this patch adds FlangOption/FC1Option enums.
Driver::printHelp is updated so that `flang-new` prints only Flang options.
The new Flang driver is disabled by default. To enable it, set
`-DBUILD_FLANG_NEW_DRIVER=ON` when configuring CMake and add clang to
`LLVM_ENABLE_PROJECTS` (e.g. -DLLVM_ENABLE_PROJECTS=“clang;flang;mlir”).

[1] “RFC: new Flang driver - next steps”
http://lists.llvm.org/pipermail/flang-dev/2020-July/000470.html
[2] “RFC: Adding a fortran mode to the clang driver for flang”
http://lists.llvm.org/pipermail/cfe-dev/2019-June/062669.html
[3] “RFC: refactoring libclangDriver/libclangFrontend to share with Flang”
http://lists.llvm.org/pipermail/cfe-dev/2020-July/066393.html
[4] https://reviews.llvm.org/rG6bf55804924d5a1d902925ad080b1a2b57c5c75c

co-authored-by: Andrzej Warzynski <[email protected]>

Reviewed By: richard.barton.arm, sameeranjoshi

Differential Revision: https://reviews.llvm.org/D86089

show more ...

[MC] Pass down argv0 & cc1 cmd-line to the back-end and store in MCTargetOptions

When targetting CodeView, the goal is to store argv0 & cc1 cmd-line in the emitted .OBJ, in order to allow a reproduc

[MC] Pass down argv0 & cc1 cmd-line to the back-end and store in MCTargetOptions

When targetting CodeView, the goal is to store argv0 & cc1 cmd-line in the emitted .OBJ, in order to allow a reproducer from the .OBJ alone.

This patch is to simplify https://reviews.llvm.org/D80833

show more ...

[clangd] Log cc1 args at verbose level.

Summary: This will help debugging driver issues.

Reviewers: kbobyrev

Subscribers: ilya-biryukov, javed.absar, MaskRay, jkorous, arphaman, kadircet, usaxena9

[clangd] Log cc1 args at verbose level.

Summary: This will help debugging driver issues.

Reviewers: kbobyrev

Subscribers: ilya-biryukov, javed.absar, MaskRay, jkorous, arphaman, kadircet, usaxena95, cfe-commits

Tags: #clang

Differential Revision: https://reviews.llvm.org/D70832

show more ...

ArrayRef'ized CompilerInvocation::CreateFromArgs

Subscribers: cfe-commits

Tags: #clang

Differential Revision: https://reviews.llvm.org/D66797

llvm-svn: 370122

[Driver] Add an option for createInvocationFromCommandLine to recover on errors

Summary:
Previously, it would always return nullptr on any error.
This change adds a parameter, controlling whether th

[Driver] Add an option for createInvocationFromCommandLine to recover on errors

Summary:
Previously, it would always return nullptr on any error.
This change adds a parameter, controlling whether the function should
attempt to return a non-null result even if unknown arguments (or other
errors were encountered).

The new behavior is only used in clangd.

Considered an alternative of changing the return value instead of adding
a new parameter, but that would require updating all callsites. Settled
with the parameter to minimize the code changes.

Reviewers: gribozavr

Reviewed By: gribozavr

Subscribers: nridge, jkorous, arphaman, kadircet, cfe-commits

Tags: #clang

Differential Revision: https://reviews.llvm.org/D66731

llvm-svn: 370033

show more ...

[Clang] Migrate llvm::make_unique to std::make_unique

Now that we've moved to C++14, we no longer need the llvm::make_unique
implementation from STLExtras.h. This patch is a mechanical replacement
o

[Clang] Migrate llvm::make_unique to std::make_unique

Now that we've moved to C++14, we no longer need the llvm::make_unique
implementation from STLExtras.h. This patch is a mechanical replacement
of (hopefully) all the llvm::make_unique instances across the monorepo.

Differential revision: https://reviews.llvm.org/D66259

llvm-svn: 368942

show more ...

Update the file headers across all of the LLVM projects in the monorepo
to reflect the new license.

We understand that people may be surprised that we're moving the header
entirely to discuss the ne

Update the file headers across all of the LLVM projects in the monorepo
to reflect the new license.

We understand that people may be surprised that we're moving the header
entirely to discuss the new license. We checked this carefully with the
Foundation's lawyer and we believe this is the correct approach.

Essentially, all code in the project is now made available by the LLVM
project under our new license, so you will see that the license headers
include that license only. Some of our contributors have contributed
code under our old license, and accordingly, we have retained a copy of
our old license notice in the top-level files in each project and
repository.

llvm-svn: 351636

show more ...

Revert "Make clang-based tools find libc++ on MacOS"

This breaks the LLDB bots.

llvm-svn: 346675

Make clang-based tools find libc++ on MacOS

Summary:
When they read compiler args from compile_commands.json.
This change allows to run clang-based tools, like clang-tidy or clangd,
built from head

Make clang-based tools find libc++ on MacOS

Summary:
When they read compiler args from compile_commands.json.
This change allows to run clang-based tools, like clang-tidy or clangd,
built from head using the compile_commands.json file produced for XCode
toolchains.

On MacOS clang can find the C++ standard library relative to the
compiler installation dir.

The logic to do this was based on resource dir as an approximation of
where the compiler is installed. This broke the tools that read
'compile_commands.json' and don't ship with the compiler, as they
typically change resource dir.

To workaround this, we now use compiler install dir detected by the driver
to better mimic the behavior of the original compiler when replaying the
compilations using other tools.

Reviewers: sammccall, arphaman, EricWF

Reviewed By: sammccall

Subscribers: ioeric, christof, kadircet, cfe-commits

Differential Revision: https://reviews.llvm.org/D54310

llvm-svn: 346652

show more ...

Lift VFS from clang to llvm (NFC)

This patch moves the virtual file system form clang to llvm so it can be
used by more projects.

Concretely the patch:
- Moves VirtualFileSystem.{h|cpp} from clang

Lift VFS from clang to llvm (NFC)

This patch moves the virtual file system form clang to llvm so it can be
used by more projects.

Concretely the patch:
- Moves VirtualFileSystem.{h|cpp} from clang/Basic to llvm/Support.
- Moves the corresponding unit test from clang to llvm.
- Moves the vfs namespace from clang::vfs to llvm::vfs.
- Formats the lines affected by this change, mostly this is the result of
the added llvm namespace.

RFC on the mailing list:
http://lists.llvm.org/pipermail/llvm-dev/2018-October/126657.html

Differential revision: https://reviews.llvm.org/D52783

llvm-svn: 344140

show more ...

Use vfs::FileSystem in ASTUnit when creating CompilerInvocation.

Summary: It used to always call into the RealFileSystem before.

Reviewers: bkramer, krasimir, klimek, bruno

Reviewed By: klimek

Su

Use vfs::FileSystem in ASTUnit when creating CompilerInvocation.

Summary: It used to always call into the RealFileSystem before.

Reviewers: bkramer, krasimir, klimek, bruno

Reviewed By: klimek

Subscribers: bruno, cfe-commits

Differential Revision: https://reviews.llvm.org/D34469

llvm-svn: 306549

show more ...

Method loadFromCommandLine should be able to report errors

Now FixedCompilationDatabase::loadFromCommandLine has no means to report
which error occurred if it fails to create compilation object. Thi

Method loadFromCommandLine should be able to report errors

Now FixedCompilationDatabase::loadFromCommandLine has no means to report
which error occurred if it fails to create compilation object. This is
a block for implementing D33013, because after that change driver will
refuse to create compilation if command line contains erroneous options.

This change adds additional argument to loadFromCommandLine, which is
assigned error message text if compilation object was not created. This is
the same way as other methods of CompilationDatabase report failure.

Differential Revision: https://reviews.llvm.org/D33272

llvm-svn: 303741

show more ...

Reapply "IntrusiveRefCntPtr -> std::shared_ptr for CompilerInvocationBase and CodeCompleteConsumer"

Aleksey Shlypanikov pointed out my mistake in migrating an explicit
unique_ptr to auto - I was exp

Reapply "IntrusiveRefCntPtr -> std::shared_ptr for CompilerInvocationBase and CodeCompleteConsumer"

Aleksey Shlypanikov pointed out my mistake in migrating an explicit
unique_ptr to auto - I was expecting the function returned a unique_ptr,
but instead it returned a raw pointer - introducing a leak.

Thanks Aleksey!

This reapplies r291184, reverted in r291249.

llvm-svn: 291270

show more ...

Revert "IntrusiveRefCntPtr -> std::shared_ptr for CompilerInvocationBase and CodeCompleteConsumer"

Caused a memory leak reported by asan. Reverting while I investigate.

This reverts commit r291184.

Revert "IntrusiveRefCntPtr -> std::shared_ptr for CompilerInvocationBase and CodeCompleteConsumer"

Caused a memory leak reported by asan. Reverting while I investigate.

This reverts commit r291184.

llvm-svn: 291249

show more ...

IntrusiveRefCntPtr -> std::shared_ptr for CompilerInvocationBase and CodeCompleteConsumer

llvm-svn: 291184

[CUDA][OpenMP] Create generic offload action

Summary:
This patch replaces the CUDA specific action by a generic offload action. The offload action may have multiple dependences classier in “host” an

[CUDA][OpenMP] Create generic offload action

Summary:
This patch replaces the CUDA specific action by a generic offload action. The offload action may have multiple dependences classier in “host” and “device”. The way this generic offloading action is used is very similar to what is done today by the CUDA implementation: it is used to set a specific toolchain and architecture to its dependences during the generation of jobs.

This patch also proposes propagating the offloading information through the action graph so that that information can be easily retrieved at any time during the generation of commands. This allows e.g. the "clang tool” to evaluate whether CUDA should be supported for the device or host and ptas to easily retrieve the target architecture.

This is an example of how the action graphs would look like (compilation of a single CUDA file with two GPU architectures)
```
0: input, "cudatests.cu", cuda, (host-cuda)
1: preprocessor, {0}, cuda-cpp-output, (host-cuda)
2: compiler, {1}, ir, (host-cuda)
3: input, "cudatests.cu", cuda, (device-cuda, sm_35)
4: preprocessor, {3}, cuda-cpp-output, (device-cuda, sm_35)
5: compiler, {4}, ir, (device-cuda, sm_35)
6: backend, {5}, assembler, (device-cuda, sm_35)
7: assembler, {6}, object, (device-cuda, sm_35)
8: offload, "device-cuda (nvptx64-nvidia-cuda:sm_35)" {7}, object
9: offload, "device-cuda (nvptx64-nvidia-cuda:sm_35)" {6}, assembler
10: input, "cudatests.cu", cuda, (device-cuda, sm_37)
11: preprocessor, {10}, cuda-cpp-output, (device-cuda, sm_37)
12: compiler, {11}, ir, (device-cuda, sm_37)
13: backend, {12}, assembler, (device-cuda, sm_37)
14: assembler, {13}, object, (device-cuda, sm_37)
15: offload, "device-cuda (nvptx64-nvidia-cuda:sm_37)" {14}, object
16: offload, "device-cuda (nvptx64-nvidia-cuda:sm_37)" {13}, assembler
17: linker, {8, 9, 15, 16}, cuda-fatbin, (device-cuda)
18: offload, "host-cuda (powerpc64le-unknown-linux-gnu)" {2}, "device-cuda (nvptx64-nvidia-cuda)" {17}, ir
19: backend, {18}, assembler
20: assembler, {19}, object
21: input, "cuda", object
22: input, "cudart", object
23: linker, {20, 21, 22}, image
```
The changes in this patch pass the existent regression tests (keeps the existent functionality) and resulting binaries execute correctly in a Power8+K40 machine.

Reviewers: echristo, hfinkel, jlebar, ABataev, tra

Subscribers: guansong, andreybokhanko, tcramer, mkuron, cfe-commits, arpith-jacob, carlo.bertolli, caomhin

Differential Revision: https://reviews.llvm.org/D18171

llvm-svn: 275645

show more ...

Rename Action::begin() to Action::input_begin().

Also introduce inputs() that reutnrs an llvm::iterator_range.
Iterating over A->inputs() is much less mysterious than
iterating over *A. No intended

Rename Action::begin() to Action::input_begin().

Also introduce inputs() that reutnrs an llvm::iterator_range.
Iterating over A->inputs() is much less mysterious than
iterating over *A. No intended behavior change.

llvm-svn: 261674

show more ...

[libclang] Add entry points that take a full command line including argv[0].

This provides both a more uniform interface and makes libclang behave like
clang tooling wrt relative paths against argv[

[libclang] Add entry points that take a full command line including argv[0].

This provides both a more uniform interface and makes libclang behave like
clang tooling wrt relative paths against argv[0]. This is necessary for
finding paths to a c++ standard library relative to a clang binary given
in a compilation database. It can also be used to find paths relative to
libclang.so if the full path to it is passed in.

Differential Revision: http://reviews.llvm.org/D14695

llvm-svn: 253466

show more ...