[flang][Driver] Refine _when_ driver diagnostics are formatted

This patch refines //when// driver diagnostics are formatted so that
`flang-new` and `flang-new -fc1` behave consistently with `clang`

[flang][Driver] Refine _when_ driver diagnostics are formatted

This patch refines //when// driver diagnostics are formatted so that
`flang-new` and `flang-new -fc1` behave consistently with `clang` and
`clang -cc1`, respectively. This change only applies to driver diagnostics.
Scanning, parsing and semantic diagnostics are separate and not covered here.

**NEW BEHAVIOUR**
To illustrate the new behaviour, consider the following input file:
```! file.f90
program m
integer :: i = k
end
```
In the following invocations, "error: Semantic errors in file.f90" _will be_
formatted:
```
$ flang-new file.f90
error: Semantic errors in file.f90
./file.f90:2:18: error: Must be a constant value
integer :: i = k
$ flang-new -fc1 -fcolor-diagnostics file.f90
error: Semantic errors in file.f90
./file.f90:2:18: error: Must be a constant value
integer :: i = k
```

However, in the following invocations, "error: Semantic errors in file.f90"
_will not be_ formatted:
```
$ flang-new -fno-color-diagnostics file.f90
error: Semantic errors in file.f90
./file.f90:2:18: error: Must be a constant value
integer :: i = k
$ flang-new -fc1 file.f90
error: Semantic errors in file.f90
./file.f90:2:18: error: Must be a constant value
integer :: i = k
```

Before this change, none of the above would be formatted. Note also that the
default behaviour in `flang-new` is different to `flang-new -fc1` (this is
consistent with Clang).

**NOTES ON IMPLEMENTATION**
Note that the diagnostic options are parsed in `createAndPopulateDiagOpt`s in
driver.cpp. That's where the driver's `DiagnosticEngine` options are set. Like
most command-line compiler driver options, these flags are "claimed" in
Flang.cpp (i.e. when creating a frontend driver invocation) by calling
`getLastArg` rather than in driver.cpp.

In Clang's Options.td, `defm color_diagnostics` is replaced with two separate
definitions: `def fcolor_diagnostics` and def fno_color_diagnostics`. That's
because originally `color_diagnostics` derived from `OptInCC1FFlag`, which is a
multiclass for opt-in options in CC1. In order to preserve the current
behaviour in `clang -cc1` (i.e. to keep `-fno-color-diagnostics` unavailable in
`clang -cc1`) and to implement similar behaviour in `flang-new -fc1`, we can't
re-use `OptInCC1FFlag`.

Formatting is only available in consoles that support it and will normally mean that
the message is printed in bold + color.

Co-authored-by: Andrzej Warzynski <[email protected]>

Reviewed By: rovka

Differential Revision: https://reviews.llvm.org/D126164

show more ...

[flang][driver] Switch to the MLIR coding style in the driver (nfc)

This patch re-factors the driver code in LLVM Flang (frontend +
compiler) to use the MLIR style. For more context, please see:
htt

[flang][driver] Switch to the MLIR coding style in the driver (nfc)

This patch re-factors the driver code in LLVM Flang (frontend +
compiler) to use the MLIR style. For more context, please see:
https://discourse.llvm.org/t/rfc-coding-style-in-the-driver/

Most changes here are rather self-explanatory. Accessors are renamed to
be more consistent with the rest of LLVM (e.g. allSource -->
getAllSources). Additionally, MLIR clang-tidy files are added in the
affected directories.

clang-tidy and clang-format files were copied from MLIR. Small
additional changes are made to silence clang-tidy/clang-format
warnings.

[1] https://mlir.llvm.org/getting_started/DeveloperGuide/

Differential Revision: https://reviews.llvm.org/D125007

show more ...

[flang][driver] Re-organise the code-gen actions (nfc)

All frontend actions that generate code (MLIR, LLVM IR/BC,
Assembly/Object Code) are re-factored as essentially one action,
`CodeGenAction`, w

[flang][driver] Re-organise the code-gen actions (nfc)

All frontend actions that generate code (MLIR, LLVM IR/BC,
Assembly/Object Code) are re-factored as essentially one action,
`CodeGenAction`, with minor specialisations. To facilate all this,
`CodeGenAction` is extended to hold `TargetMachine` and backend action
type (MLIR vs LLVM IR vs LLVM BC vs Assembly vs Object Code).

`CodeGenAction` is no longer a pure abstract class and the
corresponding `ExecuteAction` is implemented so that it covers all use
cases. All this allows a much better code re-use.

Key functionality is extracted into some helpful hooks:
* `SetUpTargetMachine`
* `GetOutputStream`
* `EmitObjectCodeHelper`
* `EmitBCHelper`
I hope that this clarifies the overall structure. I suspect that we may
need to revisit this again as the functionality grows in complexity.

Differential Revision: https://reviews.llvm.org/D124665

show more ...

[flang][driver] NFC: Make code more in line with LLVM style

This patch basically implements [1] in ExecuteCompilerInvocation.cpp. It
also:
* replaces `CreateFrontendBaseAction` with `CreateFronten

[flang][driver] NFC: Make code more in line with LLVM style

This patch basically implements [1] in ExecuteCompilerInvocation.cpp. It
also:
* replaces `CreateFrontendBaseAction` with `CreateFrontendAction`
(only one method is needed ATM, this change removes the extra
indirection)
* removes `InvalidAction` from the `ActionKind` enum (I don't think it
adds much and keeping it would mean adding a new void case in
`CreateFrontendAction`)
* sets the default frontend action in FrontendOptions.h to
`ParseSyntaxOnly` (note that this is still overridden independently
in `ParseFrontendArg` in CompilerInvocation.cpp)

No new functionality is added, hence no tests.

[1] https://llvm.org/docs/CodingStandards.html#don-t-use-default-labels-in-fully-covered-switches-over-enumerations

Differential Revision: https://reviews.llvm.org/D124245

show more ...

[flang][driver] Add support for `-mmlir`

The semantics of `-mmlir` are identical to `-mllvm`. The only notable
difference is that `-mmlir` options should be forwarded to MLIR rather
than LLVM.

Note

[flang][driver] Add support for `-mmlir`

The semantics of `-mmlir` are identical to `-mllvm`. The only notable
difference is that `-mmlir` options should be forwarded to MLIR rather
than LLVM.

Note that MLIR llvm::cl options are lazily constructed on demand (see
the definition of options in PassManagerOptions.cpp). This means that:
* MLIR global options are only visible when explicitly initialised and
displayed only when using `-mmlir --help`,
* Flang and LLVM global options are always visible and displayed when
using either `-mllvm -help` or `-mmlir --help`.

In other words, `-mmlir --help` is a superset of `-mllvm --help`. This is not
ideal, but we'd need to refactor all option definitions in Flang and
LLVM to improve this. I suggesting leaving this for later.

Differential Revision: https://reviews.llvm.org/D123297

show more ...

[flang][driver] Add support for generating LLVM bytecode files

Support for generating LLVM BC files is added in Flang's compiler and
frontend drivers. This requires the `BitcodeWriterPass` pass to b

[flang][driver] Add support for generating LLVM bytecode files

Support for generating LLVM BC files is added in Flang's compiler and
frontend drivers. This requires the `BitcodeWriterPass` pass to be run
on the input LLVM IR module and is implemented as a dedicated frontend
aciton. The new functionality as seen by the user (compiler driver):
```
flang-new -c -emit-llvm file.90
```
or (frontend driver):
```
flang-new -fc1 -emit-llvm-bc file.f90
```

The new behaviour is consistent with `clang` and `clang -cc1`.

Differential Revision: https://reviews.llvm.org/D123211

show more ...

[flang][driver] Add support for `-mllvm`

This option is added in both `flang-new` (the compiler driver) and
`flang-new -fc1` (the frontend driver). The semantics are consistent
with `clang` and `cla

[flang][driver] Add support for `-mllvm`

This option is added in both `flang-new` (the compiler driver) and
`flang-new -fc1` (the frontend driver). The semantics are consistent
with `clang` and `clang -cc1`.

As Flang does not run any LLVM passes when invoked with `-emit-llvm`
(i.e. `flang-new -S -emit-llvm <file>`), the tests use
`-S`/`-c`/`-emit-obj` instead. These options require an LLVM backend to
be run by the driver to generate the output (this makese `-mllvm`
relevant here).

Differential Revision: https://reviews.llvm.org/D121374

show more ...

[flang][driver] Add support for -S and implement -c/-emit-obj

This patch adds support for:
* `-S` in Flang's compiler and frontend drivers,
and implements:
* `-emit-obj` in Flang's frontend driv

[flang][driver] Add support for -S and implement -c/-emit-obj

This patch adds support for:
* `-S` in Flang's compiler and frontend drivers,
and implements:
* `-emit-obj` in Flang's frontend driver and `-c` in Flang's compiler
driver (this is consistent with Clang).
(these options were already available before, but only as placeholders).
The semantics of these options in Clang and Flang are identical.

The `EmitObjAction` frontend action is renamed as `BackendAction`. This
new name more accurately reflects the fact that this action will
primarily run the code-gen/backend pipeline in LLVM. It also makes more
sense as an action implementing both `-emit-obj` and `-S` (originally,
it was just `-emit-obj`).

`tripleName` from FirContext.cpp is deleted and, when a target triple is
required, `mlir::LLVM::LLVMDialect::getTargetTripleAttrName()` is used
instead. In practice, this means that `fir.triple` is replaced with
`llvm.target_triple`. The former was effectively ignored. The latter is
used when lowering from the LLVM dialect in MLIR to LLVM IR (i.e. it's
embedded in the generated LLVM IR module). The driver can then re-use
it when configuring the backend. With this change, the LLVM IR files
generated by e.g. `tco` will from now on contain the correct target
triple.

The code-gen.f90 test is replaced with code-gen-x86.f90 and
code-gen-aarch64.f90. With 2 seperate files we can verify that
`--target` is correctly taken into account. LIT configuration is updated
to enable e.g.:
```
! REQUIRES: aarch64-registered-target
```

Differential Revision: https://reviews.llvm.org/D120568

show more ...

[flang][driver] Add support for `-debug-dump-pft`

This patch adds support for dumping the pre-FIR tree in `flang-new
-fc1`, i.e. Flang's frontend driver. This flag is functionally identical
to `-pft

[flang][driver] Add support for `-debug-dump-pft`

This patch adds support for dumping the pre-FIR tree in `flang-new
-fc1`, i.e. Flang's frontend driver. This flag is functionally identical
to `-pft-test` in `bbc` and semantically similar to
`-fdebug-dump-parse-tree` from `flang-new -fc1`.

Differential Revision: https://reviews.llvm.org/D121198

show more ...

[flang][driver] Add support for `-emit-llvm`

This patch adds support for the `-emit-llvm` option in the frontend
driver (i.e. `flang-new -fc1`). Similarly to Clang, `flang-new -fc1
-emit-llvm file.f

[flang][driver] Add support for `-emit-llvm`

This patch adds support for the `-emit-llvm` option in the frontend
driver (i.e. `flang-new -fc1`). Similarly to Clang, `flang-new -fc1
-emit-llvm file.f` will generate a textual LLVM IR file.

Depends on D118985

Differential Revision: https://reviews.llvm.org/D119012

show more ...

[flang][driver] Add support for `-emit-mlir`

This patch adds support for generating MLIR files in Flang's frontend
driver (i.e. `flang-new -fc1`). `-emit-fir` is added as an alias for
`-emit-mlir`.

[flang][driver] Add support for `-emit-mlir`

This patch adds support for generating MLIR files in Flang's frontend
driver (i.e. `flang-new -fc1`). `-emit-fir` is added as an alias for
`-emit-mlir`. We may want to decide to split the two in the future.

A new parent class for code-gen frontend actions is introduced:
`CodeGenAction`. We will be using this class to encapsulate logic shared
between all code-generation actions, but not required otherwise. For
now, it will:
* run prescanning, parsing and semantic checks,
* lower the input to MLIR.
`EmitObjAction` is updated to inherit from this class. This means that
the behaviour of `flang-new -fc1 -emit-obj` is also updated (previously,
it would just exit immediately). This change required
`flang/test/Driver/syntax-only.f90` to be updated.

For `-emit-fir`, a specialisation of `CodeGenAction` is introduced:
`EmitMLIRAction`. The key logic for this class is implemented in
`EmitMLIRAction::ExecuteAction`.

Differential Revision: https://reviews.llvm.org/D118985

show more ...

[flang][driver] Add support for Frontend Plugins

Introducing a plugin API and a simple HelloWorld Plugin example.
This patch adds the `-load` and `-plugin` flags to frontend driver and
the code arou

[flang][driver] Add support for Frontend Plugins

Introducing a plugin API and a simple HelloWorld Plugin example.
This patch adds the `-load` and `-plugin` flags to frontend driver and
the code around using custom frontend actions from within a plugin
shared library object.

It also adds to the Driver-help test to check the help option with the
updated driver flags.

Additionally, the patch creates a plugin-example test to check the
HelloWorld plugin example runs correctly. As part of this, a new CMake
flag (`FLANG_BUILD_EXAMPLES`) is added to allow the example to be built
and for the test to run.

This Plugin API has only been tested on Linux.

Reviewed By: awarzynski

Differential Revision: https://reviews.llvm.org/D106137

show more ...

[flang][nfc] Fix variable names in `FrontendOptions` & `PreprocessorOptions`

As all member variables in `FrontendOptions` and `PreprocessorOptions`
are public, we should be naming them as `variable`

[flang][nfc] Fix variable names in `FrontendOptions` & `PreprocessorOptions`

As all member variables in `FrontendOptions` and `PreprocessorOptions`
are public, we should be naming them as `variable` rather than
`variable_` [1]. This patch fixes that.

Also, `FrontendOptions` & `PreprocessorOptions` are re-defined as a
structs rather than classes (all fields are meant to be public).

[1]
https://github.com/llvm/llvm-project/blob/main/flang/docs/C%2B%2Bstyle.md#naming

Differential Revision: https://reviews.llvm.org/D107062

show more ...

[flang] Produce proper "preprocessor output" for -E option

Rename the current -E option to "-E -Xflang -fno-reformat".

Add a new Parsing::EmitPreprocessedSource() routine to convert the
cooked char

[flang] Produce proper "preprocessor output" for -E option

Rename the current -E option to "-E -Xflang -fno-reformat".

Add a new Parsing::EmitPreprocessedSource() routine to convert the
cooked character stream output of the prescanner back to something
more closely resembling output from a traditional preprocessor;
call this new routine when -E appears.

The new -E output is suitable for use as fixed form Fortran source to
compilation by (one hopes) any Fortran compiler. If the original
top-level source file had been free form source, the output will be
suitable for use as free form source as well; otherwise there may be
diagnostics about missing spaces if they were indeed absent in the
original fixed form source.

Unless the -P option appears, #line directives are interspersed
with the output (but be advised, f18 will ignore these if presented
with them in a later compilation).

An effort has been made to preserve original alphabetic character case
and source indentation.

Add -P and -fno-reformat to the new drivers.

Tweak test options to avoid confusion with prior -E output; use
-fno-reformat where needed, but prefer to keep -E, sometimes
in concert with -P, on most, updating expected results accordingly.

Differential Revision: https://reviews.llvm.org/D106727

show more ...

[flang] Revert "PoC for Flang Driver Plugins"

This patch has not been reviewed and was commited by accident.

This reverts commit 788a5d4afe6407e647454a9832a7b4a27fba06bf.

PoC for Flang Driver Plugins

[OptTable] Rename PrintHelp to printHelp

To be consistent with other member functions and match the coding standard.

[flang][driver] Add `-fdebug-dump-all`

The new option will run the semantic checks and then dump the parse tree
and all the symbols. This is equivalent to running the driver twice,
once with `-fdebu

[flang][driver] Add `-fdebug-dump-all`

The new option will run the semantic checks and then dump the parse tree
and all the symbols. This is equivalent to running the driver twice,
once with `-fdebug-dump-parse-tree` and then with
the `-fdebug-dump-symbols` action flag.

Currently we wouldn't be able to achieve the same by simply running:
```
flang-new -fc1 -fdebug-dump-parse-tree -fdebug-dump-symbols <input-file>
```
That's because the new driver will only run one frontend action per
invocation (both of the flags used here are action flags). Diverging
from this design would lead to costly compromises and it's best avoided.

We may want to consider re-designing our debugging actions (and action
options) in the future so that there's more code re-use. For now, I'm
focusing on making sure that we support all the major cases requested by
our users.

Differential Revision: https://reviews.llvm.org/D104305

show more ...

[flang][driver] Add support for the "-init-only" option

Adding the `-init-only` option and corresponding frontend action to
generate a diagnostic.

`-init-only` vs `-test-io`:
`-init-only` ignores t

[flang][driver] Add support for the "-init-only" option

Adding the `-init-only` option and corresponding frontend action to
generate a diagnostic.

`-init-only` vs `-test-io`:
`-init-only` ignores the input (it never calls the prescanner)
`-test-io` is similar to `-init-only`, but does read and print the input
without calling the prescanner.

This patch also adds a Driver test to check this action.

Reviewed By: awarzynski, AMDChirag

Differential Revision: https://reviews.llvm.org/D102849

show more ...

[flang][driver] Add support for `-fget-definition`

This patch adds `-fget-definition` to `flang-new`. The semantics of this
option are identical in both drivers. The error message in the
"throwaway"

[flang][driver] Add support for `-fget-definition`

This patch adds `-fget-definition` to `flang-new`. The semantics of this
option are identical in both drivers. The error message in the
"throwaway" driver is updated so that it matches the one from
`flang-new` (which is auto-generated and cannot be changed easily).

Tests are updated accordingly. A dedicated test for error handling was
added: get-definition.f90 (for the sake of simplicity,
getdefinition01.f90 no longer tests for errors).

The `ParseFrontendArgs` function is updated so that it can return
errors. This change is required in order to report invalid values
following `-fget-definition`.

The actual implementation of `GetDefinitionAction::ExecuteAction()` was
extracted from f18.cpp (i.e. the bit that deals with
`-fget-definition`).

Depends on: https://reviews.llvm.org/D100556

Differential Revision: https://reviews.llvm.org/D100558

show more ...

[flang][driver] Add debug options not requiring semantic checks

This patch adds two debugging options in the new Flang driver
(flang-new):
*fdebug-unparse-no-sema
*fdebug-dump-parse-tree-no-sema

[flang][driver] Add debug options not requiring semantic checks

This patch adds two debugging options in the new Flang driver
(flang-new):
*fdebug-unparse-no-sema
*fdebug-dump-parse-tree-no-sema
Each of these options combines two options from the "throwaway" driver
(left: f18, right: flang-new):
* `-fdebug-uparse -fdebug-no-semantics` --> `-fdebug-unparse-no-sema`
* `-fdebug-dump-parse-tree -fdebug-no-semantics` -->
`-fdebug-dump-parse-tree-no-sema`

There are no plans to implement `-fdebug-no-semantics` in the new
driver. Such option would be too powerful. Also, it would only make
sense when combined with specific frontend actions (`-fdebug-unparse`
and `-fdebug-dump-parse-tree`). Instead, this patch adds 2 specialised
options listed above. Each of these is implemented through a dedicated
FrontendAction (also added).

The new frontend actions are implemented in terms of a new abstract base
action: `PrescanAndSemaAction`. This new base class was required so that
we can have finer control over what steps within the frontend are
executed:
* `PrescanAction`: run the _prescanner_
* `PrescanAndSemaAction`: run the _prescanner_ and the _parser_ (new
in this patch)
* `PrescanAndSemaAction`: run the _prescanner_, _parser_ and run the
_semantic checks_

This patch introduces `PrescanAndParseAction::BeginSourceFileAction`.
Apart from the semantic checks removed at the end, it is similar to
`PrescanAndSemaAction::BeginSourceFileAction`.

Differential Revision: https://reviews.llvm.org/D99645

show more ...

[flang][driver] Add support for `-fget-symbols-sources`

Adds support for `-fget-symbols-sources` in the new Flang driver. All
relevant tests are updated to use the new driver when
`FLANG_BUILD_NEW_D

[flang][driver] Add support for `-fget-symbols-sources`

Adds support for `-fget-symbols-sources` in the new Flang driver. All
relevant tests are updated to use the new driver when
`FLANG_BUILD_NEW_DRIVER` is set.

`RUN` lines in tests are updated so `-fsyntax-only`
comes before `-fget-symbols-sources`. That's because:
* both `-fsyntax-only` and `-fget-symbols-sources` are
action flags, and
* the new driver, flang-new, will only consider the right-most
action flag.
In other words, this change is needed so that the tests work with both
`f18` (requires both flags) and `flang-new` (only considers the last
action flag).

Differential Revision: https://reviews.llvm.org/D98191

show more ...

[flang][driver] Add `-fdebug-dump-parsing-log`

This patch adds `-fdebug-dump-parsing-log` in the new driver. This option is
semantically identical to `-fdebug-instrumented-parse` in `f18` (the
forme

[flang][driver] Add `-fdebug-dump-parsing-log`

This patch adds `-fdebug-dump-parsing-log` in the new driver. This option is
semantically identical to `-fdebug-instrumented-parse` in `f18` (the
former is added as an alias in `f18`).

As dumping the parsing log makes only sense for instrumented parses, we
set Fortran::parser::Options::instrumentedParse to `True` when
`-fdebug-dump-parsing-log` is used. This is consistent with `f18`.

To facilitate tweaking the configuration of the frontend based on the
action being requested, `setUpFrontendBasedOnAction` is introduced in
CompilerInvocation.cpp.

Differential Revision: https://reviews.llvm.org/D97457

show more ...

[flang][driver] Add debug measure-parse-tree and pre-fir-tree options

Add the following options:
* -fdebug-measure-parse-tree
* -fdebug-pre-fir-tree

Summary of changes:
- Add 2 new frontend actions

[flang][driver] Add debug measure-parse-tree and pre-fir-tree options

Add the following options:
* -fdebug-measure-parse-tree
* -fdebug-pre-fir-tree

Summary of changes:
- Add 2 new frontend actions: DebugMeasureParseTreeAction and DebugPreFIRTreeAction
- Add MeasurementVisitor to FrontendActions.h
- Make reportFatalSemanticErrors return true if there are any fatal errors
- Port most of the `-fdebug-pre-fir-tree` tests to use the new driver if built, otherwise use f18.

Differential Revision: https://reviews.llvm.org/D96884

show more ...

[flang][driver] Add debug dump options

Add the following options:
* -fdebug-dump-symbols
* -fdebug-dump-parse-tree
* -fdebug-dump-provenance

Summary of changes:
- Add 3 new frontend actions: DebugD

[flang][driver] Add debug dump options

Add the following options:
* -fdebug-dump-symbols
* -fdebug-dump-parse-tree
* -fdebug-dump-provenance

Summary of changes:
- Add 3 new frontend actions: DebugDumpSymbolsAction, DebugDumpParseTreeAction and DebugDumpProvenanceAction
- Add a unique pointer to the Semantics instance created in PrescanAndSemaAction
- Move fatal semantic error reporting to its own method, FrontendActions#reportFatalSemanticErrors
- Port most tests using `-fdebug-dump-symbols` and `-fdebug-dump-parse-tree` to the new driver if built, otherwise default to f18

Differential Revision: https://reviews.llvm.org/D96716

show more ...