[flang] Add & use a better visit() (take 2)

Adds flang/include/flang/Common/log2-visit.h, which defines
a Fortran::common::visit() template function that is a drop-in
replacement for std::visit().

[flang] Add & use a better visit() (take 2)

Adds flang/include/flang/Common/log2-visit.h, which defines
a Fortran::common::visit() template function that is a drop-in
replacement for std::visit(). Modifies most use sites in
the front-end and runtime to use common::visit().

The C++ standard mandates that std::visit() have O(1) execution
time, which forces implementations to build dispatch tables.
This new common::visit() is O(log2 N) in the number of alternatives
in a variant<>, but that N tends to be small and so this change
produces a fairly significant improvement in compiler build
memory requirements, a 5-10% improvement in compiler build time,
and a small improvement in compiler execution time.

Building with -DFLANG_USE_STD_VISIT causes common::visit()
to be an alias for std::visit().

Calls to common::visit() with multiple variant arguments
are referred to std::visit(), pending further work.

This change is enabled only for GCC builds with GCC >= 9;
an earlier attempt (D122441) ran into bugs in some versions of
clang and was reverted rather than simply disabled; and it is
not well tested with MSVC. In non-GCC and older GCC builds,
common::visit() is simply an alias for std::visit().

show more ...

Revert "[flang] Add & use a better visit()"

This reverts commit 2ab9990c9eb79682a4d4b183dfbc7612d3e55328. It has
caused multiple build failures:
* https://lab.llvm.org/buildbot/#/builders/177/build

Revert "[flang] Add & use a better visit()"

This reverts commit 2ab9990c9eb79682a4d4b183dfbc7612d3e55328. It has
caused multiple build failures:
* https://lab.llvm.org/buildbot/#/builders/177/builds/4346
* https://lab.llvm.org/buildbot/#/builders/180/builds/3803
* https://lab.llvm.org/buildbot/#/builders/175/builds/10419
* https://lab.llvm.org/buildbot/#/builders/191/builds/4318
* https://lab.llvm.org/buildbot/#/builders/173/builds/4274
* https://lab.llvm.org/buildbot/#/builders/181/builds/4297

All these bots failed with a time-out:
```
command timed out: 1200 seconds without output running [b'ninja', b'-j', b'32'], attempting to kill
```
I'm guessing that that's due to template instantiations failing at some
point (https://reviews.llvm.org/D122441 introduced a custom
implementation of std::visit). Everything seems fine when either:
* building on X86 with GCC or Clang (tested with GCC 9.3 and Clang 12)
* building on AArch64 with GCC (tested with GCC 11)

show more ...

[flang] Add & use a better visit()

Adds flang/include/flang/Common/visit.h, which defines
a Fortran::common::visit() template function that is a drop-in
replacement for std::visit(). Modifies most

[flang] Add & use a better visit()

Adds flang/include/flang/Common/visit.h, which defines
a Fortran::common::visit() template function that is a drop-in
replacement for std::visit(). Modifies most use sites in
the front-end and runtime to use common::visit().

The C++ standard mandates that std::visit() have O(1) execution
time, which forces implementations to build dispatch tables.
This new common::visit() is O(log2 N) in the number of alternatives
in a variant<>, but that N tends to be small and so this change
produces a fairly significant improvement in compiler build
memory requirements, a 5-10% improvement in compiler build time,
and a small improvement in compiler execution time.

Building with -DFLANG_USE_STD_VISIT causes common::visit()
to be an alias for std::visit().

Calls to common::visit() with multiple variant arguments
are referred to std::visit(), pending further work.

Differential Revision: https://reviews.llvm.org/D122441

show more ...

[flang] Avoid crash case in provenance mapping

When a contiguous range of a cooked character stream is being
mapped to a range of source provenance, the code was assuming
that the "end()" position o

[flang] Avoid crash case in provenance mapping

When a contiguous range of a cooked character stream is being
mapped to a range of source provenance, the code was assuming
that the "end()" position of the input range -- being the character
immediately after the range -- would also follow the range's
source provenance. This isn't always the case.

Modify the code to work with the true last character of the
input range (at end()-1) and to also cope with cases when that
last position truly maps to an earlier provenance, which can happen
when the prescanner has inserted a space into the cooked character
stream.

Differential Revision: https://reviews.llvm.org/D121124

show more ...

[flang] Distinguish intrinsic from non-intrinsic modules

For "USE, INTRINSIC", search only for intrinsic modules;
for "USE, NON_INTRINSIC", do not recognize intrinsic modules.
Allow modules of both

[flang] Distinguish intrinsic from non-intrinsic modules

For "USE, INTRINSIC", search only for intrinsic modules;
for "USE, NON_INTRINSIC", do not recognize intrinsic modules.
Allow modules of both kinds with the same name to be used in
the same source file (but not in the same scoping unit, a
constraint of the standard that is now enforced).

The symbol table's scope tree now has a single instance of
a scope with a new kind, IntrinsicModules, whose children are
the USE'd intrinsic modules (explicit or not). This separate
"top-level" scope is a child of the single global scope and
it allows both intrinsic and non-intrinsic modules of the same
name to exist in the symbol table. Intrinsic modules' scopes'
symbols now have the INTRINSIC attribute set.

The search path directories need to make a distinction between
regular directories and the one(s) that point(s) to intrinsic
modules. I allow for multiple intrinsic module directories in
the second search path, although only one is needed today.

Differential Revision: https://reviews.llvm.org/D118631

show more ...

[flang] Produce proper "preprocessor output" for -E option

Rename the current -E option to "-E -Xflang -fno-reformat".

Add a new Parsing::EmitPreprocessedSource() routine to convert the
cooked char

[flang] Produce proper "preprocessor output" for -E option

Rename the current -E option to "-E -Xflang -fno-reformat".

Add a new Parsing::EmitPreprocessedSource() routine to convert the
cooked character stream output of the prescanner back to something
more closely resembling output from a traditional preprocessor;
call this new routine when -E appears.

The new -E output is suitable for use as fixed form Fortran source to
compilation by (one hopes) any Fortran compiler. If the original
top-level source file had been free form source, the output will be
suitable for use as free form source as well; otherwise there may be
diagnostics about missing spaces if they were indeed absent in the
original fixed form source.

Unless the -P option appears, #line directives are interspersed
with the output (but be advised, f18 will ignore these if presented
with them in a later compilation).

An effort has been made to preserve original alphabetic character case
and source indentation.

Add -P and -fno-reformat to the new drivers.

Tweak test options to avoid confusion with prior -E output; use
-fno-reformat where needed, but prefer to keep -E, sometimes
in concert with -P, on most, updating expected results accordingly.

Differential Revision: https://reviews.llvm.org/D106727

show more ...

[flang] Fix crash from -DMACRO= with empty replacement

Such macros were exposing some edge cases in the preprocessor
regarding empty tokens.

Differential Revision: https://reviews.llvm.org/D101207

[flang] Refine symbol sorting

Replace semantics::SymbolSet with alternatives that clarify
whether the set should order its contents by source position
or not. This matters because positionally-orde

[flang] Refine symbol sorting

Replace semantics::SymbolSet with alternatives that clarify
whether the set should order its contents by source position
or not. This matters because positionally-ordered sets must
not be used for Symbols that might be subjected to name
replacement during name resolution, and address-ordered
sets must not be used (without sorting) in circumstances
where the order of their contents affects the output of the
compiler.

All set<> and map<> instances in the compiler that are keyed
by Symbols now have explicit Compare types in their template
instantiations. Symbol::operator< is no more.

Differential Revision: https://reviews.llvm.org/D98878

show more ...

[flang] Order Symbols by source provenance

In parser::AllCookedSources, implement a map from CharBlocks to
the CookedSource instances that they cover. This permits a fast
Find() operation based on

[flang] Order Symbols by source provenance

In parser::AllCookedSources, implement a map from CharBlocks to
the CookedSource instances that they cover. This permits a fast
Find() operation based on std::map::equal_range to map a CharBlock
to its enclosing CookedSource instance.

Add a creation order number to each CookedSource. This allows
AllCookedSources to provide a Precedes(x,y) predicate that is a
true source stream ordering between two CharBlocks -- x is less
than y if it is in an earlier CookedSource, or in the same
CookedSource at an earlier position.

Add a reference to the singleton SemanticsContext to each Scope.

All of this allows operator< to be implemented on Symbols by
means of a true source ordering. From a Symbol, we get to
its Scope, then to the SemanticsContext, and then use its
AllCookedSources reference to call Precedes().

Differential Revision: https://reviews.llvm.org/D98743

show more ...

[flang] Search for #include "file" in right directory (take 2)

Make the #include "file" preprocessing directive begin its
search in the same directory as the file containing the directive,
as other

[flang] Search for #include "file" in right directory (take 2)

Make the #include "file" preprocessing directive begin its
search in the same directory as the file containing the directive,
as other preprocessors and our Fortran INCLUDE statement do.

Avoid current working directory for all source files except the original.

Resolve tests.

Differential Revision: https://reviews.llvm.org/D95481

show more ...

Revert "[flang] Search for #include "file" in right directory"

This reverts commit d987b61b1dce9948801ac37704477e7c257100b1.

As pointed out in https://reviews.llvm.org/D95388, the reverted commit
c

Revert "[flang] Search for #include "file" in right directory"

This reverts commit d987b61b1dce9948801ac37704477e7c257100b1.

As pointed out in https://reviews.llvm.org/D95388, the reverted commit
causes build failures in the following Flang buildbots:
* http://lab.llvm.org:8011/#/builders/32/builds/2642
* http://lab.llvm.org:8011/#/builders/33/builds/2131
* http://lab.llvm.org:8011/#/builders/135/builds/1473
* http://lab.llvm.org:8011/#/builders/66/builds/1559
* http://lab.llvm.org:8011/#/builders/134/builds/1409
* http://lab.llvm.org:8011/#/builders/132/builds/1817
I'm guessing that the patch was only tested with
`FLANG_BUILD_NEW_DRIVER=Off` (i.e. the default). The builders listed
above set `FLANG_BUILD_NEW_DRIVER` to `On`.

Although fixing the build is relatively easy, the reverted patch
modifies the behaviour of the frontend, which breaks driver tests. In
particular, in https://reviews.llvm.org/D93453 support for `-I` was
added that depends on the current behaviour. The reverted patch
changes that behaviour. Either the tests have to be updated or the
change fine-tuned.

show more ...

[flang] Search for #include "file" in right directory

Make the #include "file" preprocessing directive begin its
search in the same directory as the file containing the directive,
as other preproces

[flang] Search for #include "file" in right directory

Make the #include "file" preprocessing directive begin its
search in the same directory as the file containing the directive,
as other preprocessors and our Fortran INCLUDE statement do.

Avoid current working directory for all source files after the original.

Differential Revision: https://reviews.llvm.org/D95388

show more ...

[flang] More clean-up of CookedSource API

The std::string holding the content of a CookedSource no longer
needs to be exposed in its API after the recent work that allows
the parsing context to hold

[flang] More clean-up of CookedSource API

The std::string holding the content of a CookedSource no longer
needs to be exposed in its API after the recent work that allows
the parsing context to hold multiple instances of a CookedSource.
So clean the API. These changes were extracted from some work in
progress that was made easier by the API changes.

Differential Revision: https://reviews.llvm.org/D87635

show more ...

[flang] Support multiple CookedSource instances

These are owned by an instance of a new class AllCookedSources.

This removes the need for a Scope to own a string containing
a module's cooked source

[flang] Support multiple CookedSource instances

These are owned by an instance of a new class AllCookedSources.

This removes the need for a Scope to own a string containing
a module's cooked source stream, and will enable errors to be
emitted when parsing module files in the future.

Differential Revision: https://reviews.llvm.org/D86891

show more ...

[flang] Reformat with latest clang-format and .clang-format

Original-commit: flang-compiler/f18@9fe84f45d7fd685051004678d6b5775dcc4c6f8f
Reviewed-on: https://github.com/flang-compiler/f18/pull/1094

[flang] Replace manual mmap with llvm::MemoryBuffer

The previous code had handling for cases when too many file descriptors may be
opened; this is not necessary with MemoryBuffer as the file descrip

[flang] Replace manual mmap with llvm::MemoryBuffer

The previous code had handling for cases when too many file descriptors may be
opened; this is not necessary with MemoryBuffer as the file descriptors are
closed after the mapping occurs. MemoryBuffer also internally handles the case
where a file is small and therefore an mmap is bad for performance; such files
are simply copied to memory after being opened.

Many places elsewhere in the code assume that the buffer is not empty, and the
old file opening code handles this by replacing an empty file with a buffer
containing a single newline. That behavior is now kept in the new MemoryBuffer
based code.

Original-commit: flang-compiler/f18@d34df8435127d847867e2c0bb157def9f20f4202
Reviewed-on: https://github.com/flang-compiler/f18/pull/1032

show more ...

[flang] [LLVMify F18] Replace the use std::ostream with LLVM streams llvm::ostream

This patch replaces the occurrence of std::ostream by llvm::raw_ostream.
In LLVM Coding Standards[1] "All new code

[flang] [LLVMify F18] Replace the use std::ostream with LLVM streams llvm::ostream

This patch replaces the occurrence of std::ostream by llvm::raw_ostream.
In LLVM Coding Standards[1] "All new code should use raw_ostream
instead of ostream".[1]

As a consequence, this patch also replaces the use of:
std::stringstream by llvm::raw_string_ostream or llvm::raw_ostream*
std::ofstream by llvm::raw_fd_ostream
std::endl by '\n' and flush()[2]
std::cout by llvm::outs() and
std::cerr by llvm::errs()

It also replaces std::strerro by llvm::sys::StrError** , but NOT in Fortran
runtime libraries

*std::stringstream were replaced by llvm::raw_ostream in all methods that
used std::stringstream as a parameter. Moreover, it removes the pointers to
these streams.

[1]https://llvm.org/docs/CodingStandards.html
[2]https://releases.llvm.org/2.5/docs/CodingStandards.html#ll_avoidendl

Signed-off-by: Caroline Concatto <[email protected]>

Running clang-format-7

Signed-off-by: Caroline Concatto <[email protected]>

Removing residue of ostream library

Signed-off-by: Caroline Concatto <[email protected]>

Original-commit: flang-compiler/f18@a3507d44b8911e6024033aa583c1dc54e0eb89fd
Reviewed-on: https://github.com/flang-compiler/f18/pull/1047

show more ...

[flang] [LLVMify F18] Compiler module folders should have capitalised names (flang-compiler/f18#980)

This patch renames the modules in f18 to use a capital letter in the
module name

Signed-off-b

[flang] [LLVMify F18] Compiler module folders should have capitalised names (flang-compiler/f18#980)

This patch renames the modules in f18 to use a capital letter in the
module name

Signed-off-by: Caroline Concatto <[email protected]>

Original-commit: flang-compiler/f18@d2eb7a1c443d1539ef12b6f027074a0eb15b1ea0
Reviewed-on: https://github.com/flang-compiler/f18/pull/980

show more ...