Use value_or instead of getValueOr (NFC)

Remove unneeded cl::ZeroOrMore for cl::opt/cl::list options

[Polly] Mark classes as final by default. NFC.

This make is obivious that a class was not intended to be derived from.

NPM analysis pass can unfortunately not marked as final because they are
deriv

[Polly] Mark classes as final by default. NFC.

This make is obivious that a class was not intended to be derived from.

NPM analysis pass can unfortunately not marked as final because they are
derived from a llvm::Checker<T> template internally by the NPM.

Also normalize the use of classes/structs
* NPM passes are structs
* Legacy passes are classes
* structs that have methods and are not a visitor pattern are classes
* structs have public inheritance by default, remove "public" keyword
* Use typedef'ed type instead of inline forward declaration

show more ...

[SCEV] Sequential/in-order `UMin` expression

As discussed in https://github.com/llvm/llvm-project/issues/53020 / https://reviews.llvm.org/D116692,
SCEV is forbidden from reasoning about 'backedge ta

[SCEV] Sequential/in-order `UMin` expression

As discussed in https://github.com/llvm/llvm-project/issues/53020 / https://reviews.llvm.org/D116692,
SCEV is forbidden from reasoning about 'backedge taken count'
if the branch condition is a poison-safe logical operation,
which is conservatively correct, but is severely limiting.

Instead, we should have a way to express those
poison blocking properties in SCEV expressions.

The proposed semantics is:
```
Sequential/in-order min/max SCEV expressions are non-commutative variants
of commutative min/max SCEV expressions. If none of their operands
are poison, then they are functionally equivalent, otherwise,
if the operand that represents the saturation point* of given expression,
comes before the first poison operand, then the whole expression is not poison,
but is said saturation point.
```
* saturation point - the maximal/minimal possible integer value for the given type

The lowering is straight-forward:
```
compare each operand to the saturation point,
perform sequential in-order logical-or (poison-safe!) ordered reduction
over those checks, and if reduction returned true then return
saturation point else return the naive min/max reduction over the operands
```
https://alive2.llvm.org/ce/z/Q7jxvH (2 ops)
https://alive2.llvm.org/ce/z/QCRrhk (3 ops)
Note that we don't need to check the last operand: https://alive2.llvm.org/ce/z/abvHQS
Note that this is not commutative: https://alive2.llvm.org/ce/z/FK9e97

That allows us to handle the patterns in question.

Reviewed By: nikic, reames

Differential Revision: https://reviews.llvm.org/D116766

show more ...

[Polly] Introduce caching for the isErrorBlock function. NFC.

Compilation of the file insn-attrtab.c of the SPEC CPU 2017 502.gcc_r
benchmark takes excessive time (> 30min) with Polly enabled. Most

[Polly] Introduce caching for the isErrorBlock function. NFC.

Compilation of the file insn-attrtab.c of the SPEC CPU 2017 502.gcc_r
benchmark takes excessive time (> 30min) with Polly enabled. Most time
is spent in the isErrorBlock function querying the DominatorTree.
The isErrorBlock is invoked redundantly over the course of ScopDetection
and ScopBuilder. This patch introduces a caching mechanism for its
result.

Instead of a free function, isErrorBlock is moved to ScopDetection where
its cache map resides. This also means that many functions directly or
indirectly calling isErrorBlock are not "const" anymore. The
DetectionContextMap was marked as "mutable", but IMHO it never should
have been since it stores the detection result.

502.gcc_r only takes excessive time with the new pass manager. The
reason seeams to be that it invalidates the ScopDetection analysis more
often than the legacy pass manager, for unknown reasons.

show more ...

[Polly] Break early when the result is known. NFC.

[Polly][ManualOpt] Match interpretation of unroll metadata to LoopUnrolls's.

We previously had a different interpretation of unroll transformation
attributes than how LoopUnroll interpreted it. In p

[Polly][ManualOpt] Match interpretation of unroll metadata to LoopUnrolls's.

We previously had a different interpretation of unroll transformation
attributes than how LoopUnroll interpreted it. In particular,
llvm.loop.unroll.enable was needed explicitly to enable it and disabling
metadata was ignored.
Additionally, it required that either full unrolling or an unroll factor
to be specified or fail otherwise. An unroll factor is still required,
but the transformation is ignored with the hope that LoopUnroll is going
to apply the unrolling, since Polly currently does not implement an
heuristic.

Fixes llvm.org/PR50109

show more ...

[Polly][Optimizer] Apply user-directed unrolling.

Make Polly look for unrolling metadata (https://llvm.org/docs/TransformMetadata.html#loop-unrolling) that is usually only interpreted by the LoopUnr

[Polly][Optimizer] Apply user-directed unrolling.

Make Polly look for unrolling metadata (https://llvm.org/docs/TransformMetadata.html#loop-unrolling) that is usually only interpreted by the LoopUnroll pass and apply it to the SCoP's schedule.

While not that useful by itself (there already is an unroll pass), it introduces mechanism to apply arbitrary loop transformation directives in arbitrary order to the schedule. Transformations are applied until no more directives are found. Since ISL's rescheduling would discard the manual transformations and it is assumed that when the user specifies the sequence of transformations, they do not want any other transformations to apply. Applying user-directed transformations can be controlled using the `-polly-pragma-based-opts` switch and is enabled by default.

This does not influence the SCoP detection heuristic. As a consequence, loop that do not fulfill SCoP requirements or the initial profitability heuristic will be ignored. `-polly-process-unprofitable` can be used to disable the latter.

Other than manually editing the IR, there is currently no way for the user to add loop transformations in an order other than the order in the default pipeline, or transformations other than the one supported by clang's LoopHint. See the `unroll_double.ll` test as example that clang currently is unable to emit. My own extension of `#pragma clang loop` allowing an arbitrary order and additional transformations is available here: https://github.com/meinersbur/llvm-project/tree/pragma-clang-loop. An effort to upstream this functionality as `#pragma clang transform` (because `#pragma clang loop` has an implicit transformation order defined by the loop pipeline) is D69088.

Additional transformations from my downstream pragma-clang-loop branch are tiling, interchange, reversal, unroll-and-jam, thread-parallelization and array packing. Unroll was chosen because it uses already-defined metadata and does not require correctness checks.

Reviewed By: sebastiankreutzer

Differential Revision: https://reviews.llvm.org/D97977

show more ...

[Polly] Track defined behavior for PHI predecessor computation.

ZoneAlgorithms's computePHI relies on being provided with consistent a
schedule to compute the statement prodecessors of a statement c

[Polly] Track defined behavior for PHI predecessor computation.

ZoneAlgorithms's computePHI relies on being provided with consistent a
schedule to compute the statement prodecessors of a statement containing
PHINodes. Otherwise unexpected results such as PHI nodes with multiple
predecessors can occur which would result in problems in the
algorithms expecting consistent data.

In the added test case, statement instances are scrubbed from the
SCoP their execution would result in undefined behavior (Due to a nsw
overflow). As already being undefined behavior in LLVM-IR, neither
AssumedContext nor InvalidContext are updated, giving computePHI no
means to avoid these cases.

Intoduce a new SCoP property, the DefinedBehaviorContext, that among
the runtime-checked conditions, also tracks the assumptions not needing
a runtime check, in particular those affecting the assumed control flow.
This replaces the manual combination of the 3 other contexts that was
already done in computePHI and setNewAccessRelation. Currently, the only
additional assumption is that loop induction variables will nsw flag for
not wrap, but potentially more can be added. Use in
hasFeasibleRuntimeContext, isl::ast_build and gisting are other
potential uses.

To limit computational complexity, the DefinedBehaviorContext is not
availabe if it grows too large (atm hardcoded to 8 disjuncts).

Possible other fixes include bailing out in computePHI when
inconsistencies are detected, choose an arbitrary value for inconsistent
cases (since it is undefined behavior anyways), or make the code
receiving the result from ComputePHI handle inconsistent data. All of
them reduce the quality of implementation having to bail out more often
and disabling the ability to assert on actually wrong results.

This fixes llvm.org/PR48783.

show more ...

[SCEV] Introduce SCEVPtrToIntExpr (PR46786)

And use it to model LLVM IR's `ptrtoint` cast.

This is essentially an alternative to D88806, but with no chance for
all the problems it caused due to hav

[SCEV] Introduce SCEVPtrToIntExpr (PR46786)

And use it to model LLVM IR's `ptrtoint` cast.

This is essentially an alternative to D88806, but with no chance for
all the problems it caused due to having the cast as implicit there.
(see rG7ee6c402474a2f5fd21c403e7529f97f6362fdb3)

As we've established by now, there are at least two reasons why we want this:
* It will allow SCEV to actually model the `ptrtoint` casts
and their operands, instead of treating them as `SCEVUnknown`
* It should help with initial problem of PR46786 - this should eventually allow us
to not loose pointer-ness of an expression in more cases

As discussed in [[ https://bugs.llvm.org/show_bug.cgi?id=46786 | PR46786 ]], in principle,
we could just extend `SCEVUnknown` with a `is ptrtoint` cast, because `ScalarEvolution::getPtrToIntExpr()`
should sink the cast as far down into the expression as possible,
so in the end we should always end up with `SCEVPtrToIntExpr` of `SCEVUnknown`.

But i think that it isn't the best solution, because it doesn't really matter
from memory consumption side - there probably won't be *that* many `SCEVPtrToIntExpr`s
for it to matter, and it allows for much better discoverability.

Reviewed By: mkazantsev

Differential Revision: https://reviews.llvm.org/D89456

show more ...

Reland "[SCEVExpander] Add option to preserve LCSSA directly."

This reverts the revert commit dc2867576886247cbe351e7c63618c09ab6af808.

It includes a fix for Polly, which uses SCEVExpander on IR th

Reland "[SCEVExpander] Add option to preserve LCSSA directly."

This reverts the revert commit dc2867576886247cbe351e7c63618c09ab6af808.

It includes a fix for Polly, which uses SCEVExpander on IR that is not
in LCSSA form. Set PreserveLCSSA = false in that case, to ensure we do
not introduce LCSSA phis where there were none before.

show more ...

[Polly] Update ScalarEvolutionExpander.h include.

[NFC] [DA] Refactoring getIndexExpressionsFromGEP

Summary:
This patch moves the getIndexExpressionsFromGEP function from polly
into ScalarEvolution so that both polly and DependenceAnalysis can
use

[NFC] [DA] Refactoring getIndexExpressionsFromGEP

Summary:
This patch moves the getIndexExpressionsFromGEP function from polly
into ScalarEvolution so that both polly and DependenceAnalysis can
use it for the purpose of subscript delinearization when the array
sizes are not parametric.

Authored By: bmahjour

Reviewer: Meinersbur, sebpop, fhahn, dmgreen, grosser, etiotto, bollu

Reviewed By: Meinersbur

Subscribers: hiraditya, arphaman, Whitney, ppc-slack, llvm-commits

Tags: #llvm

Differential Revision: https://reviews.llvm.org/D73995

show more ...

[Polly] Run polly-update-format after commit 55cfb1. NFC.

[Polly] Fix build after IRBuilder changes

Simply dropping the createPollyIRBuilder() function here, because
it doesn't do much. Also directly initialize Expander in
ScopExpander instead of going thr

[Polly] Fix build after IRBuilder changes

Simply dropping the createPollyIRBuilder() function here, because
it doesn't do much. Also directly initialize Expander in
ScopExpander instead of going through the copy-constructor.

show more ...

[NFC][ScopBuilder] Move RecordedAssumptions vector to ScopBuilder

Scope of changes:

1) Moved RecordedAssumptions vector to ScopBuilder. RecordedAssumptions are used only for Scop constructions.
2)

[NFC][ScopBuilder] Move RecordedAssumptions vector to ScopBuilder

Scope of changes:

1) Moved RecordedAssumptions vector to ScopBuilder. RecordedAssumptions are used only for Scop constructions.
2) Moved definition of RecordedAssumptionsTy to ScopHelper. It is required both by ScopBuilder and SCEVAffinator.
3) Add new function recordAssumption to ScopHelper. One of its argument is a reference to RecordedAssumption vector. This function is used by ScopBuilder and SCEVAffinator.
4) All RecordedAssumptions are created by ScopBuilder. isl::pw_aff
objects for corresponding SCEVs are created inside ScopBuilder. Scop
functions do not record any assumptions. Scop can use isl::pw_aff
objects which were created by ScopBuilder.
5) Removed functions for handling RecordedAssumptions from Scop class.
6) Removed constness from getScopArrayInfo functions.
7) Replaced SCEVVisitor struct from SCEVAffinator with taylored version, which allow to pass pointer to RecordedAssumptions as function argument.

Differential Revision: https://reviews.llvm.org/D68056

show more ...

[NFC][ScopBuilder] Move buildSchedule and its callees to ScopBuilder or ScopHelper

Scope of changes:
1. Moved buildSchedule functions to ScopBuilder.
2. Moved combineInSequence function to ScopBuild

[NFC][ScopBuilder] Move buildSchedule and its callees to ScopBuilder or ScopHelper

Scope of changes:
1. Moved buildSchedule functions to ScopBuilder.
2. Moved combineInSequence function to ScopBuilder.
3. Moved mapToDimension function to ScopBuilder.
4. Moved LoopStackTy to ScopBuilder.
5. Moved getLoopSurroundingScop to ScopHelper.
6. Moved getNumBlocksInLoop to ScopHelper.
7. Moved getNumBlocksInRegionNode to ScopHelper.
8. Moved getRegionNodeLoop to ScopHelper.

Differential Revision: https://reviews.llvm.org/D64223

llvm-svn: 366377

show more ...

[polly][SCEV] Expand SCEV matcher cases for new smin/umin ops

These were added in rL360159, but I neglected to update polly at the
same time.

llvm-svn: 360238

Apply include-what-you-use #include removal suggestions. NFC.

This removes unused includes (and forward declarations) as
suggested by include-what-you-use. If a transitive include of a removed
inclu

Apply include-what-you-use #include removal suggestions. NFC.

This removes unused includes (and forward declarations) as
suggested by include-what-you-use. If a transitive include of a removed
include is required to compile a file, I added the required header (or
forward declaration if suggested by include-what-you-use).

This should reduce compilation time and reduce the number of iterative
recompilations when a header was changed.

llvm-svn: 357209

show more ...

Update the file headers across all of the LLVM projects in the monorepo
to reflect the new license.

We understand that people may be surprised that we're moving the header
entirely to discuss the ne

Update the file headers across all of the LLVM projects in the monorepo
to reflect the new license.

We understand that people may be surprised that we're moving the header
entirely to discuss the new license. We checked this carefully with the
Foundation's lawyer and we believe this is the correct approach.

Essentially, all code in the project is now made available by the LLVM
project under our new license, so you will see that the license headers
include that license only. Some of our contributors have contributed
code under our old license, and accordingly, we have retained a copy of
our old license notice in the top-level files in each project and
repository.

llvm-svn: 351636

show more ...

[TI removal] Make `getTerminator()` return a generic `Instruction`.

This removes the primary remaining API producing `TerminatorInst` which
will reduce the rate at which code is introduced trying to

[TI removal] Make `getTerminator()` return a generic `Instruction`.

This removes the primary remaining API producing `TerminatorInst` which
will reduce the rate at which code is introduced trying to use it and
generally make it much easier to remove the remaining APIs across the
codebase.

Also clean up some of the stragglers that the previous mechanical update
of variables missed.

Users of LLVM and out-of-tree code generally will need to update any
explicit variable types to handle this. Replacing `TerminatorInst` with
`Instruction` (or `auto`) almost always works. Most of these edits were
made in prior commits using the perl one-liner:
```
perl -i -ple 's/TerminatorInst(\b.* = .*getTerminator\(\))/Instruction\1/g'
```

This also my break some rare use cases where people overload for both
`Instruction` and `TerminatorInst`, but these should be easily fixed by
removing the `TerminatorInst` overload.

llvm-svn: 344504

show more ...

[ScopHelper] Provide support for recognising collective invariant loads

Summary: This patch aims to provide support for detecting load patterns which are collectively invariant but right now `isHois

[ScopHelper] Provide support for recognising collective invariant loads

Summary: This patch aims to provide support for detecting load patterns which are collectively invariant but right now `isHoistableLoad()` is checking each load instruction individually which cannot detect the load pattern as a whole.

Patch by: Sahil Girish Yerawar

Reviewers: bollu, philip.pfaffe, Meinersbur

Reviewed By: philip.pfaffe, Meinersbur

Differential Revision: https://reviews.llvm.org/D48026

llvm-svn: 335949

show more ...

[ScopHelper] Cache ScopExpander results.

The number of SCEV expressions is usually linear in the number of IR
instructions being modeled. However, a naive SCEV visitor is not. For
an expression lik

[ScopHelper] Cache ScopExpander results.

The number of SCEV expressions is usually linear in the number of IR
instructions being modeled. However, a naive SCEV visitor is not. For
an expression like x*x, "x" will be visited twice. If x is itself an
expression like x*x, that will be visited twice, etc, and the overall
runtime is O(2^N) in the number of SCEV expressions.

To prevent this from happening, add a cache, so we only visit each SCEV
expression once.

Not sure this is the best solution. Maybe we can instead check whether
the SCEV is scop-invariant (in which case we never need to map the
value). But we don't have a utility for that at the moment.

Differential Revision: https://reviews.llvm.org/D47087

llvm-svn: 335783

show more ...

Allow arbitrary function calls for debugging purposes.

Add the switch -polly-debug-func to define the name of a debug
function. This function is ignored for any validity check.

Its purpose is to al

Allow arbitrary function calls for debugging purposes.

Add the switch -polly-debug-func to define the name of a debug
function. This function is ignored for any validity check.

Its purpose is to allow to observe a value after transformation by a
SCoP, and to follow which statements are executed in which order. For
instance, consider the following code:

static void dbg_printf(int sum, int i) {
fprintf(stderr, "The value of sum is %d, i=%d\n", sum, i);
fflush(stderr);
}

void func(int n) {
int sum = 0;
for (int i = 0; i < 16; i+=1) {
sum += i;
dbg_printf(sum, i);
}
}

Executing this after Polly's codegen with -polly-debug-func=dbg_printf
reveals the new execution order and the assumed values at that point of
execution.

Differential Revision: https://reviews.llvm.org/D45728

llvm-svn: 330466

show more ...

Remove immediate dominator heuristic for error block detection.

This patch removes the heuristic in
- Polly :: lib/Support/ScopHelper.cpp

The heuristic forces blocks that directly follow a loop hea

Remove immediate dominator heuristic for error block detection.

This patch removes the heuristic in
- Polly :: lib/Support/ScopHelper.cpp

The heuristic forces blocks that directly follow a loop header to not to be considered error blocks.
It was introduced in r249611 with the following commit message:

> This replaces the support for user defined error functions by a
> heuristic that tries to determine if a call to a non-pure function
> should be considered "an error". If so the block is assumed not to be
> executed at runtime. While treating all non-pure function calls as
> errors will allow a lot more regions to be analyzed, it will also
> cause us to dismiss a lot again due to an infeasible runtime context.
> This patch tries to limit that effect. A non-pure function call is
> considered an error if it is executed only in conditionally with
> regards to a cheap but simple heuristic.

In the code below `CCK_Abort2()` would be considered as an error block, but not `CCK_Abort1()` due to this heuristic.
```
for (int i = 0; i < n; i+=1) {
if (ErrorCondition1)
CCK_Abort1(); // No __attribute__((noreturn))
if (ErrorCondition2)
CCK_Abort2(); // No __attribute__((noreturn))
}
```

This does not seem useful. Checking error conditions in the beginning of some work is quite common. It causes a switch default-case to be not considered an error block in SPEC's cactuBSSN. The comment justifying the heuristic mentions a "load", which does not seem to be applicable here. It has been proposed to remove the heuristic.

In addition, the patch fixes the following test cases:
- Polly :: ScopDetect/mod_ref_read_pointer.ll
- Polly :: ScopInfo/max-loop-depth.ll
- Polly :: ScopInfo/mod_ref_access_pointee_arguments.ll
- Polly :: ScopInfo/mod_ref_read_pointee_arguments.ll
- Polly :: ScopInfo/mod_ref_read_pointer.ll
- Polly :: ScopInfo/mod_ref_read_pointers.ll

The test cases failed after removing the heuristic.

Differential Revision: https://reviews.llvm.org/D45274

Contributed-by: Lorenzo Chelini <[email protected]>
llvm-svn: 329548

show more ...