Use value_or instead of getValueOr (NFC)

[Polly] Mark classes as final by default. NFC.

This make is obivious that a class was not intended to be derived from.

NPM analysis pass can unfortunately not marked as final because they are
deriv

[Polly] Mark classes as final by default. NFC.

This make is obivious that a class was not intended to be derived from.

NPM analysis pass can unfortunately not marked as final because they are
derived from a llvm::Checker<T> template internally by the NPM.

Also normalize the use of classes/structs
* NPM passes are structs
* Legacy passes are classes
* structs that have methods and are not a visitor pattern are classes
* structs have public inheritance by default, remove "public" keyword
* Use typedef'ed type instead of inline forward declaration

show more ...

[polly] Fix type in function name. NFC.

[Polly][SchedOpt] Account for prevectorization of multiple statements.

A prevectorized loop may contain multiple statements, in which case
isl_schedule_node_band_sink will sink the vector band to mu

[Polly][SchedOpt] Account for prevectorization of multiple statements.

A prevectorized loop may contain multiple statements, in which case
isl_schedule_node_band_sink will sink the vector band to multiple
leaves. Instead of statically assuming a specific tree structure after
sinking, add a SIMD marker to all inner bands.

Fixes llvm.org/PR52637

show more ...

[Polly][Isl] Use the function unsignedFromIslSize to manage a isl::size object. NFCI

This is part of an effort to reduce the differences between the custom C++ bindings used right now by polly in li

[Polly][Isl] Use the function unsignedFromIslSize to manage a isl::size object. NFCI

This is part of an effort to reduce the differences between the custom C++ bindings used right now by polly in lib/External/isl/include/isl/isl-noxceptions.h and the official isl C++ interface.
In the official interface the type `isl::size` cannot be casted to an unsigned without previously having checked if it contains a valid value with the function `isl::size::is_error()`.
For this reason two helping functions have been added:
- `IslAssert`: assert that no errors are present in debug builds and just disables the mandatory error check in non-debug builds
- `unisgnedFromIslSIze`: cast the `isl::size` object to `unsigned`

Changes made:
- Add the functions `IslAssert` and `unsignedFromIslSize`
- Add the utility function `rangeIslSize()`
- Retype `MaxDisjunctsInDomain` from `int` to `unsigned`
- Retype `RunTimeChecksMaxAccessDisjuncts` from `int` to `unsigned`
- Retype `MaxDimensionsInAccessRange` from `int` to `unsigned`
- Replaced some usages of `isl_size` to `unsigned` since we aim not to use `isl_size` anymore
- `isl-noexceptions.h` has been generated by https://github.com/patacca/isl/commit/e704f73c88f0b4d88e62e447bdb732cf5914094b

No functional change intended.

Reviewed By: Meinersbur

Differential Revision: https://reviews.llvm.org/D113101

show more ...

[Polly] Fix unused variable in non-assert builds. NFC.

[Polly] Add greedy fusion algorithm.

When the option -polly-loopfusion-greedy is set, the ScheduleOptimizer
tries to aggressively fuse any band it can and does not violate any
dependences.

As part

[Polly] Add greedy fusion algorithm.

When the option -polly-loopfusion-greedy is set, the ScheduleOptimizer
tries to aggressively fuse any band it can and does not violate any
dependences.

As part if the implementation, the functionalty for copying a band
into an new schedule was extracted out of the ScheduleTreeRewriter.

show more ...

[Polly] Implement user-directed loop distribution/fission.

This is a simple version without the possibility to define distribute
points or followup-transformations. However, it is the first
transfor

[Polly] Implement user-directed loop distribution/fission.

This is a simple version without the possibility to define distribute
points or followup-transformations. However, it is the first
transformation that has to check whether the transformation is correct.

It interprets the same metadata as the LoopDistribute pass.

Re-apply after revert in c7bcd72a38bcf99e03e4651ed5204d1a1f2bf695 with
fix: Take isBand out of #ifndef NDEBUG since it now is used
unconditionally.

show more ...

Revert "[Polly] Implement user-directed loop distribution/fission."

This reverts commit 52c30adc7dfe6334b71adf256d81f70e7b976143 which
breaks the build when NDEBUG is defined.

[Polly] Implement user-directed loop distribution/fission.

This is a simple version without the possibility to define distribute
points or followup-transformations. However, it is the first
transfor

[Polly] Implement user-directed loop distribution/fission.

This is a simple version without the possibility to define distribute
points or followup-transformations. However, it is the first
transformation that has to check whether the transformation is correct.

It interprets the same metadata as the LoopDistribute pass.

show more ...

[Polly] Use subtyped isl::schedule_nodes for ScheduleTreeVisitor. NFC.

Change pass-by-const-ref to pass-by-value as objects are recreated
due to custom up-/down-casting anwyway.

[Polly][Isl] Move to the new-polly-generator branch version of isl-noexceptions.h. NFCI

This is part of an effort to reduce the differences between the custom C++ bindings used right now by polly in

[Polly][Isl] Move to the new-polly-generator branch version of isl-noexceptions.h. NFCI

This is part of an effort to reduce the differences between the custom C++ bindings used right now by polly in `lib/External/isl/include/isl/isl-noxceptions.h` and the official isl C++ interface.

With this commit we are moving from the `polly-generator` branch to the `new-polly-generator` branch that is more mantainable and is based on the official C++ interface `cpp-checked.h`.

Changes made:
- There are now many sublcasses for `isl::ast_node` representing different isl types. Use `isl::ast_node_for`, `isl::ast_node_user`, `isl::ast_node_block` and `isl::ast_node_mark` where needed.
- There are now many sublcasses for `isl::schedule_node` representing different isl types. Use `isl::schedule_node_mark`, `isl::schedule_node_extension`, `isl::schedule_node_band` and `isl::schedule_node_filter` where needed.
- Replace the `isl::*::dump` with `dumpIslObj` since the isl dump method is not exposed in the C++ interface.
- `isl::schedule_node::get_child` has been renamed to `isl::schedule_node::child`
- `isl::pw_multi_aff::get_pw_aff` has been renamed to `isl::pw_multi_aff::at`
- The constructor `isl::union_map(isl::union_pw_multi_aff)` has been replaced with the static method `isl::union_map::from()`
- Replace usages of `isl::val::add_ui` with `isl::val::add`
- `isl::union_set_list::alloc` is now a constructor
- All the `isl_size` values are now wrapped inside the class `isl::size` use `isl::size::release` to get the internal `isl_size` value where needed.
- `isl-noexceptions.h` has been generated by https://github.com/patacca/isl/commit/73f5ed1f4d1f72582f731590ef9e43d9ab1956ad

No functional change intended.

Reviewed By: Meinersbur

Differential Revision: https://reviews.llvm.org/D107225

show more ...

[Polly][Isl] Use isl::union_map::unite() instead of isl::union_map::add_map(). NFC

This is part of an effort to reduce the differences between the custom C++ bindings used right now by polly in `lib

[Polly][Isl] Use isl::union_map::unite() instead of isl::union_map::add_map(). NFC

This is part of an effort to reduce the differences between the custom C++ bindings used right now by polly in `lib/External/isl/include/isl/isl-noxceptions.h` and the official isl C++ interface.

Changes made:
- Use `isl::union_map::unite()` instead of `isl::union_map::add_map()`
- `isl-noexceptions.h` has been generated by this https://github.com/patacca/isl/commit/3f43ae29fa2a22936a583b85b2fe8d439f805d8d

Depends on D106059

Reviewed By: Meinersbur

Differential Revision: https://reviews.llvm.org/D106061

show more ...

[Polly][Isl] Stop generating isl::union_{set,map} from isl::space. NFC

This is part of an effort to reduce the differences between the custom C++ bindings used right now by polly in `lib/External/is

[Polly][Isl] Stop generating isl::union_{set,map} from isl::space. NFC

This is part of an effort to reduce the differences between the custom C++ bindings used right now by polly in `lib/External/isl/include/isl/isl-noxceptions.h` and the official isl C++ interface.

Changes made:
- Stop generating `isl::union_set` and isl::union_map` from `isl::space` and instead generate them from `isl::ctx`
- Disable clang-format on `isl-noexceptions.h`
- Removed `isl::union_{set,map}` generator from `isl::space` from `isl-noexceptions.h`
- `isl-noexceptions.h` has been generated by this https://github.com/patacca/isl/commit/87c3413b6f1d62ca3dddf716352f90a0b8533353

Reviewed By: Meinersbur

Differential Revision: https://reviews.llvm.org/D106059

show more ...

[Polly][Isl] Use isl::*::ctx instead of isl::*::get_ctx. NFC

This is part of an effort to reduce the differences between the custom C++ bindings used right now by polly in `lib/External/isl/include/

[Polly][Isl] Use isl::*::ctx instead of isl::*::get_ctx. NFC

This is part of an effort to reduce the differences between the custom C++ bindings used right now by polly in `lib/External/isl/include/isl/isl-noxceptions.h` and the official isl C++ interface.

Changes made:
- Use `isl::*::ctx()` instead of `isl::*::get_ctx()` (for example `isl::space::ctx()` instead of `isl::space::get_ctx()`)
- Add `isl::` namespace in front of isl types to avoid confusion (for example `isl::space::ctx` and `isl::ctx`
- `isl-noexceptions.h` has been generated by this https://github.com/patacca/isl/commit/b64e33c62d3fe7db506f6630f9e935b663f9c5a4

Reviewed By: Meinersbur

Differential Revision: https://reviews.llvm.org/D105691

show more ...

[Polly][Isl] Use isl::set::tuple_dim, isl::map::domain_tuple_dim and isl::map::range_tuple_dim. NFC

This is part of an effort to reduce the differences between the custom C++ bindings used right now

[Polly][Isl] Use isl::set::tuple_dim, isl::map::domain_tuple_dim and isl::map::range_tuple_dim. NFC

This is part of an effort to reduce the differences between the custom C++ bindings used right now by polly in `lib/External/isl/include/isl/isl-noxceptions.h` and the official isl C++ interface.

Changes made:
- Use `isl::set::tuple_dim` instead of `isl::set::dim` and `isl::set::n_dim`
- Use `isl::map::domain_tuple_dim` instead of `isl::map::dim`
- Use `isl::map::range_tuple_dim` instead of `isl::map::dim`
- isl-noexceptions.h has been generated by this https://github.com/patacca/isl/commit/45576e1b4260f91946e4cf819485f57bd2ed5490

Note that not all the usage of `isl::{set,map}::dim` where replaced

Reviewed By: Meinersbur

Differential Revision: https://reviews.llvm.org/D104994

show more ...

[Polly][Isl] Removing explicit operator bool() from isl C++ bindings. NFC.

This is part of an effort to reduce the differences between the custom C++ bindings used right now by polly in `lib/Externa

[Polly][Isl] Removing explicit operator bool() from isl C++ bindings. NFC.

This is part of an effort to reduce the differences between the custom C++ bindings used right now by polly in `lib/External/isl/include/isl/isl-noxceptions.h` and the official isl C++ interface.

Changes made:
- Removing explicit operator bool() from all the classes in the isl C++ bindings.
- Replace each call to operator bool() to method `is_null()`.
- isl-noexceptions.h has been generated by this https://github.com/patacca/isl/commit/27396daac5a5ee8228d25511a12f4a814c92ba8f

Reviewed By: Meinersbur

Differential Revision: https://reviews.llvm.org/D103976

show more ...

[Polly] Move MatMul optimization into its own file. NFC.

Functions shared between generalized matrix-multiplication optimization
and other post-reschedule optimizations (tiling, prevect) are moved i

[Polly] Move MatMul optimization into its own file. NFC.

Functions shared between generalized matrix-multiplication optimization
and other post-reschedule optimizations (tiling, prevect) are moved into
the schedule tree transformation utility ScheduleTreeTransform.

show more ...

Preprocessor conditionalize some assert-only functions to suppress -Wunused-function

[Polly][Unroll] Fix unroll_double test.

We enumerated the cross product Domain x Scatter, but sorted only be the
scatter key. In case there are are multiple statement instances per
scatter value, th

[Polly][Unroll] Fix unroll_double test.

We enumerated the cross product Domain x Scatter, but sorted only be the
scatter key. In case there are are multiple statement instances per
scatter value, the order between statement instances of the same loop
iteration was undefined.

Propertly enumerate and sort only by the scatter value, and group the
domains using the scatter dimension again.

Thanks to Leonard Chan for the report.

show more ...

[Polly][Optimizer] Apply user-directed unrolling.

Make Polly look for unrolling metadata (https://llvm.org/docs/TransformMetadata.html#loop-unrolling) that is usually only interpreted by the LoopUnr

[Polly][Optimizer] Apply user-directed unrolling.

Make Polly look for unrolling metadata (https://llvm.org/docs/TransformMetadata.html#loop-unrolling) that is usually only interpreted by the LoopUnroll pass and apply it to the SCoP's schedule.

While not that useful by itself (there already is an unroll pass), it introduces mechanism to apply arbitrary loop transformation directives in arbitrary order to the schedule. Transformations are applied until no more directives are found. Since ISL's rescheduling would discard the manual transformations and it is assumed that when the user specifies the sequence of transformations, they do not want any other transformations to apply. Applying user-directed transformations can be controlled using the `-polly-pragma-based-opts` switch and is enabled by default.

This does not influence the SCoP detection heuristic. As a consequence, loop that do not fulfill SCoP requirements or the initial profitability heuristic will be ignored. `-polly-process-unprofitable` can be used to disable the latter.

Other than manually editing the IR, there is currently no way for the user to add loop transformations in an order other than the order in the default pipeline, or transformations other than the one supported by clang's LoopHint. See the `unroll_double.ll` test as example that clang currently is unable to emit. My own extension of `#pragma clang loop` allowing an arbitrary order and additional transformations is available here: https://github.com/meinersbur/llvm-project/tree/pragma-clang-loop. An effort to upstream this functionality as `#pragma clang transform` (because `#pragma clang loop` has an implicit transformation order defined by the loop pipeline) is D69088.

Additional transformations from my downstream pragma-clang-loop branch are tiling, interchange, reversal, unroll-and-jam, thread-parallelization and array packing. Unroll was chosen because it uses already-defined metadata and does not require correctness checks.

Reviewed By: sebastiankreutzer

Differential Revision: https://reviews.llvm.org/D97977

show more ...

[ScheduleTreeTransform] Silence compiler warning. NFC.

Use size_t for position which is the return type type ArrayRef::size()
it is compared to.

llvm-svn: 362724

[ScheduleOptimizer] Hoist extension nodes after schedule optimization.

Extension nodes make schedule trees are less flexible: Many operations,
such as rescheduling, do not work on such schedule tree

[ScheduleOptimizer] Hoist extension nodes after schedule optimization.

Extension nodes make schedule trees are less flexible: Many operations,
such as rescheduling, do not work on such schedule trees with extension.
As such, some functionality such as determining parallel loops in isl's
AST are disabled.

Currently, only the pattern-matching generalized matrix-matrix
multiplication optimization adds extension nodes (to add copy-in
statements).

This patch removes all extension nodes as the last step of the schedule
optimization by hoisting the extension node's added domain up to the
root domain node. All following passes can assume that schedule trees
work without restrictions, including the parallelism test. Mark the
outermost loop of the optimized matrix-matrix multiplication as parallel
such that -polly-parallel is able to parallelize that loop.

Differential Revision: https://reviews.llvm.org/D58202

llvm-svn: 362257

show more ...