[LAA] Require AddRecs to be in the innermost loop for diff-checks.

The simpler diff-checks require pointers with add-recs from the same
innermost loop, but this property wasn't check completely. Add

[LAA] Require AddRecs to be in the innermost loop for diff-checks.

The simpler diff-checks require pointers with add-recs from the same
innermost loop, but this property wasn't check completely. Add the
missing check to ensure both addrecs are in the innermost loop.

Fixes #57315.

(cherry picked from commit 9405af1c850139f6ddd6d35ff395756fe2aef31f)

show more ...

[LV] Add another test for incorrect runtime check generation.

Add a variation of @nested_loop_outer_iv_addrec_invariant_in_inner with
the dependence sink and source swapped to extend test coverage.

[LV] Add another test for incorrect runtime check generation.

Add a variation of @nested_loop_outer_iv_addrec_invariant_in_inner with
the dependence sink and source swapped to extend test coverage.

Also simplifies the test by removing an unneeded reduction.

(cherry picked from commit e117137af0a37ac309d40db57ec7162ef3d22c29)

show more ...

[LV] Add test for incorrect runtime check generation #57315.

Test for PR57315 based on a test provided by @kpdev42.

(cherry picked from commit 6e56779e6bc168a3acd14f9bf2c4fd3fd9d86bd1)

[LV] Convert runtime diff check test to use opaque pointers.

Modernize the test to make it easier to extend with up-to-date IR.

(cherry picked from commit 3b135ef446d56e8542d55a1b716e9413f50da7aa)

[LAA,LV] Add initial support for pointer-diff memory checks.

This patch adds initial support for a pointer diff based runtime check
scheme for vectorization. This scheme requires fewer computations

[LAA,LV] Add initial support for pointer-diff memory checks.

This patch adds initial support for a pointer diff based runtime check
scheme for vectorization. This scheme requires fewer computations and
checks than the existing full overlap checking, if it is applicable.

The main idea is to only check if source and sink of a dependency are
far enough apart so the accesses won't overlap in the vector loop. To do
so, it is sufficient to compute the difference and compare it to the
`VF * UF * AccessSize`. It is sufficient to check
`(Sink - Src) <u VF * UF * AccessSize` to rule out a backwards
dependence in the vector loop with the given VF and UF. If Src >=u Sink,
there is not dependence preventing vectorization, hence the overflow
should not matter and using the ULT should be sufficient.

Note that the initial version is restricted in multiple ways:

1. Pointers must only either be read or written, by a single
instruction (this allows re-constructing source/sink for
dependences with the available information)
2. Source and sink pointers must be add-recs, with matching steps
3. The step must be a constant.
3. abs(step) == AccessSize.

Most of those restrictions can be relaxed in the future.

See https://github.com/llvm/llvm-project/issues/53590.

Reviewed By: dmgreen

Differential Revision: https://reviews.llvm.org/D119078

show more ...

[LV] Add addiitonal tests for pointer difference memory checks.

Additional tests for D119078.