[LSR] Allow already invariant operand for ICmpZero matching [try 2]

Changes since initial commit:

* Wrapping a pointer in an SCEV unknown hides the base, and SCEV is only able to compute a subtract

[LSR] Allow already invariant operand for ICmpZero matching [try 2]

Changes since initial commit:

* Wrapping a pointer in an SCEV unknown hides the base, and SCEV is only able to compute a subtraction when the bases are known to be equal. This results in a SCEVCouldNotCompute flowing forward and triggering asserts. Test case added in d767b392.
* isLoopInvariant returns true for instructions outside the loop, but not necessarily *above* the loop. Since this code is allowed to visit uses of an IV outside of a loop, we have to make sure the operands of the compare are both invariant and dominating the header. Test case added in 2aed3cdb.

Original commit message follows...

The ICmpZero matching is checking to see if the expression is loop invariant per SCEV and expandable. This allows expressions inside the loop which can be made loop invariant to be seamlessly expanded, but is overly conservative for expressions which already *are* loop invariant.

As a simple justification for why this is correct, consider a loop invariant urem as RHS vs an alternate function with that same urem wrapped inside a helper call. Why would it be legal to match the later, but not the former?

Differential Revision: https://reviews.llvm.org/D129793

show more ...

Revert "[LSR] Allow already invariant operand for ICmpZero matching"

This reverts commit 9153515a7bea9fb9dd4c76f70053a170bf825f35. Builtbot crash was reported in the commit thread, reverting while

Revert "[LSR] Allow already invariant operand for ICmpZero matching"

This reverts commit 9153515a7bea9fb9dd4c76f70053a170bf825f35. Builtbot crash was reported in the commit thread, reverting while investigating.

show more ...

[LSR] Allow already invariant operand for ICmpZero matching

The ICmpZero matching is checking to see if the expression is loop invariant per SCEV and expandable. This allows expressions inside the l

[LSR] Allow already invariant operand for ICmpZero matching

The ICmpZero matching is checking to see if the expression is loop invariant per SCEV and expandable. This allows expressions inside the loop which can be made loop invariant to be seamlessly expanded, but is overly conservative for expressions which already *are* loop invariant.

As a simple justification for why this is correct, consider a loop invariant urem as RHS vs an alternate function with that same urem wrapped inside a helper call. Why would it be legal to match the later, but not the former?

Differential Revision: https://reviews.llvm.org/D129793

show more ...

[LSR] Create SCEVExpander earlier, use member isSafeToExpand() (NFC)

This is a followup to D129630, which switches LSR to the member
isSafeToExpand() variant, and removes the freestanding function.

[LSR] Create SCEVExpander earlier, use member isSafeToExpand() (NFC)

This is a followup to D129630, which switches LSR to the member
isSafeToExpand() variant, and removes the freestanding function.

This is done by creating the SCEVExpander early (already during the
analysis phase). Because the SCEVExpander is now available for the
whole lifetime of LSRInstance, I've also made it into a member
variable, rather than passing it around in even more places.

Differential Revision: https://reviews.llvm.org/D129769

show more ...

[SCEVExpander] Make CanonicalMode handing in isSafeToExpand() more robust (PR50506)

isSafeToExpand() for addrecs depends on whether the SCEVExpander
will be used in CanonicalMode. At least one calle

[SCEVExpander] Make CanonicalMode handing in isSafeToExpand() more robust (PR50506)

isSafeToExpand() for addrecs depends on whether the SCEVExpander
will be used in CanonicalMode. At least one caller currently gets
this wrong, resulting in PR50506.

Fix this by a) making the CanonicalMode argument on the freestanding
functions required and b) adding member functions on SCEVExpander
that automatically take the SCEVExpander mode into account. We can
use the latter variant nearly everywhere, and thus make sure that
there is no chance of CanonicalMode mismatch.

Fixes https://github.com/llvm/llvm-project/issues/50506.

Differential Revision: https://reviews.llvm.org/D129630

show more ...

[llvm] Use value instead of getValue (NFC)

[LSR] Fix bug for optimizing unused IVs to final values

This is a fix for a crash reported for https://reviews.llvm.org/D118808
The fix is to only consider PHINodes which are induction phis.
Fixes #

[LSR] Fix bug for optimizing unused IVs to final values

This is a fix for a crash reported for https://reviews.llvm.org/D118808
The fix is to only consider PHINodes which are induction phis.
Fixes #55529

Differential Revision: https://reviews.llvm.org/D125990

show more ...

Revert "Don't use Optional::hasValue (NFC)"

This reverts commit aa8feeefd3ac6c78ee8f67bf033976fc7d68bc6d.

Don't use Optional::hasValue (NFC)

[LSR] Don't allow zero quotient as scale ref. PR56160

Scale reg should never be zero, so when the quotient is zero, we
cannot assign it there. Limit this transform to avoid this situation.

Differen

[LSR] Don't allow zero quotient as scale ref. PR56160

Scale reg should never be zero, so when the quotient is zero, we
cannot assign it there. Limit this transform to avoid this situation.

Differential Revision: https://reviews.llvm.org/D128339
Reviewed By: eopXD

show more ...

[LSR] Early exit for RateFormula when it is already losing. NFC

This patch does not effect any behavior of the current code.

The codebase implicitly implies that `Cost::RateFormula` is only called

[LSR] Early exit for RateFormula when it is already losing. NFC

This patch does not effect any behavior of the current code.

The codebase implicitly implies that `Cost::RateFormula` is only called
when the `Cost` is not in losing status, or else there may be possible
to trigger the assertion of `Cost::isValid`.

The intention here is to prevent mis-use where future development
allow `Cost` that is already loser to call `Cost::RateFormula` - Early
exit when `Cost` is already losing.

Reviewed By: Meinersbur, #loopoptwg

Differential Revision: https://reviews.llvm.org/D125670

show more ...

[LSR][TTI][PowerPC][SystemZ][X86] Add const-ness to TTI::isLSRCostLess. NFC

Reviewed By: Meinersbur

Differential Revision: https://reviews.llvm.org/D126350

[Debuginfo][LSR] Add salvaging variadic dbg.value intrinsics [2/2]

Reland 3f2b76ec90b5f108272a3072a1345ba55d8ec75b with the test corrected
to require x86-registered-target.

Differential Revision: h

[Debuginfo][LSR] Add salvaging variadic dbg.value intrinsics [2/2]

Reland 3f2b76ec90b5f108272a3072a1345ba55d8ec75b with the test corrected
to require x86-registered-target.

Differential Revision: https://reviews.llvm.org/D120169

show more ...

Revert "[Debuginfo][LSR] Add salvaging variadic dbg.value intrinsics [2/2]"

This reverts commit 3f2b76ec90b5f108272a3072a1345ba55d8ec75b.

[Debuginfo][LSR] Add salvaging variadic dbg.value intrinsics [2/2]

Reland commit 74273d575f9938d751a1c67862cffe553fe2de8b following a fix
for a memory leak. The DVIRecoveryRecord vectors now use uni

[Debuginfo][LSR] Add salvaging variadic dbg.value intrinsics [2/2]

Reland commit 74273d575f9938d751a1c67862cffe553fe2de8b following a fix
for a memory leak. The DVIRecoveryRecord vectors now use unique_ptr.

Differential Revision: https://reviews.llvm.org/D120169

show more ...

Revert "[Debuginfo][LSR] Add salvaging variadic dbg.value intrinsics [2/2]"

This reverts commit 74273d575f9938d751a1c67862cffe553fe2de8b.

Buildbot: https://lab.llvm.org/buildbot/#/builders/5/builds

Revert "[Debuginfo][LSR] Add salvaging variadic dbg.value intrinsics [2/2]"

This reverts commit 74273d575f9938d751a1c67862cffe553fe2de8b.

Buildbot: https://lab.llvm.org/buildbot/#/builders/5/builds/22795
Failing with memory leak.

show more ...

[Debuginfo][LSR] Add salvaging variadic dbg.value intrinsics [2/2]

This relands commit 8f550368b169b0a3d4ebc3e65efec49ec1e0798a.

The test is amended with REQUIRES: x86-registered-target, in line wi

[Debuginfo][LSR] Add salvaging variadic dbg.value intrinsics [2/2]

This relands commit 8f550368b169b0a3d4ebc3e65efec49ec1e0798a.

The test is amended with REQUIRES: x86-registered-target, in line with
the other debuginfo-scev-salvage tests.

Differential Revision: https://reviews.llvm.org/D120169

show more ...

Revert [Debuginfo][LSR] Add salvaging variadic dbg.value intrinsics[2/2]

This reverts commit 8f550368b169b0a3d4ebc3e65efec49ec1e0798a.

[Debuginfo][LSR] Add salvaging variadic dbg.value intrinsics [2/2]

Second of two patches to extend SCEV-based salvaging to dbg.value
intrinsics that have multiple location ops pre-LSR. This second p

[Debuginfo][LSR] Add salvaging variadic dbg.value intrinsics [2/2]

Second of two patches to extend SCEV-based salvaging to dbg.value
intrinsics that have multiple location ops pre-LSR. This second patch
adds the core implementation.

Reviewers: @StephenTozer, @djtodoro

Differential Revision: https://reviews.llvm.org/D120169

show more ...

[Debuginfo][LSR] Add salvaging variadic dbg.value intrinsics [1/2] [NFC]

First of two patches that extends SCEV-based salvaging to enable
salvaging of dbg.value instrinsics that have multiple locati

[Debuginfo][LSR] Add salvaging variadic dbg.value intrinsics [1/2] [NFC]

First of two patches that extends SCEV-based salvaging to enable
salvaging of dbg.value instrinsics that have multiple locations ops
before the Loop Strength Reduction pass.

The existing single-op SCEV-based salvaging can generate variadic
dbg.value intrinsics in order to salvage a dbg.value that has a single
location op. If a dbg.value has multiple location ops before LSR, and
LSR optimises away one or more of the location operands, then currently
no salvaging will be attempted.

Salvaging can now be added, but first this patch cleans up consistency
in both the code and comments, and applies some refactoring to make
application of the new salvaging implementation more straightforward.

- Use SCEVDbgValueBuilder for both types of recovery expressions:
IV-offset based and iteration count based.
- Combine the functions that write the final DIExpression.
- Move some static functions into member functions.

Reviewers: @Orlando

Differential Revision: https://reviews.llvm.org/D120168

show more ...

[LSR] Update outdated comment

[LSR] Optimize unused IVs to final values in the exit block

Loop Strength Reduce sometimes optimizes away all uses of an induction variable
from a loop but leaves the IV increments. When the only re

[LSR] Optimize unused IVs to final values in the exit block

Loop Strength Reduce sometimes optimizes away all uses of an induction variable
from a loop but leaves the IV increments. When the only remaining use of the IV
is the PHI in the exit block, this patch will call rewriteLoopExitValues to
replace the exit block PHI with the final value of the IV to skip the updates
in each loop iteration.

Differential Revision: https://reviews.llvm.org/D118808

show more ...

[LSR] Fixup canonicalization formula and its checker.

According to definition of canonical form, it is a canonical
if scale reg does not contain addrec for loop L then none of bases
should contain a

[LSR] Fixup canonicalization formula and its checker.

According to definition of canonical form, it is a canonical
if scale reg does not contain addrec for loop L then none of bases
should contain addrec for this loop.

The critical word here is "contains".

Current checker of canonical form checks not "containing" property
but "is". So it does not check whether it contains but whether it is.

Fix the checker and canonicalizing utility to follow definition.

Without this fix in the test attached the base formula looking as
reg((-1 * {0,+,8}<nuw><nsw><%bb2>)<nsw>) + 1*reg((8 * (%arg /u 8))<nuw>)
is considered as conanocial while base contains an addrec.
And modified formula we want to insert
reg({0,+,8}<nuw><nsw><%bb2>) + 1*reg((-8 * (%arg /u 8)))
is considered as not canonical.

Reviewed By: mkazantsev
Subscribers: llvm-commits
Differential Revision: https://reviews.llvm.org/D122457

show more ...

Cleanup includes: Transform/Scalar

Estimated impact on preprocessor output line:
before: 1062981579
after: 1062494547

Discourse thread: https://discourse.llvm.org/t/include-what-you-use-include-cl

Cleanup includes: Transform/Scalar

Estimated impact on preprocessor output line:
before: 1062981579
after: 1062494547

Discourse thread: https://discourse.llvm.org/t/include-what-you-use-include-cleanup
Differential Revision: https://reviews.llvm.org/D120817

show more ...

Reduce dependencies on llvm/BinaryFormat/Dwarf.h

This header is very large (3M Lines once expended) and was included in location
where dwarf-specific information were not needed.

More specifically,

Reduce dependencies on llvm/BinaryFormat/Dwarf.h

This header is very large (3M Lines once expended) and was included in location
where dwarf-specific information were not needed.

More specifically, this commit suppresses the dependencies on
llvm/BinaryFormat/Dwarf.h in two headers: llvm/IR/IRBuilder.h and
llvm/IR/DebugInfoMetadata.h. As these headers (esp. the former) are widely used,
this has a decent impact on number of preprocessed lines generated during
compilation of LLVM, as showcased below.

This is achieved by moving some definitions back to the .cpp file, no
performance impact implied[0].

As a consequence of that patch, downstream user may need to manually some extra
files:

llvm/IR/IRBuilder.h no longer includes llvm/BinaryFormat/Dwarf.h
llvm/IR/DebugInfoMetadata.h no longer includes llvm/BinaryFormat/Dwarf.h

In some situations, codes maybe relying on the fact that
llvm/BinaryFormat/Dwarf.h was including llvm/ADT/Triple.h, this hidden
dependency now needs to be explicit.

$ clang++ -E -Iinclude -I../llvm/include ../llvm/lib/Transforms/Scalar/*.cpp -std=c++14 -fno-rtti -fno-exceptions | wc -l
after: 10978519
before: 11245451

Related Discourse thread: https://llvm.discourse.group/t/include-what-you-use-include-cleanup
[0] https://llvm-compile-time-tracker.com/compare.php?from=fa7145dfbf94cb93b1c3e610582c495cb806569b&to=995d3e326ee1d9489145e20762c65465a9caeab4&stat=instructions

Differential Revision: https://reviews.llvm.org/D118781

show more ...