[InstCombine] Improve check for catchswitch BBs (NFC)

Reviewed By: nikic

Differential Revision: https://reviews.llvm.org/D127810

[InstCombine] Don't slice up PHIs when pred BB has catchswitch

If an integer PHI has an illegal type (according to the data layout) and
it is only used by `trunc` or `trunc(lshr)` operations, we spl

[InstCombine] Don't slice up PHIs when pred BB has catchswitch

If an integer PHI has an illegal type (according to the data layout) and
it is only used by `trunc` or `trunc(lshr)` operations, we split the PHI
into various instructions in its predecessors:
https://github.com/llvm/llvm-project/blob/6d1543a16797fa07eecea7e542df5b42422fc721/llvm/lib/Transforms/InstCombine/InstCombinePHI.cpp#L1536-L1543

So this can produce code like the following:
Before:
```
pred:
...

bb:
%p = phi i8 [ %somevalue, %pred ], ...
...
%tobool = trunc i8 %p to i1
use %tobool
...
```
In this code, `%p` has an illegal integer type, `i8`, and its only used
in a `trunc` instruction later. In this case this pass puts extraction
code in its predecessors:

After:
```
pred:
...
%t = and i8 %somevalue, 1
%extract = icmp ne i8 %t, 0

bb:
%p.new = phi i1 [ %extract, %pred ], ...
use %p.new instead of %tobool
```

But this doesn't work if `pred` is a `catchswitch` BB because it cannot
have any non-PHI instructions. This CL ensures we bail out in that case.

Fixes https://github.com/llvm/llvm-project/issues/55803.

Reviewed By: dschuff

Differential Revision: https://reviews.llvm.org/D127699

show more ...

[NFC] format InstructionSimplify & lowerCaseFunctionNames

Clang-format InstructionSimplify and convert all "FunctionName"s to
"functionName". This patch does touch a lot of files but gets done with

[NFC] format InstructionSimplify & lowerCaseFunctionNames

Clang-format InstructionSimplify and convert all "FunctionName"s to
"functionName". This patch does touch a lot of files but gets done with
the cleanup of InstructionSimplify in one commit.

This is the alternative to the less invasive clang-format only patch: D126783

Reviewed By: spatel, rengolin

Differential Revision: https://reviews.llvm.org/D126889

show more ...

[InstCombine] Fix const folding of switches with default case

In case phi was in the default block it could lead to multi-edge.
Fixes #55721.

Differential Revision: https://reviews.llvm.org/D126650

Reapply [InstCombine] Support switch in phi to cond fold

Reapply with an explicit check for multi-edges, as the expected
behavior of multi-edge dominance is unclear (D120811).

-----

For conditiona

Reapply [InstCombine] Support switch in phi to cond fold

Reapply with an explicit check for multi-edges, as the expected
behavior of multi-edge dominance is unclear (D120811).

-----

For conditional branches, we know the value is i1 0 or i1 1 along
the outgoing edges. For switches we can apply exactly the same
optimization, just with the known values determined by the switch
cases.

show more ...

InstCombine: Can't fold a phi arg load into the phi if the load is from a swifterror address

`swifterror` addresses are only allowed as operands to load, store, and
calls.

The following transforma

InstCombine: Can't fold a phi arg load into the phi if the load is from a swifterror address

`swifterror` addresses are only allowed as operands to load, store, and
calls.

The following transformation is not allowed. It would create a phi with a
`swifterror` address operand.

```
%addr = alloca swifterror i8*
br %cond, label %bb1, label %b22

bb1:
%val1 = load i8*, i8** %addr
br exit

bb2:
%val2 = load i8*, i8** %addr
br exit

exit:
%val = phi [%val1, %bb1] [%val2, %bb2]
```

=>

```
%addr = alloca swifterror i8*
br %cond, label %bb1, label %b22

bb1:
br exit

bb2:
br exit

exit:
%val_addr = phi [%addr, %bb1] [%addr, %bb2]
%val2 = load i8*, i8** %val_addr
```

rdar://89865485

Differential Revision: https://reviews.llvm.org/D121217

show more ...

Revert "[InstCombine] Support switch in phi to cond fold"

This reverts commit 0817ce86b540f909eade6a8d7370e1b47e863a70.

Seeing some ppc64le stage2 failures, reverting to investigate.

[InstCombine] Support switch in phi to cond fold

For conditional branches, we know the value is i1 0 or i1 1 along
the outgoing edges. For switches we can apply exactly the same
optimization, just w

[InstCombine] Support switch in phi to cond fold

For conditional branches, we know the value is i1 0 or i1 1 along
the outgoing edges. For switches we can apply exactly the same
optimization, just with the known values determined by the switch
cases.

show more ...

[InstCombine] Support phi to cond fold with more than two preds

This transform can still be applied if there are more than two
phi inputs, as long as phi inputs with the same value are dominated
by

[InstCombine] Support phi to cond fold with more than two preds

This transform can still be applied if there are more than two
phi inputs, as long as phi inputs with the same value are dominated
by the same idom edge.

show more ...

[InstCombine] Check source element type in phi of gep fold

Rather than checking that the type is the same (which is always
the case, given how these are part of the same phi) check that the
source e

[InstCombine] Check source element type in phi of gep fold

Rather than checking that the type is the same (which is always
the case, given how these are part of the same phi) check that the
source element type is the same. With opaque pointers, this is no
longer implied.

show more ...

[InstCombine] Do not combine atomic and non-atomic loads

Before this change, InstCombine was willing to fold atomic and
non-atomic loads through a PHI node as long as the first PHI argument
is not a

[InstCombine] Do not combine atomic and non-atomic loads

Before this change, InstCombine was willing to fold atomic and
non-atomic loads through a PHI node as long as the first PHI argument
is not an atomic load. The combined load would be non-atomic, which is
incorrect.

Fix this by only combining the loads in a PHI node when all of the
arguments are non-atomic loads.

Thanks to Eli Friedman for pointing out the bug at
https://github.com/llvm/llvm-project/issues/50777#issuecomment-981045342!

Fixes #50777

Differential Revision: https://reviews.llvm.org/D115113

show more ...

[InstCombine] Use range for loops (NFC)

Preliminary clean-up for D115113

Differential Revision: https://reviews.llvm.org/D116086

[InstCombine] Uppercase some variable names (NFC)

Uppercase some variable names, per LLVM coding standards. This change
intentionally does not rename every miscased variable, as a follow-up
change (

[InstCombine] Uppercase some variable names (NFC)

Uppercase some variable names, per LLVM coding standards. This change
intentionally does not rename every miscased variable, as a follow-up
change ( D116086 ) intends to eliminate many of those by switching
loops to range for loops.

Differential Revision: https://reviews.llvm.org/D118553

show more ...

[InstCombine] Drop outdated alignment comment (NFC)

Loads always have an alignment now, so this is no longer relevant.

[InstCombine] Fold phi ( inttoptr/ptrtoint x ) to phi (x)

The inttoptr/ptrtoint roundtrip optimization is not always correct.
We are working towards removing this optimization and adding support to

[InstCombine] Fold phi ( inttoptr/ptrtoint x ) to phi (x)

The inttoptr/ptrtoint roundtrip optimization is not always correct.
We are working towards removing this optimization and adding support to specific cases where this optimization works.

In this patch, we focus on phi-node operands with inttoptr casts.
We know that ptrtoint( inttoptr( ptrtoint x) ) is same as ptrtoint (x).
So, we want to remove this roundtrip cast which goes through phi-node.

Reviewed By: aqjune

Differential Revision: https://reviews.llvm.org/D106289

show more ...

[InstCombine] Don't combine PHI before catchswitch

This tries to bail out if the PHI is in a `catchswitch` BB in
InstCombine. A PHI cannot be combined into a non-PHI instruction if it
is in a `catch

[InstCombine] Don't combine PHI before catchswitch

This tries to bail out if the PHI is in a `catchswitch` BB in
InstCombine. A PHI cannot be combined into a non-PHI instruction if it
is in a `catchswitch` BB, because `catchswitch` BB cannot have any
non-PHI instruction other than `catchswitch` itself.

The given test case started crashing after D98058.

Reviewed By: lebedev.ri, rnk

Differential Revision: https://reviews.llvm.org/D105309

show more ...

[InstCombine] Use poison constant to represent the result of unreachable instrs

This patch updates InstCombine to use poison constant to represent the resulting value of (either semantically or synt

[InstCombine] Use poison constant to represent the result of unreachable instrs

This patch updates InstCombine to use poison constant to represent the resulting value of (either semantically or syntactically) unreachable instrs, or a don't-care value of an unreachable store instruction.

This allows more aggressive folding of unused results, as shown in llvm/test/Transforms/InstCombine/getelementptr.ll .

Reviewed By: nikic

Differential Revision: https://reviews.llvm.org/D104602

show more ...

[InstCombine] Simplify phis with incoming pointer-casts.

If the incoming values of a phi are pointer casts of the same original
value, replace the phi with a single cast. Such redundant phis are
som

[InstCombine] Simplify phis with incoming pointer-casts.

If the incoming values of a phi are pointer casts of the same original
value, replace the phi with a single cast. Such redundant phis are
somewhat common after loop-rotate and removing them can avoid some
unnecessary code bloat, e.g. because an iteration of a loop is peeled
off to make the phi invariant. It should also simplify further analysis
on its own.

InstCombine already uses stripPointerCasts in a couple of places and
also simplifies phis based on the incoming values, so the patch should
fit in the existing scope.

The patch causes binary changes in 47 out of 237 benchmarks in
MultiSource/SPEC2000/SPEC2006 with -O3 -flto on X86.

Reviewed By: lebedev.ri

Differential Revision: https://reviews.llvm.org/D98058

show more ...

[CSSPGO] Unblock optimizations with pseudo probe instrumentation.

The IR/MIR pseudo probe intrinsics don't get materialized into real machine instructions and therefore they don't incur runtime cost

[CSSPGO] Unblock optimizations with pseudo probe instrumentation.

The IR/MIR pseudo probe intrinsics don't get materialized into real machine instructions and therefore they don't incur runtime cost directly. However, they come with indirect cost by blocking certain optimizations. Some of the blocking are intentional (such as blocking code merge) for better counts quality while the others are accidental. This change unblocks perf-critical optimizations that do not affect counts quality. They include:

1. IR InstCombine, sinking load operation to shorten lifetimes.
2. MIR LiveRangeShrink, similar to #1
3. MIR TwoAddressInstructionPass, i.e, opeq transform
4. MIR function argument copy elision
5. IR stack protection. (though not perf-critical but nice to have).

Reviewed By: wmi

Differential Revision: https://reviews.llvm.org/D95982

show more ...

Add missing namespace closure comments. NFCI.

Fixes some clang-tidy llvm-namespace-comment warnings.

[NFC][InstCombine] visitPHINode(): cleanup PHI CSE instruction replacement

As @nikic is pointing out in https://reviews.llvm.org/rGbf21ce7b908e#inline-4647
this must be sufficient otherwise `Elimina

[NFC][InstCombine] visitPHINode(): cleanup PHI CSE instruction replacement

As @nikic is pointing out in https://reviews.llvm.org/rGbf21ce7b908e#inline-4647
this must be sufficient otherwise `EliminateDuplicatePHINodes()`
would have hit issues with it already.

show more ...

[InstCombine] Take 3: Perform trivial PHI CSE

The original take 1 was 6102310d814ad73eab60a88b21dd70874f7a056f,
which taught InstSimplify to do that, which seemed better at time,
since we got EarlyC

[InstCombine] Take 3: Perform trivial PHI CSE

The original take 1 was 6102310d814ad73eab60a88b21dd70874f7a056f,
which taught InstSimplify to do that, which seemed better at time,
since we got EarlyCSE support for free.

However, it was proven that we can not do that there,
the simplified-to PHI would not be reachable from the original PHI,
and that is not something InstSimplify is allowed to do,
as noted in the commit ed90f15efb40d26b5d3ead3bb8e9e284218e0186
that reverted it:
> It appears to cause compilation non-determinism and caused stage3 mismatches.

Then there was take 2 3e69871ab5a66fb55913a2a2f5e7f5b42899a4c9,
which was InstCombine-specific, but it again showed stage2-stage3 differences,
and reverted in bdaa3f86a040b138c58de41d73d35b76fdec1380.
This is quite alarming.

Here, let's try to change how we find existing PHI candidate:
due to the worklist order, and the way PHI nodes are inserted
(it may be inserted as the first one, or maybe not), let's look at *all*
PHI nodes in the block.

Effects on vanilla llvm test-suite + RawSpeed:
```
| statistic name | baseline | proposed | Δ | % | \|%\| |
|----------------------------------------------------|-----------|-----------|-------:|---------:|---------:|
| asm-printer.EmittedInsts | 7942329 | 7942457 | 128 | 0.00% | 0.00% |
| assembler.ObjectBytes | 254295632 | 254312480 | 16848 | 0.01% | 0.01% |
| correlated-value-propagation.NumPhis | 18412 | 18347 | -65 | -0.35% | 0.35% |
| early-cse.NumCSE | 2183283 | 2183267 | -16 | 0.00% | 0.00% |
| early-cse.NumSimplify | 550105 | 541842 | -8263 | -1.50% | 1.50% |
| instcombine.NumAggregateReconstructionsSimplified | 73 | 4506 | 4433 | 6072.60% | 6072.60% |
| instcombine.NumCombined | 3640311 | 3644419 | 4108 | 0.11% | 0.11% |
| instcombine.NumDeadInst | 1778204 | 1783205 | 5001 | 0.28% | 0.28% |
| instcombine.NumPHICSEs | 0 | 22490 | 22490 | 0.00% | 0.00% |
| instcombine.NumWorklistIterations | 2023272 | 2024400 | 1128 | 0.06% | 0.06% |
| instcount.NumCallInst | 1758395 | 1758802 | 407 | 0.02% | 0.02% |
| instcount.NumInvokeInst | 59478 | 59502 | 24 | 0.04% | 0.04% |
| instcount.NumPHIInst | 330557 | 330545 | -12 | 0.00% | 0.00% |
| instcount.TotalBlocks | 1077138 | 1077220 | 82 | 0.01% | 0.01% |
| instcount.TotalFuncs | 101442 | 101441 | -1 | 0.00% | 0.00% |
| instcount.TotalInsts | 8831946 | 8832606 | 660 | 0.01% | 0.01% |
| simplifycfg.NumHoistCommonCode | 24186 | 24187 | 1 | 0.00% | 0.00% |
| simplifycfg.NumInvokes | 4300 | 4410 | 110 | 2.56% | 2.56% |
| simplifycfg.NumSimpl | 1019813 | 999767 | -20046 | -1.97% | 1.97% |
```
So it fires 22490 times, which is less than ~24k the take 1 did,
but more than what take 2 did (22228 times)
.
It allows foldAggregateConstructionIntoAggregateReuse() to actually work
after PHI-of-extractvalue folds did their thing. Previously SimplifyCFG
would have done this PHI CSE, of all places. Additionally, allows some
more `invoke`->`call` folds to happen (+110, +2.56%).

All in all, expectedly, this catches less things overall,
but all the motivational cases are still caught, so all good.

show more ...

Revert "[InstCombine] Take 2: Perform trivial PHI CSE"

While the original variant with doing this in InstSimplify (rightfully)
caused questions and ultimately was detected to be a culprit
of stage2-

Revert "[InstCombine] Take 2: Perform trivial PHI CSE"

While the original variant with doing this in InstSimplify (rightfully)
caused questions and ultimately was detected to be a culprit
of stage2-stage3 mismatch, it was expected that
InstCombine-based implementation would be fine.

But apparently it's not, as
http://lab.llvm.org:8011/builders/clang-with-thin-lto-ubuntu/builds/24095/steps/compare-compilers/logs/stdio
suggests.

Which suggests that somewhere in InstCombine there is a loop
over nondeterministically sorted container, which causes
different worklist ordering.

This reverts commit 3e69871ab5a66fb55913a2a2f5e7f5b42899a4c9.

show more ...

[InstCombine] Return replaceInstUsesWith() result (NFC)

Follow the usual usage pattern for this function and return the
result.

[InstCombine] visitPHINode(): use InstCombiner::replaceInstUsesWith() instead of RAUW

As noted in post-commit review, we really shouldn't use RAUW in InstCombine
because we should consistently updat

[InstCombine] visitPHINode(): use InstCombiner::replaceInstUsesWith() instead of RAUW

As noted in post-commit review, we really shouldn't use RAUW in InstCombine
because we should consistently update Worklist to avoid extra iterations.

show more ...