[NFC] Switch a few uses of undef to poison as placeholders for unreachble code

[Scalarizer] No need to gather a scattered extracted element

ExtractElement does not produce a vector out of a vector, so there's no need to
call a gather once done.

Fix #54469

Credits to npopov@r

[Scalarizer] No need to gather a scattered extracted element

ExtractElement does not produce a vector out of a vector, so there's no need to
call a gather once done.

Fix #54469

Credits to [email protected] for the original approach.

Differential Revision: https://reviews.llvm.org/D126012

show more ...

[llvm] Use value_or instead of getValueOr (NFC)

[LV][SLP] Mark fptosi_sat as vectorizable

This adds fptosi_sat and fptoui_sat to the list of trivially
vectorizable functions, mainly so that the loop vectorizer can vectorize
the instruction. Marki

[LV][SLP] Mark fptosi_sat as vectorizable

This adds fptosi_sat and fptoui_sat to the list of trivially
vectorizable functions, mainly so that the loop vectorizer can vectorize
the instruction. Marking them as trivially vectorizable also allows them
to be SLP vectorized, and Scalarized.

The signature of a fptosi_sat requires two type overrides
(@llvm.fptosi.sat.v2i32.v2f32), unlike other intrinsics that often only
take a single. This patch alters hasVectorInstrinsicOverloadedScalarOpd
to isVectorIntrinsicWithOverloadTypeAtArg, so that it can mark the first
operand of the intrinsic as a overloaded (but not scalar) operand.

Differential Revision: https://reviews.llvm.org/D124358

show more ...

[NFC] Rename Instrinsic to Intrinsic

Expose ScalarizerPass options to C++ (not just commandline)

Context: I needed this for https://github.com/google/iree/pull/8474 .
I found that TSan instrumentation expects vector sizes to be <= 16,

Expose ScalarizerPass options to C++ (not just commandline)

Context: I needed this for https://github.com/google/iree/pull/8474 .
I found that TSan instrumentation expects vector sizes to be <= 16,
and in my project (IREE) we have tests with higher vector sizes.
That left some test functions uninstrumented, resulting in crashes as
instrumented code called into them.

Differential Revision: https://reviews.llvm.org/D121182

show more ...

[Scalarizer] Avoid pointer element type accesses

Pass through the load/store type to the Scatterer instead.

Cleanup includes: Transform/Scalar

Estimated impact on preprocessor output line:
before: 1062981579
after: 1062494547

Discourse thread: https://discourse.llvm.org/t/include-what-you-use-include-cl

Cleanup includes: Transform/Scalar

Estimated impact on preprocessor output line:
before: 1062981579
after: 1062494547

Discourse thread: https://discourse.llvm.org/t/include-what-you-use-include-cleanup
Differential Revision: https://reviews.llvm.org/D120817

show more ...

[NFC] Remove uses of PointerType::getElementType()

Instead use either Type::getPointerElementType() or
Type::getNonOpaquePointerElementType().

This is part of D117885, in preparation for deprecatin

[NFC] Remove uses of PointerType::getElementType()

Instead use either Type::getPointerElementType() or
Type::getNonOpaquePointerElementType().

This is part of D117885, in preparation for deprecating the API.

show more ...

[Scalarizer] Do not insert instructions between PHI nodes and debug intrinsics.

The scalarizer pass seems to be inserting instructions in-between PHI nodes or debug intrinsics that end up staying at

[Scalarizer] Do not insert instructions between PHI nodes and debug intrinsics.

The scalarizer pass seems to be inserting instructions in-between PHI nodes or debug intrinsics that end up staying at the end of the pass, resulting in malformed IR and violating assumptions.

This patch adds a check to make sure the `extractelement` instructions that it adds are correctly placed after all PHI nodes and debug intrinsics.

Patch by vettoreldaniele.

Reviewed By: bjope

Differential Revision: https://reviews.llvm.org/D112472

show more ...

[Transforms] Migrate from getNumArgOperands to arg_size (NFC)

Note that getNumArgOperands is considered a legacy name. See
llvm/include/llvm/IR/InstrTypes.h for details.

Update @llvm.powi to handle different int sizes for the exponent

This can be seen as a follow up to commit 0ee439b705e82a4fe20e2,
that changed the second argument of __powidf2, __powisf2 and
__powit

Update @llvm.powi to handle different int sizes for the exponent

This can be seen as a follow up to commit 0ee439b705e82a4fe20e2,
that changed the second argument of __powidf2, __powisf2 and
__powitf2 in compiler-rt from si_int to int. That was to align with
how those runtimes are defined in libgcc.
One thing that seem to have been missing in that patch was to make
sure that the rest of LLVM also handle that the argument now depends
on the size of int (not using the si_int machine mode for 32-bit).
When using __builtin_powi for a target with 16-bit int clang crashed.
And when emitting libcalls to those rtlib functions, typically when
lowering @llvm.powi), the backend would always prepare the exponent
argument as an i32 which caused miscompiles when the rtlib was
compiled with 16-bit int.

The solution used here is to use an overloaded type for the second
argument in @llvm.powi. This way clang can use the "correct" type
when lowering __builtin_powi, and then later when emitting the libcall
it is assumed that the type used in @llvm.powi matches the rtlib
function.

One thing that needed some extra attention was that when vectorizing
calls several passes did not support that several arguments could
be overloaded in the intrinsics. This patch allows overload of a
scalar operand by adding hasVectorInstrinsicOverloadedScalarOpd, with
an entry for powi.

Differential Revision: https://reviews.llvm.org/D99439

show more ...

[Scalarizer] Use poison as insertelement's placeholder

This patch makes Scalarizer to use poison as insertelement's placeholder.

It contains two changes in Scalarizer.cpp, and the both changes does

[Scalarizer] Use poison as insertelement's placeholder

This patch makes Scalarizer to use poison as insertelement's placeholder.

It contains two changes in Scalarizer.cpp, and the both changes does not change the semantics of the optimized program.
It is because the placeholder value (poison) is already completely hidden by following insertelement instructions.

The first change at visitBitCastInst() creates poison vector of MidTy and consecutively inserts FanIn times,
which is # of elems of MidTy.
The second change at ScalarizerVisitor::finish() creates poison with Op->getType(), and it is filled with
Count insertelements.

The test diffs show that the poison value is never exposed after insertelements.

Reviewed By: nikic

Differential Revision: https://reviews.llvm.org/D93989

show more ...

[Scalarizer] Avoid changing name of non-instructions

The "takeName" logic in ScalarizerVisitor::gather did not consider
that the value vector could refer to non-instructions, such as
global variable

[Scalarizer] Avoid changing name of non-instructions

The "takeName" logic in ScalarizerVisitor::gather did not consider
that the value vector could refer to non-instructions, such as
global variables. This patch make sure that we avoid changing the
name of a value if it isn't an instruction.

Reviewed By: lebedev.ri

Differential Revision: https://reviews.llvm.org/D87685

show more ...

[Scalarizer] Avoid updating the name of globals

The "takeName" logic at the end of ScalarizerVisitor::finish
could end up renaming global variables when having simplified
and extractelement instruct

[Scalarizer] Avoid updating the name of globals

The "takeName" logic at the end of ScalarizerVisitor::finish
could end up renaming global variables when having simplified
and extractelement instruction to simply pick a single vector
element. If the input vector to the extractelement instruction
held pointers to global variables we ended up renaming the global
variable.
The patch make sure we only take the name of the replaced Op when
we have added new instructions that might need a useful name.

Reviewed By: lebedev.ri

Differential Revision: https://reviews.llvm.org/D86472

show more ...

[SVE] Remove calls to VectorType::getNumElements from Scalar

Reviewers: efriedma, fhahn, reames, kmclaughlin, sdesmalen

Reviewed By: sdesmalen

Subscribers: tschuett, hiraditya, rkruppe, psnobl, da

[SVE] Remove calls to VectorType::getNumElements from Scalar

Reviewers: efriedma, fhahn, reames, kmclaughlin, sdesmalen

Reviewed By: sdesmalen

Subscribers: tschuett, hiraditya, rkruppe, psnobl, dantrushin, llvm-commits

Tags: #llvm

Differential Revision: https://reviews.llvm.org/D82243

show more ...

[Scalarizer] When gathering scattered scalar, don't replace it with itself

The (previously-crashing) test-case would cause us to seemingly-harmlessly
replace some use with something else, but we can

[Scalarizer] When gathering scattered scalar, don't replace it with itself

The (previously-crashing) test-case would cause us to seemingly-harmlessly
replace some use with something else, but we can't replace it with itself,
so we would crash.

show more ...

[Scalarizer] Centralize instruction DCE

As reported in https://reviews.llvm.org/D83101#2133062
the new visitInsertElementInst()/visitExtractElementInst() functionality
is causing miscompiles (previo

[Scalarizer] Centralize instruction DCE

As reported in https://reviews.llvm.org/D83101#2133062
the new visitInsertElementInst()/visitExtractElementInst() functionality
is causing miscompiles (previously-crashing test added)

It is due to the fact how the infra of Scalarizer is dealing with DCE,
it was not updated or was it ready for such scalar value forwarding.
It always assumed that the moment we "scalarized" something,
it can go away, and did so with prejudice.

But that is no longer safe/okay to do.

Instead, let's prevent it from ever shooting itself into foot,
and let's just accumulate the instructions-to-be-deleted
in a vector, and collectively cleanup (those that are *actually* dead)
them all at the end.

All existing tests are not reporting any new garbage leftovers,
but maybe it's test coverage issue.

show more ...

[Scalarizer] visit{Insert,Extract}ElementInst(): avoid call arg evaluation order deps

Compilers may evaluate call arguments in different order,
which would result in different order of IR, which wou

[Scalarizer] visit{Insert,Extract}ElementInst(): avoid call arg evaluation order deps

Compilers may evaluate call arguments in different order,
which would result in different order of IR, which would break the tests.

Spotted thanks to Dmitri Gribenko!

show more ...

[Scalarizer] ExtractElement handling w/ variable insert index (PR46524)

Summary:
Similar to D82961.

Reviewers: bjope, cameron.mcinally, arsenm, jdoerfert

Reviewed By: jdoerfert

Subscribers: arpha

[Scalarizer] ExtractElement handling w/ variable insert index (PR46524)

Summary:
Similar to D82961.

Reviewers: bjope, cameron.mcinally, arsenm, jdoerfert

Reviewed By: jdoerfert

Subscribers: arphaman, wdng, hiraditya, llvm-commits

Tags: #llvm

Differential Revision: https://reviews.llvm.org/D82970

show more ...

[Scalarizer] InsertElement handling w/ variable insert index (PR46524)

Summary:
I'm interested in taking the original C++ input,
for which we currently are stuck with an alloca
and producing roughly

[Scalarizer] InsertElement handling w/ variable insert index (PR46524)

Summary:
I'm interested in taking the original C++ input,
for which we currently are stuck with an alloca
and producing roughly the lower IR,
with neither an alloca nor a vector ops:
https://godbolt.org/z/cRRWaJ

For that, as intermediate step, i'd to somehow perform scalarization.
As per @arsenmn suggestion, i'm trying to see if scalarizer can help me
avoid writing a bicycle.

I'm not sure if it's really intentional that variable insert is not handled currently.
If it really is, and is supposed to stay that way (?), i guess i could guard it..

See [[ https://bugs.llvm.org/show_bug.cgi?id=46524 | PR46524 ]].

Reviewers: bjope, cameron.mcinally, arsenm, jdoerfert

Reviewed By: jdoerfert

Subscribers: arphaman, uabelho, wdng, hiraditya, llvm-commits

Tags: #llvm

Differential Revision: https://reviews.llvm.org/D82961

show more ...

[Scalarizer] ExtractElement handling w/ constant extract index

Summary:
It appears to be better IR-wise to aggressively scalarize it,
rather than relying on gathering it, and leaving it as-is.

Revi

[Scalarizer] ExtractElement handling w/ constant extract index

Summary:
It appears to be better IR-wise to aggressively scalarize it,
rather than relying on gathering it, and leaving it as-is.

Reviewers: jdoerfert, bjope, arsenm, cameron.mcinally

Reviewed By: jdoerfert

Subscribers: arphaman, wdng, hiraditya, llvm-commits

Tags: #llvm

Differential Revision: https://reviews.llvm.org/D83101

show more ...

[Scalarizer] InsertElement handling w/ constant insert index

Summary: As it can be clearly seen from the diff, this results in nicer IR.

Reviewers: jdoerfert, arsenm, bjope, cameron.mcinally

Revie

[Scalarizer] InsertElement handling w/ constant insert index

Summary: As it can be clearly seen from the diff, this results in nicer IR.

Reviewers: jdoerfert, arsenm, bjope, cameron.mcinally

Reviewed By: jdoerfert

Subscribers: arphaman, wdng, hiraditya, llvm-commits

Tags: #llvm

Differential Revision: https://reviews.llvm.org/D83102

show more ...

[Alignment][NFC] VectorLayout now uses Align internally

By rewritting `ScalarizerVisitor::getVectorLayout` in such a way it returns `VectorLayout` (or `None`) it becomes obvious that `VectorLayout::

[Alignment][NFC] VectorLayout now uses Align internally

By rewritting `ScalarizerVisitor::getVectorLayout` in such a way it returns `VectorLayout` (or `None`) it becomes obvious that `VectorLayout::VecAlign` cannot be `0`.

This patch is part of a series to introduce an Alignment type.
See this thread for context: http://lists.llvm.org/pipermail/llvm-dev/2019-July/133851.html
See this patch for the introduction of the type: https://reviews.llvm.org/D64790

Differential Revision: https://reviews.llvm.org/D82981

show more ...

[SVE] Eliminate calls to default-false VectorType::get() from Scalar

Reviewers: efriedma, kmclaughlin, sdesmalen, fhahn, bkramer, anna, gchatelet, c-rhodes, david-arm, fpetrogalli

Reviewed By: davi

[SVE] Eliminate calls to default-false VectorType::get() from Scalar

Reviewers: efriedma, kmclaughlin, sdesmalen, fhahn, bkramer, anna, gchatelet, c-rhodes, david-arm, fpetrogalli

Reviewed By: david-arm

Subscribers: tschuett, hiraditya, rkruppe, psnobl, dantrushin, llvm-commits

Tags: #llvm

Differential Revision: https://reviews.llvm.org/D80336

show more ...