[PowerPC] emit VSX instructions instead of VMX instructions for vector loads and stores

This patch changes the PowerPC backend to generate VSX load/store instructions
for all vector loads/stores on

[PowerPC] emit VSX instructions instead of VMX instructions for vector loads and stores

This patch changes the PowerPC backend to generate VSX load/store instructions
for all vector loads/stores on Power8 and earlier (LE) instead of VMX
load/store instructions. The reason for this change is because VMX instructions
require the vector to be 16-byte aligned. So, a vector load/store will fail with
VMX instructions if the vector is misaligned. Also, `gcc` generates VSX
instructions in this situation which allow for unaligned access but require a
swap instruction after loading/before storing. This is not an issue for BE
because we already emit VSX instructions since no swap is required. And this is
not an issue on Power9 and up since we have access to `lxv[x]`/`stxv[x]` which
allow for unaligned access and do not require swaps.

This patch also delays the VSX load/store for LE combines until after
LegalizeOps to prioritize other load/store combines.

Reviewed By: #powerpc, stefanp

Differential Revision: https://reviews.llvm.org/D127309

show more ...

[PowerPC] Fix shift amount of xxsldwi when performing vector int_to_double

POC
```
// main.c
#include <stdio.h>
#include <altivec.h>
extern vector double foo(vector int s);
int main() {
vector int

[PowerPC] Fix shift amount of xxsldwi when performing vector int_to_double

POC
```
// main.c
#include <stdio.h>
#include <altivec.h>
extern vector double foo(vector int s);
int main() {
vector int s = {0, 1, 0, 4};
vector double vd;
vd = foo(s);
printf("%lf %lf\n", vd[0], vd[1]);
return 0;
}
// poc.c
vector double foo(vector int s) {
int x1 = s[1];
int x3 = s[3];
double d1 = x1;
double d3 = x3;
vector double x = { d1, d3 };
return x;
}
```
Compiled with `poc.c main.c -mcpu=pwr8 -O3` on BE machine.
Current clang gives
```
4.000000 1.000000
```
while xlc gives
```
1.000000 4.000000
```
Xlc's output should be correct.

Reviewed By: shchenz, #powerpc

Differential Revision: https://reviews.llvm.org/D107428

show more ...

[PowerPC] Update Refactored Load/Store Implementation, XForm VSX Patterns, and Tests

This patch includes the following updates to the load/store refactoring effort introduced in D93370:
- Update va

[PowerPC] Update Refactored Load/Store Implementation, XForm VSX Patterns, and Tests

This patch includes the following updates to the load/store refactoring effort introduced in D93370:
- Update various VSX patterns that use to "force" an XForm, to instead just XForm.
This allows the ability for the patterns to compute the most optimal addressing
mode (and to produce a DForm instruction when possible)
- Update pattern and test case for the LXVD2X/STXVD2X intrinsics
- Update LIT test cases that use to use the XForm instruction to use the DForm instruction

Differential Revision: https://reviews.llvm.org/D95115

show more ...

[PowerPC] Improve codegen for vector fp to int widening conversions

We currently do not utilize instructions that convert single
precision vectors to doubleword integer vectors. These conversions
co

[PowerPC] Improve codegen for vector fp to int widening conversions

We currently do not utilize instructions that convert single
precision vectors to doubleword integer vectors. These conversions
come up in code occasionally and this improvement allows us to
open code some functions that need to be added to altivec.h.

show more ...

[PowerPC] Canonicalize shuffles on big endian targets as well

Extend shuffle canonicalization and conversion of shuffles fed by vectorized
scalars to big endian subtargets. For big endian subtargets

[PowerPC] Canonicalize shuffles on big endian targets as well

Extend shuffle canonicalization and conversion of shuffles fed by vectorized
scalars to big endian subtargets. For big endian subtargets, loads and direct
moves of scalars into vector registers put the data in the correct element for
SCALAR_TO_VECTOR if the data type is 8 bytes wide. However, if the data type is
narrower, the value still ends up in the wrong place - althouth a different
wrong place than on little endian targets.

This patch extends the combine that keeps values where they are if they feed a
shuffle to big endian targets.

Differential revision: https://reviews.llvm.org/D100478

show more ...

[PowerPC] Canonicalize shuffles to match more single-instruction masks on LE

We currently miss a number of opportunities to emit single-instruction
VMRG[LH][BHW] instructions for shuffles on little

[PowerPC] Canonicalize shuffles to match more single-instruction masks on LE

We currently miss a number of opportunities to emit single-instruction
VMRG[LH][BHW] instructions for shuffles on little endian subtargets. Although
this in itself is not a huge performance opportunity since loading the permute
vector for a VPERM can always be pulled out of loops, producing such merge
instructions is useful to downstream optimizations.
Since VPERM is essentially opaque to all subsequent optimizations, we want to
avoid it as much as possible. Other permute instructions have semantics that can
be reasoned about much more easily in later optimizations.

This patch does the following:
- Canonicalize shuffles so that the first element comes from the first vector
(since that's what most of the mask matching functions want)
- Switch the elements that come from splat vectors so that they match the
corresponding elements from the other vector (to allow for merges)
- Adds debugging messages for when a shuffle is matched to a VPERM so that
anyone interested in improving this further can get the info for their code

Differential revision: https://reviews.llvm.org/D77448

show more ...

[PowerPC] Ignore implicit register operands for MCInst

Summary:
When doing the conversion: MachineInst -> MCInst, we should ignore the
implicit operands, it will expose more opportunity for InstiAli

[PowerPC] Ignore implicit register operands for MCInst

Summary:
When doing the conversion: MachineInst -> MCInst, we should ignore the
implicit operands, it will expose more opportunity for InstiAlias.

Reviewed By: steven.zhang

Differential Revision: https://reviews.llvm.org/D77118

show more ...

[NFC][PowerPC] Fix register class for patterns using XXPERMDIs

There are a few patterns where we use a superclass for inputs to this
instruction rather than the correct class. This can sometimes lea

[NFC][PowerPC] Fix register class for patterns using XXPERMDIs

There are a few patterns where we use a superclass for inputs to this
instruction rather than the correct class. This can sometimes lead to
unncessary copies.

show more ...

[PowerPC] Improve handling of some BUILD_VECTOR nodes

An analysis of real world code turned up a number of patterns with BUILD_VECTOR
of nodes resulting from operations on extracted vector elements

[PowerPC] Improve handling of some BUILD_VECTOR nodes

An analysis of real world code turned up a number of patterns with BUILD_VECTOR
of nodes resulting from operations on extracted vector elements for which we
produce poor code. This addresses those cases. No attempt is made for
completeness as that would entail a large amount of work for something that
there is no evidence of in real code.

Differential revision: https://reviews.llvm.org/D72660

show more ...

[PowerPC] Remove allow-deprecated-dag-overlap and fix broken tests

Summary:
This is found during review of https://reviews.llvm.org/D67088.

CHECK-DAG is non-overlapping after https://reviews.llvm.o

[PowerPC] Remove allow-deprecated-dag-overlap and fix broken tests

Summary:
This is found during review of https://reviews.llvm.org/D67088.

CHECK-DAG is non-overlapping after https://reviews.llvm.org/D47106.
-allow-deprecated-dag-overlap was introduced to temporary accept old
behavior.

But it actually hide some broken tests, eg: `test/CodeGen/PowerPC/swaps-le-1.ll`
The codegen has changed, but the CHECK-DAG still PASS due to allowing `overlap`.

This patch remove the deprecated options, and fix the broken tests.

Reviewers: #powerpc, hfinkel, nemanjai, steven.zhang, shchenz

Reviewed By: shchenz

Subscribers: shchenz, llvm-commits

Tags: #llvm

Differential Revision: https://reviews.llvm.org/D69733

show more ...

[PowerPC][NFC] Remove deprecated Function Attrs comments

[PowerPC] Exploit single instruction load-and-splat for word and doubleword

We currently produce a load, followed by (possibly a move for integers and) a
splat as separate instructions. VSX has alwa

[PowerPC] Exploit single instruction load-and-splat for word and doubleword

We currently produce a load, followed by (possibly a move for integers and) a
splat as separate instructions. VSX has always had a splatting load for
doublewords, but as of Power9, we have it for words as well. This patch just
exploits these instructions.

Differential revision: https://reviews.llvm.org/D63624

llvm-svn: 372139

show more ...

[PowerPC][NFC] Remove duplicate tests in build-vector-test.ll

AllOnes has been split into build-vector-allones.ll.

llvm-svn: 368900

recommit:[PowerPC] Eliminate loads/swap feeding swap/store for vector type by using big-endian load/store

In PowerPC, there is instruction to load vector in big endian element order when it's in lit

recommit:[PowerPC] Eliminate loads/swap feeding swap/store for vector type by using big-endian load/store

In PowerPC, there is instruction to load vector in big endian element order when it's in little endian target.
So we can combine vector load + reverse into big endian load to eliminate the swap instruction.
Also combine vector reverse + store into big endian store.

Differential Revision: https://reviews.llvm.org/D65063

llvm-svn: 367516

show more ...

revert r367382 because buildbot failure

llvm-svn: 367388

[PowerPC] Eliminate loads/swap feeding swap/store for vector type by using big-endian load/store

In PowerPC, there is instruction to load vector in big endian element order when it's in little endia

[PowerPC] Eliminate loads/swap feeding swap/store for vector type by using big-endian load/store

In PowerPC, there is instruction to load vector in big endian element order when it's in little endian target.
So we can combine vector load + reverse into big endian load to eliminate the swap instruction.
Also combine vector reverse + store into big endian store.

llvm-svn: 367382

show more ...

[MachineScheduler] checkResourceLimit boundary condition update

When we call checkResourceLimit in bumpCycle or bumpNode, and we
know the resource count has just reached the limit (the equations
ar

[MachineScheduler] checkResourceLimit boundary condition update

When we call checkResourceLimit in bumpCycle or bumpNode, and we
know the resource count has just reached the limit (the equations
are equal). We should return true to mark that we are resource
limited for next schedule, or else we might continue to schedule
in favor of latency for 1 more schedule and create a schedule that
actually overbook the resource.

When we call checkResourceLimit to estimate the resource limite before
scheduling, we don't need to return true even if the equations are
equal, as it shouldn't limit the schedule for it .

Differential Revision: https://reviews.llvm.org/D62345

llvm-svn: 362805

show more ...

[PowerPC] P9 Scheduling Model: dispatching rule fixes

This is to address some of the problems in existing P9 resource modeling,
especially about the dispatching rules.

Instead of using a hypothetic

[PowerPC] P9 Scheduling Model: dispatching rule fixes

This is to address some of the problems in existing P9 resource modeling,
especially about the dispatching rules.

Instead of using a hypothetical DISPATCHER , we try to use the number of
actual dispatch slots, and define SchedWriteRes to model dispatch rules,
then update instruction classes according to dispatch rules.

All the dispatch rules and instruction classes update are made according
to POWER9 User Manual.

Differential Revision: https://reviews.llvm.org/D61873

llvm-svn: 362509

show more ...

[PowerPC] [ISEL] select x-form instruction for unaligned offset
Differential Revision: https://reviews.llvm.org/D62173

llvm-svn: 361346

[PowerPC][NFC] Update build-vector-tests.ll using utils/update_llc_test_checks.py

build-vector-tests.ll is a huge testcase, it is hard to maintain: eg:
any fundamental changes might need to update h

[PowerPC][NFC] Update build-vector-tests.ll using utils/update_llc_test_checks.py

build-vector-tests.ll is a huge testcase, it is hard to maintain: eg:
any fundamental changes might need to update hundreds of lines. We should
leverage the script to maintain it.

This patch simply run utils/update_llc_test_checks.py on it. There
should be no missing test points.

llvm-svn: 360175

show more ...

[Power9] Enable the Out-of-Order scheduling model for P9 hw

When switched to the MI scheduler for P9, the hardware is modeled as out of order.
However, inside the MI Scheduler algorithm, we still us

[Power9] Enable the Out-of-Order scheduling model for P9 hw

When switched to the MI scheduler for P9, the hardware is modeled as out of order.
However, inside the MI Scheduler algorithm, we still use the in-order scheduling model
as the MicroOpBufferSize isn't set. The MI scheduler take it as the hw cannot buffer
the op. So, only when all the available instructions issued, the pending instruction
could be scheduled. That is not true for our P9 hw in fact.

This patch is trying to enable the Out-of-Order scheduling model. The buffer size 44 is
picked from the P9 hw spec, and the perf test indicate that, its value won't hurt the cpu2017.

With this patch, there are 3 specs improved over 3% and 1 spec deg over 3%. The detail is as follows:

x264_r: +6.95%
cactuBSSN_r: +6.94%
lbm_r: +4.11%
xz_r: -3.85%

And the GEOMEAN for all the C/C++ spec in spec2017 is about 0.18% improved.

Reviewer: Nemanjai
Differential Revision: https://reviews.llvm.org/D55810

llvm-svn: 350285

show more ...

[PowerPC] Improve BUILD_VECTOR of 4 i32s

Currently, for this node:
vector int test(int a, int b, int c, int d) {
return (vector int) { a, b, c, d };
}

we get this on Power9:
mtvsrdd 34, 5

[PowerPC] Improve BUILD_VECTOR of 4 i32s

Currently, for this node:
vector int test(int a, int b, int c, int d) {
return (vector int) { a, b, c, d };
}

we get this on Power9:
mtvsrdd 34, 5, 3
mtvsrdd 35, 6, 4
vmrgow 2, 3, 2

and this on Power8:
mtvsrwz 0, 3
mtvsrwz 1, 5
mtvsrwz 2, 4
mtvsrwz 3, 6
xxmrghd 34, 1, 0
xxmrghd 35, 3, 2
vmrgow 2, 3, 2

This can be improved to this on LE Power9:
rldimi 3, 4, 32, 0
rldimi 5, 6, 32, 0
mtvsrdd 34, 5, 3

and this on LE Power8
rldimi 3, 4, 32, 0
rldimi 5, 6, 32, 0
mtvsrd 34, 3
mtvsrd 35, 5
xxpermdi 34, 35, 34, 0

This patch updates the TD pattern to generate the optimized sequence for both
Power8 and Power9 on LE and BE.

Differential Revision: https://reviews.llvm.org/D53494

llvm-svn: 345414

show more ...

[PowerPC] Improve codegen for vector loads using scalar_to_vector

This patch aims to improve the codegen for vector loads involving the
scalar_to_vector (load X) sequence. Initially, ld->mv instruct

[PowerPC] Improve codegen for vector loads using scalar_to_vector

This patch aims to improve the codegen for vector loads involving the
scalar_to_vector (load X) sequence. Initially, ld->mv instructions were used
for scalar_to_vector (load X), so this patch allows scalar_to_vector (load X)
to utilize:

LXSD and LXSDX for i64 and f64
LXSIWAX for i32 (sign extension to i64)
LXSIWZX for i32 and f64

Committing on behalf of Amy Kwan.
Differential Revision: https://reviews.llvm.org/D48950

llvm-svn: 339260

show more ...

[PowerPC] Do not round values prior to converting to integer

Adding the FP_ROUND nodes when combining FP_TO_[SU]INT of elements
feeding a BUILD_VECTOR into an FP_TO_[SU]INT of the built vector
loses

[PowerPC] Do not round values prior to converting to integer

Adding the FP_ROUND nodes when combining FP_TO_[SU]INT of elements
feeding a BUILD_VECTOR into an FP_TO_[SU]INT of the built vector
loses precision. This patch removes the code that adds these nodes
to true f64 operands. It also adds patterns required to ensure
the code is still vectorized rather than converting individual
elements and inserting into a vector.

Fixes https://bugs.llvm.org/show_bug.cgi?id=38342

Differential Revision: https://reviews.llvm.org/D50121

llvm-svn: 338658

show more ...

[FileCheck] Add -allow-deprecated-dag-overlap to failing llvm tests

See https://reviews.llvm.org/D47106 for details.

Reviewed By: probinson

Differential Revision: https://reviews.llvm.org/D47171

[FileCheck] Add -allow-deprecated-dag-overlap to failing llvm tests

See https://reviews.llvm.org/D47106 for details.

Reviewed By: probinson

Differential Revision: https://reviews.llvm.org/D47171

This commit drops that patch's changes to:

llvm/test/CodeGen/NVPTX/f16x2-instructions.ll
llvm/test/CodeGen/NVPTX/param-load-store.ll

For some reason, the dos line endings there prevent me from commiting
via the monorepo. A follow-up commit (not via the monorepo) will
finish the patch.

llvm-svn: 336843

show more ...