Add support for atomic memory copy lowering

Currently, the utility supports lowering of non atomic memory transfer routines only. This patch adds support for atomic version of memcopy. This may be u

Add support for atomic memory copy lowering

Currently, the utility supports lowering of non atomic memory transfer routines only. This patch adds support for atomic version of memcopy. This may be useful for targets not supporting atomic memcopy.

Reviewed By: arsenm

Differential Revision: https://reviews.llvm.org/D118443

show more ...

Preserve aliasing info during memory intrinsics lowering

By specification, source and destination of llvm.memcpy.* must either be equal or non-overlapping. This semantics is hard or impossible to f

Preserve aliasing info during memory intrinsics lowering

By specification, source and destination of llvm.memcpy.* must either be equal or non-overlapping. This semantics is hard or impossible to figure out once lowered. This patch explicitly marks loads from source and stores to destination as not aliasing if source and destination is known to be not equal.

Reviewed By: arsenm

Differential Revision: https://reviews.llvm.org/D118441

show more ...

[LowerMemIntrinsics] Explicitly use i8 type in memmove lowering

By convention, memcpy/memmove intrinsics are always used with i8
pointers (though this is not enforced), so in practice this code
was

[LowerMemIntrinsics] Explicitly use i8 type in memmove lowering

By convention, memcpy/memmove intrinsics are always used with i8
pointers (though this is not enforced), so in practice this code
was always using an i8 type. Make that explicit.

Of course, i8 is not a very profitable choice, and this code could
be more performant by picking an appropriate larger type. But that
would require additional test coverage and correctness review, and
certainly shouldn't be a decision based on the pointer element type.

show more ...

[NFC] Remove uses of PointerType::getElementType()

Instead use either Type::getPointerElementType() or
Type::getNonOpaquePointerElementType().

This is part of D117885, in preparation for deprecatin

[NFC] Remove uses of PointerType::getElementType()

Instead use either Type::getPointerElementType() or
Type::getNonOpaquePointerElementType().

This is part of D117885, in preparation for deprecating the API.

show more ...

[LowerMemIntrinsics] Typo fix.

Utils: Mostly convert memcpy expansion to use Align

The TTI hooks aren't converted. I also think the intrinsics should
have mandatory alignment and never return MaybeAlign.

Utils: Always set alignment when expanding mem intrinsics

This was creating natural aligned loads and stores, which may not be
the case. The target could request a wider type load with less
alignmen

Utils: Always set alignment when expanding mem intrinsics

This was creating natural aligned loads and stores, which may not be
the case. The target could request a wider type load with less
alignment.

show more ...

TTI: Add addrspace parameters to memcpy lowering functions

[opaque pointer types] Pass value type to GetElementPtr creation.

This cleans up all GetElementPtr creation in LLVM to explicitly pass a
value type rather than deriving it from the pointer's element

[opaque pointer types] Pass value type to GetElementPtr creation.

This cleans up all GetElementPtr creation in LLVM to explicitly pass a
value type rather than deriving it from the pointer's element-type.

Differential Revision: https://reviews.llvm.org/D57173

llvm-svn: 352913

show more ...

[opaque pointer types] Pass value type to LoadInst creation.

This cleans up all LoadInst creation in LLVM to explicitly pass the
value type rather than deriving it from the pointer's element-type.

[opaque pointer types] Pass value type to LoadInst creation.

This cleans up all LoadInst creation in LLVM to explicitly pass the
value type rather than deriving it from the pointer's element-type.

Differential Revision: https://reviews.llvm.org/D57172

llvm-svn: 352911

show more ...

Update the file headers across all of the LLVM projects in the monorepo
to reflect the new license.

We understand that people may be surprised that we're moving the header
entirely to discuss the ne

Update the file headers across all of the LLVM projects in the monorepo
to reflect the new license.

We understand that people may be surprised that we're moving the header
entirely to discuss the new license. We checked this carefully with the
Foundation's lawyer and we believe this is the correct approach.

Essentially, all code in the project is now made available by the LLVM
project under our new license, so you will see that the license headers
include that license only. Some of our contributors have contributed
code under our old license, and accordingly, we have retained a copy of
our old license notice in the top-level files in each project and
repository.

llvm-svn: 351636

show more ...

[TI removal] Remove `TerminatorInst` from BasicBlockUtils.h

This requires updating a number of .cpp files to adapt to the new API.
I've just systematically updated all uses of `TerminatorInst` withi

[TI removal] Remove `TerminatorInst` from BasicBlockUtils.h

This requires updating a number of .cpp files to adapt to the new API.
I've just systematically updated all uses of `TerminatorInst` within
these files te `Instruction` so thta I won't have to touch them again in
the future.

llvm-svn: 344498

show more ...

[LowerMemIntrinsics] Update uses of deprecated MemIntrinsic::getAlignment API (NFC)

Summary:
This change is part of step five in the series of changes to remove alignment argument from
memcpy/memmov

[LowerMemIntrinsics] Update uses of deprecated MemIntrinsic::getAlignment API (NFC)

Summary:
This change is part of step five in the series of changes to remove alignment argument from
memcpy/memmove/memset in favour of alignment attributes. In particular, this changes the
LowerMemIntrinsics pass to cease using the old getAlignment() API of MemoryIntrinsic in
favour of getting source & dest specific alignments through the new API.

Steps:
Step 1) Remove alignment parameter and create alignment parameter attributes for
memcpy/memmove/memset. ( rL322965, rC322964, rL322963 )
Step 2) Expand the IRBuilder API to allow creation of memcpy/memmove with differing
source and dest alignments. ( rL323597 )
Step 3) Update Clang to use the new IRBuilder API. ( rC323617 )
Step 4) Update Polly to use the new IRBuilder API. ( rL323618 )
Step 5) Update LLVM passes that create memcpy/memmove calls to use the new IRBuilder API,
and those that use use MemIntrinsicInst::[get|set]Alignment() to use [get|set]DestAlignment()
and [get|set]SourceAlignment() instead. ( rL323886, rL323891, rL324148, rL324273 )
Step 6) Remove the single-alignment IRBuilder API for memcpy/memmove, and the
MemIntrinsicInst::[get|set]Alignment() methods.

Reference
http://lists.llvm.org/pipermail/llvm-dev/2015-August/089384.html
http://lists.llvm.org/pipermail/llvm-commits/Week-of-Mon-20151109/312083.html

llvm-svn: 324278

show more ...

[Memcpy Loop Lowering] Remove the fixed int8 lowering.

Switch over to the lowering that uses target supplied operand types.

Differential Revision: https://reviews.llvm.org/D41201

llvm-svn: 320989

[Memcpy Loop Lowering] Only calculate residual size/bytes copied when needed.

If the loop operand type is int8 then there will be no residual loop for the
unknown size expansion. Dont create the res

[Memcpy Loop Lowering] Only calculate residual size/bytes copied when needed.

If the loop operand type is int8 then there will be no residual loop for the
unknown size expansion. Dont create the residual-size and bytes-copied values
when they are not needed.

llvm-svn: 320929

show more ...

[Memcpy Loop Lowering] Insert loop BB inbetween the split BB.

The original memcpy expansion inserted the loop basic block inbetween
the 2 new basic blocks created by splitting the original block the

[Memcpy Loop Lowering] Insert loop BB inbetween the split BB.

The original memcpy expansion inserted the loop basic block inbetween
the 2 new basic blocks created by splitting the original block the memcpy
call was in. This commit makes the new memcpy expansion do the same to keep the
layout of the IR matching between the old and new implementations.

Differential Review: https://reviews.llvm.org/D41197

llvm-svn: 320848

show more ...

Extend memcpy expansion in Transform/Utils to handle wider operand types.

Adds loop expansions for known-size and unknown-sized memcpy calls, allowing the
target to provide the operand types through

Extend memcpy expansion in Transform/Utils to handle wider operand types.

Adds loop expansions for known-size and unknown-sized memcpy calls, allowing the
target to provide the operand types through TTI callbacks. The default values
for the TTI callbacks use int8 operand types and matches the existing behaviour
if they aren't overridden by the target.

Differential revision: https://reviews.llvm.org/D32536

llvm-svn: 307346

show more ...

Recommit "r306541 - Add zero-length check to memcpy/memset load store loop expansion""

With fix for use-after-free errors. We can't add the new branch and
remove the old one until we are done with t

Recommit "r306541 - Add zero-length check to memcpy/memset load store loop expansion""

With fix for use-after-free errors. We can't add the new branch and
remove the old one until we are done with the Builder constructed for
the block.

llvm-svn: 306937

show more ...

Revert "r306541 - Add zero-length check to memcpy/memset load store loop expansion"

Segfaults in non-optimized builds. I'll get a stack trace and a
reproducer to Teresa.

llvm-svn: 306793

Add zero-length check to memcpy/memset load store loop expansion

Summary:
I was testing using this expansion logic in other cases besides
NVPTX, and found some runtime failures due to the lack of a

Add zero-length check to memcpy/memset load store loop expansion

Summary:
I was testing using this expansion logic in other cases besides
NVPTX, and found some runtime failures due to the lack of a check
for a zero length memcpy/memset before the loop. There is already
such a check in the memmove expansion code though.

Reviewers: hfinkel

Subscribers: jholewinski, wdng, llvm-commits

Differential Revision: https://reviews.llvm.org/D34707

llvm-svn: 306541

show more ...

Sort the remaining #include lines in include/... and lib/....

I did this a long time ago with a janky python script, but now
clang-format has built-in support for this. I fed clang-format every
line

Sort the remaining #include lines in include/... and lib/....

I did this a long time ago with a janky python script, but now
clang-format has built-in support for this. I fed clang-format every
line with a #include and let it re-sort things according to the precise
LLVM rules for include ordering baked into clang-format these days.

I've reverted a number of files where the results of sorting includes
isn't healthy. Either places where we have legacy code relying on
particular include ordering (where possible, I'll fix these separately)
or where we have particular formatting around #include lines that
I didn't want to disturb in this patch.

This patch is *entirely* mechanical. If you get merge conflicts or
anything, just ignore the changes in this patch and run clang-format
over your #include lines in the files.

Sorry for any noise here, but it is important to keep these things
stable. I was seeing an increasing number of patches with irrelevant
re-ordering of #include lines because clang-format was used. This patch
at least isolates that churn, makes it easy to skip when resolving
conflicts, and gets us to a clean baseline (again).

llvm-svn: 304787

show more ...

NVPTX: Extract mem intrinsic expansions into utilities

llvm-svn: 294490