[clang][CodeGen] add fn_ret_thunk_extern to synthetic fns

Follow up fix to
commit 2240d72f15f3 ("[X86] initial -mfunction-return=thunk-extern
support")
https://reviews.llvm.org/D129572

@nathanchanc

[clang][CodeGen] add fn_ret_thunk_extern to synthetic fns

Follow up fix to
commit 2240d72f15f3 ("[X86] initial -mfunction-return=thunk-extern
support")
https://reviews.llvm.org/D129572

@nathanchance reported that -mfunction-return=thunk-extern was failing
to annotate the asan and tsan contructors.
https://lore.kernel.org/llvm/Ys7pLq+tQk5xEa%[email protected]/

I then noticed the same occurring for gcov synthetic functions.

Similar to
commit 2786e67 ("[IR][sanitizer] Add module flag "frame-pointer" and set
it for cc1 -mframe-pointer={non-leaf,all}")
define a new module level MetaData, "fn_ret_thunk_extern", then when set
adds the fn_ret_thunk_extern IR Fn Attr to synthetically created
Functions.

Fixes https://github.com/llvm/llvm-project/issues/56514

Reviewed By: MaskRay

Differential Revision: https://reviews.llvm.org/D129709

show more ...

[DirectX] add thread/group id DXIL operations.

Add DXIL operation for thread/group id operations.

ID Name Description
93 ThreadId reads the thread ID
94 Gro

[DirectX] add thread/group id DXIL operations.

Add DXIL operation for thread/group id operations.

ID Name Description
93 ThreadId reads the thread ID
94 GroupId reads the group ID (SV_GroupID)
95 ThreadIdInGroup reads the thread ID within the group (SV_GroupThreadID)
96 FlattenedThreadIdInGroup provides a flattened index for a given thread within a given group (SV_GroupIndex)

Also add llvm intrinsic which map to these intrinsics to DXIL operation.

Reviewed By: beanz

Differential Revision: https://reviews.llvm.org/D127990

show more ...

Rename GCCBuiltin into ClangBuiltin

This patch is needed because developers expect "GCCBuiltin" items to be the GCC intrinsics equivalent and not the Clang internals.

Reviewed By: #libc_abi, RKSimo

Rename GCCBuiltin into ClangBuiltin

This patch is needed because developers expect "GCCBuiltin" items to be the GCC intrinsics equivalent and not the Clang internals.

Reviewed By: #libc_abi, RKSimon, xbolva00

Differential Revision: https://reviews.llvm.org/D127460

show more ...

Don't use Optional::hasValue (NFC)

[ValueTypes] Define MVTs for v128i2/v64i4 as well as i2 and i4.

Adds MVT::v128i2, MVT::v64i4, and implied MVT::i2, MVT::i4.

Keeps MVT::i2, MVT::i4 lowering actions as expand, which should be
remove

[ValueTypes] Define MVTs for v128i2/v64i4 as well as i2 and i4.

Adds MVT::v128i2, MVT::v64i4, and implied MVT::i2, MVT::i4.

Keeps MVT::i2, MVT::i4 lowering actions as expand, which should be
removed once targets set this explicitly.

Adjusts 11 lit tests to reflect slightly different behavior during
DAG combine.

Differential Revision: https://reviews.llvm.org/D125247

show more ...

Revert "[ValueTypes] Define MVTs for v128i2/v64i4 as well as i2 and i4."

This reverts commit 430ac5c3029c52e391e584c6d4447e6e361fae99.

Due to failures in Clang tests.

Differential Revision: https:

Revert "[ValueTypes] Define MVTs for v128i2/v64i4 as well as i2 and i4."

This reverts commit 430ac5c3029c52e391e584c6d4447e6e361fae99.

Due to failures in Clang tests.

Differential Revision: https://reviews.llvm.org/D125247

show more ...

[ValueTypes] Define MVTs for v128i2/v64i4 as well as i2 and i4.

Adds MVT::v128i2, MVT::v64i4, and implied MVT::i2, MVT::i4.

Keeps MVT::i2, MVT::i4 lowering actions as `expand`, which should be
remo

[ValueTypes] Define MVTs for v128i2/v64i4 as well as i2 and i4.

Adds MVT::v128i2, MVT::v64i4, and implied MVT::i2, MVT::i4.

Keeps MVT::i2, MVT::i4 lowering actions as `expand`, which should be
removed once targets set this explicitly.

Adjusts 11 lit tests to reflect slightly different behavior during
DAG combine.

Differential Revision: https://reviews.llvm.org/D125247

show more ...

Assert on polymorphic pointer intrinsic param

Opaque pointers cannot be polymorphic on the pointed type given their
lack thereof. However they are currently accepted by tablegen but the
intrinsic si

Assert on polymorphic pointer intrinsic param

Opaque pointers cannot be polymorphic on the pointed type given their
lack thereof. However they are currently accepted by tablegen but the
intrinsic signature verifier trips when verifying any further
polymorphic type because the opaque pointer codepath for pointers will
not push the pointed type in ArgTys.

This commit adds an assert to easily catch such cases instead of having
the generic signature match failure.

Reviewed By: #opaque-pointers, nikic

Differential Revision: https://reviews.llvm.org/D125764

show more ...

[IR] Require intrinsic struct return type to be anonymous

This is an alternative to D122376. Rather than working around the
problem, this patch requires that struct return types in intrinsics
are an

[IR] Require intrinsic struct return type to be anonymous

This is an alternative to D122376. Rather than working around the
problem, this patch requires that struct return types in intrinsics
are anonymous/literal and adds auto-upgrade code to convert
existing uses of intrinsics with named struct types.

This ensures that the mapping between intrinsic name and
intrinsic function type is actually bijective, as it is supposed
to be.

This also fixes https://github.com/llvm/llvm-project/issues/37891.

Differential Revision: https://reviews.llvm.org/D122471

show more ...

[VP] Preserve address space of pointer for strided load/store intrinsics.

This adds LLVMAnyPointerToElt to use instead of LLVMPointerToElt.
This allows us to preserve the address space as part of th

[VP] Preserve address space of pointer for strided load/store intrinsics.

This adds LLVMAnyPointerToElt to use instead of LLVMPointerToElt.
This allows us to preserve the address space as part of the type
overload for the intrinsic, but still require the vector element
type to match the pointer type.

Reviewed By: nikic

Differential Revision: https://reviews.llvm.org/D122042

show more ...

[IR] Allow matching pointer to vector with opaque pointers.

Allows for skipping the pointer to vector type if opaque pointers
are enabled and the matching pointer is a vector pointer when
matching a

[IR] Allow matching pointer to vector with opaque pointers.

Allows for skipping the pointer to vector type if opaque pointers
are enabled and the matching pointer is a vector pointer when
matching an intrinsic signature in the verifier.

No test added since lacking a target using intrinsic with pointer
to vector arguments.

Differential Revision: https://reviews.llvm.org/D122203

show more ...

[IR][IPSCCP] Treat different function type as address taken (PR54258)

Without opaque pointers, this code currently treats a call through
a bitcast as the function being address taken, and IPSCCP rel

[IR][IPSCCP] Treat different function type as address taken (PR54258)

Without opaque pointers, this code currently treats a call through
a bitcast as the function being address taken, and IPSCCP relies
on this for correctness. Match the same behavior under opaque
pointers by checking that the function types are the same.

Fixes https://github.com/llvm/llvm-project/issues/54258.

show more ...

Extend the `uwtable` attribute with unwind table kind

We have the `clang -cc1` command-line option `-funwind-tables=1|2` and
the codegen option `VALUE_CODEGENOPT(UnwindTables, 2, 0) ///< Unwind
tabl

Extend the `uwtable` attribute with unwind table kind

We have the `clang -cc1` command-line option `-funwind-tables=1|2` and
the codegen option `VALUE_CODEGENOPT(UnwindTables, 2, 0) ///< Unwind
tables (1) or asynchronous unwind tables (2)`. However, this is
encoded in LLVM IR by the presence or the absence of the `uwtable`
attribute, i.e. we lose the information whether to generate want just
some unwind tables or asynchronous unwind tables.

Asynchronous unwind tables take more space in the runtime image, I'd
estimate something like 80-90% more, as the difference is adding
roughly the same number of CFI directives as for prologues, only a bit
simpler (e.g. `.cfi_offset reg, off` vs. `.cfi_restore reg`). Or even
more, if you consider tail duplication of epilogue blocks.
Asynchronous unwind tables could also restrict code generation to
having only a finite number of frame pointer adjustments (an example
of *not* having a finite number of `SP` adjustments is on AArch64 when
untagging the stack (MTE) in some cases the compiler can modify `SP`
in a loop).
Having the CFI precise up to an instruction generally also means one
cannot bundle together CFI instructions once the prologue is done,
they need to be interspersed with ordinary instructions, which means
extra `DW_CFA_advance_loc` commands, further increasing the unwind
tables size.

That is to say, async unwind tables impose a non-negligible overhead,
yet for the most common use cases (like C++ exceptions), they are not
even needed.

This patch extends the `uwtable` attribute with an optional
value:
- `uwtable` (default to `async`)
- `uwtable(sync)`, synchronous unwind tables
- `uwtable(async)`, asynchronous (instruction precise) unwind tables

Reviewed By: MaskRay

Differential Revision: https://reviews.llvm.org/D114543

show more ...

Cleanup header dependencies in LLVMCore

Based on the output of include-what-you-use.

This is a big chunk of changes. It is very likely to break downstream code
unless they took a lot of care in avo

Cleanup header dependencies in LLVMCore

Based on the output of include-what-you-use.

This is a big chunk of changes. It is very likely to break downstream code
unless they took a lot of care in avoiding hidden ehader dependencies, something
the LLVM codebase doesn't do that well :-/

I've tried to summarize the biggest change below:

- llvm/include/llvm-c/Core.h: no longer includes llvm-c/ErrorHandling.h
- llvm/IR/DIBuilder.h no longer includes llvm/IR/DebugInfo.h
- llvm/IR/IRBuilder.h no longer includes llvm/IR/IntrinsicInst.h
- llvm/IR/LLVMRemarkStreamer.h no longer includes llvm/Support/ToolOutputFile.h
- llvm/IR/LegacyPassManager.h no longer include llvm/Pass.h
- llvm/IR/Type.h no longer includes llvm/ADT/SmallPtrSet.h
- llvm/IR/PassManager.h no longer includes llvm/Pass.h nor llvm/Support/Debug.h

And the usual count of preprocessed lines:
$ clang++ -E -Iinclude -I../llvm/include ../llvm/lib/IR/*.cpp -std=c++14 -fno-rtti -fno-exceptions | wc -l
before: 6400831
after: 6189948

200k lines less to process is no that bad ;-)

Discourse thread on the topic: https://llvm.discourse.group/t/include-what-you-use-include-cleanup

Differential Revision: https://reviews.llvm.org/D118652

show more ...

[IR] Handle opaque pointers in PtrToArgument mangling

It appears that this mangling type is currently unused. Make it
compatible with opaque pointers in case it becomes used again...

[NFC] Remove uses of PointerType::getElementType()

Instead use either Type::getPointerElementType() or
Type::getNonOpaquePointerElementType().

This is part of D117885, in preparation for deprecatin

[NFC] Remove uses of PointerType::getElementType()

Instead use either Type::getPointerElementType() or
Type::getNonOpaquePointerElementType().

This is part of D117885, in preparation for deprecating the API.

show more ...

Use a sorted array instead of a map to store AttrBuilder string attributes

Using and std::map<SmallString, SmallString> for target dependent attributes is
inefficient: it makes its constructor sligh

Use a sorted array instead of a map to store AttrBuilder string attributes

Using and std::map<SmallString, SmallString> for target dependent attributes is
inefficient: it makes its constructor slightly heavier, and involves extra
allocation for each new string attribute. Storing the attribute key/value as
strings implies extra allocation/copy step.

Use a sorted vector instead. Given the low number of attributes generally
involved, this is cheaper, as showcased by

https://llvm-compile-time-tracker.com/compare.php?from=5de322295f4ade692dc4f1823ae4450ad3c48af2&to=05bc480bf641a9e3b466619af43a2d123ee3f71d&stat=instructions

Differential Revision: https://reviews.llvm.org/D116599

show more ...

Introduce the AttributeMask class

This class is solely used as a lightweight and clean way to build a set of
attributes to be removed from an AttrBuilder. Previously AttrBuilder was used
both for bu

Introduce the AttributeMask class

This class is solely used as a lightweight and clean way to build a set of
attributes to be removed from an AttrBuilder. Previously AttrBuilder was used
both for building and removing, which introduced odd situation like creation of
Attribute with dummy value because the only relevant part was the attribute
kind.

Differential Revision: https://reviews.llvm.org/D116110

show more ...

[WebAssembly] Implementation of intrinsic for ref.null and HeapType removal

This patch implements the intrinsic for ref.null.
In the process of implementing int_wasm_ref_null_func() and
int_wasm_ref

[WebAssembly] Implementation of intrinsic for ref.null and HeapType removal

This patch implements the intrinsic for ref.null.
In the process of implementing int_wasm_ref_null_func() and
int_wasm_ref_null_extern() intrinsics, it removes the redundant
HeapType.

This also causes the textual assembler syntax for ref.null to
change. Instead of receiving an argument: `func` or `extern`, the
instruction mnemonic is either ref.null_func or ref.null_extern,
without the need for a further operand.

Reviewed By: tlively

Differential Revision: https://reviews.llvm.org/D114979

show more ...

[IR,TableGen] Add support for vec3 intrinsic arguments

Add generic support for vec3 types, and in particular define
llvm_v3f32_ty which will be used by AMDGPU's
llvm.amdgcn.image.bvh.intersect.ray i

[IR,TableGen] Add support for vec3 intrinsic arguments

Add generic support for vec3 types, and in particular define
llvm_v3f32_ty which will be used by AMDGPU's
llvm.amdgcn.image.bvh.intersect.ray intrinsic.

Differential Revision: https://reviews.llvm.org/D114956

show more ...

[NFC] Use Optional<ProfileCount> to model invalid counts

ProfileCount could model invalid values, but a user had no indication
that the getCount method could return bogus data. Optional<ProfileCount

[NFC] Use Optional<ProfileCount> to model invalid counts

ProfileCount could model invalid values, but a user had no indication
that the getCount method could return bogus data. Optional<ProfileCount>
addresses that, because the user must dereference the optional. In
addition, the patch removes concept duplication.

Differential Revision: https://reviews.llvm.org/D113839

show more ...

[PowerPC] Implement longdouble pack/unpack builtins

Implement two builtins to pack/unpack IBM extended long double float,
according to GCC 'Basic PowerPC Builtin Functions Available ISA 2.05'.

Revi

[PowerPC] Implement longdouble pack/unpack builtins

Implement two builtins to pack/unpack IBM extended long double float,
according to GCC 'Basic PowerPC Builtin Functions Available ISA 2.05'.

Reviewed By: jsji

Differential Revision: https://reviews.llvm.org/D112055

show more ...

Reland [IR] Increase max alignment to 4GB

Currently the max alignment representable is 1GB, see D108661.
Setting the align of an object to 4GB is desirable in some cases to make sure the lower 32 bi

Reland [IR] Increase max alignment to 4GB

Currently the max alignment representable is 1GB, see D108661.
Setting the align of an object to 4GB is desirable in some cases to make sure the lower 32 bits are clear which can be used for some optimizations, e.g. https://crbug.com/1016945.

This uses an extra bit in instructions that carry an alignment. We can store 15 bits of "free" information, and with this change some instructions (e.g. AtomicCmpXchgInst) use 14 bits.
We can increase the max alignment representable above 4GB (up to 2^62) since we're only using 33 of the 64 values, but I've just limited it to 4GB for now.

The one place we have to update the bitcode format is for the alloca instruction. It stores its alignment into 5 bits of a 32 bit bitfield. I've added another field which is 8 bits and should be future proof for a while. For backward compatibility, we check if the old field has a value and use that, otherwise use the new field.

Updating clang's max allowed alignment will come in a future patch.

Reviewed By: hans

Differential Revision: https://reviews.llvm.org/D110451

show more ...

[IR] Remove unused parameter (NFC)

[OpaquePtr] Forbid mixing typed and opaque pointers

Currently, opaque pointers are supported in two forms: The
-force-opaque-pointers mode, where all pointers are opaque and
typed pointers do not ex

[OpaquePtr] Forbid mixing typed and opaque pointers

Currently, opaque pointers are supported in two forms: The
-force-opaque-pointers mode, where all pointers are opaque and
typed pointers do not exist. And as a simple ptr type that can
coexist with typed pointers.

This patch removes support for the mixed mode. You either get
typed pointers, or you get opaque pointers, but not both. In the
(current) default mode, using ptr is forbidden. In -opaque-pointers
mode, all pointers are opaque.

The motivation here is that the mixed mode introduces additional
issues that don't exist in fully opaque mode. D105155 is an example
of a design problem. Looking at D109259, it would probably need
additional work to support mixed mode (e.g. to generate GEPs for
typed base but opaque result). Mixed mode will also end up
inserting many casts between i8* and ptr, which would require
significant additional work to consistently avoid.

I don't think the mixed mode is particularly valuable, as it
doesn't align with our end goal. The only thing I've found it to
be moderately useful for is adding some opaque pointer tests in
between typed pointer tests, but I think we can live without that.

Differential Revision: https://reviews.llvm.org/D109290

show more ...