[X86] initial -mfunction-return=thunk-extern support

Adds support for:
* `-mfunction-return=<value>` command line flag, and
* `__attribute__((function_return("<value>")))` function attribute

Where

[X86] initial -mfunction-return=thunk-extern support

Adds support for:
* `-mfunction-return=<value>` command line flag, and
* `__attribute__((function_return("<value>")))` function attribute

Where the supported <value>s are:
* keep (disable)
* thunk-extern (enable)

thunk-extern enables clang to change ret instructions into jmps to an
external symbol named __x86_return_thunk, implemented as a new
MachineFunctionPass named "x86-return-thunks", keyed off the new IR
attribute fn_ret_thunk_extern.

The symbol __x86_return_thunk is expected to be provided by the runtime
the compiled code is linked against and is not defined by the compiler.
Enabling this option alone doesn't provide mitigations without
corresponding definitions of __x86_return_thunk!

This new MachineFunctionPass is very similar to "x86-lvi-ret".

The <value>s "thunk" and "thunk-inline" are currently unsupported. It's
not clear yet that they are necessary: whether the thunk pattern they
would emit is beneficial or used anywhere.

Should the <value>s "thunk" and "thunk-inline" become necessary,
x86-return-thunks could probably be merged into x86-retpoline-thunks
which has pre-existing machinery for emitting thunks (which could be
used to implement the <value> "thunk").

Has been found to build+boot with corresponding Linux
kernel patches. This helps the Linux kernel mitigate RETBLEED.
* CVE-2022-23816
* CVE-2022-28693
* CVE-2022-29901

See also:
* "RETBLEED: Arbitrary Speculative Code Execution with Return
Instructions."
* AMD SECURITY NOTICE AMD-SN-1037: AMD CPU Branch Type Confusion
* TECHNICAL GUIDANCE FOR MITIGATING BRANCH TYPE CONFUSION REVISION 1.0
2022-07-12
* Return Stack Buffer Underflow / Return Stack Buffer Underflow /
CVE-2022-29901, CVE-2022-28693 / INTEL-SA-00702

SystemZ may eventually want to support "thunk-extern" and "thunk"; both
options are used by the Linux kernel's CONFIG_EXPOLINE.

This functionality has been available in GCC since the 8.1 release, and
was backported to the 7.3 release.

Many thanks for folks that provided discrete review off list due to the
embargoed nature of this hardware vulnerability. Many Bothans died to
bring us this information.

Link: https://www.youtube.com/watch?v=IF6HbCKQHK8
Link: https://github.com/llvm/llvm-project/issues/54404
Link: https://gcc.gnu.org/legacy-ml/gcc-patches/2018-01/msg01197.html
Link: https://www.intel.com/content/www/us/en/developer/articles/technical/software-security-guidance/advisory-guidance/return-stack-buffer-underflow.html
Link: https://arstechnica.com/information-technology/2022/07/intel-and-amd-cpus-vulnerable-to-a-new-speculative-execution-attack/?comments=1
Link: https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=ce114c866860aa9eae3f50974efc68241186ba60
Link: https://www.intel.com/content/www/us/en/security-center/advisory/intel-sa-00702.html
Link: https://www.intel.com/content/www/us/en/security-center/advisory/intel-sa-00707.html

Reviewed By: aaron.ballman, craig.topper

Differential Revision: https://reviews.llvm.org/D129572

show more ...

[SanitizerBounds] Add support for NoSanitizeBounds function

Currently adding attribute no_sanitize("bounds") isn't disabling
-fsanitize=local-bounds (also enabled in -fsanitize=bounds). The Clang
fr

[SanitizerBounds] Add support for NoSanitizeBounds function

Currently adding attribute no_sanitize("bounds") isn't disabling
-fsanitize=local-bounds (also enabled in -fsanitize=bounds). The Clang
frontend handles fsanitize=array-bounds which can already be disabled by
no_sanitize("bounds"). However, instrumentation added by the
BoundsChecking pass in the middle-end cannot be disabled by the
attribute.

The fix is very similar to D102772 that added the ability to selectively
disable sanitizer pass on certain functions.

In this patch, if no_sanitize("bounds") is provided, an additional
function attribute (NoSanitizeBounds) is attached to IR to let the
BoundsChecking pass know we want to disable local-bounds checking. In
order to support this feature, the IR is extended (similar to D102772)
to make Clang able to preserve the information and let BoundsChecking
pass know bounds checking is disabled for certain function.

Reviewed By: melver

Differential Revision: https://reviews.llvm.org/D119816

show more ...

Extend the `uwtable` attribute with unwind table kind

We have the `clang -cc1` command-line option `-funwind-tables=1|2` and
the codegen option `VALUE_CODEGENOPT(UnwindTables, 2, 0) ///< Unwind
tabl

Extend the `uwtable` attribute with unwind table kind

We have the `clang -cc1` command-line option `-funwind-tables=1|2` and
the codegen option `VALUE_CODEGENOPT(UnwindTables, 2, 0) ///< Unwind
tables (1) or asynchronous unwind tables (2)`. However, this is
encoded in LLVM IR by the presence or the absence of the `uwtable`
attribute, i.e. we lose the information whether to generate want just
some unwind tables or asynchronous unwind tables.

Asynchronous unwind tables take more space in the runtime image, I'd
estimate something like 80-90% more, as the difference is adding
roughly the same number of CFI directives as for prologues, only a bit
simpler (e.g. `.cfi_offset reg, off` vs. `.cfi_restore reg`). Or even
more, if you consider tail duplication of epilogue blocks.
Asynchronous unwind tables could also restrict code generation to
having only a finite number of frame pointer adjustments (an example
of *not* having a finite number of `SP` adjustments is on AArch64 when
untagging the stack (MTE) in some cases the compiler can modify `SP`
in a loop).
Having the CFI precise up to an instruction generally also means one
cannot bundle together CFI instructions once the prologue is done,
they need to be interspersed with ordinary instructions, which means
extra `DW_CFA_advance_loc` commands, further increasing the unwind
tables size.

That is to say, async unwind tables impose a non-negligible overhead,
yet for the most common use cases (like C++ exceptions), they are not
even needed.

This patch extends the `uwtable` attribute with an optional
value:
- `uwtable` (default to `async`)
- `uwtable(sync)`, synchronous unwind tables
- `uwtable(async)`, asynchronous (instruction precise) unwind tables

Reviewed By: MaskRay

Differential Revision: https://reviews.llvm.org/D114543

show more ...

[IR] Allow the 'align' param attr on vectors of pointers

This patch extends the available uses of the 'align' parameter attribute
to include vectors of pointers. The attribute specifies pointer
alig

[IR] Allow the 'align' param attr on vectors of pointers

This patch extends the available uses of the 'align' parameter attribute
to include vectors of pointers. The attribute specifies pointer
alignment element-wise.

This change was previously requested and discussed in D87304.

The vector predication (VP) intrinsics intend to use this for scatter
and gather operations, as they lack the explicit alignment parameter
that the masked versions use.

Reviewed By: nikic

Differential Revision: https://reviews.llvm.org/D115161

show more ...

[BitcodeReader] propagateAttributeTypes(): fix opaque pointer handling

Can't get the pointee type of an opaque pointer,
but in that case said attributes must already be typed,
so just don't try to r

[BitcodeReader] propagateAttributeTypes(): fix opaque pointer handling

Can't get the pointee type of an opaque pointer,
but in that case said attributes must already be typed,
so just don't try to rewrite them if they already are.

show more ...

[IR] Remove unbounded as possible value for vscale_range minimum

The default for min is changed to 1. The behaviour of -mvscale-{min,max}
in Clang is also changed such that 16 is the max vscale when

[IR] Remove unbounded as possible value for vscale_range minimum

The default for min is changed to 1. The behaviour of -mvscale-{min,max}
in Clang is also changed such that 16 is the max vscale when targeting
SVE and no max is specified.

Reviewed By: sdesmalen, paulwalker-arm

Differential Revision: https://reviews.llvm.org/D113294

show more ...

Fix typo of colon to semicolon in lit tests

[clang][Codegen] Introduce the disable_sanitizer_instrumentation attribute

The purpose of __attribute__((disable_sanitizer_instrumentation)) is to
prevent all kinds of sanitizer instrumentation appl

[clang][Codegen] Introduce the disable_sanitizer_instrumentation attribute

The purpose of __attribute__((disable_sanitizer_instrumentation)) is to
prevent all kinds of sanitizer instrumentation applied to a certain
function, Objective-C method, or global variable.

The no_sanitize(...) attribute drops instrumentation checks, but may
still insert code preventing false positive reports. In some cases
though (e.g. when building Linux kernel with -fsanitize=kernel-memory
or -fsanitize=thread) the users may want to avoid any kind of
instrumentation.

Differential Revision: https://reviews.llvm.org/D108029

show more ...

[IR] Add elementtype attribute

This implements the elementtype attribute specified in D105407. It
just adds the attribute and the specified verifier rules, but
doesn't yet make use of it anywhere.

[IR] Add elementtype attribute

This implements the elementtype attribute specified in D105407. It
just adds the attribute and the specified verifier rules, but
doesn't yet make use of it anywhere.

Differential Revision: https://reviews.llvm.org/D106008

show more ...

[BitCode] Add noprofile to getAttrFromCode()

After D104475 / D104658, building the Linux kernel with ThinLTO is
broken:

ld.lld: error: Unknown attribute kind (73) (Producer: 'LLVM13.0.0git'
Reader:

[BitCode] Add noprofile to getAttrFromCode()

After D104475 / D104658, building the Linux kernel with ThinLTO is
broken:

ld.lld: error: Unknown attribute kind (73) (Producer: 'LLVM13.0.0git'
Reader: 'LLVM 13.0.0git')

getAttrFromCode() has never handled this attribute so it is written
during the ThinLTO phase but it cannot be handled during the linking
phase.

Add noprofile to getAttrFromCode() so that disassembly works properly.

Reviewed By: MaskRay

Differential Revision: https://reviews.llvm.org/D104995

show more ...

[SanitizeCoverage] Add support for NoSanitizeCoverage function attribute

We really ought to support no_sanitize("coverage") in line with other
sanitizers. This came up again in discussions on the Li

[SanitizeCoverage] Add support for NoSanitizeCoverage function attribute

We really ought to support no_sanitize("coverage") in line with other
sanitizers. This came up again in discussions on the Linux-kernel
mailing lists, because we currently do workarounds using objtool to
remove coverage instrumentation. Since that support is only on x86, to
continue support coverage instrumentation on other architectures, we
must support selectively disabling coverage instrumentation via function
attributes.

Unfortunately, for SanitizeCoverage, it has not been implemented as a
sanitizer via fsanitize= and associated options in Sanitizers.def, but
rolls its own option fsanitize-coverage. This meant that we never got
"automatic" no_sanitize attribute support.

Implement no_sanitize attribute support by special-casing the string
"coverage" in the NoSanitizeAttr implementation. To keep the feature as
unintrusive to existing IR generation as possible, define a new negative
function attribute NoSanitizeCoverage to propagate the information
through to the instrumentation pass.

Fixes: https://bugs.llvm.org/show_bug.cgi?id=49035

Reviewed By: vitalybuka, morehouse

Differential Revision: https://reviews.llvm.org/D102772

show more ...

IR+AArch64: add a "swiftasync" argument attribute.

This extends any frame record created in the function to include that
parameter, passed in X22.

The new record looks like [X22, FP, LR] in memory,

IR+AArch64: add a "swiftasync" argument attribute.

This extends any frame record created in the function to include that
parameter, passed in X22.

The new record looks like [X22, FP, LR] in memory, and FP is stored with 0b0001
in bits 63:60 (CodeGen assumes they are 0b0000 in normal operation). The effect
of this is that tools walking the stack should expect to see one of three
values there:

* 0b0000 => a normal, non-extended record with just [FP, LR]
* 0b0001 => the extended record [X22, FP, LR]
* 0b1111 => kernel space, and a non-extended record.

All other values are currently reserved.

If compiling for arm64e this context pointer is address-discriminated with the
discriminator 0xc31a and the DB (process-specific) key.

There is also an "i8** @llvm.swift.async.context.addr()" intrinsic providing
front-ends access to this slot (and forcing its creation initialized to nullptr
if necessary).

show more ...

Reapply "OpaquePtr: Turn inalloca into a type attribute"

This reverts commit 07e46367baeca96d84b03fa215b41775f69d5989.

Revert "Reapply "OpaquePtr: Turn inalloca into a type attribute""

Reverting because test 'Bindings/Go/go.test' is failing on most
buildbots.

This reverts commit fc9df309917e57de704f3ce4372138a8d4a2

Revert "Reapply "OpaquePtr: Turn inalloca into a type attribute""

Reverting because test 'Bindings/Go/go.test' is failing on most
buildbots.

This reverts commit fc9df309917e57de704f3ce4372138a8d4a23d7a.

show more ...

Reapply "OpaquePtr: Turn inalloca into a type attribute"

This reverts commit 20d5c42e0ef5d252b434bcb610b04f1cb79fe771.

Revert "OpaquePtr: Turn inalloca into a type attribute"

This reverts commit 4fefed65637ec46c8c2edad6b07b5569ac61e9e5.
Broke check-clang everywhere.

OpaquePtr: Turn inalloca into a type attribute

I think byval/sret and the others are close to being able to rip out
the code to support the missing type case. A lot of this code is
shared with inall

OpaquePtr: Turn inalloca into a type attribute

I think byval/sret and the others are close to being able to rip out
the code to support the missing type case. A lot of this code is
shared with inalloca, so catch this up to the others so that can
happen.

show more ...

[IR] Add vscale_range IR function attribute

This attribute represents the minimum and maximum values vscale can
take. For now this attribute is not hooked up to anything during
codegen, this will be

[IR] Add vscale_range IR function attribute

This attribute represents the minimum and maximum values vscale can
take. For now this attribute is not hooked up to anything during
codegen, this will be added in the future when such codegen is
considered stable.

Additionally hook up the -msve-vector-bits=<x> clang option to emit this
attribute.

Differential Revision: https://reviews.llvm.org/D98030

show more ...

[IR][PGO] Add hot func attribute and use hot/cold attribute in func section

Clang FE currently has hot/cold function attribute. But we only have
cold function attribute in LLVM IR.

This patch adds

[IR][PGO] Add hot func attribute and use hot/cold attribute in func section

Clang FE currently has hot/cold function attribute. But we only have
cold function attribute in LLVM IR.

This patch adds support of hot function attribute to LLVM IR. This
attribute will be used in setting function section prefix/suffix.
Currently .hot and .unlikely suffix only are added in PGO (Sample PGO)
compilation (through isFunctionHotInCallGraph and
isFunctionColdInCallGraph).

This patch changes the behavior. The new behavior is:
(1) If the user annotates a function as hot or isFunctionHotInCallGraph
is true, this function will be marked as hot. Otherwise,
(2) If the user annotates a function as cold or
isFunctionColdInCallGraph is true, this function will be marked as
cold.

The changes are:
(1) user annotated function attribute will used in setting function
section prefix/suffix.
(2) hot attribute overwrites profile count based hotness.
(3) profile count based hotness overwrite user annotated cold attribute.

The intention for these changes is to provide the user a way to mark
certain function as hot in cases where training input is hard to cover
all the hot functions.

Differential Revision: https://reviews.llvm.org/D92493

show more ...

[IR] Fixed the typo in attributes test

Fixed the typo introduced in D90275.

Differential Revision: https://reviews.llvm.org/D93420

[clang][IR] Add support for leaf attribute

This patch adds support for leaf attribute as an optimization hint
in Clang/LLVM.

Differential Revision: https://reviews.llvm.org/D90275

OpaquePtr: Bulk update tests to use typed sret

OpaquePtr: Bulk update tests to use typed byval

Upgrade of the IR text tests should be the only thing blocking making
typed byval mandatory. Partially done through regex and partially
manual.

Revert "[BitCode] decode nossp fn attr"

This reverts commit 0b11d018cc2f2c6bea5dac8dc72140cdb502ca02.

Going with a simpler approach.

[BitCode] decode nossp fn attr

I missed this in https://reviews.llvm.org/D87956.

Reviewed By: void

Differential Revision: https://reviews.llvm.org/D90177