[llvm][Bitcode] Add bitcode reader/writer for DSOLocalEquivalent

This is necessary for compilation with [thin]lto.

Differential Revision: https://reviews.llvm.org/D96170

Encode alignment attribute for `cmpxchg`

This is a follow up patch to D83136 adding the align attribute to `cmpxchg`.
See also D83465 for `atomicrmw`.

Differential Revision: https://reviews.llvm.or

Encode alignment attribute for `cmpxchg`

This is a follow up patch to D83136 adding the align attribute to `cmpxchg`.
See also D83465 for `atomicrmw`.

Differential Revision: https://reviews.llvm.org/D87443

show more ...

Encode alignment attribute for `atomicrmw`

This is a follow up patch to D83136 adding the align attribute to `atomicwmw`.

Differential Revision: https://reviews.llvm.org/D83465

[ThinLTO] Add Visibility bits to GlobalValueSummary::GVFlags

Imported functions and variable get the visibility from the module supplying the
definition. However, non-imported definitions do not ge

[ThinLTO] Add Visibility bits to GlobalValueSummary::GVFlags

Imported functions and variable get the visibility from the module supplying the
definition. However, non-imported definitions do not get the visibility from
(ELF) the most constraining visibility among all modules (Mach-O) the visibility
of the prevailing definition.

This patch

* adds visibility bits to GlobalValueSummary::GVFlags
* computes the result visibility and propagates it to all definitions

Protected/hidden can imply dso_local which can enable some optimizations (this
is stronger than GVFlags::DSOLocal because the implied dso_local can be
leveraged for ELF -shared while default visibility dso_local has to be cleared
for ELF -shared).

Note: we don't have summaries for declarations, so for ELF if a declaration has
the most constraining visibility, the result visibility may not be that one.

Differential Revision: https://reviews.llvm.org/D92900

show more ...

Support for instrumenting only selected files or functions

This change implements support for applying profile instrumentation
only to selected files or functions. The implementation uses the
saniti

Support for instrumenting only selected files or functions

This change implements support for applying profile instrumentation
only to selected files or functions. The implementation uses the
sanitizer special case list format to select which files and functions
to instrument, and relies on the new noprofile IR attribute to exclude
functions from instrumentation.

Differential Revision: https://reviews.llvm.org/D94820

show more ...

Revert "Support for instrumenting only selected files or functions"

This reverts commit 4edf35f11a9e20bd5df3cb47283715f0ff38b751 because
the test fails on Windows bots.

Support for instrumenting only selected files or functions

This change implements support for applying profile instrumentation
only to selected files or functions. The implementation uses the
saniti

Support for instrumenting only selected files or functions

This change implements support for applying profile instrumentation
only to selected files or functions. The implementation uses the
sanitizer special case list format to select which files and functions
to instrument, and relies on the new noprofile IR attribute to exclude
functions from instrumentation.

Differential Revision: https://reviews.llvm.org/D94820

show more ...

[llvm] Use llvm::sort (NFC)

[llvm] Use llvm::append_range (NFC)

[X86] Add x86_amx type for intel AMX.

The x86_amx is used for AMX intrisics. <256 x i32> is bitcast to x86_amx when
it is used by AMX intrinsics, and x86_amx is bitcast to <256 x i32> when it
is use

[X86] Add x86_amx type for intel AMX.

The x86_amx is used for AMX intrisics. <256 x i32> is bitcast to x86_amx when
it is used by AMX intrinsics, and x86_amx is bitcast to <256 x i32> when it
is used by load/store instruction. So amx intrinsics only operate on type x86_amx.
It can help to separate amx intrinsics from llvm IR instructions (+-*/).
Thank Craig for the idea. This patch depend on https://reviews.llvm.org/D87981.

Differential Revision: https://reviews.llvm.org/D91927

show more ...

[DebugInfo] Support Fortran 'use <external module>' statement.

The main change is to add a 'IsDecl' field to DIModule so
that when IsDecl is set to true, the debug info entry generated
for the modul

[DebugInfo] Support Fortran 'use <external module>' statement.

The main change is to add a 'IsDecl' field to DIModule so
that when IsDecl is set to true, the debug info entry generated
for the module would be marked as a declaration. That way, the debugger
would look up the definition of the module in the gloabl scope.

Please see the comments in llvm/test/DebugInfo/X86/dimodule.ll
for what the debug info entries would look like.

Differential Revision: https://reviews.llvm.org/D93462

show more ...

[IR][PGO] Add hot func attribute and use hot/cold attribute in func section

Clang FE currently has hot/cold function attribute. But we only have
cold function attribute in LLVM IR.

This patch adds

[IR][PGO] Add hot func attribute and use hot/cold attribute in func section

Clang FE currently has hot/cold function attribute. But we only have
cold function attribute in LLVM IR.

This patch adds support of hot function attribute to LLVM IR. This
attribute will be used in setting function section prefix/suffix.
Currently .hot and .unlikely suffix only are added in PGO (Sample PGO)
compilation (through isFunctionHotInCallGraph and
isFunctionColdInCallGraph).

This patch changes the behavior. The new behavior is:
(1) If the user annotates a function as hot or isFunctionHotInCallGraph
is true, this function will be marked as hot. Otherwise,
(2) If the user annotates a function as cold or
isFunctionColdInCallGraph is true, this function will be marked as
cold.

The changes are:
(1) user annotated function attribute will used in setting function
section prefix/suffix.
(2) hot attribute overwrites profile count based hotness.
(3) profile count based hotness overwrite user annotated cold attribute.

The intention for these changes is to provide the user a way to mark
certain function as hot in cases where training input is hard to cover
all the hot functions.

Differential Revision: https://reviews.llvm.org/D92493

show more ...

[clang][IR] Add support for leaf attribute

This patch adds support for leaf attribute as an optimization hint
in Clang/LLVM.

Differential Revision: https://reviews.llvm.org/D90275

Adding PoisonValue for representing poison value explicitly in IR

Define ConstantData::PoisonValue.
Add support for poison value to LLLexer/LLParser/BitcodeReader/BitcodeWriter.
Add support for pois

Adding PoisonValue for representing poison value explicitly in IR

Define ConstantData::PoisonValue.
Add support for poison value to LLLexer/LLParser/BitcodeReader/BitcodeWriter.
Add support for poison value to llvm-c interface.
Add support for poison value to OCaml binding.
Add m_Poison in PatternMatch.

Differential Revision: https://reviews.llvm.org/D71126

show more ...

Revert "[IR] add fn attr for no_stack_protector; prevent inlining on mismatch"

This reverts commit b7926ce6d7a83cdf70c68d82bc3389c04009b841.

Going with a simpler approach.

[ThinLTO] Fix .llvmcmd emission

llvm::EmbedBitcodeInModule needs (what used to be called) EmbedMarker
set, in order to emit .llvmcmd. EmbedMarker is really about embedding the
command line, so renam

[ThinLTO] Fix .llvmcmd emission

llvm::EmbedBitcodeInModule needs (what used to be called) EmbedMarker
set, in order to emit .llvmcmd. EmbedMarker is really about embedding the
command line, so renamed the parameter accordingly, too.

This was not caught at test because the check-prefix was incorrect, but
FileCheck does not report that when multiple prefixes are provided. A
separate patch will address that.

Differential Revision: https://reviews.llvm.org/D90278

show more ...

[DebugInfo] Support for DW_TAG_generic_subrange

This is needed to support fortran assumed rank arrays which
have runtime rank.

Summary:
Fortran assumed rank arrays have dynamic rank. DWARF TAG
DW

[DebugInfo] Support for DW_TAG_generic_subrange

This is needed to support fortran assumed rank arrays which
have runtime rank.

Summary:
Fortran assumed rank arrays have dynamic rank. DWARF TAG
DW_TAG_generic_subrange is needed to support that.

Testing:
unit test cases added (hand-written)
check llvm
check debug-info

Reviewed By: aprantl

Differential Revision: https://reviews.llvm.org/D89218

show more ...

[NFC][ThinLTO] Change command line passing to EmbedBitcodeInModule

Changing to pass by ref - less null checks to worry about.

Differential Revision: https://reviews.llvm.org/D90330

[IR] add fn attr for no_stack_protector; prevent inlining on mismatch

It's currently ambiguous in IR whether the source language explicitly
did not want a stack a stack protector (in C, via function

[IR] add fn attr for no_stack_protector; prevent inlining on mismatch

It's currently ambiguous in IR whether the source language explicitly
did not want a stack a stack protector (in C, via function attribute
no_stack_protector) or doesn't care for any given function.

It's common for code that manipulates the stack via inline assembly or
that has to set up its own stack canary (such as the Linux kernel) would
like to avoid stack protectors in certain functions. In this case, we've
been bitten by numerous bugs where a callee with a stack protector is
inlined into an __attribute__((__no_stack_protector__)) caller, which
generally breaks the caller's assumptions about not having a stack
protector. LTO exacerbates the issue.

While developers can avoid this by putting all no_stack_protector
functions in one translation unit together and compiling those with
-fno-stack-protector, it's generally not very ergonomic or as
ergonomic as a function attribute, and still doesn't work for LTO. See also:
https://lore.kernel.org/linux-pm/[email protected]/
https://lore.kernel.org/lkml/[email protected]/T/#u

Typically, when inlining a callee into a caller, the caller will be
upgraded in its level of stack protection (see adjustCallerSSPLevel()).
By adding an explicit attribute in the IR when the function attribute is
used in the source language, we can now identify such cases and prevent
inlining. Block inlining when the callee and caller differ in the case that one
contains `nossp` when the other has `ssp`, `sspstrong`, or `sspreq`.

Fixes pr/47479.

Reviewed By: void

Differential Revision: https://reviews.llvm.org/D87956

show more ...

[IR] Adds mustprogress as a LLVM IR attribute

This adds the LLVM IR attribute `mustprogress` as defined in LangRef through D86233. This attribute will be applied to functions with in languages like

[IR] Adds mustprogress as a LLVM IR attribute

This adds the LLVM IR attribute `mustprogress` as defined in LangRef through D86233. This attribute will be applied to functions with in languages like C++ where forward progress is guaranteed. Functions without this attribute are not required to make progress.

Reviewed By: nikic

Differential Revision: https://reviews.llvm.org/D85393

show more ...

[DebugInfo] Support for DWARF attribute DW_AT_rank

This patch adds support for DWARF attribute DW_AT_rank.

Summary:
Fortran assumed rank arrays have dynamic rank. DWARF attribute
DW_AT_rank is ne

[DebugInfo] Support for DWARF attribute DW_AT_rank

This patch adds support for DWARF attribute DW_AT_rank.

Summary:
Fortran assumed rank arrays have dynamic rank. DWARF attribute
DW_AT_rank is needed to support that.

Testing:
unit test cases added (hand-written)
check llvm
check debug-info

Reviewed By: aprantl

Differential Revision: https://reviews.llvm.org/D89141

show more ...

Flush bitcode incrementally for LTO output

Bitcode writer does not flush buffer until the end by default. This is
fine to small bitcode files. When -flto,--plugin-opt=emit-llvm,-gmlt are
used, the f

Flush bitcode incrementally for LTO output

Bitcode writer does not flush buffer until the end by default. This is
fine to small bitcode files. When -flto,--plugin-opt=emit-llvm,-gmlt are
used, the final bitcode file is large, for example, >8G. Keeping all
data in memory consumes a lot of memory.

This change allows bitcode writer flush data to disk early when buffered
data size is above some threshold. This is only enabled when lld emits
LLVM bitcode.

One issue to address is backpatching bitcode: subblock length, function
body indexes, meta data indexes need to backfill. If buffer can be
flushed partially, we introduced raw_fd_stream that supports
read/seek/write, and enables backpatching bitcode flushed in disk.

Reviewed-by: tejohnson, MaskRay

Differential Revision: https://reviews.llvm.org/D86905

show more ...

[NFC][ThinLTO] Let llvm::EmbedBitcodeInModule handle serialization.

llvm::EmbedBitcodeInModule handles serializing the passed-in module, if
the provided MemoryBufferRef is invalid. This is already t

[NFC][ThinLTO] Let llvm::EmbedBitcodeInModule handle serialization.

llvm::EmbedBitcodeInModule handles serializing the passed-in module, if
the provided MemoryBufferRef is invalid. This is already the path taken
in one of the uses of the API - clang::EmbedBitcode, when called from
BackendConsumer::HandleTranslationUnit - so might as well do the same
here and reduce (by very little) code duplication.

The only difference this patch introduces is that the serialization happens
with ShouldPreserveUseListOrder set to true.

Differential Revision: https://reviews.llvm.org/D87339

show more ...

Set alignment of .llvmbc and .llvmcmd to 1

Otherwise their alignment is dependent on the size of the section. If the size
is large than 16, the alignment will be 16.

16 is a bad choice for both .l

Set alignment of .llvmbc and .llvmcmd to 1

Otherwise their alignment is dependent on the size of the section. If the size
is large than 16, the alignment will be 16.

16 is a bad choice for both .llvmbc and .llvmcmd because the padding between two
contributions from input sections is of a variable size.

A bitstream is actually guaranteed to be 4-byte aligned, but consumers don't
need this property.

show more ...

[SVE] Make ElementCount members private

This patch changes ElementCount so that the Min and Scalable
members are now private and can only be accessed via the get
functions getKnownMinValue() and isS

[SVE] Make ElementCount members private

This patch changes ElementCount so that the Min and Scalable
members are now private and can only be accessed via the get
functions getKnownMinValue() and isScalable(). In addition I've
added some other member functions for more commonly used operations.
Hopefully this makes the class more useful and will reduce the
need for calling getKnownMinValue().

Differential Revision: https://reviews.llvm.org/D86065

show more ...