[WPD] Use new llvm.public.type.test intrinsic for potentially publicly visible classes

Turning on opaque pointers has uncovered an issue with WPD where we currently pattern match away `assume(type.t

[WPD] Use new llvm.public.type.test intrinsic for potentially publicly visible classes

Turning on opaque pointers has uncovered an issue with WPD where we currently pattern match away `assume(type.test)` in WPD so that a later LTT doesn't resolve the type test to undef and introduce an `assume(false)`. The pattern matching can fail in cases where we transform two `assume(type.test)`s into `assume(phi(type.test.1, type.test.2))`.

Currently we create `assume(type.test)` for all virtual calls that might be devirtualized. This is to support `-Wl,--lto-whole-program-visibility`.

To prevent this, all virtual calls that may not be in the same LTO module instead use a new `llvm.public.type.test` intrinsic in place of the `llvm.type.test`. Then when we know if `-Wl,--lto-whole-program-visibility` is passed or not, we can either replace all `llvm.public.type.test` with `llvm.type.test`, or replace all `llvm.public.type.test` with `true`. This prevents WPD from trying to pattern match away `assume(type.test)` for public virtual calls when failing the pattern matching will result in miscompiles.

Reviewed By: tejohnson

Differential Revision: https://reviews.llvm.org/D128955

show more ...

Cleanup header dependencies in LLVMCore

Based on the output of include-what-you-use.

This is a big chunk of changes. It is very likely to break downstream code
unless they took a lot of care in avo

Cleanup header dependencies in LLVMCore

Based on the output of include-what-you-use.

This is a big chunk of changes. It is very likely to break downstream code
unless they took a lot of care in avoiding hidden ehader dependencies, something
the LLVM codebase doesn't do that well :-/

I've tried to summarize the biggest change below:

- llvm/include/llvm-c/Core.h: no longer includes llvm-c/ErrorHandling.h
- llvm/IR/DIBuilder.h no longer includes llvm/IR/DebugInfo.h
- llvm/IR/IRBuilder.h no longer includes llvm/IR/IntrinsicInst.h
- llvm/IR/LLVMRemarkStreamer.h no longer includes llvm/Support/ToolOutputFile.h
- llvm/IR/LegacyPassManager.h no longer include llvm/Pass.h
- llvm/IR/Type.h no longer includes llvm/ADT/SmallPtrSet.h
- llvm/IR/PassManager.h no longer includes llvm/Pass.h nor llvm/Support/Debug.h

And the usual count of preprocessed lines:
$ clang++ -E -Iinclude -I../llvm/include ../llvm/lib/IR/*.cpp -std=c++14 -fno-rtti -fno-exceptions | wc -l
before: 6400831
after: 6189948

200k lines less to process is no that bad ;-)

Discourse thread on the topic: https://llvm.discourse.group/t/include-what-you-use-include-cleanup

Differential Revision: https://reviews.llvm.org/D118652

show more ...

[LTO] Ignore unreachable virtual functions in WPD in hybrid LTO.

Differential Revision: https://reviews.llvm.org/D115492

[ThinLTO] Add noRecurse and noUnwind thinlink function attribute propagation

Thinlink provides an opportunity to propagate function attributes across modules, enabling additional propagation opportu

[ThinLTO] Add noRecurse and noUnwind thinlink function attribute propagation

Thinlink provides an opportunity to propagate function attributes across modules, enabling additional propagation opportunities.

This change propagates (currently default off, turn on with `disable-thinlto-funcattrs=1`) noRecurse and noUnwind based off of function summaries of the prevailing functions in bottom-up call-graph order. Testing on clang self-build:
1. There's a 35-40% increase in noUnwind functions due to the additional propagation opportunities.
2. Throughput is measured at 10-15% increase in thinlink time which itself is 1.5% of E2E link time.

Implementation-wise this adds the following summary function attributes:
1. noUnwind: function is noUnwind
2. mayThrow: function contains a non-call instruction that `Instruction::mayThrow` returns true on (e.g. windows SEH instructions)
3. hasUnknownCall: function contains calls that don't make it into the summary call-graph thus should not be propagated from (e.g. indirect for now, could add no-opt functions as well)

Testing:
Clang self-build passes and 2nd stage build passes check-all
ninja check-all with newly added tests passing

Reviewed By: tejohnson

Differential Revision: https://reviews.llvm.org/D36850

show more ...

[ThinLTO] Update combined index for SamplePGO indirect calls to locals

In ThinLTO for locals we normally compute the GUID from the name after
prepending the source path to get a unique global id. Sa

[ThinLTO] Update combined index for SamplePGO indirect calls to locals

In ThinLTO for locals we normally compute the GUID from the name after
prepending the source path to get a unique global id. SamplePGO indirect
call profiles contain the target GUID without this uniquification,
however (unless compiling with -funique-internal-linkage-names).

In order to correctly handle the call edges added to the combined index
for these indirect calls, during importing and bitcode writing we
consult a map of original to full GUID to identify the actual callee.
However, for a large application this was consuming a lot of compile
time as we need to do this repeatedly (especially during importing where
we may traverse call edges multiple times).

To fix this implement a suggestion in one of the FIXME comments, and
actually modify the call edges during a single traversal after the index
is built to perform the fixups once. I combined this fixup with the dead
code analysis performed on the index in order to avoid adding an
additional walk of the index. The dead code analysis is the first
analysis performed on the index.

This reduced the time required for a large thin link with SamplePGO by
about 20%.

No new test added, but I confirmed that there are existing tests that
will fail when no fixup is performed.

Differential Revision: https://reviews.llvm.org/D110374

show more ...

[IR] Use range-based for loops (NFC)

[LTO] Perform DSOLocal propagation in combined index

Perform DSOLocal propagation within summary list of every GV. This
avoids the repeated query of this information during function
importing.

Diff

[LTO] Perform DSOLocal propagation in combined index

Perform DSOLocal propagation within summary list of every GV. This
avoids the repeated query of this information during function
importing.

Differential Revision: https://reviews.llvm.org/D96398

show more ...

[ThinLTO] Add Visibility bits to GlobalValueSummary::GVFlags

Imported functions and variable get the visibility from the module supplying the
definition. However, non-imported definitions do not ge

[ThinLTO] Add Visibility bits to GlobalValueSummary::GVFlags

Imported functions and variable get the visibility from the module supplying the
definition. However, non-imported definitions do not get the visibility from
(ELF) the most constraining visibility among all modules (Mach-O) the visibility
of the prevailing definition.

This patch

* adds visibility bits to GlobalValueSummary::GVFlags
* computes the result visibility and propagates it to all definitions

Protected/hidden can imply dso_local which can enable some optimizations (this
is stronger than GVFlags::DSOLocal because the implied dso_local can be
leveraged for ELF -shared while default visibility dso_local has to be cleared
for ELF -shared).

Note: we don't have summaries for declarations, so for ELF if a declaration has
the most constraining visibility, the result visibility may not be that one.

Differential Revision: https://reviews.llvm.org/D92900

show more ...

[Analysis, CodeGen, IR] Use contains (NFC)

[ThinLTO] Compile time improvement to propagateAttributes

I found that propagateAttributes was ~23% of a thin link's run time
(almost 4x higher than the second hottest function). The main reason is

[ThinLTO] Compile time improvement to propagateAttributes

I found that propagateAttributes was ~23% of a thin link's run time
(almost 4x higher than the second hottest function). The main reason is
that it re-examines a global var each time it is referenced. This
becomes unnecessary once it is marked both non read only and non write
only. I added a set to avoid doing redundant work, which dropped the
runtime of that thin link by almost 15%.

I made a smaller efficiency improvement (no measurable impact) to skip
all summaries for a VI if the first copy is dead. I added an assert to
ensure that all copies are dead if any is. The code in
computeDeadSymbols marks all summaries for a VI as live. There is one
corner case where it was skipping marking an alias as live, that I
fixed. However, since the code earlier marked all copies of a preserved
GUID's VI as live, and each 'visit' marks all copies live, the only case
where this could make a difference is summaries that were marked live
when they were built initially, and that is only a few special compiler
generated symbols and inline assembly symbols, so it likely is never
provoked in practice.

Differential Revision: https://reviews.llvm.org/D84985

show more ...

Revert "[StackSafety,NFC] Remove unneded constexpr"

This reverts commit 38470baa542bde038340b7d10a0ed2c25fac1bfa because it
breaks builds with lld and gold linkers.

[StackSafety,NFC] Remove unneded constexpr

Differential Revision: https://reviews.llvm.org/D80908

[StackSafety] Add info into function summary

Summary:
This patch adds optional field into function summary,
implements asm and bitcode serialization. YAML
serialization is omitted and can be added l

[StackSafety] Add info into function summary

Summary:
This patch adds optional field into function summary,
implements asm and bitcode serialization. YAML
serialization is omitted and can be added later if
needed.

This patch includes this information into summary only
if module contains at least one sanitize_memtag function.
In a near future MTE is the user of the analysis.
Later if needed we can provede more direct control
on when information is included into summary.

Reviewers: eugenis

Subscribers: hiraditya, steven_wu, dexonsmith, arphaman, llvm-commits

Tags: #llvm

Differential Revision: https://reviews.llvm.org/D80908

show more ...

[ADT][NFC] SCCIterator: Change hasLoop() to hasCycle()

[Assembler] Emit summary index flags

Differential revision: https://reviews.llvm.org/D74420

Reenable "Always import constants" after compile time fixes

Summary:
Reenables importing of constants by default, which was disabled in
D73724 due to excessive thin link times. These inefficiencies

Reenable "Always import constants" after compile time fixes

Summary:
Reenables importing of constants by default, which was disabled in
D73724 due to excessive thin link times. These inefficiencies were
fixed in D73851.

I re-measured thin link times for a number of binaries that had compile
time explosions with importing of constants previously and confirmed
they no longer have any notable increases with it enabled.

Reviewers: wmi, evgeny777

Subscribers: hiraditya, arphaman, llvm-commits

Tags: #llvm

Differential Revision: https://reviews.llvm.org/D74512

show more ...

[ThinLTO] Disable "Always import constants" due to compile time issues

Summary:
Disable the always importing of constants introduced in D70404 by
default under a new internal option, since it is cau

[ThinLTO] Disable "Always import constants" due to compile time issues

Summary:
Disable the always importing of constants introduced in D70404 by
default under a new internal option, since it is causing order of
magnitude compile time regressions during the thin link. Will continue
investigating why the regressions occur.

Reviewers: evgeny777, wmi

Subscribers: mehdi_amini, inglorion, hiraditya, steven_wu, dexonsmith, arphaman, llvm-commits

Tags: #llvm

Differential Revision: https://reviews.llvm.org/D73724

show more ...

[ThinLTO] Always import constants

This patch imports constant variables even when they can't be internalized
(which results in promotion). This offers some extra constant folding
opportunities.

Dif

[ThinLTO] Always import constants

This patch imports constant variables even when they can't be internalized
(which results in promotion). This offers some extra constant folding
opportunities.

Differential revision: https://reviews.llvm.org/D70404

show more ...

[ThinLTO] Show preserved symbols in DOT files

Differential revision: https://reviews.llvm.org/D71608

[IR] Use a reference in a range-based for

This avoids unneeded copies when using a range-based for loops.

This avoids new warnings due to D68912 adds -Wrange-loop-analysis to -Wall.

Differential R

[IR] Use a reference in a range-based for

This avoids unneeded copies when using a range-based for loops.

This avoids new warnings due to D68912 adds -Wrange-loop-analysis to -Wall.

Differential Revision: https://reviews.llvm.org/D70870

show more ...

[ThinLTO] Add option to disable readonly/writeonly attribute propagation

Summary:
Add an option to allow the attribute propagation on the index to be
disabled, to allow a workaround for issues (such

[ThinLTO] Add option to disable readonly/writeonly attribute propagation

Summary:
Add an option to allow the attribute propagation on the index to be
disabled, to allow a workaround for issues (such as that fixed by
D70977).

Also move the setting of the WithAttributePropagation flag on the index
into propagateAttributes(), and remove some old stale code that predated
this flag and cleared the maybe read/write only bits when we need to
disable the propagation (previously only when importing disabled, now
also when the new option disables it).

Reviewers: evgeny777, steven_wu

Subscribers: mehdi_amini, inglorion, hiraditya, dexonsmith, arphaman, llvm-commits

Tags: #llvm

Differential Revision: https://reviews.llvm.org/D70984

show more ...

Fix error message missed in commit dde589389fcb8b5098f7a47f1b781b27d29a0cac.

Patch by Andrew Browne <[email protected]>

Reviewers: tejohnson, evgeny777

Reviewed By: tejohnson

Subscribers: arpha

Fix error message missed in commit dde589389fcb8b5098f7a47f1b781b27d29a0cac.

Patch by Andrew Browne <[email protected]>

Reviewers: tejohnson, evgeny777

Reviewed By: tejohnson

Subscribers: arphaman, hiraditya, llvm-commits

Tags: #llvm

Differential Revision: https://reviews.llvm.org/D70195

show more ...

ThinLTO : Import always_inline functions irrespective of the threshold

Summary: A user can force a function to be inlined by specifying the always_inline attribute. Currently, thinlto implementation

ThinLTO : Import always_inline functions irrespective of the threshold

Summary: A user can force a function to be inlined by specifying the always_inline attribute. Currently, thinlto implementation is not aware of always_inline functions and does not guarantee import of such functions, which in turn can prevent inlining of such functions.

Patch by Bharathi Seshadri <[email protected]>

Reviewers: tejohnson

Reviewed By: tejohnson

Subscribers: mehdi_amini, inglorion, hiraditya, steven_wu, dexonsmith, arphaman, llvm-commits

Tags: #llvm

Differential Revision: https://reviews.llvm.org/D70014

show more ...

[ThinLTO] Fix bug when importing writeonly variables

Patch enables import of write-only variables with non-trivial initializers
to fix linker errors. Initializers of imported variables are converted

[ThinLTO] Fix bug when importing writeonly variables

Patch enables import of write-only variables with non-trivial initializers
to fix linker errors. Initializers of imported variables are converted to
'zeroinitializer' to avoid promotion of referenced objects.

Differential revision: https://reviews.llvm.org/D70006

show more ...

[ThinLTO] Import readonly vars with refs

Patch allows importing declarations of functions and variables, referenced
by the initializer of some other readonly variable.
Differential revision: https:/

[ThinLTO] Import readonly vars with refs

Patch allows importing declarations of functions and variables, referenced
by the initializer of some other readonly variable.
Differential revision: https://reviews.llvm.org/D69561

show more ...