[ThinLTO] Add Visibility bits to GlobalValueSummary::GVFlags

Imported functions and variable get the visibility from the module supplying the
definition. However, non-imported definitions do not ge

[ThinLTO] Add Visibility bits to GlobalValueSummary::GVFlags

Imported functions and variable get the visibility from the module supplying the
definition. However, non-imported definitions do not get the visibility from
(ELF) the most constraining visibility among all modules (Mach-O) the visibility
of the prevailing definition.

This patch

* adds visibility bits to GlobalValueSummary::GVFlags
* computes the result visibility and propagates it to all definitions

Protected/hidden can imply dso_local which can enable some optimizations (this
is stronger than GVFlags::DSOLocal because the implied dso_local can be
leveraged for ELF -shared while default visibility dso_local has to be cleared
for ELF -shared).

Note: we don't have summaries for declarations, so for ELF if a declaration has
the most constraining visibility, the result visibility may not be that one.

Differential Revision: https://reviews.llvm.org/D92900

show more ...

[LLVMgold.so] -plugin-opt=save-temps: save combined module to .lto.o instead of .o

This matches LLD and fixes https://sourceware.org/bugzilla/show_bug.cgi?id=26262#c1

.o is a bad choice for save-te

[LLVMgold.so] -plugin-opt=save-temps: save combined module to .lto.o instead of .o

This matches LLD and fixes https://sourceware.org/bugzilla/show_bug.cgi?id=26262#c1

.o is a bad choice for save-temps output because it is easy to override the bitcode file (*.o)

```
# Use bfd for the example, -fuse-ld=gold is similar.
clang -flto -c a.c # generate bitcode file a.o
clang -fuse-ld=bfd -flto a.o -o a -Wl,-plugin-opt=save-temps # override a.o

# The user repeats the command but get surprised, because a.o is now a combined module.
clang -fuse-ld=bfd -flto a.o -o a -Wl,-plugin-opt=save-temps
```

Reviewed By: tejohnson

Differential Revision: https://reviews.llvm.org/D84132

show more ...

Test update for a7fa35a629e85a72b8cf07a8f95c7c09d9663808

[LTO] Suppress emission of empty combined module by default

Summary:
That unless the user requested an output object (--lto-obj-path), the an
unused empty combined module is not emitted.

This chang

[LTO] Suppress emission of empty combined module by default

Summary:
That unless the user requested an output object (--lto-obj-path), the an
unused empty combined module is not emitted.

This changed is helpful for some target (ex. RISCV-V) which encoded the
ABI info in IR module flags (target-abi). Empty unused module has no ABI
info so the linker would get the linking error during merging
incompatible ABIs.

Reviewers: tejohnson, espindola, MaskRay

Subscribers: emaste, inglorion, arichardson, hiraditya, simoncook, MaskRay, steven_wu, dexonsmith, PkmX, dang, lenary, s.egerton, luismarques, llvm-commits

Tags: #llvm

Differential Revision: https://reviews.llvm.org/D78988

show more ...

[Object] Change ELFObjectFile<ELFT>::getFileFormatName() to use BFD names

Follow-up for D74433

What the function returns are almost standard BFD names, except that "ELF" is
in uppercase instead of

[Object] Change ELFObjectFile<ELFT>::getFileFormatName() to use BFD names

Follow-up for D74433

What the function returns are almost standard BFD names, except that "ELF" is
in uppercase instead of lowercase.

This patch changes "ELF" to "elf" and changes ARM/AArch64 to use their BFD names.
MIPS and PPC64 have endianness differences as well, but this patch does not intend to address them.

Advantages:

* llvm-objdump: the "file format " line matches GNU objdump on ARM/AArch64 objects
* "file format " line can be extracted and fed into llvm-objcopy -O literally.
(https://github.com/ClangBuiltLinux/linux/issues/779 has such a use case)

Affected tools: llvm-readobj, llvm-objdump, llvm-dwarfdump, MCJIT (internal implementation detail, not exposed)

Reviewed By: jhenderson

Differential Revision: https://reviews.llvm.org/D76046

show more ...

[LLVMgold][test] Fix llvm-nm test after D72658

Differential Revision: https://reviews.llvm.org/D73014

[ThinLTO] Auto-hide prevailing linkonce_odr only when all copies eligible

Summary:
We hit undefined references building with ThinLTO when one source file
contained explicit instantiations of a templ

[ThinLTO] Auto-hide prevailing linkonce_odr only when all copies eligible

Summary:
We hit undefined references building with ThinLTO when one source file
contained explicit instantiations of a template method (weak_odr) but
there were also implicit instantiations in another file (linkonce_odr),
and the latter was the prevailing copy. In this case the symbol was
marked hidden when the prevailing linkonce_odr copy was promoted to
weak_odr. It led to unsats when the resulting shared library was linked
with other code that contained a reference (expecting to be resolved due
to the explicit instantiation).

Add a CanAutoHide flag to the GV summary to allow the thin link to
identify when all copies are eligible for auto-hiding (because they were
all originally linkonce_odr global unnamed addr), and only do the
auto-hide in that case.

Most of the changes here are due to plumbing the new flag through the
bitcode and llvm assembly, and resulting test changes. I augmented the
existing auto-hide test to check for this situation.

Reviewers: pcc

Subscribers: mehdi_amini, inglorion, eraman, dexonsmith, arphaman, dang, llvm-commits, steven_wu, wmi

Tags: #llvm

Differential Revision: https://reviews.llvm.org/D59709

llvm-svn: 360466

show more ...

[llvm-nm] Update all tests to redirect stderr to stdout

This addresses the breakage introduced in r343887.

llvm-svn: 343896

[llvm-nm] Print an explicit "no symbols" message when an object file has no symbols

Summary:
GNU nm (and other nm implementations, such as "go tool nm") prints an explicit "no symbols" message when

[llvm-nm] Print an explicit "no symbols" message when an object file has no symbols

Summary:
GNU nm (and other nm implementations, such as "go tool nm") prints an explicit "no symbols" message when an object file has no symbols. Currently llvm-nm just doesn't print anything. Adding an explicit "no symbols" message will allow llvm-nm to be used in place of nm: some scripts and build processes use `nm <file> | grep "no symbols"` as a test to see if a file has no symbols. It will also be more familiar to anyone used to nm.

That said, the format implemented here is slightly different, in that it doesn't print the tool name in the message (which IMHO is not useful to include).

Demo:
```
$ for nm in nm bin/llvm-nm ; do echo "nm implementation: $nm"; $nm /tmp/foo{1,2}.o; echo; done
nm implementation: nm

/tmp/foo1.o:
nm: /tmp/foo1.o: no symbols

/tmp/foo2.o:
0000000000000000 T foo2

nm implementation: bin/llvm-nm

/tmp/foo1.o:
no symbols

/tmp/foo2.o:
0000000000000000 T foo2
```

Reviewers: MaskRay

Reviewed By: MaskRay

Subscribers: llvm-commits

Differential Revision: https://reviews.llvm.org/D52810

llvm-svn: 343742

show more ...

Remove absolute path in test

My test change in r336148 accidentally included an absolute path, clean
that up to fix bot failures.

llvm-svn: 336151

[ThinLTO] Fix printing of module paths for distributed backend indexes

Summary:
In the individual index files emitted for distributed ThinLTO backends,
the module path ids are not contiguous. Assign

[ThinLTO] Fix printing of module paths for distributed backend indexes

Summary:
In the individual index files emitted for distributed ThinLTO backends,
the module path ids are not contiguous. Assign slots to module paths in
order to handle this better and also to get contiguous numbering in the
summary assembly.

Reviewers: davidxl, dexonsmith

Subscribers: mehdi_amini, inglorion, eraman, llvm-commits, steven_wu

Differential Revision: https://reviews.llvm.org/D48698

llvm-svn: 336148

show more ...

[ThinLTO] Serialize WithGlobalValueDeadStripping index flag for distributed backends

Summary:
A recent fix to drop dead symbols (r323633) did not work for ThinLTO
distributed backends because we los

[ThinLTO] Serialize WithGlobalValueDeadStripping index flag for distributed backends

Summary:
A recent fix to drop dead symbols (r323633) did not work for ThinLTO
distributed backends because we lose the WithGlobalValueDeadStripping
set on the index during the thin link. This patch adds a new flags
record to the bitcode format for the index, and serializes this flag
for the combined index (it would always be 0 for the per-module index
generated by the compile step, so no need to serialize the new flags
record there until/unless we add another flag that applies to the
per-module indexes).

Generally this flag should always be set for the distributed backends,
which are necessarily performed after the thin link. However, if we were
to simply set this flag on the index applied to the distributed backends
(invoked via clang), we would lose the ability to disable dead stripping
via -compute-dead=false for debugging purposes.

Reviewers: grimar, pcc

Subscribers: mehdi_amini, inglorion, eraman, llvm-commits

Differential Revision: https://reviews.llvm.org/D42799

llvm-svn: 324444

show more ...

[LTO] Make processing of combined module more consistent

Summary:
1. Use stream 0 only for combined module. Previously if combined module was not
processes ThinLTO used the stream for own output. Ho

[LTO] Make processing of combined module more consistent

Summary:
1. Use stream 0 only for combined module. Previously if combined module was not
processes ThinLTO used the stream for own output. However small changes in input,
could trigger combined module and shuffle outputs making life of llvm::LTO harder.

2. Always process combined module and write output to stream 0. Processing empty
combined module is cheap and allows llvm::LTO users to avoid implementing processing
which is already done in llvm::LTO.

Subscribers: mehdi_amini, inglorion, eraman, hiraditya

Differential Revision: https://reviews.llvm.org/D41267

llvm-svn: 320905

show more ...

[LLVMgold] Use platform dependent extension in tests

Differential revision: https://reviews.llvm.org/D41238

llvm-svn: 320710

Bitcode: Add a string table to the bitcode format.

Add a top-level STRTAB block containing a string table blob, and start storing
strings for module codes FUNCTION, GLOBALVAR, ALIAS, IFUNC and COMDA

Bitcode: Add a string table to the bitcode format.

Add a top-level STRTAB block containing a string table blob, and start storing
strings for module codes FUNCTION, GLOBALVAR, ALIAS, IFUNC and COMDAT in
the string table.

This change allows us to share names between globals and comdats as well
as between modules, and improves the efficiency of loading bitcode files by
no longer using a bit encoding for symbol names. Once we start writing the
irsymtab to the bitcode file we will also be able to share strings between
it and the module.

On my machine, link time for Chromium for Linux with ThinLTO decreases by
about 7% for no-op incremental builds or about 1% for full builds. Total
bitcode file size decreases by about 3%.

As discussed on llvm-dev:
http://lists.llvm.org/pipermail/llvm-dev/2017-April/111732.html

Differential Revision: https://reviews.llvm.org/D31838

llvm-svn: 300464

show more ...

gold-plugin: Fix test case.

llvm-svn: 293137

Fix x86 gold tests on non-x86 targets.

These tests are missing a target triple and the -m elf_x86_64 gold option,
which makes them fail on non-x86 targets.

Differential revision: https://reviews.ll

Fix x86 gold tests on non-x86 targets.

These tests are missing a target triple and the -m elf_x86_64 gold option,
which makes them fail on non-x86 targets.

Differential revision: https://reviews.llvm.org/D28285

Reviewers: tejohnson
llvm-svn: 290965

show more ...

[LTO] Reject modules without datalayout.

Also, udpate the ~60 failing tests in the tree which did
not contain a valid datalayout.
This fixes PR31123. lld will be updated in a following patch,
immedi

[LTO] Reject modules without datalayout.

Also, udpate the ~60 failing tests in the tree which did
not contain a valid datalayout.
This fixes PR31123. lld will be updated in a following patch,
immediately after this is committed.

Differential Revision: https://reviews.llvm.org/D27082

llvm-svn: 289719

show more ...

Tentatively fix gold-plugin test: ThinLTO objects start at offset 0 now.

Annoyingly, incremental builds don't detect these kind of issue.

llvm-svn: 279612

[gold/ThinLTO] Restore ThinLTO file management in gold plugin

Summary:
The gold-plugin changes added along with the new LTO API in r278338 had
the effect of removing the management of the PluginInpu

[gold/ThinLTO] Restore ThinLTO file management in gold plugin

Summary:
The gold-plugin changes added along with the new LTO API in r278338 had
the effect of removing the management of the PluginInputFile that
ensured the files weren't released back to gold until the backend
threads were complete. Add back the old file handling.

Fixes PR29020.

Reviewers: mehdi_amini

Subscribers: mehdi_amini, llvm-commits, hjl.tools

Differential Revision: https://reviews.llvm.org/D23721

llvm-svn: 279356

show more ...

Restore "Resolution-based LTO API."

This restores commit r278330, with fixes for a few bot failures:
- Fix a late change I had made to the save temps output file that I
missed due to existing file

Restore "Resolution-based LTO API."

This restores commit r278330, with fixes for a few bot failures:
- Fix a late change I had made to the save temps output file that I
missed due to existing files sitting on my disk
- Fix a bunch of Windows bot failures with "ambiguous call to overloaded
function" due to confusion between llvm::make_unique vs
std::make_unique (preface the new make_unique calls with "llvm::")
- Attempt to fix a modules bot failure by adding a missing include
to LTO/Config.h.

Original change:

Resolution-based LTO API.

Summary:
This introduces a resolution-based LTO API. The main advantage of this API over
existing APIs is that it allows the linker to supply a resolution for each
symbol in each object, rather than the combined object as a whole. This will
become increasingly important for use cases such as ThinLTO which require us
to process symbol resolutions in a more complicated way than just adjusting
linkage.

Patch by Peter Collingbourne.

Reviewers: rafael, tejohnson, mehdi_amini

Subscribers: lhames, tejohnson, mehdi_amini, llvm-commits

Differential Revision: https://reviews.llvm.org/D20268

llvm-svn: 278338

show more ...

Revert "Resolution-based LTO API."

This reverts commit r278330.

I made a change to the save temps output that is causing issues with the
bots. Didn't realize this because I had older output files s

Revert "Resolution-based LTO API."

This reverts commit r278330.

I made a change to the save temps output that is causing issues with the
bots. Didn't realize this because I had older output files sitting on
disk in my test output directory.

llvm-svn: 278331

show more ...

Resolution-based LTO API.

Summary:
This introduces a resolution-based LTO API. The main advantage of this API over
existing APIs is that it allows the linker to supply a resolution for each
symbol i

Resolution-based LTO API.

Summary:
This introduces a resolution-based LTO API. The main advantage of this API over
existing APIs is that it allows the linker to supply a resolution for each
symbol in each object, rather than the combined object as a whole. This will
become increasingly important for use cases such as ThinLTO which require us
to process symbol resolutions in a more complicated way than just adjusting
linkage.

Patch by Peter Collingbourne.

Reviewers: rafael, tejohnson, mehdi_amini

Subscribers: lhames, tejohnson, mehdi_amini, llvm-commits

Differential Revision: https://reviews.llvm.org/D20268

Address review comments

llvm-svn: 278330

show more ...

[ThinLTO/gold] Enable symbol resolution in distributed backend case

While testing a follow-on change to enable index-based symbol resolution
and internalization in the distributed backends, I realiz

[ThinLTO/gold] Enable symbol resolution in distributed backend case

While testing a follow-on change to enable index-based symbol resolution
and internalization in the distributed backends, I realized that a test
case change I made in r275247 was only required because we were not
analyzing symbols in the claimed files in thinlto-index-only mode.

In the fixed test case there should be no internalization because we are
linking in -shared mode, so f() is in fact exported, which is detected
properly when we analyze symbols in thinlto-index-only mode. Note that
this is not (yet) a correctness issue (because we are not yet performing
the index-based linkage optimizations in the distributed backends -
that's coming in a follow-on patch).

llvm-svn: 275277

show more ...

[ThinLTO/gold] ThinLTO internalization fixes

Internalization was missing cases where we originally had a local symbol
that was promoted eagerly but not actually exported. This is because we
were onl

[ThinLTO/gold] ThinLTO internalization fixes

Internalization was missing cases where we originally had a local symbol
that was promoted eagerly but not actually exported. This is because we
were only internalizing the set of global (non-local) symbols that were
PREVAILAING_DEF_IRONLY. Instead, collect the set of global symbols that
are referenced outside of a single IR file, and skip internalization for
those.

llvm-svn: 275247

show more ...