[Propeller] Encode address offsets of basic blocks relative to the end of the previous basic blocks.

This is a resurrection of D106421 with the change that it keeps backward-compatibility. This mean

[Propeller] Encode address offsets of basic blocks relative to the end of the previous basic blocks.

This is a resurrection of D106421 with the change that it keeps backward-compatibility. This means decoding the previous version of `LLVM_BB_ADDR_MAP` will work. This is required as the profile mapping tool is not released with LLVM (AutoFDO). As suggested by @jhenderson we rename the original section type value to `SHT_LLVM_BB_ADDR_MAP_V0` and assign a new value to the `SHT_LLVM_BB_ADDR_MAP` section type. The new encoding adds a version byte to each function entry to specify the encoding version for that function. This patch also adds a feature byte to be used with more flexibility in the future. An use-case example for the feature field is encoding multi-section functions more concisely using a different format.

Conceptually, the new encoding emits basic block offsets and sizes as label differences between each two consecutive basic block begin and end label. When decoding, offsets must be aggregated along with basic block sizes to calculate the final offsets of basic blocks relative to the function address.

This encoding uses smaller values compared to the existing one (offsets relative to function symbol).
Smaller values tend to occupy fewer bytes in ULEB128 encoding. As a result, we get about 17% total reduction in the size of the bb-address-map section (from about 11MB to 9MB for the clang PGO binary).
The extra two bytes (version and feature fields) incur a small 3% size overhead to the `LLVM_BB_ADDR_MAP` section size.

Reviewed By: jhenderson

Differential Revision: https://reviews.llvm.org/D121346

show more ...

[llvm] Use value_or instead of getValueOr (NFC)

Cleanup LLVMDWARFDebugInfo

As usual with that header cleanup series, some implicit dependencies now need to
be explicit:

llvm/DebugInfo/DWARF/DWARFContext.h no longer includes:
- "llvm/DebugInfo/DW

Cleanup LLVMDWARFDebugInfo

As usual with that header cleanup series, some implicit dependencies now need to
be explicit:

llvm/DebugInfo/DWARF/DWARFContext.h no longer includes:
- "llvm/DebugInfo/DWARF/DWARFAcceleratorTable.h"
- "llvm/DebugInfo/DWARF/DWARFCompileUnit.h"
- "llvm/DebugInfo/DWARF/DWARFDebugAbbrev.h"
- "llvm/DebugInfo/DWARF/DWARFDebugAranges.h"
- "llvm/DebugInfo/DWARF/DWARFDebugFrame.h"
- "llvm/DebugInfo/DWARF/DWARFDebugLoc.h"
- "llvm/DebugInfo/DWARF/DWARFDebugMacro.h"
- "llvm/DebugInfo/DWARF/DWARFGdbIndex.h"
- "llvm/DebugInfo/DWARF/DWARFSection.h"
- "llvm/DebugInfo/DWARF/DWARFTypeUnit.h"
- "llvm/DebugInfo/DWARF/DWARFUnitIndex.h"

Plus llvm/Support/Errc.h not included by a bunch of llvm/DebugInfo/DWARF/DWARF*.h files

Preprocessed lines to build llvm on my setup:
after: 1065629059
before: 1066621848

Which is a great diff!

Discourse thread: https://discourse.llvm.org/t/include-what-you-use-include-cleanup
Differential Revision: https://reviews.llvm.org/D119723

show more ...

Fix ambiguous overload build failure

LLVM (llvmorg-14-init) under Debian sid using latest gcc (Debian
10.3.0-9) 10.3.0 fails due to ambiguous overload on operators == and !=:

/root/src/llvm/src/llv

Fix ambiguous overload build failure

LLVM (llvmorg-14-init) under Debian sid using latest gcc (Debian
10.3.0-9) 10.3.0 fails due to ambiguous overload on operators == and !=:

/root/src/llvm/src/llvm/tools/obj2yaml/elf2yaml.cpp:212:22:
error: ambiguous overload for 'operator!='
(operand types are 'llvm::ELFYAML::ELF_SHF' and 'int')

/root/src/llvm/src/llvm/tools/obj2yaml/elf2yaml.cpp:204:32:
error: ambiguous overload for 'operator!='
(operand types are 'const llvm::yaml::Hex64' and 'int')

/root/src/llvm/src/llvm/lib/CodeGen/LiveDebugValues/VarLocBasedImpl.cpp:629:35:
error: ambiguous overload for 'operator=='
(operand types are 'const uint64_t' {aka 'const long unsigned int'} and
'llvm::Register')

Reviewed by: StephenTozer, jmorse, Higuoxing

Differential Revision: https://reviews.llvm.org/D109534

show more ...

[LLD][LLVM] CG Graph profile using relocations

Currently when .llvm.call-graph-profile is created by llvm it explicitly encodes the symbol indices. This section is basically a black box for post pro

[LLD][LLVM] CG Graph profile using relocations

Currently when .llvm.call-graph-profile is created by llvm it explicitly encodes the symbol indices. This section is basically a black box for post processing tools. For example, if we run strip -s on the object files the symbol table changes, but indices in that section do not. In non-visible behavior indices point to wrong symbols. The visible behavior indices point outside of Symbol table: "invalid symbol index".

This patch changes the format by using R_*_NONE relocations to indicate the from/to symbols. The Frequency (Weight) will still be in the .llvm.call-graph-profile, but symbol information will be in relocation section. In LLD information from both sections is used to reconstruct call graph profile. Relocations themselves will never be applied.

With this approach post processing tools that handle relocations correctly work for this section also. Tools can add/remove symbols and as long as they handle relocation sections with this approach information stays correct.

Doing a quick experiment with clang-13.
The size went up from 107KB to 322KB, aggregate of all the input sections. Size of clang-13 binary is ~118MB. For users of -fprofile-use/-fprofile-sample-use the size of object files will go up slightly, it will not impact final binary size.

Reviewed By: jhenderson, MaskRay

Differential Revision: https://reviews.llvm.org/D104080

show more ...

[obj2yaml] Address D104035 review comments

Accidentally missed from commit 5c1639fe064b.

Differential Revision: https://reviews.llvm.org/D104035

[yaml2obj][obj2yaml] Support custom ELF section header string table name

This patch adds support for a new field in the FileHeader, which states
the name to use for the section header string table.

[yaml2obj][obj2yaml] Support custom ELF section header string table name

This patch adds support for a new field in the FileHeader, which states
the name to use for the section header string table. This also allows
combining the string table with another string table in the object, e.g.
the symbol name string table. The field is optional. By default,
.shstrtab will continue to be used.

This partially fixes https://bugs.llvm.org/show_bug.cgi?id=50506.

Reviewed by: Higuoxing

Differential Revision: https://reviews.llvm.org/D104035

show more ...

[obj2yaml,yaml2obj] Add NumBlocks to the BBAddrMapEntry yaml field.

As discussed in D95511, this allows us to encode invalid BBAddrMap
sections to be used in more rigorous testing.

Reviewed By: jhe

[obj2yaml,yaml2obj] Add NumBlocks to the BBAddrMapEntry yaml field.

As discussed in D95511, this allows us to encode invalid BBAddrMap
sections to be used in more rigorous testing.

Reviewed By: jhenderson

Differential Revision: https://reviews.llvm.org/D96831

show more ...

[obj2yaml,yaml2obj] Add NumBlocks to the BBAddrMapEntry yaml field.

As discussed in D95511, this allows us to encode invalid BBAddrMap
sections to be used in more rigorous testing.

Reviewed By: jhe

[obj2yaml,yaml2obj] Add NumBlocks to the BBAddrMapEntry yaml field.

As discussed in D95511, this allows us to encode invalid BBAddrMap
sections to be used in more rigorous testing.

Reviewed By: jhenderson

Differential Revision: https://reviews.llvm.org/D96831

show more ...

[yaml2obj] Handle NT_* string values in for ELF note types

This is required for D74393.

Reviewed By: jhenderson

Differential Revision: https://reviews.llvm.org/D95953

[obj2yaml, yaml2obj] Use Hex64 for BBAddressMap fields.

This patch let the yaml encoding use Hex64 values for NumBlocks, BB AddressOffset, BB Size, and BB Metadata.
Additionally, it changes the deco

[obj2yaml, yaml2obj] Use Hex64 for BBAddressMap fields.

This patch let the yaml encoding use Hex64 values for NumBlocks, BB AddressOffset, BB Size, and BB Metadata.
Additionally, it changes the decoded values in elf2yaml to uint64_t to match DataExtractor::getULEB128 return type.

Reviewed By: jhenderson

Differential Revision: https://reviews.llvm.org/D95767

show more ...

[yaml2obj, obj2yaml] - Implement section header table as a special Chunk.

This was discussed in D93678 thread.
Currently we have one special chunk - Fill.

This patch re implements the "SectionHeade

[yaml2obj, obj2yaml] - Implement section header table as a special Chunk.

This was discussed in D93678 thread.
Currently we have one special chunk - Fill.

This patch re implements the "SectionHeaderTable" key to become a special chunk too.
With that we are able to place the section header table at any location,
just like we place sections.

Differential revision: https://reviews.llvm.org/D95140

show more ...

[yaml2obj/obj2yaml] - Improve dumping/creating of ELF versioning sections.

This makes the following improvements.

For `SHT_GNU_versym`:
* yaml2obj: set `sh_link` to index of `.dynsym` section auto

[yaml2obj/obj2yaml] - Improve dumping/creating of ELF versioning sections.

This makes the following improvements.

For `SHT_GNU_versym`:
* yaml2obj: set `sh_link` to index of `.dynsym` section automatically.
For `SHT_GNU_verdef`:
* yaml2obj: set `sh_link` to index of `.dynstr` section automatically.
* yaml2obj: set `sh_info` field automatically.
* obj2yaml: don't dump the `Info` field when its value matches the number of version definitions.
For `SHT_GNU_verneed`:
* yaml2obj: set `sh_link` to index of `.dynstr` section automatically.
* yaml2obj: set `sh_info` field automatically.
* obj2yaml: don't dump the `Info` field when its value matches the number of version dependencies.

Also, simplifies few test cases.

Differential revision: https://reviews.llvm.org/D94956

show more ...

[yaml2obj/obj2yaml] - Refine handling of SHT_GNU_verdef sections.

This patch:
1) Makes `Version`, `Flags`, `VersionNdx` and `Hash` fields to be `Optional<>`.
2) Disallows dumping version definitions

[yaml2obj/obj2yaml] - Refine handling of SHT_GNU_verdef sections.

This patch:
1) Makes `Version`, `Flags`, `VersionNdx` and `Hash` fields to be `Optional<>`.
2) Disallows dumping version definitions that have `vd_version != 1`.
`vd_version` identifies the version of the structure itself.
(https://refspecs.linuxfoundation.org/LSB_5.0.0/LSB-Core-generic/LSB-Core-generic/symversion.html,
https://docs.oracle.com/cd/E19683-01/816-7777/chapter6-80869/index.html)
3) Stops dumping default values for `Version`, `Flags`, `VersionNdx` and `Hash` fields.
4) Refines testing.

Differential revision: https://reviews.llvm.org/D94659

show more ...

[llvm] Use std::any_of (NFC)

[obj2yaml,yaml2obj] - Refine how we set/dump the sh_entsize field.

This reuses the code from yaml2obj (moves it to ELFYAML.h).
With it we can set the `sh_entsize` in a single place in `obj2yaml`.

N

[obj2yaml,yaml2obj] - Refine how we set/dump the sh_entsize field.

This reuses the code from yaml2obj (moves it to ELFYAML.h).
With it we can set the `sh_entsize` in a single place in `obj2yaml`.

Note that it also fixes a bug of `yaml2obj`: we do not
set the `sh_entsize` field for the `SHT_ARM_EXIDX` section properly.

Differential revision: https://reviews.llvm.org/D93858

show more ...

[obj2yaml] - Don't crash when an object has an empty symbol table.

Currently we crash when we have an object with SHT_SYMTAB/SHT_DYNSYM sections
of size 0.

With this patch instead of the crash we s

[obj2yaml] - Don't crash when an object has an empty symbol table.

Currently we crash when we have an object with SHT_SYMTAB/SHT_DYNSYM sections
of size 0.

With this patch instead of the crash we start to dump them properly.

Differential revision: https://reviews.llvm.org/D93697

show more ...

[obj2yaml,yaml2obj] - Fix issues with creating/dumping group sections.

We have the following issues related to group sections:
1) yaml2obj is unable to set the custom `sh_entsize` value, because the

[obj2yaml,yaml2obj] - Fix issues with creating/dumping group sections.

We have the following issues related to group sections:
1) yaml2obj is unable to set the custom `sh_entsize` value, because the `EntSize`
key is currently ignored.
2) obj2yaml is unable to dump the group section which `sh_entsize != 4`.
3) obj2yaml always dumps the "EntSize" for group sections, though
usually we are trying to omit dumping default values when dumping keys.
I.e. we should not print the "EntSize" key when `sh_entsize` == 4.

This patch fixes (1),(3) and adds the test case to document the behavior of (2).

Differential revision: https://reviews.llvm.org/D93854

show more ...

[obj2yaml] - Fix the crash in getUniquedSectionName().

`getUniquedSectionName(const Elf_Shdr *Sec)` assumes that
`Sec` is not `nullptr`.

I've found one place in `getUniquedSymbolName` where it is
n

[obj2yaml] - Fix the crash in getUniquedSectionName().

`getUniquedSectionName(const Elf_Shdr *Sec)` assumes that
`Sec` is not `nullptr`.

I've found one place in `getUniquedSymbolName` where it is
not true (because of that we crash when trying to dump
unnamed null section symbols).

Patch fixes the crash and changes the signature of the
`getUniquedSectionName` section to accept a reference.

Differential revision: https://reviews.llvm.org/D93754

show more ...

[obj2yaml] - Dump the content of a broken hash table properly.

This is similar to D93760.

When something is wrong with the hash table header we dump
its context as a raw data.

Currently we have th

[obj2yaml] - Dump the content of a broken hash table properly.

This is similar to D93760.

When something is wrong with the hash table header we dump
its context as a raw data.

Currently we have the calculation overflow issue and it is possible to
bypass the validation we have (and crash).

The patch fixes it.

Differential revision: https://reviews.llvm.org/D93799

show more ...

[libObject] - Add more ELF types to LLVM_ELF_IMPORT_TYPES_ELFT define (ELFTypes.h).

This allows to get rid of lots for typedefs/usings from many places.

Differential revision: https://reviews.llvm.

[libObject] - Add more ELF types to LLVM_ELF_IMPORT_TYPES_ELFT define (ELFTypes.h).

This allows to get rid of lots for typedefs/usings from many places.

Differential revision: https://reviews.llvm.org/D93801

show more ...

[obj2yaml] - Dump the content of a broken GNU hash table properly.

When something is wrong with the GNU hash table header we dump
its context as a raw data.

Currently we have the calculation overfl

[obj2yaml] - Dump the content of a broken GNU hash table properly.

When something is wrong with the GNU hash table header we dump
its context as a raw data.

Currently we have the calculation overflow issue and it is possible to
bypass the validation we have (and crash).

The patch fixes it.

Differential revision: https://reviews.llvm.org/D93760

show more ...

[yaml2obj/obj2yaml] - Make Value/Size fields of Symbol optional.

When a field is optional we can use the `=<none>` syntax in macros.
This patch makes `Value`/`Size` fields of `Symbol` optional
and a

[yaml2obj/obj2yaml] - Make Value/Size fields of Symbol optional.

When a field is optional we can use the `=<none>` syntax in macros.
This patch makes `Value`/`Size` fields of `Symbol` optional
and adds test cases for them.

Differential revision: https://reviews.llvm.org/D93010

show more ...

[obj2yaml] - Support dumping objects that have multiple SHT_SYMTAB_SHNDX sections.

It is allowed to have multiple `SHT_SYMTAB_SHNDX` sections, though
we currently don't implement it.

The current im

[obj2yaml] - Support dumping objects that have multiple SHT_SYMTAB_SHNDX sections.

It is allowed to have multiple `SHT_SYMTAB_SHNDX` sections, though
we currently don't implement it.

The current implementation assumes that there is a maximum of one SHT_SYMTAB_SHNDX
section and that it is always linked with .symtab section.

This patch drops this limitations.

Differential revision: https://reviews.llvm.org/D92644

show more ...

[lib/Object, tools] - Make ELFObjectFile::getELFFile return reference.

We always have an object, so we don't have to return a pointer.

Differential revision: https://reviews.llvm.org/D92560