[Propeller] Encode address offsets of basic blocks relative to the end of the previous basic blocks.

This is a resurrection of D106421 with the change that it keeps backward-compatibility. This mean

[Propeller] Encode address offsets of basic blocks relative to the end of the previous basic blocks.

This is a resurrection of D106421 with the change that it keeps backward-compatibility. This means decoding the previous version of `LLVM_BB_ADDR_MAP` will work. This is required as the profile mapping tool is not released with LLVM (AutoFDO). As suggested by @jhenderson we rename the original section type value to `SHT_LLVM_BB_ADDR_MAP_V0` and assign a new value to the `SHT_LLVM_BB_ADDR_MAP` section type. The new encoding adds a version byte to each function entry to specify the encoding version for that function. This patch also adds a feature byte to be used with more flexibility in the future. An use-case example for the feature field is encoding multi-section functions more concisely using a different format.

Conceptually, the new encoding emits basic block offsets and sizes as label differences between each two consecutive basic block begin and end label. When decoding, offsets must be aggregated along with basic block sizes to calculate the final offsets of basic blocks relative to the function address.

This encoding uses smaller values compared to the existing one (offsets relative to function symbol).
Smaller values tend to occupy fewer bytes in ULEB128 encoding. As a result, we get about 17% total reduction in the size of the bb-address-map section (from about 11MB to 9MB for the clang PGO binary).
The extra two bytes (version and feature fields) incur a small 3% size overhead to the `LLVM_BB_ADDR_MAP` section size.

Reviewed By: jhenderson

Differential Revision: https://reviews.llvm.org/D121346

show more ...

[llvm] Don't use Optional::getValue (NFC)

[llvm] Use value_or instead of getValueOr (NFC)

Fixed some errors detected by PVS Studio

[LLD][LLVM] CG Graph profile using relocations

Currently when .llvm.call-graph-profile is created by llvm it explicitly encodes the symbol indices. This section is basically a black box for post pro

[LLD][LLVM] CG Graph profile using relocations

Currently when .llvm.call-graph-profile is created by llvm it explicitly encodes the symbol indices. This section is basically a black box for post processing tools. For example, if we run strip -s on the object files the symbol table changes, but indices in that section do not. In non-visible behavior indices point to wrong symbols. The visible behavior indices point outside of Symbol table: "invalid symbol index".

This patch changes the format by using R_*_NONE relocations to indicate the from/to symbols. The Frequency (Weight) will still be in the .llvm.call-graph-profile, but symbol information will be in relocation section. In LLD information from both sections is used to reconstruct call graph profile. Relocations themselves will never be applied.

With this approach post processing tools that handle relocations correctly work for this section also. Tools can add/remove symbols and as long as they handle relocation sections with this approach information stays correct.

Doing a quick experiment with clang-13.
The size went up from 107KB to 322KB, aggregate of all the input sections. Size of clang-13 binary is ~118MB. For users of -fprofile-use/-fprofile-sample-use the size of object files will go up slightly, it will not impact final binary size.

Reviewed By: jhenderson, MaskRay

Differential Revision: https://reviews.llvm.org/D104080

show more ...

[obj2yaml] Address D104035 review comments

Accidentally missed from commit 5c1639fe064b.

Differential Revision: https://reviews.llvm.org/D104035

[yaml2obj][obj2yaml] Support custom ELF section header string table name

This patch adds support for a new field in the FileHeader, which states
the name to use for the section header string table.

[yaml2obj][obj2yaml] Support custom ELF section header string table name

This patch adds support for a new field in the FileHeader, which states
the name to use for the section header string table. This also allows
combining the string table with another string table in the object, e.g.
the symbol name string table. The field is optional. By default,
.shstrtab will continue to be used.

This partially fixes https://bugs.llvm.org/show_bug.cgi?id=50506.

Reviewed by: Higuoxing

Differential Revision: https://reviews.llvm.org/D104035

show more ...

[yaml2obj] Fix bug when referencing items in SectionHeaderTable

There was an off-by-one error caused by an index (which included an
index for the null section header) being used to check against the

[yaml2obj] Fix bug when referencing items in SectionHeaderTable

There was an off-by-one error caused by an index (which included an
index for the null section header) being used to check against the size
of a list of sections (which didn't include the null section header).

This is a partial fix for https://bugs.llvm.org/show_bug.cgi?id=50506.

Reviewed by: MaskRay

Differential Revision: https://reviews.llvm.org/D104098

show more ...

Read NumBlocks as uint64_t.

This fixes the Buildbot failure happened due to a sloppy merge.

[obj2yaml,yaml2obj] Add NumBlocks to the BBAddrMapEntry yaml field.

As discussed in D95511, this allows us to encode invalid BBAddrMap
sections to be used in more rigorous testing.

Reviewed By: jhe

[obj2yaml,yaml2obj] Add NumBlocks to the BBAddrMapEntry yaml field.

As discussed in D95511, this allows us to encode invalid BBAddrMap
sections to be used in more rigorous testing.

Reviewed By: jhenderson

Differential Revision: https://reviews.llvm.org/D96831

show more ...

[obj2yaml,yaml2obj] Add NumBlocks to the BBAddrMapEntry yaml field.

As discussed in D95511, this allows us to encode invalid BBAddrMap
sections to be used in more rigorous testing.

Reviewed By: jhe

[obj2yaml,yaml2obj] Add NumBlocks to the BBAddrMapEntry yaml field.

As discussed in D95511, this allows us to encode invalid BBAddrMap
sections to be used in more rigorous testing.

Reviewed By: jhenderson

Differential Revision: https://reviews.llvm.org/D96831

show more ...

[yaml2obj] - Allow empty SectionHeaderTable definitions.

Currently we don't allow the following definition:

```
Sections:
- Type: SectionHeaderTable
- Name: .foo
Type: SHT_PROGBITS
```

We

[yaml2obj] - Allow empty SectionHeaderTable definitions.

Currently we don't allow the following definition:

```
Sections:
- Type: SectionHeaderTable
- Name: .foo
Type: SHT_PROGBITS
```

We report an error: "SectionHeaderTable can't be empty. Use 'NoHeaders' key to drop the section header table".

It was implemented in this way earlier, when `SectionHeaderTable`
was a dedicated key outside of the `Sections` list. And we did not
allow to select where the table is written.

Currently it makes sense to allow it, because a user might
want to place the default section header table at an arbitrary position,
e.g. before other sections. In this case it is not convenient and error prone
to require specifying all sections:

```
Sections:
- Type: SectionHeaderTable
Sections:
- Name: .foo
- Name: .strtab
- Name: .shstrtab
- Name: .foo
Type: SHT_PROGBITS
```

This patch allows empty SectionHeaderTable definitions.

Differential revision: https://reviews.llvm.org/D95341

show more ...

[yaml2obj][obj2yaml] - Improve how we set/dump the sh_entsize field.

We already set the `sh_entsize` field in a single place
for all non-implicit sections.

This patch reorders the logic slightly an

[yaml2obj][obj2yaml] - Improve how we set/dump the sh_entsize field.

We already set the `sh_entsize` field in a single place
for all non-implicit sections.

This patch reorders the logic slightly and with it
we finally have the only one place where the `sh_entsize` is set.

obj2yaml will not dump the `EntSize` key for `SHT_DYNSYM/SHT_SYMTAB` sections anymore,
when the value of `sh_entsize` is equal to `sizeof(Elf_Sym)`

Note that this also seems revealed an issue in llvm-objcopy:
Previously yaml2obj set the `sh_entsize` for the `.symtab` section to 0x18,
now we it sets it for `SHT_SYMTAB` sections, i.e. by type.
But the `llvm-objcopy/ELF/only-keep-debug.test` has a `.symtab` section of type `SHT_STRTAB`,
and now yaml2obj sets the `sh_entsize` to 0 for it.
I had to update the corresponding check lines for `ES`, but the behavior of
`llvm-objcopy` should be fixed instead I think.
I've added a TODO and a comment.

Differential revision: https://reviews.llvm.org/D95364

show more ...

[yaml2obj] - Refine how we set the sh_link field. NFCI.

This refactors the logic that sets the `sh_link` field.
With this patch we set it in a single place for all sections.

Differential revision:

[yaml2obj] - Refine how we set the sh_link field. NFCI.

This refactors the logic that sets the `sh_link` field.
With this patch we set it in a single place for all sections.

Differential revision: https://reviews.llvm.org/D95354

show more ...

[yaml2obj, obj2yaml] - Implement section header table as a special Chunk.

This was discussed in D93678 thread.
Currently we have one special chunk - Fill.

This patch re implements the "SectionHeade

[yaml2obj, obj2yaml] - Implement section header table as a special Chunk.

This was discussed in D93678 thread.
Currently we have one special chunk - Fill.

This patch re implements the "SectionHeaderTable" key to become a special chunk too.
With that we are able to place the section header table at any location,
just like we place sections.

Differential revision: https://reviews.llvm.org/D95140

show more ...

[yaml2obj/obj2yaml] - Improve dumping/creating of ELF versioning sections.

This makes the following improvements.

For `SHT_GNU_versym`:
* yaml2obj: set `sh_link` to index of `.dynsym` section auto

[yaml2obj/obj2yaml] - Improve dumping/creating of ELF versioning sections.

This makes the following improvements.

For `SHT_GNU_versym`:
* yaml2obj: set `sh_link` to index of `.dynsym` section automatically.
For `SHT_GNU_verdef`:
* yaml2obj: set `sh_link` to index of `.dynstr` section automatically.
* yaml2obj: set `sh_info` field automatically.
* obj2yaml: don't dump the `Info` field when its value matches the number of version definitions.
For `SHT_GNU_verneed`:
* yaml2obj: set `sh_link` to index of `.dynstr` section automatically.
* yaml2obj: set `sh_info` field automatically.
* obj2yaml: don't dump the `Info` field when its value matches the number of version dependencies.

Also, simplifies few test cases.

Differential revision: https://reviews.llvm.org/D94956

show more ...

[yaml2obj/obj2yaml] - Refine handling of SHT_GNU_verdef sections.

This patch:
1) Makes `Version`, `Flags`, `VersionNdx` and `Hash` fields to be `Optional<>`.
2) Disallows dumping version definitions

[yaml2obj/obj2yaml] - Refine handling of SHT_GNU_verdef sections.

This patch:
1) Makes `Version`, `Flags`, `VersionNdx` and `Hash` fields to be `Optional<>`.
2) Disallows dumping version definitions that have `vd_version != 1`.
`vd_version` identifies the version of the structure itself.
(https://refspecs.linuxfoundation.org/LSB_5.0.0/LSB-Core-generic/LSB-Core-generic/symversion.html,
https://docs.oracle.com/cd/E19683-01/816-7777/chapter6-80869/index.html)
3) Stops dumping default values for `Version`, `Flags`, `VersionNdx` and `Hash` fields.
4) Refines testing.

Differential revision: https://reviews.llvm.org/D94659

show more ...

[obj2yaml,yaml2obj] - Refine how we set/dump the sh_entsize field.

This reuses the code from yaml2obj (moves it to ELFYAML.h).
With it we can set the `sh_entsize` in a single place in `obj2yaml`.

N

[obj2yaml,yaml2obj] - Refine how we set/dump the sh_entsize field.

This reuses the code from yaml2obj (moves it to ELFYAML.h).
With it we can set the `sh_entsize` in a single place in `obj2yaml`.

Note that it also fixes a bug of `yaml2obj`: we do not
set the `sh_entsize` field for the `SHT_ARM_EXIDX` section properly.

Differential revision: https://reviews.llvm.org/D93858

show more ...

[obj2yaml,yaml2obj] - Fix issues with creating/dumping group sections.

We have the following issues related to group sections:
1) yaml2obj is unable to set the custom `sh_entsize` value, because the

[obj2yaml,yaml2obj] - Fix issues with creating/dumping group sections.

We have the following issues related to group sections:
1) yaml2obj is unable to set the custom `sh_entsize` value, because the `EntSize`
key is currently ignored.
2) obj2yaml is unable to dump the group section which `sh_entsize != 4`.
3) obj2yaml always dumps the "EntSize" for group sections, though
usually we are trying to omit dumping default values when dumping keys.
I.e. we should not print the "EntSize" key when `sh_entsize` == 4.

This patch fixes (1),(3) and adds the test case to document the behavior of (2).

Differential revision: https://reviews.llvm.org/D93854

show more ...

[libObject] - Add more ELF types to LLVM_ELF_IMPORT_TYPES_ELFT define (ELFTypes.h).

This allows to get rid of lots for typedefs/usings from many places.

Differential revision: https://reviews.llvm.

[libObject] - Add more ELF types to LLVM_ELF_IMPORT_TYPES_ELFT define (ELFTypes.h).

This allows to get rid of lots for typedefs/usings from many places.

Differential revision: https://reviews.llvm.org/D93801

show more ...

[yaml2obj/obj2yaml] - Make Value/Size fields of Symbol optional.

When a field is optional we can use the `=<none>` syntax in macros.
This patch makes `Value`/`Size` fields of `Symbol` optional
and a

[yaml2obj/obj2yaml] - Make Value/Size fields of Symbol optional.

When a field is optional we can use the `=<none>` syntax in macros.
This patch makes `Value`/`Size` fields of `Symbol` optional
and adds test cases for them.

Differential revision: https://reviews.llvm.org/D93010

show more ...

[yaml2obj,obj2yaml] - Make Symbol::Section field optional.

This is similar to what we did earlier for fields of the Section class.

When a field is optional we can use the =<none> syntax in macros.

[yaml2obj,obj2yaml] - Make Symbol::Section field optional.

This is similar to what we did earlier for fields of the Section class.

When a field is optional we can use the =<none> syntax in macros.

This was splitted from D92478.

Differential revision: https://reviews.llvm.org/D92565

show more ...

[obj2yaml] - Dump section offsets in some cases.

Currently we never dump the `sh_offset` key.
Though it sometimes an important information.

To reduce the noise this patch implements the following l

[obj2yaml] - Dump section offsets in some cases.

Currently we never dump the `sh_offset` key.
Though it sometimes an important information.

To reduce the noise this patch implements the following logic:
1) The "Offset" key for the first section is always emitted.
2) If we can derive the offset for a next section naturally,
then the "Offset" key is omitted.

By "naturally" I mean that section[X] offset is expected to be:
```
offsetOf(section[X]) == alignTo(section[X - 1].sh_offset + section[X - 1].sh_size, section[X].sh_addralign)
```

So, when it has the expected value, we omit it from the output.

Differential revision: https://reviews.llvm.org/D91152

show more ...

[yaml2obj] - ProgramHeaders: introduce FirstSec/LastSec instead of Sections list.

Imagine we have a YAML declaration of few sections: `foo1`, `<unnamed 2>`, `foo3`, `foo4`.

To put them into segment

[yaml2obj] - ProgramHeaders: introduce FirstSec/LastSec instead of Sections list.

Imagine we have a YAML declaration of few sections: `foo1`, `<unnamed 2>`, `foo3`, `foo4`.

To put them into segment we can do (1*):

```
Sections:
- Section: foo1
- Section: foo4
```

or we can use (2*):

```
Sections:
- Section: foo1
- Section: foo3
- Section: foo4
```

or (3*) :

```
Sections:
- Section: foo1
## "(index 2)" here is a name that we automatically created for a unnamed section.
- Section: (index 2)
- Section: foo3
- Section: foo4
```

It looks really confusing that we don't have to list all of sections.

At first I've tried to make this rule stricter and report an error when there is a gap
(i.e. when a section is included into segment, but not listed explicitly).
This did not work perfect, because such approach conflicts with unnamed sections/fills (see (3*)).

This patch drops "Sections" key and introduces 2 keys instead: `FirstSec` and `LastSec`.
Both are optional.

Differential revision: https://reviews.llvm.org/D90458

show more ...

[obj2yaml] [yaml2obj] Add yaml support for SHT_LLVM_BB_ADDR_MAP section.

YAML support allows us to better test the feature in the subsequent patches. The implementation is quite similar to the .stac

[obj2yaml] [yaml2obj] Add yaml support for SHT_LLVM_BB_ADDR_MAP section.

YAML support allows us to better test the feature in the subsequent patches. The implementation is quite similar to the .stack_sizes section.

Reviewed By: jhenderson, grimar

Differential Revision: https://reviews.llvm.org/D88717

show more ...