[Object] Fix updating darwin archives

When creating an archive, llvm-ar looks at the host to determine the
archive format to use, on Apple platforms this means it uses the
K_DARWIN format. K_DARWIN

[Object] Fix updating darwin archives

When creating an archive, llvm-ar looks at the host to determine the
archive format to use, on Apple platforms this means it uses the
K_DARWIN format. K_DARWIN is _virtually_ equivalent to K_BSD, expect for
some very slight differences around padding, timestamps in deterministic
mode, and 64 bit formats. When updating an archive using llvm-ar, or
llvm-objcopy, Archive would try to determine the kind, but it was not
possible to get K_DARWIN in the initialization of the archive, because
they're virtually inciting usable from K_BSD, especially since the
slight differences only apply in very specific cases. This leads to
linker failures when the alignment workaround is not applied to an
archive copied with llvm-objcopy. This change teaches Archive to infer
the K_DARWIN type in the cases where it's possible and the first object
in the archive is a macho object. This avoids using the host triple to
determine this to not affect cross compiling.

Ideally we would eliminate the separate K_DARWIN type entirely since
it's not a truly separate archive type, but then we'd have to force the
macho workarounds on the BSD format generally. This might be acceptable
but then it would be unclear how to handle this case without forcing the
K_DARWIN64 format on all BSD users:

```
if (LastOffset >= Sym64Threshold) {
if (Kind == object::Archive::K_DARWIN)
Kind = object::Archive::K_DARWIN64;
else
Kind = object::Archive::K_GNU64;
}
```

The logic used to determine if the object is macho is derived from the
logic llvm-ar uses.

Previous context:

- 111cd669e90e5b2132187d36f8b141b11a671a8b
- 23a76be5adcaa768ba538f8a4514a7afccf61988

Differential Revision: https://reviews.llvm.org/D124895

show more ...

[AIX] support write operation of big archive.

SUMMARY

1. Enable supporting the write operation of big archive.
2. the first commit come from https://reviews.llvm.org/D104367
3. refactor the first c

[AIX] support write operation of big archive.

SUMMARY

1. Enable supporting the write operation of big archive.
2. the first commit come from https://reviews.llvm.org/D104367
3. refactor the first commit and implement writing symbol table.
4. fixed the bugs and add more test cases in the second commit.

Reviewers: James Henderson
Differential Revision: https://reviews.llvm.org/D123949

show more ...

Cleanup LLVMObject headers

Most notably,

llvm/Object/Binary.h no longer includes llvm/Support/MemoryBuffer.h
llvm/Object/MachOUniversal*.h no longer include llvm/Object/Archive.h
llvm/Object/TapiUn

Cleanup LLVMObject headers

Most notably,

llvm/Object/Binary.h no longer includes llvm/Support/MemoryBuffer.h
llvm/Object/MachOUniversal*.h no longer include llvm/Object/Archive.h
llvm/Object/TapiUniversal.h no longer includes llvm/Object/TapiFile.h

llvm-project preprocessed size:
before: 1068185081
after: 1068324320

Discourse thread: https://discourse.llvm.org/t/include-what-you-use-include-cleanup
Differential Revision: https://reviews.llvm.org/D119457

show more ...

[AIX] Support of Big archive (read)

Summary:

The patch is based on the EGuesnet's implement of the "Support of Big archive (read)
the first commit of the patch is come from https://reviews.llvm.org

[AIX] Support of Big archive (read)

Summary:

The patch is based on the EGuesnet's implement of the "Support of Big archive (read)
the first commit of the patch is come from https://reviews.llvm.org/D100651.

the rest of commits of the patch

1 Addressed the comments on the https://reviews.llvm.org/D100651
2 according to https://www.ibm.com/docs/en/aix/7.2?topic=formats-ar-file-format-big
using the "fl_fstmoff" for the first object file number, using "char ar_nxtmem[20]" to get next object file ,
using the "char fl_lstmoff[20]" for the last of the object file will fix the following problems:
2.1 can not correct reading a archive files which has padding data between too object file
2.2 can not correct reading a archive files from which some object file has be deleted

3 introduce a new derived class BigArchive for big ar file.

Reviewers: James Henderson
Differential Revision: https://reviews.llvm.org/D111889

show more ...

Revert "[AIX] Support of Big archive (read)"

This appears to be causing the following build failures on green
dragon during stage2 builds on macOS:

/System/Volumes/Data/jenkins/workspace/apple-cla

Revert "[AIX] Support of Big archive (read)"

This appears to be causing the following build failures on green
dragon during stage2 builds on macOS:

/System/Volumes/Data/jenkins/workspace/apple-clang-stage2-configure-RA_osceola/clang.roots/BuildRecords/clang-9999.99_install/Objects/obj-llvm/./bin/clang++ -fno-stack-protector -fno-common -Wno-profile-instr-unprofiled -Wno-unknown-warning-option -Werror=unguarded-availability-new -fPIC -fvisibility-inlines-hidden -Werror=date-time -Werror=unguarded-availability-new -fmodules -fmodules-cache-path=/System/Volumes/Data/jenkins/workspace/apple-clang-stage2-configure-RA_osceola/clang.roots/BuildRecords/clang-9999.99_install/Objects/obj-llvm/tools/clang/stage2-bins/module.cache -fcxx-modules -Xclang -fmodules-local-submodule-visibility -gmodules -Wall -Wextra -Wno-unused-parameter -Wwrite-strings -Wcast-qual -Wmissing-field-initializers -pedantic -Wno-long-long -Wc++98-compat-extra-semi -Wimplicit-fallthrough -Wcovered-switch-default -Wno-class-memaccess -Wno-noexcept-type -Wnon-virtual-dtor -Wdelete-non-virtual-dtor -Wsuggest-override -Wstring-conversion -Wmisleading-indentation -fdiagnostics-color -O2 -gline-tables-only -DNDEBUG -arch x86_64 -arch arm64 -arch arm64e -isysroot /Volumes/Xcode/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX12.0.sdk -mmacosx-version-min=10.10 -Wl,-search_paths_first -Wl,-headerpad_max_install_names -Wl,-dead_strip tools/llvm-cov/CMakeFiles/llvm-cov.dir/llvm-cov.cpp.o tools/llvm-cov/CMakeFiles/llvm-cov.dir/gcov.cpp.o tools/llvm-cov/CMakeFiles/llvm-cov.dir/CodeCoverage.cpp.o tools/llvm-cov/CMakeFiles/llvm-cov.dir/CoverageExporterJson.cpp.o tools/llvm-cov/CMakeFiles/llvm-cov.dir/CoverageExporterLcov.cpp.o tools/llvm-cov/CMakeFiles/llvm-cov.dir/CoverageFilters.cpp.o tools/llvm-cov/CMakeFiles/llvm-cov.dir/CoverageReport.cpp.o tools/llvm-cov/CMakeFiles/llvm-cov.dir/CoverageSummaryInfo.cpp.o tools/llvm-cov/CMakeFiles/llvm-cov.dir/SourceCoverageView.cpp.o tools/llvm-cov/CMakeFiles/llvm-cov.dir/SourceCoverageViewHTML.cpp.o tools/llvm-cov/CMakeFiles/llvm-cov.dir/SourceCoverageViewText.cpp.o tools/llvm-cov/CMakeFiles/llvm-cov.dir/TestingSupport.cpp.o -o bin/llvm-cov -Wl,-rpath,@loader_path/../lib lib/libLLVMCore.a lib/libLLVMSupport.a lib/libLLVMObject.a lib/libLLVMCoverage.a lib/libLLVMProfileData.a lib/libLLVMDebugInfoDWARF.a lib/libLLVMObject.a lib/libLLVMBitReader.a lib/libLLVMCore.a lib/libLLVMRemarks.a lib/libLLVMBitstreamReader.a lib/libLLVMMCParser.a lib/libLLVMTextAPI.a lib/libLLVMMC.a lib/libLLVMBinaryFormat.a lib/libLLVMDebugInfoCodeView.a lib/libLLVMSupport.a -lm /Volumes/Xcode/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX12.0.sdk/usr/lib/libz.tbd /Volumes/Xcode/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX12.0.sdk/usr/lib/libcurses.tbd lib/libLLVMDemangle.a && cd /System/Volumes/Data/jenkins/workspace/apple-clang-stage2-configure-RA_osceola/clang.roots/BuildRecords/clang-9999.99_install/Objects/obj-llvm/tools/clang/stage2-bins/tools/llvm-cov && xcrun dsymutil -o=llvm-cov.dSYM /System/Volumes/Data/jenkins/workspace/apple-clang-stage2-configure-RA_osceola/clang.roots/BuildRecords/clang-9999.99_install/Objects/obj-llvm/tools/clang/stage2-bins/bin/llvm-cov
Undefined symbols for architecture x86_64:
"llvm::object::CommonArchiveMemberHeader<llvm::object::BigArMemHdrType>::getRawAccessMode() const", referenced from:
vtable for llvm::object::BigArchiveMemberHeader in libLLVMObject.a(Archive.cpp.o)
"llvm::object::CommonArchiveMemberHeader<llvm::object::BigArMemHdrType>::getRawUID() const", referenced from:
vtable for llvm::object::BigArchiveMemberHeader in libLLVMObject.a(Archive.cpp.o)
"llvm::object::CommonArchiveMemberHeader<llvm::object::BigArMemHdrType>::getRawGID() const", referenced from:
vtable for llvm::object::BigArchiveMemberHeader in libLLVMObject.a(Archive.cpp.o)
"llvm::object::CommonArchiveMemberHeader<llvm::object::UnixArMemHdrType>::getRawAccessMode() const", referenced from:
vtable for llvm::object::ArchiveMemberHeader in libLLVMObject.a(Archive.cpp.o)
"llvm::object::CommonArchiveMemberHeader<llvm::object::UnixArMemHdrType>::getRawLastModified() const", referenced from:
vtable for llvm::object::ArchiveMemberHeader in libLLVMObject.a(Archive.cpp.o)
"llvm::object::CommonArchiveMemberHeader<llvm::object::BigArMemHdrType>::getRawLastModified() const", referenced from:
vtable for llvm::object::BigArchiveMemberHeader in libLLVMObject.a(Archive.cpp.o)
"llvm::object::CommonArchiveMemberHeader<llvm::object::BigArMemHdrType>::getOffset() const", referenced from:
vtable for llvm::object::BigArchiveMemberHeader in libLLVMObject.a(Archive.cpp.o)
"llvm::object::CommonArchiveMemberHeader<llvm::object::UnixArMemHdrType>::getRawUID() const", referenced from:
vtable for llvm::object::ArchiveMemberHeader in libLLVMObject.a(Archive.cpp.o)
"llvm::object::CommonArchiveMemberHeader<llvm::object::UnixArMemHdrType>::getRawGID() const", referenced from:
vtable for llvm::object::ArchiveMemberHeader in libLLVMObject.a(Archive.cpp.o)
"llvm::object::CommonArchiveMemberHeader<llvm::object::UnixArMemHdrType>::getOffset() const", referenced from:
vtable for llvm::object::ArchiveMemberHeader in libLLVMObject.a(Archive.cpp.o)
ld: symbol(s) not found for architecture x86_64

https://smooshbase.apple.com/ci/job/apple-clang-stage2-configure-RA_osceola/30276/console

show more ...

[AIX] Support of Big archive (read)

Summary:

The patch is based on the EGuesnet's implement of the "Support of Big archive (read)
the first commit of the patch is come from https://reviews.llvm.org

[AIX] Support of Big archive (read)

Summary:

The patch is based on the EGuesnet's implement of the "Support of Big archive (read)
the first commit of the patch is come from https://reviews.llvm.org/D100651.

the rest of commits of the patch

1 Addressed the comments on the https://reviews.llvm.org/D100651
2 according to https://www.ibm.com/docs/en/aix/7.2?topic=formats-ar-file-format-big
using the "fl_fstmoff" for the first object file number, using "char ar_nxtmem[20]" to get next object file ,
using the "char fl_lstmoff[20]" for the last of the object file will fix the following problems:
2.1 can not correct reading a archive files which has padding data between too object file
2.2 can not correct reading a archive files from which some object file has be deleted

3 introduce a new derived class BigArchive for big ar file.

Reviewers: James Henderson
Differential Revision: https://reviews.llvm.org/D111889

show more ...

Revert "[AIX] Support of Big archive (read)"

This reverts commit 3130134d6e4823b5ee7619288a4b7e1e60831a82.

[AIX] Support of Big archive (read)

Summary:

The patch is based on the EGuesnet's implement of the "Support of Big archive (read)
the first commit of the patch is come from https://reviews.llvm.org

[AIX] Support of Big archive (read)

Summary:

The patch is based on the EGuesnet's implement of the "Support of Big archive (read)
the first commit of the patch is come from https://reviews.llvm.org/D100651.

the rest of commits of the patch

1 Addressed the comments on the https://reviews.llvm.org/D100651
2 according to https://www.ibm.com/docs/en/aix/7.2?topic=formats-ar-file-format-big
using the "fl_fstmoff" for the first object file number, using "char ar_nxtmem[20]" to get next object file ,
using the "char fl_lstmoff[20]" for the last of the object file will fix the following problems:
2.1 can not correct reading a archive files which has padding data between too object file
2.2 can not correct reading a archive files from which some object file has be deleted

3 introduce a new derived class BigArchive for big ar file.

Reviewers: James Henderson
Differential Revision: https://reviews.llvm.org/D111889

show more ...

[llvm] Add null-termination capability to SmallVectorMemoryBuffer

Most of `MemoryBuffer` interfaces expose a `RequiresNullTerminator` parameter that's being used to:
* determine how to open a file (

[llvm] Add null-termination capability to SmallVectorMemoryBuffer

Most of `MemoryBuffer` interfaces expose a `RequiresNullTerminator` parameter that's being used to:
* determine how to open a file (`mmap` vs `open`),
* assert newly initialized buffer indeed has an implicit null terminator.

This patch adds the paramater to the `SmallVectorMemoryBuffer` constructors, meaning:
* null terminator can now be added to `SmallVector`s that didn't have one before,
* `SmallVectors` that had a null terminator before keep it even after the move.

In line with existing code, the new parameter is defaulted to `true`. This patch makes sure all calls to the `SmallVectorMemoryBuffer` constructor set it to `false` to preserve the current semantics.

Reviewed By: dexonsmith

Differential Revision: https://reviews.llvm.org/D115331

show more ...

[llvm-ar][Object] Fix detection of need for 64-bit archive symbol tables

The code to detect the requirement for 64-bit offsets in the archive
symbol table was not correctly accounting for the archiv

[llvm-ar][Object] Fix detection of need for 64-bit archive symbol tables

The code to detect the requirement for 64-bit offsets in the archive
symbol table was not correctly accounting for the archive file signature
and the size of all the contents of the symbol table itself, e.g. the
symbol table's header and string table. Also was not considering the
variation in symbol table formats. This could result in the creation of
large archives with a corrupt symbol table.

Change the testing environment variable SYM64_THRESHOLD to be an
absolute value rather than a power of 2 in order to enable precise
testing of this detection code.

Differential Revision: https://reviews.llvm.org/D89891

show more ...

[Archive] Don't throw away errors for malformed archive members

When adding an archive member with a problem, e.g. a new bitcode with an
old archiver, containing an unsupported attribute, or an ELF

[Archive] Don't throw away errors for malformed archive members

When adding an archive member with a problem, e.g. a new bitcode with an
old archiver, containing an unsupported attribute, or an ELF file with a
malformed symbol table, the archiver would throw away the error and
simply add the member to the archive without any symbol entries. This
meant that the resultant archive could be silently unusable when not
using --whole-archive, and result in unexpected undefined symbols.

This change fixes this issue by addressing two FIXMEs and only throwing
away not-an-object errors. However, this meant that some LLD tests which
didn't need symbol tables and were using invalid members deliberately to
test the linker's malformed input handling no longer worked, so this
patch also stops the archiver from looking for symbols in an object if
it doesn't require a symbol table, and updates the tests accordingly.

Differential Revision: https://reviews.llvm.org/D88288

Reviewed by: grimar, rupprecht, MaskRay

show more ...

[llvm-libtool-darwin] Refactor ArchiveWriter

Refactoring function `writeArchive` in ArchiveWriter. Added a new
function `writeArchiveBuffer` that returns the archive in a memory
buffer instead of wr

[llvm-libtool-darwin] Refactor ArchiveWriter

Refactoring function `writeArchive` in ArchiveWriter. Added a new
function `writeArchiveBuffer` that returns the archive in a memory
buffer instead of writing it out to the disk. This refactor is necessary
so as to allow `llvm-libtool-darwin` to write universal files containing
archives.

Reviewed by jhenderson, MaskRay, smeenai

Differential Revision: https://reviews.llvm.org/D84858

show more ...

[Object] Change uint32_t getSymbolFlags() to Expected<uint32_t> getSymbolFlags().

This change enables getSymbolFlags() to return errors which benefit error reporting in clients.

Differential Revisi

[Object] Change uint32_t getSymbolFlags() to Expected<uint32_t> getSymbolFlags().

This change enables getSymbolFlags() to return errors which benefit error reporting in clients.

Differential Revision: https://reviews.llvm.org/D77860

show more ...

Drop a StringMap.h include, NFC

$ diff -u <(sort thedeps-before.txt) <(sort thedeps-after.txt) \
| grep '^[-+] ' | sort | uniq -c | sort -nr
231 - llvm/include/llvm/ADT/StringMap.h
17

Drop a StringMap.h include, NFC

$ diff -u <(sort thedeps-before.txt) <(sort thedeps-after.txt) \
| grep '^[-+] ' | sort | uniq -c | sort -nr
231 - llvm/include/llvm/ADT/StringMap.h
171 - llvm/include/llvm/Support/AllocatorBase.h
142 - llvm/include/llvm/Support/PointerLikeTypeTraits.h

show more ...

Revert "Remove redundant "std::move"s in return statements"

The build failed with

error: call to deleted constructor of 'llvm::Error'

errors.

This reverts commit 1c2241a7936bf85aa68aef94bd40c3b

Revert "Remove redundant "std::move"s in return statements"

The build failed with

error: call to deleted constructor of 'llvm::Error'

errors.

This reverts commit 1c2241a7936bf85aa68aef94bd40c3ba77d8ddf2.

show more ...

Remove redundant "std::move"s in return statements

Make llvm::StringRef to std::string conversions explicit.

This is how it should've been and brings it more in line with
std::string_view. There should be no functional change here.

This is mostly m

Make llvm::StringRef to std::string conversions explicit.

This is how it should've been and brings it more in line with
std::string_view. There should be no functional change here.

This is mostly mechanical from a custom clang-tidy check, with a lot of
manual fixups. It uncovers a lot of minor inefficiencies.

This doesn't actually modify StringRef yet, I'll do that in a follow-up.

show more ...

[Alignment][NFC] Remove unneeded llvm:: scoping on Align types

llvm-svn: 373081

[Alignment] Move OffsetToAlignment to Alignment.h

Summary:
This is patch is part of a series to introduce an Alignment type.
See this thread for context: http://lists.llvm.org/pipermail/llvm-dev/201

[Alignment] Move OffsetToAlignment to Alignment.h

Summary:
This is patch is part of a series to introduce an Alignment type.
See this thread for context: http://lists.llvm.org/pipermail/llvm-dev/2019-July/133851.html
See this patch for the introduction of the type: https://reviews.llvm.org/D64790

Reviewers: courbet, JDevlieghere, alexshap, rupprecht, jhenderson

Subscribers: sdardis, nemanjai, hiraditya, kbarton, jakehehrlich, jrtc27, MaskRay, atanasyan, jsji, seiya, cfe-commits, llvm-commits

Tags: #clang, #llvm

Differential Revision: https://reviews.llvm.org/D67499

llvm-svn: 371742

show more ...

[llvm-ar] Fix support for archives with members larger than 4GB

llvm-ar outputs a strange error message when handling archives with
members larger than 4GB due to not checking file size when passing

[llvm-ar] Fix support for archives with members larger than 4GB

llvm-ar outputs a strange error message when handling archives with
members larger than 4GB due to not checking file size when passing the
value as an unsigned 32 bit integer. This overflow issue caused
malformed archives to be created.:

https://bugs.llvm.org/show_bug.cgi?id=38058

This change allows for members above 4GB and will error in a case that
is over the formats size limit, a 10 digit decimal integer.

Differential Revision: https://reviews.llvm.org/D65093

llvm-svn: 366813

show more ...

[Support] Move llvm::MemoryBuffer to sys::fs::file_t

Summary:
On Windows, Posix integer file descriptors are a compatibility layer
over native file handles provided by the C runtime. There is a hard

[Support] Move llvm::MemoryBuffer to sys::fs::file_t

Summary:
On Windows, Posix integer file descriptors are a compatibility layer
over native file handles provided by the C runtime. There is a hard
limit on the maximum number of file descriptors that a process can open,
and the limit is 8192. LLD typically doesn't run into this limit because
it opens input files, maps them into memory, and then immediately closes
the file descriptor. This prevents it from running out of FDs.

For various reasons, I'd like to open handles to every input file and
keep them open during linking. That requires migrating MemoryBuffer over
to taking open native file handles instead of integer FDs.

Reviewers: aganea, Bigcheese

Reviewed By: aganea

Subscribers: smeenai, silvas, mehdi_amini, hiraditya, steven_wu, dexonsmith, dang, llvm-commits, zturner

Tags: #llvm

Differential Revision: https://reviews.llvm.org/D63453

llvm-svn: 365588

show more ...

[llvm-ar] Reapply Fix relative thin archive path handling

Includes a fix for an introduced build failure due to a post c++11 use of std::mismatch.

This fixes some thin archive relative path issues

[llvm-ar] Reapply Fix relative thin archive path handling

Includes a fix for an introduced build failure due to a post c++11 use of std::mismatch.

This fixes some thin archive relative path issues, paths are shortened where possible and paths are output correctly when using the display table command.

Differential Revision: https://reviews.llvm.org/D59491

llvm-svn: 362484

show more ...

Revert "[llvm-ar] Fix relative thin archive path handling"

This reverts commit r362407. It broke compilation of
llvm/lib/Object/ArchiveWriter.cpp:

error: type 'llvm::sys::path::const_iterator' doe

Revert "[llvm-ar] Fix relative thin archive path handling"

This reverts commit r362407. It broke compilation of
llvm/lib/Object/ArchiveWriter.cpp:

error: type 'llvm::sys::path::const_iterator' does not provide a call
operator

llvm-svn: 362413

show more ...

[llvm-ar] Fix relative thin archive path handling

This fixes some thin archive relative path issues, paths are shortened where possible and paths are output correctly when using the display table co

[llvm-ar] Fix relative thin archive path handling

This fixes some thin archive relative path issues, paths are shortened where possible and paths are output correctly when using the display table command.

Differential Revision: https://reviews.llvm.org/D59491

llvm-svn: 362407

show more ...

[Object] Change SymbolicFile::printSymbolName to use Error

llvm-svn: 360414