[lld-macho] Add support for -w

This flag suppresses warnings produced by the linker. In ld64 this has
an interesting interaction with -fatal_warnings, it silences the
warnings but the link still fai

[lld-macho] Add support for -w

This flag suppresses warnings produced by the linker. In ld64 this has
an interesting interaction with -fatal_warnings, it silences the
warnings but the link still fails. Instead of doing that here we still
print the warning and eagerly fail the link in case both are passed,
this seems more reasonable so users can understand why the link fails.

Differential Revision: https://reviews.llvm.org/D127564

show more ...

[lld] Make error handling functions opaque

The inline `lld::error` expands to two function calls `errorHandler` and `error`
where the latter is opaque. Move the functions to .cpp files to decrease c

[lld] Make error handling functions opaque

The inline `lld::error` expands to two function calls `errorHandler` and `error`
where the latter is opaque. Move the functions to .cpp files to decrease code
size.

My x86-64 lld executable is 9KiB smaller.

Reviewed By: #lld-macho, thakis

Differential Revision: https://reviews.llvm.org/D120002

show more ...

[lld] Add module name to LTO inline asm diagnostic

Close #52781: for LTO, the inline asm diagnostic uses `<inline asm>` as the file
name (lib/CodeGen/AsmPrinter/AsmPrinterInlineAsm.cpp) and it is un

[lld] Add module name to LTO inline asm diagnostic

Close #52781: for LTO, the inline asm diagnostic uses `<inline asm>` as the file
name (lib/CodeGen/AsmPrinter/AsmPrinterInlineAsm.cpp) and it is unclear which
module has the issue.

With this patch, we will see the module name (say `asm.o`) before `<inline asm>` with ThinLTO.

```
% clang -flto=thin -c asm.c && myld.lld asm.o -e f
ld.lld: error: asm.o <inline asm>:1:2: invalid instruction mnemonic 'invalid'
invalid
^~~~~~~
```

For regular LTO, unfortunately the original module name is lost and we only get
ld-temp.o.

Reviewed By: #lld-macho, ychen, Jez Ng

Differential Revision: https://reviews.llvm.org/D118434

show more ...

Re-land [LLD] Remove global state in lldCommon

Move all variables at file-scope or function-static-scope into a hosting structure (lld::CommonLinkerContext) that lives at lldMain()-scope. Drivers wi

Re-land [LLD] Remove global state in lldCommon

Move all variables at file-scope or function-static-scope into a hosting structure (lld::CommonLinkerContext) that lives at lldMain()-scope. Drivers will inherit from this structure and add their own global state, in the same way as for the existing COFFLinkerContext.

See discussion in https://lists.llvm.org/pipermail/llvm-dev/2021-June/151184.html

The previous land f860fe362282ed69b9d4503a20e5d20b9a041189 caused issues in https://lab.llvm.org/buildbot/#/builders/123/builds/8383, fixed by 22ee510dac9440a74b2e5b3fe3ff13ccdbf55af3.

Differential Revision: https://reviews.llvm.org/D108850

show more ...

Revert [LLD] Remove global state in lldCommon

It seems to be causing issues on https://lab.llvm.org/buildbot/#/builders/123/builds/8383

[LLD] Remove global state in lldCommon

Move all variables at file-scope or function-static-scope into a hosting structure (lld::CommonLinkerContext) that lives at lldMain()-scope. Drivers will inher

[LLD] Remove global state in lldCommon

Move all variables at file-scope or function-static-scope into a hosting structure (lld::CommonLinkerContext) that lives at lldMain()-scope. Drivers will inherit from this structure and add their own global state, in the same way as for the existing COFFLinkerContext.

See discussion in https://lists.llvm.org/pipermail/llvm-dev/2021-June/151184.html

Differential Revision: https://reviews.llvm.org/D108850

show more ...

[lld/mac] Write -v output to stderr

This matches ld64, and it's conceivable that projects try to read
this information off stderr for that reason.

--version keeps writing to stdout.

Differential R

[lld/mac] Write -v output to stderr

This matches ld64, and it's conceivable that projects try to read
this information off stderr for that reason.

--version keeps writing to stdout.

Differential Revision: https://reviews.llvm.org/D113020

show more ...

[lld] Buffer writes when composing a single diagnostic

llvm::errs() is unbuffered. On a POSIX platform, composing a diagnostic
string may invoke the ::write syscall multiple times, which can be slow

[lld] Buffer writes when composing a single diagnostic

llvm::errs() is unbuffered. On a POSIX platform, composing a diagnostic
string may invoke the ::write syscall multiple times, which can be slow.
Buffer writes to a temporary SmallString when composing a single diagnostic to
reduce the number of ::write syscalls to one (also easier to read under
strace/truss).

For an invocation of ld.lld with 62000+ lines of
`ld.lld: warning: symbol ordering file: no such symbol: ` warnings (D87121),
the buffering decreases the write time from 1s to 0.4s (for /dev/tty) and
from 0.4s to 0.1s (for a tmpfs file). This can speed up
`relocation R_X86_64_PC32 out of range` diagnostic printing as well
with `--noinhibit-exec --no-fatal-warnings`.

Reviewed By: jhenderson

Differential Revision: https://reviews.llvm.org/D87272

show more ...

[LLD][COFF] When using LLD-as-a-library, always prevent re-entrance on failures

This is a follow-up for D70378 (Cover usage of LLD as a library).

While debugging an intermittent failure on a bot, I

[LLD][COFF] When using LLD-as-a-library, always prevent re-entrance on failures

This is a follow-up for D70378 (Cover usage of LLD as a library).

While debugging an intermittent failure on a bot, I recalled this scenario which
causes the issue:

1.When executing lld/test/ELF/invalid/symtab-sh-info.s L45, we reach
lld::elf::Obj-File::ObjFile() which goes straight into its base ELFFileBase(),
then ELFFileBase::init().
2.At that point fatal() is thrown in lld/ELF/InputFiles.cpp L381, leaving a
half-initialized ObjFile instance.
3.We then end up in lld::exitLld() and since we are running with LLD_IN_TEST, we
hapily restore the control flow to CrashRecoveryContext::RunSafely() then back
in lld::safeLldMain().
4.Before this patch, we called errorHandler().reset() just after, and this
attempted to reset the associated SpecificAlloc<ObjFile<ELF64LE>>. That tried
to free the half-initialized ObjFile instance, and more precisely its
ObjFile::dwarf member.

Sometimes that worked, sometimes it failed and was catched by the
CrashRecoveryContext. This scenario was the reason we called
errorHandler().reset() through a CrashRecoveryContext.

But in some rare cases, the above repro somehow corrupted the heap, creating a
stack overflow. When the CrashRecoveryContext's filter (that is,
__except (ExceptionFilter(GetExceptionInformation()))) tried to handle the
exception, it crashed again since the stack was exhausted -- and that took the
whole application down. That is the issue seen on the bot. Locally it happens
about 1 times out of 15.

Now this situation can happen anywhere in LLD. Since catching stack overflows is
not a reliable scenario ATM when using CrashRecoveryContext, we're now
preventing further re-entrance when such failures occur, by signaling
lld::SafeReturn::canRunAgain=false. When running with LLD_IN_TEST=2 (or above),
only one iteration will be executed, instead of two.

Differential Revision: https://reviews.llvm.org/D88348

show more ...

[lld] Provide a hook to customize undefined symbols error handling

This is a follow up to https://reviews.llvm.org/D87758, implementing the missing
symbol part, as done by binutils.

Differential Re

[lld] Provide a hook to customize undefined symbols error handling

This is a follow up to https://reviews.llvm.org/D87758, implementing the missing
symbol part, as done by binutils.

Differential Revision: https://reviews.llvm.org/D89687

show more ...

Fix 'default label in switch which covers all enumeration values' warning

Provide a hook to customize missing library error handling

Make it possible for lld users to provide a custom script that would help to
find missing libraries. A possible scenario could be:

% c

Provide a hook to customize missing library error handling

Make it possible for lld users to provide a custom script that would help to
find missing libraries. A possible scenario could be:

% clang /tmp/a.c -fuse-ld=lld -loauth -Wl,--error-handling-script=/tmp/addLibrary.py
unable to find library -loauth
looking for relevant packages to provides that library

liboauth-0.9.7-4.el7.i686
liboauth-devel-0.9.7-4.el7.i686
liboauth-0.9.7-4.el7.x86_64
liboauth-devel-0.9.7-4.el7.x86_64
pix-1.6.1-3.el7.x86_64

Where addLibrary would be called with the missing library name as first argument
(in that case addLibrary.py oauth)

Differential Revision: https://reviews.llvm.org/D87758

show more ...

[LLD][COFF] Cover usage of LLD-as-a-library in tests

In lit tests, we run each LLD invocation twice (LLD_IN_TEST=2), without shutting down the process in-between. This ensures a full cleanup is prop

[LLD][COFF] Cover usage of LLD-as-a-library in tests

In lit tests, we run each LLD invocation twice (LLD_IN_TEST=2), without shutting down the process in-between. This ensures a full cleanup is properly done between runs.
Only active for the COFF driver for now. Other drivers still use LLD_IN_TEST=1 which executes just one iteration with full cleanup, like before.
When the environment variable LLD_IN_TEST is unset, a shortcut is taken, only one iteration is executed, no cleanup for faster exit, like before.
A public API, lld::safeLldMain(), is also available when using LLD as a library.

Differential Revision: https://reviews.llvm.org/D70378

show more ...

[Support] Move LLD's parallel algorithm wrappers to support

Essentially takes the lld/Common/Threads.h wrappers and moves them to
the llvm/Support/Paralle.h algorithm header.

The changes are:
- Rem

[Support] Move LLD's parallel algorithm wrappers to support

Essentially takes the lld/Common/Threads.h wrappers and moves them to
the llvm/Support/Paralle.h algorithm header.

The changes are:
- Remove policy parameter, since all clients use `par`.
- Rename the methods to `parallelSort` etc to match LLVM style, since
they are no longer C++17 pstl compatible.
- Move algorithms from llvm::parallel:: to llvm::, since they have
"parallel" in the name and are no longer overloads of the regular
algorithms.
- Add range overloads
- Use the sequential algorithm directly when 1 thread is requested
(skips task grouping)
- Fix the index type of parallelForEachN to size_t. Nobody in LLVM was
using any other parameter, and it made overload resolution hard for
for_each_n(par, 0, foo.size(), ...) because 0 is int, not size_t.

Remove Threads.h and update LLD for that.

This is a prerequisite for parallel public symbol processing in the PDB
library, which is in LLVM.

Reviewed By: MaskRay, aganea

Differential Revision: https://reviews.llvm.org/D79390

show more ...

Make llvm::StringRef to std::string conversions explicit.

This is how it should've been and brings it more in line with
std::string_view. There should be no functional change here.

This is mostly m

Make llvm::StringRef to std::string conversions explicit.

This is how it should've been and brings it more in line with
std::string_view. There should be no functional change here.

This is mostly mechanical from a custom clang-tidy check, with a lot of
manual fixups. It uncovers a lot of minor inefficiencies.

This doesn't actually modify StringRef yet, I'll do that in a follow-up.

show more ...

[LLD] Avoid exiting with a locked mutex NFC

In ErrorHandler::error(), rearrange code to avoid calling exitLld with
the mutex locked. Acquire mutex lock when flushing the output streams in
exitLld.

[LLD] Avoid exiting with a locked mutex NFC

In ErrorHandler::error(), rearrange code to avoid calling exitLld with
the mutex locked. Acquire mutex lock when flushing the output streams in
exitLld.

Differential Revision: https://reviews.llvm.org/D73281

show more ...

[Support] ThreadPoolExecutor fixes for Windows/MinGW

Changed ThreadPoolExecutor to no longer use detached threads and instead
to join threads on destruction. This is to prevent intermittent crashing

[Support] ThreadPoolExecutor fixes for Windows/MinGW

Changed ThreadPoolExecutor to no longer use detached threads and instead
to join threads on destruction. This is to prevent intermittent crashing
on Windows when doing a normal full exit, e.g. via exit().

Changed ThreadPoolExecutor to be a ManagedStatic so that it can be
stopped on llvm_shutdown(). Without this, it would only be stopped in
the destructor when doing a full exit. This is required to avoid
intermittent crashing on Windows due to a race condition between the
ThreadPoolExecutor starting up threads and the process doing a fast
exit, e.g. via _exit().

The Windows crashes appear to only occur with the MSVC static runtimes
and are more frequent with the debug static runtime.

These changes also prevent intermittent deadlocks on exit with the MinGW
runtime.

Differential Revision: https://reviews.llvm.org/D70447

show more ...

LLD: Don't use the stderrOS stream in link before it's reassigned.

Remove the lld::enableColors function, as it just obscures which
stream it's affecting, and replace with explicit calls to the stre

LLD: Don't use the stderrOS stream in link before it's reassigned.

Remove the lld::enableColors function, as it just obscures which
stream it's affecting, and replace with explicit calls to the stream's
enable_colors.

Also, assign the stderrOS and stdoutOS globals first in link function,
just to ensure nothing might use them.

(Either change individually fixes the issue of using the old
stream, but both together seems best.)

Follow-up to b11386f9be9b2dc7276a758d64f66833da10bdea.

Differential Revision: https://reviews.llvm.org/D70492

show more ...

Make it possible to redirect not only errs() but also outs()

This change is for those who use lld as a library. Context:
https://reviews.llvm.org/D70287

This patch adds a new parmeter to lld::*::li

Make it possible to redirect not only errs() but also outs()

This change is for those who use lld as a library. Context:
https://reviews.llvm.org/D70287

This patch adds a new parmeter to lld::*::link() so that we can pass
an raw_ostream object representing stdout. Previously, lld::*::link()
took only an stderr object.

Justification for making stdoutOS and stderrOS mandatory: I wanted to
make link() functions to take stdout and stderr in that order.
However, if we change the function signature from

bool link(ArrayRef<const char *> args, bool canExitEarly,
raw_ostream &stderrOS = llvm::errs());

to

bool link(ArrayRef<const char *> args, bool canExitEarly,
raw_ostream &stdoutOS = llvm::outs(),
raw_ostream &stderrOS = llvm::errs());

, then the meaning of existing code that passes stderrOS silently
changes (stderrOS would be interpreted as stdoutOS). So, I chose to
make existing code not to compile, so that developers can fix their
code.

Differential Revision: https://reviews.llvm.org/D70292

show more ...

[ELF] For VS-style diagnostics, prefer printing full paths in the header.

The filename part in the message header is used by Visual Studio
to fill Error List so that a user can click on an item and

[ELF] For VS-style diagnostics, prefer printing full paths in the header.

The filename part in the message header is used by Visual Studio
to fill Error List so that a user can click on an item and jump
to the mentioned location. If we use only the name of a source file
and not the full path, Visual Studio might be unable to find the right
file or, even worse, show a wrong one.

Differential Revision: https://reviews.llvm.org/D65875

llvm-svn: 368409

show more ...

[ELF] Fix splitting messages for duplicate symbols.

D65213 (rL367536) does not work for the case when a source file path
includes subdirectories.

Differential Revision: https://reviews.llvm.org/D65

[ELF] Fix splitting messages for duplicate symbols.

D65213 (rL367536) does not work for the case when a source file path
includes subdirectories.

Differential Revision: https://reviews.llvm.org/D65810

llvm-svn: 368153

show more ...

Simplify error message output. NFC.

Differential Revision: https://reviews.llvm.org/D65855

llvm-svn: 368144

Re-submit r367649: Improve raw_ostream so that you can "write" colors using operator<<

The original patch broke buildbots, perhaps because it changed the
default setting whether colors are enabled o

Re-submit r367649: Improve raw_ostream so that you can "write" colors using operator<<

The original patch broke buildbots, perhaps because it changed the
default setting whether colors are enabled or not.

llvm-svn: 368131

show more ...

Revert r367649: Improve raw_ostream so that you can "write" colors using operator<<

This reverts commit r367649 in an attempt to unbreak Windows bots.

llvm-svn: 367658

Add an assert() to catch possible regexp errors.

llvm-svn: 367651