[LLD] [COFF] Add support for a new, mingw specific embedded directive -exclude-symbols:

This is an entirely new embedded directive - extending the GNU ld
command line option --exclude-symbols to be

[LLD] [COFF] Add support for a new, mingw specific embedded directive -exclude-symbols:

This is an entirely new embedded directive - extending the GNU ld
command line option --exclude-symbols to be usable in embedded
directives too.

(GNU ld.bfd also got support for the same new directive, currently in
the latest git version, after the 2.39 branch.)

This works as an inverse to the regular embedded dllexport directives,
for cases when autoexport of all eligible symbols is performed.

Differential Revision: https://reviews.llvm.org/D130120

(cherry picked from commit 5d513ef6cf4646e64bbb1d5f8610afd530964588)

show more ...

Use llvm::sort instead of std::sort where possible

llvm::sort is beneficial even when we use the iterator-based overload,
since it can optionally shuffle the elements (to detect
non-determinism). Ho

Use llvm::sort instead of std::sort where possible

llvm::sort is beneficial even when we use the iterator-based overload,
since it can optionally shuffle the elements (to detect
non-determinism). However llvm::sort is not usable everywhere, for
example, in compiler-rt.

Reviewed By: nhaehnle

Differential Revision: https://reviews.llvm.org/D130406

show more ...

[LLD] [COFF] Improve the error message for too many exported symbols

Print the actual number of symbols that would have been exported
too, which helps assessing the situation.

Differential Revision

[LLD] [COFF] Improve the error message for too many exported symbols

Print the actual number of symbols that would have been exported
too, which helps assessing the situation.

Differential Revision: https://reviews.llvm.org/D130117

show more ...

Re-land [LLD] Remove global state in lldCommon

Move all variables at file-scope or function-static-scope into a hosting structure (lld::CommonLinkerContext) that lives at lldMain()-scope. Drivers wi

Re-land [LLD] Remove global state in lldCommon

Move all variables at file-scope or function-static-scope into a hosting structure (lld::CommonLinkerContext) that lives at lldMain()-scope. Drivers will inherit from this structure and add their own global state, in the same way as for the existing COFFLinkerContext.

See discussion in https://lists.llvm.org/pipermail/llvm-dev/2021-June/151184.html

The previous land f860fe362282ed69b9d4503a20e5d20b9a041189 caused issues in https://lab.llvm.org/buildbot/#/builders/123/builds/8383, fixed by 22ee510dac9440a74b2e5b3fe3ff13ccdbf55af3.

Differential Revision: https://reviews.llvm.org/D108850

show more ...

Revert [LLD] Remove global state in lldCommon

It seems to be causing issues on https://lab.llvm.org/buildbot/#/builders/123/builds/8383

[LLD] Remove global state in lldCommon

Move all variables at file-scope or function-static-scope into a hosting structure (lld::CommonLinkerContext) that lives at lldMain()-scope. Drivers will inher

[LLD] Remove global state in lldCommon

Move all variables at file-scope or function-static-scope into a hosting structure (lld::CommonLinkerContext) that lives at lldMain()-scope. Drivers will inherit from this structure and add their own global state, in the same way as for the existing COFFLinkerContext.

See discussion in https://lists.llvm.org/pipermail/llvm-dev/2021-June/151184.html

Differential Revision: https://reviews.llvm.org/D108850

show more ...

Enable pdbpagesize to allow support for PDB file sizes > 4GB

Enable the pdbpagesize flag to allow linking of PDB files > 4GB.
Also includes a couple small fixes to change to uint64_t to support the

Enable pdbpagesize to allow support for PDB file sizes > 4GB

Enable the pdbpagesize flag to allow linking of PDB files > 4GB.
Also includes a couple small fixes to change to uint64_t to support the
larger file sizes. I updated the max file size check in MSFBuilder.cpp
to take into account the page size.

Differential Revision: https://reviews.llvm.org/D115051

show more ...

[lld/coff] Add parsing for /pdbpagesize: flag

It's not used for anything yet, but we now accept `/pdbpagesize:4096`
(the default behavior) and we give arguably more useful diagnostics
for other valu

[lld/coff] Add parsing for /pdbpagesize: flag

It's not used for anything yet, but we now accept `/pdbpagesize:4096`
(the default behavior) and we give arguably more useful diagnostics
for other values.

It's plumbed through to the MSF layer, so just uncommenting out
the bit in DriverUtils.cpp that rejects args other than 4096 is enough
to try other values.

Differential Revision: https://reviews.llvm.org/D112871

show more ...

[lld/COFF] Improve handling of the /manifestdependency: flag

If multiple /manifestdependency: flags are passed, they are
naively deduped, but after that each of them should have an
effect, instead o

[lld/COFF] Improve handling of the /manifestdependency: flag

If multiple /manifestdependency: flags are passed, they are
naively deduped, but after that each of them should have an
effect, instead of just the last one.

Also, /manifestdependency: flags are allowed in .drectve sections
(from `#pragma comment(linker, ...`). To make the interaction between
/manifestdependency: flags enabling manifest by default but
/manifest:no overriding this work, add an explict ManifestKind::Default
state to represent no explicit /manifest flag being passed.
To make /manifestdependency: flags from input file .drectve sections
work with /manifest:embed, delay embedded manifest emission until
after input files have been read.

Differential Revision: https://reviews.llvm.org/D108628

show more ...

[OptTable] Rename PrintHelp to printHelp

To be consistent with other member functions and match the coding standard.

[lld] Rename StringRef _lower() method calls to _insensitive()

[LLD] Implement /guard:[no]ehcont

Reviewed By: rnk

Differential Revision: https://reviews.llvm.org/D99078

[SystemZ][z/OS][Windows] Add new OF_TextWithCRLF flag and use this flag instead of OF_Text

Problem:
On SystemZ we need to open text files in text mode. On Windows, files opened in text mode adds a C

[SystemZ][z/OS][Windows] Add new OF_TextWithCRLF flag and use this flag instead of OF_Text

Problem:
On SystemZ we need to open text files in text mode. On Windows, files opened in text mode adds a CRLF '\r\n' which may not be desirable.

Solution:
This patch adds two new flags

- OF_CRLF which indicates that CRLF translation is used.
- OF_TextWithCRLF = OF_Text | OF_CRLF indicates that the file is text and uses CRLF translation.

Developers should now use either the OF_Text or OF_TextWithCRLF for text files and OF_None for binary files. If the developer doesn't want carriage returns on Windows, they should use OF_Text, if they do want carriage returns on Windows, they should use OF_TextWithCRLF.

So this is the behaviour per platform with my patch:

z/OS:
OF_None: open in binary mode
OF_Text : open in text mode
OF_TextWithCRLF: open in text mode

Windows:
OF_None: open file with no carriage return
OF_Text: open file with no carriage return
OF_TextWithCRLF: open file with carriage return

The Major change is in llvm/lib/Support/Windows/Path.inc to only set text mode if the OF_CRLF is set.
```
if (Flags & OF_CRLF)
CrtOpenFlags |= _O_TEXT;
```

These following files are the ones that still use OF_Text which I left unchanged. I modified all these except raw_ostream.cpp in recent patches so I know these were previously in Binary mode on Windows.
./llvm/lib/Support/raw_ostream.cpp
./llvm/lib/TableGen/Main.cpp
./llvm/tools/dsymutil/DwarfLinkerForBinary.cpp
./llvm/unittests/Support/Path.cpp
./clang/lib/StaticAnalyzer/Core/HTMLDiagnostics.cpp
./clang/lib/Frontend/CompilerInstance.cpp
./clang/lib/Driver/Driver.cpp
./clang/lib/Driver/ToolChains/Clang.cpp

Reviewed By: MaskRay

Differential Revision: https://reviews.llvm.org/D99426

show more ...

[NFC] Reordering parameters in getFile and getFileOrSTDIN

In future patches I will be setting the IsText parameter frequently so I will refactor the args to be in the following order. I have removed

[NFC] Reordering parameters in getFile and getFileOrSTDIN

In future patches I will be setting the IsText parameter frequently so I will refactor the args to be in the following order. I have removed the FileSize parameter because it is never used.

```
static ErrorOr<std::unique_ptr<MemoryBuffer>>
getFile(const Twine &Filename, bool IsText = false,
bool RequiresNullTerminator = true, bool IsVolatile = false);

static ErrorOr<std::unique_ptr<MemoryBuffer>>
getFileOrSTDIN(const Twine &Filename, bool IsText = false,
bool RequiresNullTerminator = true);

static ErrorOr<std::unique_ptr<MB>>
getFileAux(const Twine &Filename, uint64_t MapSize, uint64_t Offset,
bool IsText, bool RequiresNullTerminator, bool IsVolatile);

static ErrorOr<std::unique_ptr<WritableMemoryBuffer>>
getFile(const Twine &Filename, bool IsVolatile = false);
```

Reviewed By: jhenderson

Differential Revision: https://reviews.llvm.org/D99182

show more ...

Avoid a possible one-byte OOB read off of .drectve sections

Pointed out by Ryan Prichard

[lld/mac] Implement basic typo correction for flags

Also use "unknown flag 'flag'" instead of "unknown flag: flag" for
consistency with the other ports.

Differential Revision: https://reviews.llvm.

[lld/mac] Implement basic typo correction for flags

Also use "unknown flag 'flag'" instead of "unknown flag: flag" for
consistency with the other ports.

Differential Revision: https://reviews.llvm.org/D91970

show more ...

[LLD] [COFF] Add a private option for setting the os version separately from subsystem version

The MinGW driver has separate options for OS and subsystem version.
Having this available in lld-link a

[LLD] [COFF] Add a private option for setting the os version separately from subsystem version

The MinGW driver has separate options for OS and subsystem version.
Having this available in lld-link allows the MinGW driver to both match
GNU ld better and simplifies the code for merging two (potentially
mismatching) arguments into one.

Differential Revision: https://reviews.llvm.org/D88802

show more ...

[LLD] [COFF] Fix parsing version numbers with leading zeros

Parse the components as decimal, instead of decuding the base from
the string. This avoids ambiguity if the second number contains leading

[LLD] [COFF] Fix parsing version numbers with leading zeros

Parse the components as decimal, instead of decuding the base from
the string. This avoids ambiguity if the second number contains leading
zeros, which previously were parsed as indicating an octal number.

MS link.exe doesn't support hexadecimal numbers in the version numbers,
neither in /version nor in /subsystem.

Differential Revision: https://reviews.llvm.org/D88801

show more ...

[LLD] [COFF] Error out if creating a DLL with too many exported symbols

The PE/DLL format has a limit on 64k exported symbols per DLL; make
sure to check this.

Differential Revision: https://review

[LLD] [COFF] Error out if creating a DLL with too many exported symbols

The PE/DLL format has a limit on 64k exported symbols per DLL; make
sure to check this.

Differential Revision: https://reviews.llvm.org/D86701

show more ...

[COFF] Use a global option table to avoid reconstructing it

Otherwise an ArgumentParser is constructed for every directive section,
and that involves copying the entire table of options into a vecto

[COFF] Use a global option table to avoid reconstructing it

Otherwise an ArgumentParser is constructed for every directive section,
and that involves copying the entire table of options into a vector.
There is no need for this, just have one option table.

show more ...

[COFF] Add and use a zero-copy tokenizer for .drectve

This generalizes the main Windows command line tokenizer to be able to
produce StringRef substrings as well as freshly copied C strings. The
imp

[COFF] Add and use a zero-copy tokenizer for .drectve

This generalizes the main Windows command line tokenizer to be able to
produce StringRef substrings as well as freshly copied C strings. The
implementation is still shared with the normal tokenizer, which is
important, because we have unit tests for that.

.drective sections can be very long. They can potentially list up to
every symbol in the object file by name. It is worth avoiding these
string copies.

This saves a lot of memory when linking chrome.dll with PGO
instrumentation:

BEFORE AFTER % IMP
peak memory: 6657.76MB 4983.54MB -25%
real: 4m30.875s 2m26.250s -46%

The time improvement may not be real, my machine was noisy while running
this, but that the peak memory usage improvement should be real.

This change may also help apps that heavily use dllexport annotations,
because those also use linker directives in object files. Apps that do
not use many directives are unlikely to be affected.

Reviewed By: thakis

Differential Revision: https://reviews.llvm.org/D79262

show more ...

[COFF] Add a fastpath for /INCLUDE: in .drective sections

This speeds up linking chrome.dll with PGO instrumentation by 13%
(154271ms -> 134033ms).

LLVM's Option library is very slow. In particular

[COFF] Add a fastpath for /INCLUDE: in .drective sections

This speeds up linking chrome.dll with PGO instrumentation by 13%
(154271ms -> 134033ms).

LLVM's Option library is very slow. In particular, it allocates at least
one large-ish heap object (Arg) for every argument. When PGO
instrumentation is enabled, all the __profd_* symbols are added to the
@llvm.used list, which compiles down to these /INCLUDE: directives. This
means we have O(#symbols) directives to parse in the section, so we end
up allocating an Arg for every function symbol in the object file. This
is unnecessary.

To address the issue and speed up the link, extend the fast path that we
already have for /EXPORT:, which has similar scaling issues.

I promise that I took a hard look at optimizing the Option library, but
its data structures are very general and would need a lot of cleanup. We
have accumulated lots of optional features (option groups, aliases,
multiple values) over the years, and these are now properties of every
parsed argument, when the vast majority of arguments do not use these
features.

Reviewed By: thakis

Differential Revision: https://reviews.llvm.org/D78845

show more ...

Make llvm::StringRef to std::string conversions explicit.

This is how it should've been and brings it more in line with
std::string_view. There should be no functional change here.

This is mostly m

Make llvm::StringRef to std::string conversions explicit.

This is how it should've been and brings it more in line with
std::string_view. There should be no functional change here.

This is mostly mechanical from a custom clang-tidy check, with a lot of
manual fixups. It uncovers a lot of minor inefficiencies.

This doesn't actually modify StringRef yet, I'll do that in a follow-up.

show more ...

LLD: Don't use the stderrOS stream in link before it's reassigned.

Remove the lld::enableColors function, as it just obscures which
stream it's affecting, and replace with explicit calls to the stre

LLD: Don't use the stderrOS stream in link before it's reassigned.

Remove the lld::enableColors function, as it just obscures which
stream it's affecting, and replace with explicit calls to the stream's
enable_colors.

Also, assign the stderrOS and stdoutOS globals first in link function,
just to ensure nothing might use them.

(Either change individually fixes the issue of using the old
stream, but both together seems best.)

Follow-up to b11386f9be9b2dc7276a758d64f66833da10bdea.

Differential Revision: https://reviews.llvm.org/D70492

show more ...

Make it possible to redirect not only errs() but also outs()

This change is for those who use lld as a library. Context:
https://reviews.llvm.org/D70287

This patch adds a new parmeter to lld::*::li

Make it possible to redirect not only errs() but also outs()

This change is for those who use lld as a library. Context:
https://reviews.llvm.org/D70287

This patch adds a new parmeter to lld::*::link() so that we can pass
an raw_ostream object representing stdout. Previously, lld::*::link()
took only an stderr object.

Justification for making stdoutOS and stderrOS mandatory: I wanted to
make link() functions to take stdout and stderr in that order.
However, if we change the function signature from

bool link(ArrayRef<const char *> args, bool canExitEarly,
raw_ostream &stderrOS = llvm::errs());

to

bool link(ArrayRef<const char *> args, bool canExitEarly,
raw_ostream &stdoutOS = llvm::outs(),
raw_ostream &stderrOS = llvm::errs());

, then the meaning of existing code that passes stderrOS silently
changes (stderrOS would be interpreted as stdoutOS). So, I chose to
make existing code not to compile, so that developers can fix their
code.

Differential Revision: https://reviews.llvm.org/D70292

show more ...