[Debugify] Port verify-debuginfo-preserve to NewPM

Debugify in OriginalDebugInfo mode, introduced with D82545,
runs only with legacy PassManager.

This patch enables this utility for the NewPM.

Dif

[Debugify] Port verify-debuginfo-preserve to NewPM

Debugify in OriginalDebugInfo mode, introduced with D82545,
runs only with legacy PassManager.

This patch enables this utility for the NewPM.

Differential Revision: https://reviews.llvm.org/D115351

show more ...

[Debugify] Limit number of processed functions for original mode

Debugify in OriginalDebugInfo mode, does (DebugInfo) collect-before-pass & check-after-pass
for each instruction, which is pretty exp

[Debugify] Limit number of processed functions for original mode

Debugify in OriginalDebugInfo mode, does (DebugInfo) collect-before-pass & check-after-pass
for each instruction, which is pretty expensive. When used to analyze DebugInfo losses
in large projects (like LLVM), this raises the build time unacceptably.
This patch introduces a limit for the number of processed functions per compile unit.
By default, the limit is set to UINT_MAX (practically unlimited), and by using the introduced
option -debugify-func-limit the limit could be set to any positive integer number.

Differential revision: https://reviews.llvm.org/D115714

show more ...

[Utils] stripDebugifyMetadata - use cast<> instead of dyn_cast_or_null<> to avoid dereference of nullptr

The pointer is dereferenced immediately, so assert the cast is correct instead of returning n

[Utils] stripDebugifyMetadata - use cast<> instead of dyn_cast_or_null<> to avoid dereference of nullptr

The pointer is dereferenced immediately, so assert the cast is correct instead of returning nullptr

show more ...

[Debugify] Use DebugifyLevel in Debugify original mode

Before this patch the DebugifyLevel option was used for
the synthetic mode, so after this, it will be used in
the original mode as well.

Diffe

[Debugify] Use DebugifyLevel in Debugify original mode

Before this patch the DebugifyLevel option was used for
the synthetic mode, so after this, it will be used in
the original mode as well.

Differential Revision: https://reviews.llvm.org/D115623

show more ...

[Debugify] Optimize debugify original mode

Before we start addressing the issue with having
a lot of false positives when using debugify in
the original mode, we have made a few patches that
should

[Debugify] Optimize debugify original mode

Before we start addressing the issue with having
a lot of false positives when using debugify in
the original mode, we have made a few patches that
should speed up the execution of the testing
utility Passes.

For example, when testing a large project
(let's say LLVM project itself), we can face
a lot of potential DI issues. Usually, we use
-verify-each-debuginfo-preserve (that is very
similar to -debugify-each) -- it collects
DI metadata before each Pass, and after the Pass
it checks if the Pass preserved the DI metadata.
However, we can speed up this process, since we
don't need to collect DI metadata before each
Pass -- we could use the DI metadata that are
collected after the previous Pass from
the pipeline as an input for the next Pass.

This patch speeds up the utility for ~2x.

Differential Revision: https://reviews.llvm.org/D115622

show more ...

[Debugify] Use WeakWH map collected before Pass when checking loc drop

This fixes a typo/bug when checking for pointer reuse when testing
DI location preservation in the Debugify original mode (when

[Debugify] Use WeakWH map collected before Pass when checking loc drop

This fixes a typo/bug when checking for pointer reuse when testing
DI location preservation in the Debugify original mode (when
checking -g generated Debug Info).

Differential Revision: https://reviews.llvm.org/D115621

show more ...

NFC: [Debugify] Fix a typo when checking variables in the original mode

Recommit: "[Debugify][Original DI] Test dbg var loc preservation""

[Debugify][Original DI] Test dbg var loc preservation

This is an improvement of [0]. This adds checking of
original ll

Recommit: "[Debugify][Original DI] Test dbg var loc preservation""

[Debugify][Original DI] Test dbg var loc preservation

This is an improvement of [0]. This adds checking of
original llvm.dbg.values()/declares() instructions in
optimizations.

We have picked a real issue that has been found with
this (actually, picked one variable location missing
from [1] and resolved the issue), and the result is
the fix for that -- D100844.

Before applying the D100844, using the options from [0]
(but with this patch applied) on the compilation of GDB 7.11,
the final HTML report for the debug-info issues can be found
at [1] (please scroll down, and look for
"Summary of Variable Location Bugs"). After applying
the D100844, the numbers has improved a bit -- please take
a look into [2].

[0] https://llvm.org/docs/HowToUpdateDebugInfo.html#\
test-original-debug-info-preservation-in-optimizations
[1] https://djolertrk.github.io/di-check-before-adce-fix/
[2] https://djolertrk.github.io/di-check-after-adce-fix/

Differential Revision: https://reviews.llvm.org/D100845

The Unit test was failing because the pass from the test that
modifies the IR, in its runOnFunction() didn't return 'true',
so the expensive-check configuration triggered an assertion.

show more ...

Revert "[Debugify][Original DI] Test dbg var loc preservation"

This reverts commit 76f375f3d9d6902820ffc21200e454926748c678.

This will be pushed again, after investigating a test failure:
https://l

Revert "[Debugify][Original DI] Test dbg var loc preservation"

This reverts commit 76f375f3d9d6902820ffc21200e454926748c678.

This will be pushed again, after investigating a test failure:
https://lab.llvm.org/buildbot/#/builders/16/builds/11254

show more ...

[Debugify][Original DI] Test dbg var loc preservation

This is an improvement of [0]. This adds checking of
original llvm.dbg.values()/declares() instructions in
optimizations.

We have picked a real

[Debugify][Original DI] Test dbg var loc preservation

This is an improvement of [0]. This adds checking of
original llvm.dbg.values()/declares() instructions in
optimizations.

We have picked a real issue that has been found with
this (actually, picked one variable location missing
from [1] and resolved the issue), and the result is
the fix for that -- D100844.

Before applying the D100844, using the options from [0]
(but with this patch applied) on the compilation of GDB 7.11,
the final HTML report for the debug-info issues can be found
at [1] (please scroll down, and look for
"Summary of Variable Location Bugs"). After applying
the D100844, the numbers has improved a bit -- please take
a look into [2].

[0] https://llvm.org/docs/HowToUpdateDebugInfo.html\
[1] https://djolertrk.github.io/di-check-before-adce-fix/
[2] https://djolertrk.github.io/di-check-after-adce-fix/

Differential Revision: https://reviews.llvm.org/D100845

show more ...

[Transforms][Debugify] Fix "Missing line" false alarm on PHI nodes

This is a fix for https://bugs.llvm.org/show_bug.cgi?id=49959

The "Missing line" false alarm was introduced in D75242.

Patch by Y

[Transforms][Debugify] Fix "Missing line" false alarm on PHI nodes

This is a fix for https://bugs.llvm.org/show_bug.cgi?id=49959

The "Missing line" false alarm was introduced in D75242.

Patch by Yilong Guo<[email protected]>

Differential Revision: https://reviews.llvm.org/D100446

show more ...

[Support] Don't include VirtualFileSystem.h in CommandLine.h

CommandLine.h is indirectly included in ~50% of TUs when building
clang, and VirtualFileSystem.h is large.

(Already remarked by jhenders

[Support] Don't include VirtualFileSystem.h in CommandLine.h

CommandLine.h is indirectly included in ~50% of TUs when building
clang, and VirtualFileSystem.h is large.

(Already remarked by jhenderson on D70769.)

No behavior change.

Differential Revision: https://reviews.llvm.org/D100957

show more ...

[SystemZ][z/OS][Windows] Add new OF_TextWithCRLF flag and use this flag instead of OF_Text

Problem:
On SystemZ we need to open text files in text mode. On Windows, files opened in text mode adds a C

[SystemZ][z/OS][Windows] Add new OF_TextWithCRLF flag and use this flag instead of OF_Text

Problem:
On SystemZ we need to open text files in text mode. On Windows, files opened in text mode adds a CRLF '\r\n' which may not be desirable.

Solution:
This patch adds two new flags

- OF_CRLF which indicates that CRLF translation is used.
- OF_TextWithCRLF = OF_Text | OF_CRLF indicates that the file is text and uses CRLF translation.

Developers should now use either the OF_Text or OF_TextWithCRLF for text files and OF_None for binary files. If the developer doesn't want carriage returns on Windows, they should use OF_Text, if they do want carriage returns on Windows, they should use OF_TextWithCRLF.

So this is the behaviour per platform with my patch:

z/OS:
OF_None: open in binary mode
OF_Text : open in text mode
OF_TextWithCRLF: open in text mode

Windows:
OF_None: open file with no carriage return
OF_Text: open file with no carriage return
OF_TextWithCRLF: open file with carriage return

The Major change is in llvm/lib/Support/Windows/Path.inc to only set text mode if the OF_CRLF is set.
```
if (Flags & OF_CRLF)
CrtOpenFlags |= _O_TEXT;
```

These following files are the ones that still use OF_Text which I left unchanged. I modified all these except raw_ostream.cpp in recent patches so I know these were previously in Binary mode on Windows.
./llvm/lib/Support/raw_ostream.cpp
./llvm/lib/TableGen/Main.cpp
./llvm/tools/dsymutil/DwarfLinkerForBinary.cpp
./llvm/unittests/Support/Path.cpp
./clang/lib/StaticAnalyzer/Core/HTMLDiagnostics.cpp
./clang/lib/Frontend/CompilerInstance.cpp
./clang/lib/Driver/Driver.cpp
./clang/lib/Driver/ToolChains/Clang.cpp

Reviewed By: MaskRay

Differential Revision: https://reviews.llvm.org/D99426

show more ...

Reapply "[DebugInfo] Handle multiple variable location operands in IR"

Fixed section of code that iterated through a SmallDenseMap and added
instructions in each iteration, causing non-deterministic

Reapply "[DebugInfo] Handle multiple variable location operands in IR"

Fixed section of code that iterated through a SmallDenseMap and added
instructions in each iteration, causing non-deterministic code; replaced
SmallDenseMap with MapVector to prevent non-determinism.

This reverts commit 01ac6d1587e8613ba4278786e8341f8b492ac941.

show more ...

Revert "[DebugInfo] Handle multiple variable location operands in IR"

This caused non-deterministic compiler output; see comment on the
code review.

> This patch updates the various IR passes to co

Revert "[DebugInfo] Handle multiple variable location operands in IR"

This caused non-deterministic compiler output; see comment on the
code review.

> This patch updates the various IR passes to correctly handle dbg.values with a
> DIArgList location. This patch does not actually allow DIArgLists to be produced
> by salvageDebugInfo, and it does not affect any pass after codegen-prepare.
> Other than that, it should cover every IR pass.
>
> Most of the changes simply extend code that operated on a single debug value to
> operate on the list of debug values in the style of any_of, all_of, for_each,
> etc. Instances of setOperand(0, ...) have been replaced with with
> replaceVariableLocationOp, which takes the value that is being replaced as an
> additional argument. In places where this value isn't readily available, we have
> to track the old value through to the point where it gets replaced.
>
> Differential Revision: https://reviews.llvm.org/D88232

This reverts commit df69c69427dea7f5b3b3a4d4564bc77b0926ec88.

show more ...

[Debugify][OriginalDIMode] Export the report into JSON file

By using the original-di check with debugify in the combination with
the llvm/utils/llvm-original-di-preservation.py it becomes very user

[Debugify][OriginalDIMode] Export the report into JSON file

By using the original-di check with debugify in the combination with
the llvm/utils/llvm-original-di-preservation.py it becomes very user
friendly tool. An example of the HTML page with the issues
related to debug info can be found at [0].

[0] https://djolertrk.github.io/di-checker-html-report-example/

Differential Revision: https://reviews.llvm.org/D82546

show more ...

[DebugInfo] Handle multiple variable location operands in IR

This patch updates the various IR passes to correctly handle dbg.values with a
DIArgList location. This patch does not actually allow DIA

[DebugInfo] Handle multiple variable location operands in IR

This patch updates the various IR passes to correctly handle dbg.values with a
DIArgList location. This patch does not actually allow DIArgLists to be produced
by salvageDebugInfo, and it does not affect any pass after codegen-prepare.
Other than that, it should cover every IR pass.

Most of the changes simply extend code that operated on a single debug value to
operate on the list of debug values in the style of any_of, all_of, for_each,
etc. Instances of setOperand(0, ...) have been replaced with with
replaceVariableLocationOp, which takes the value that is being replaced as an
additional argument. In places where this value isn't readily available, we have
to track the old value through to the point where it gets replaced.

Differential Revision: https://reviews.llvm.org/D88232

show more ...

Reland "[Debugify] Make the debugify aware of the original (-g) Debug Info"

As discussed on the RFC [0], I am sharing the set of patches that
enables checking of original Debug Info metadata

Reland "[Debugify] Make the debugify aware of the original (-g) Debug Info"

As discussed on the RFC [0], I am sharing the set of patches that
enables checking of original Debug Info metadata preservation in
optimizations. The proof-of-concept/proposal can be found at [1].

The implementation from the [1] was full of duplicated code,
so this set of patches tries to merge this approach into the existing
debugify utility.

For example, the utility pass in the original-debuginfo-check
mode could be invoked as follows:

$ opt -verify-debuginfo-preserve -pass-to-test sample.ll

Since this is very initial stage of the implementation,
there is a space for improvements such as:
- Add support for the new pass manager
- Add support for metadata other than DILocations and DISubprograms

[0] https://groups.google.com/forum/#!msg/llvm-dev/QOyF-38YPlE/G213uiuwCAAJ
[1] https://github.com/djolertrk/llvm-di-checker

Differential Revision: https://reviews.llvm.org/D82545

The test that was failing is now forced to use the old PM.

show more ...

Revert "[Debugify] Make the debugify aware of the original (-g) Debug Info"

This reverts rG8ee7c7e02953.
One test is failing, I'll reland this as soon as possible.

[Debugify] Make the debugify aware of the original (-g) Debug Info

As discussed on the RFC [0], I am sharing the set of patches that
enables checking of original Debug Info metadata preservation in

[Debugify] Make the debugify aware of the original (-g) Debug Info

As discussed on the RFC [0], I am sharing the set of patches that
enables checking of original Debug Info metadata preservation in
optimizations. The proof-of-concept/proposal can be found at [1].

The implementation from the [1] was full of duplicated code,
so this set of patches tries to merge this approach into the existing
debugify utility.

For example, the utility pass in the original-debuginfo-check
mode could be invoked as follows:

$ opt -verify-debuginfo-preserve -pass-to-test sample.ll

Since this is very initial stage of the implementation,
there is a space for improvements such as:
- Add support for the new pass manager
- Add support for metadata other than DILocations and DISubprograms

[0] https://groups.google.com/forum/#!msg/llvm-dev/QOyF-38YPlE/G213uiuwCAAJ
[1] https://github.com/djolertrk/llvm-di-checker

Differential Revision: https://reviews.llvm.org/D82545

show more ...

[llvm] Construct SmallVector with iterator ranges (NFC)

Reland "[MachineDebugify] Insert synthetic DBG_VALUE instructions"

This reverts commit 841f9c937f6e593c926a26aedf054436eb807fe6.
The change landed many months ago; something else broke those tests.

Revert "[MachineDebugify] Insert synthetic DBG_VALUE instructions"

This reverts commit 2a5675f11d3bc803a245c0e2a3b47491c8f8a065.
The tests it adds fail: https://reviews.llvm.org/D78135#2453736

[Debugify] Skip debugifying on special/immutable passes

With a function pass manager, it would insert debuginfo metadata before
getting to function passes while processing the pass manager, causing

[Debugify] Skip debugifying on special/immutable passes

With a function pass manager, it would insert debuginfo metadata before
getting to function passes while processing the pass manager, causing
debugify to skip while running the function passes.

Skip special passes + verifier + printing passes. Compared to the legacy
implementation of -debugify-each, this additionally skips verifier
passes. Probably no need to update the legacy version since it will be
obsolete soon.

This fixes 2 instcombine tests using -debugify-each under NPM.

Reviewed By: MaskRay

Differential Revision: https://reviews.llvm.org/D91558

show more ...

[Debugify] Port -debugify-each to NewPM

Preemptively switch 2 tests to the new PM

Reviewed By: aeubanks

Differential Revision: https://reviews.llvm.org/D90365