[FileCheck] Implement -dump-input-context

This patch is motivated by discussions at each of:

* <https://reviews.llvm.org/D81422>
* <http://lists.llvm.org/pipermail/llvm-dev/2020-June/142369.html>

[FileCheck] Implement -dump-input-context

This patch is motivated by discussions at each of:

* <https://reviews.llvm.org/D81422>
* <http://lists.llvm.org/pipermail/llvm-dev/2020-June/142369.html>

When input is dumped as specified by `-dump-input=fail`, this patch
filters the dump to show only input lines that are the starting lines
of error diagnostics plus the number of contextual lines specified
`-dump-input-context` (defaults to 5).

When `-dump-input=always`, there might be not be any errors, so all
input lines are printed, as without this patch.

Here's some sample output with `-dump-input-context=3 -vv`:

```
<<<<<<
.
.
.
13: foo
14: foo
15: hello world
check:1 ^~~~~~~~~~~
16: foo
check:2'0 X~~ error: no match found
17: foo
check:2'0 ~~~
18: foo
check:2'0 ~~~
19: foo
check:2'0 ~~~
.
.
.
27: foo
check:2'0 ~~~
28: foo
check:2'0 ~~~
29: foo
check:2'0 ~~~
30: goodbye word
check:2'0 ~~~~~~~~~~~~
check:2'1 ? possible intended match
31: foo
check:2'0 ~~~
32: foo
check:2'0 ~~~
33: foo
check:2'0 ~~~
.
.
.
>>>>>>
```

Reviewed By: mehdi_amini, arsenm, jhenderson, rsmith, SjoerdMeijer, Meinersbur, lattner

Differential Revision: https://reviews.llvm.org/D82203

show more ...

[FileCheck] Improve -dump-input documentation

Document the default of `fail` in `-help`. Extend `-dump-input=help`
to help users find related command-line options, but let `-help`
provide their ful

[FileCheck] Improve -dump-input documentation

Document the default of `fail` in `-help`. Extend `-dump-input=help`
to help users find related command-line options, but let `-help`
provide their full documentation.

Reviewed By: probinson

Differential Revision: https://reviews.llvm.org/D83091

show more ...

[FileCheck] Permit multiple -v or -vv

`FILECHECK_OPTS` was implemented so that a test runner, such as CI,
can specify FileCheck debugging options, such as `-v` and `-vv`.
However, if a test suite ha

[FileCheck] Permit multiple -v or -vv

`FILECHECK_OPTS` was implemented so that a test runner, such as CI,
can specify FileCheck debugging options, such as `-v` and `-vv`.
However, if a test suite has a FileCheck call that already specifies
`-v` or `-vv`, then that call will fail if `FILECHECK_OPTS` also
specifies it.

For `-vv`, this problem already exists:

`clang/test/CodeGen/aarch64-v8.2a-fp16-intrinsics-constrained.c`

It's not yet clear if the `-vv` in that test was intentional, but this
usage shouldn't fail anyway. It's already true that FileCheck permits
`-vv` and `-v` together even though `-vv` implies `-v`.

Compare D70784, which fixed the same problem for `-dump-input`.

Reviewed By: jhenderson, thopre

Differential Revision: https://reviews.llvm.org/D82601

show more ...

[FileCheck][NFC] Remove redundant DumpInputDefault

Reviewed By: mehdi_amini, jhenderson

Differential Revision: https://reviews.llvm.org/D82480

Change filecheck default to dump input on failure

Having the input dumped on failure seems like a better
default: I debugged FileCheck tests for a while without knowing
about this option, which real

Change filecheck default to dump input on failure

Having the input dumped on failure seems like a better
default: I debugged FileCheck tests for a while without knowing
about this option, which really helps to understand failures.

Remove `-dump-input-on-failure` and the environment variable
FILECHECK_DUMP_INPUT_ON_FAILURE which are now obsolete.

Differential Revision: https://reviews.llvm.org/D81422

show more ...

[FileCheck] Support comment directives

Sometimes you want to disable a FileCheck directive without removing
it entirely, or you want to write comments that mention a directive by
name. The `COM:` d

[FileCheck] Support comment directives

Sometimes you want to disable a FileCheck directive without removing
it entirely, or you want to write comments that mention a directive by
name. The `COM:` directive makes it easy to do this. For example,
you might have:

```
; X32: pinsrd_1:
; X32: pinsrd $1, 4(%esp), %xmm0

; COM: FIXME: X64 isn't working correctly yet for this part of codegen, but
; COM: X64 will have something similar to X32:
; COM:
; COM: X64: pinsrd_1:
; COM: X64: pinsrd $1, %edi, %xmm0
```

Without this patch, you need to use some combination of rewording and
directive syntax mangling to prevent FileCheck from recognizing the
commented occurrences of `X32:` and `X64:` above as directives.
Moreover, FileCheck diagnostics have been proposed that might complain
about the occurrences of `X64` that don't have the trailing `:`
because they look like directive typos:

<http://lists.llvm.org/pipermail/llvm-dev/2020-April/140610.html>

I think dodging all these problems can prove tedious for test authors,
and directive syntax mangling already makes the purpose of existing
test code unclear. `COM:` can avoid all these problems.

This patch also updates the small set of existing tests that define
`COM` as a check prefix:

- clang/test/CodeGen/default-address-space.c
- clang/test/CodeGenOpenCL/addr-space-struct-arg.cl
- clang/test/Driver/hip-device-libs.hip
- llvm/test/Assembler/drop-debug-info-nonzero-alloca.ll

I think lit should support `COM:` as well. Perhaps `clang -verify`
should too.

Reviewed By: jhenderson, thopre

Differential Revision: https://reviews.llvm.org/D79276

show more ...

[FileCheck] Make invalid prefix diagnostics more precise

This will prove especially helpful after D79276, which introduces
comment prefixes. Specifically, identifying whether there's a
uniqueness v

[FileCheck] Make invalid prefix diagnostics more precise

This will prove especially helpful after D79276, which introduces
comment prefixes. Specifically, identifying whether there's a
uniqueness violation will be helpful as prefixes will be required to
be unique across both check prefixes and comment prefixes.

Also, remove a related comment about `cl::list` that no longer seems
relevant now that FileCheck is also a library.

Reviewed By: jhenderson, thopre

Differential Revision: https://reviews.llvm.org/D79375

show more ...

Revert "[FileCheck] Make invalid prefix diagnostics more precise"

This reverts commit a78e13745d4ee4a42e41ebbe698159f651515fc5 to try to
fix a bot:

http://lab.llvm.org:8011/builders/llvm-clang-x86_

Revert "[FileCheck] Make invalid prefix diagnostics more precise"

This reverts commit a78e13745d4ee4a42e41ebbe698159f651515fc5 to try to
fix a bot:

http://lab.llvm.org:8011/builders/llvm-clang-x86_64-expensive-checks-win/builds/23489

show more ...

Revert "[FileCheck] Support comment directives"

This reverts commit 9a9a5f9893c8db05cebc8818eb8485bff61f7c74 to try to
fix a bot:

http://lab.llvm.org:8011/builders/llvm-clang-x86_64-expensive-check

Revert "[FileCheck] Support comment directives"

This reverts commit 9a9a5f9893c8db05cebc8818eb8485bff61f7c74 to try to
fix a bot:

http://lab.llvm.org:8011/builders/llvm-clang-x86_64-expensive-checks-win/builds/23489

show more ...

[FileCheck] Support comment directives

Sometimes you want to disable a FileCheck directive without removing
it entirely, or you want to write comments that mention a directive by
name. The `COM:` d

[FileCheck] Support comment directives

Sometimes you want to disable a FileCheck directive without removing
it entirely, or you want to write comments that mention a directive by
name. The `COM:` directive makes it easy to do this. For example,
you might have:

```
; X32: pinsrd_1:
; X32: pinsrd $1, 4(%esp), %xmm0

; COM: FIXME: X64 isn't working correctly yet for this part of codegen, but
; COM: X64 will have something similar to X32:
; COM:
; COM: X64: pinsrd_1:
; COM: X64: pinsrd $1, %edi, %xmm0
```

Without this patch, you need to use some combination of rewording and
directive syntax mangling to prevent FileCheck from recognizing the
commented occurrences of `X32:` and `X64:` above as directives.
Moreover, FileCheck diagnostics have been proposed that might complain
about the occurrences of `X64` that don't have the trailing `:`
because they look like directive typos:

<http://lists.llvm.org/pipermail/llvm-dev/2020-April/140610.html>

I think dodging all these problems can prove tedious for test authors,
and directive syntax mangling already makes the purpose of existing
test code unclear. `COM:` can avoid all these problems.

This patch also updates the small set of existing tests that define
`COM` as a check prefix:

- clang/test/CodeGen/default-address-space.c
- clang/test/CodeGenOpenCL/addr-space-struct-arg.cl
- clang/test/Driver/hip-device-libs.hip
- llvm/test/Assembler/drop-debug-info-nonzero-alloca.ll

I think lit should support `COM:` as well. Perhaps `clang -verify`
should too.

Reviewed By: jhenderson, thopre

Differential Revision: https://reviews.llvm.org/D79276

show more ...

[FileCheck] Make invalid prefix diagnostics more precise

This will prove especially helpful after D79276, which introduces
comment prefixes. Specifically, identifying whether there's a
uniqueness v

[FileCheck] Make invalid prefix diagnostics more precise

This will prove especially helpful after D79276, which introduces
comment prefixes. Specifically, identifying whether there's a
uniqueness violation will be helpful as prefixes will be required to
be unique across both check prefixes and comment prefixes.

Also, remove a related comment about `cl::list` that no longer seems
relevant now that FileCheck is also a library.

Reviewed By: jhenderson, thopre

Differential Revision: https://reviews.llvm.org/D79375

show more ...

[FileCheck] - Refactor the code related to string arrays. NFCI.

There are few `std::vector<std::string>` members in
`FileCheckRequest`. This patch changes these arrays to `std::vector<StringRef>`
an

[FileCheck] - Refactor the code related to string arrays. NFCI.

There are few `std::vector<std::string>` members in
`FileCheckRequest`. This patch changes these arrays to `std::vector<StringRef>`
and refactors the code related to cleanup/improve/simplify it.

Differential revision: https://reviews.llvm.org/D78202

show more ...

[FileCheck] Fix --dump-input annotation sort per input line

Without this patch, `--dump-input` annotations on a single input line
are sorted by the associated directive's check-file line. That seem

[FileCheck] Fix --dump-input annotation sort per input line

Without this patch, `--dump-input` annotations on a single input line
are sorted by the associated directive's check-file line. That seemed
fine because that's often identical to the order in which FileCheck
looks for matches for those directives.

The first problem is that an `--implicit-check-not` pattern has no
check-file line. The logical equivalent is sorting in command-line
order, but that's not implemented.

The second problem is that, unlike a directive, an
`--implicit-check-not` pattern applies at many points, between many
different pairs of directives. However, sorting in command-line order
gathers all its associated diagnostics together at one point in an
input line's list of annotations.

In general, it seems to be easier to understand FileCheck's logic when
annotations on a single input line are sorted in the order FileCheck
produced the associated diagnostics, so this patch makes that change.
As documented in the patch, the annotation sort order is also
especially relevant to `CHECK-LABEL`, `CHECK-NOT`, and `CHECK-DAG`, so
this patch updates or extends tests to check the sort makes sense for
them. (However, the sort for `CHECK-DAG` annotations should not
actually be altered by this patch.)

Reviewed By: thopre

Differential Revision: https://reviews.llvm.org/D77607

show more ...

[FileCheck] Fix --dump-input implicit pattern location

Currently, `--dump-input` implies that all `--implicit-check-not`
patterns appear on line 1 by printing annotations like:

```
1: foo ba

[FileCheck] Fix --dump-input implicit pattern location

Currently, `--dump-input` implies that all `--implicit-check-not`
patterns appear on line 1 by printing annotations like:

```
1: foo bar baz
not:1 !~~ error: no match expected
```

This patch changes that to:

```
1: foo bar baz
not:imp1 !~~ error: no match expected
```

`imp1` indicates the first `--implicit-check-not` pattern.

Reviewed By: thopre

Differential Revision: https://reviews.llvm.org/D77605

show more ...

Improve error message of FileCheck when stdin is empty

Summary: Replace '-' in the error message with <stdin>. This is also consistent with another error message in the code.

Reviewers: jhenderson,

Improve error message of FileCheck when stdin is empty

Summary: Replace '-' in the error message with <stdin>. This is also consistent with another error message in the code.

Reviewers: jhenderson, probinson, jdenny, grimar, arichardson

Reviewed By: jhenderson

Subscribers: thopre, llvm-commits

Tags: #llvm

Differential Revision: https://reviews.llvm.org/D73793

show more ...

[FileCheck] Given multiple -dump-input, prefer most verbose

Problem: `FILECHECK_OPTS` was implemented so that a test runner, such
as a bot, can specify FileCheck debugging options, such as
`-dump-in

[FileCheck] Given multiple -dump-input, prefer most verbose

Problem: `FILECHECK_OPTS` was implemented so that a test runner, such
as a bot, can specify FileCheck debugging options, such as
`-dump-input=fail`. However, some existing test suites have FileCheck
calls that already specify `-dump-input=fail` or `-dump-input=always`.
Without this patch, such tests fail under such a test runner because
FileCheck doesn't accept multiple occurrences of `-dump-input`.

Solution: This patch permits multiple occurrences of `-dump-input` by
assigning precedence to its values in the following descending order:
`help`, `always`, `fail`, and `never`. That is, any occurrence of
`help` always obtains help, and otherwise the behavior is similar to
`-v` vs. `-vv` in that the option specifying the greatest verbosity
has precedence.

Rationale: My justification for the new behavior is as follows. I
have not experienced use cases where, either as a test runner or as a
test author, I want to **limit** the permitted debugging verbosity
(except as a test author in FileCheck's or lit's test suites where the
FileCheck debugging output itself is under test, but the solution
there is `env FILECHECK_OPTS=`, and I imagine we should use the same
solution anywhere else this need might occur). Of course, as either a
test runner or test author, it is useful to **increase** debugging
verbosity.

Reviewed By: probinson

Differential Revision: https://reviews.llvm.org/D70784

show more ...

[FileCheck] Implement --ignore-case option.

The FileCheck utility is enhanced to support a `--ignore-case`
option. This is useful in cases where the output of Unix tools
differs in case (e.g. case n

[FileCheck] Implement --ignore-case option.

The FileCheck utility is enhanced to support a `--ignore-case`
option. This is useful in cases where the output of Unix tools
differs in case (e.g. case not specified by Posix).

Reviewers: Bigcheese, jakehehrlich, rupprecht, espindola, alexshap, jhenderson, MaskRay

Differential Revision: https://reviews.llvm.org/D68146

llvm-svn: 374538

show more ...

Revert "[FileCheck] Implement --ignore-case option."

This reverts commit r374339. It broke tests:
http://lab.llvm.org:8011/builders/clang-x86_64-debian-fast/builds/19066

llvm-svn: 374359

[FileCheck] Implement --ignore-case option.

The FileCheck utility is enhanced to support a `--ignore-case`
option. This is useful in cases where the output of Unix tools
differs in case (e.g. case n

[FileCheck] Implement --ignore-case option.

The FileCheck utility is enhanced to support a `--ignore-case`
option. This is useful in cases where the output of Unix tools
differs in case (e.g. case not specified by Posix).

Reviewers: Bigcheese, jakehehrlich, rupprecht, espindola, alexshap, jhenderson, MaskRay

Differential Revision: https://reviews.llvm.org/D68146

llvm-svn: 374339

show more ...

[FileCheck] Remove implementation types from API

Summary:
Remove use of FileCheckPatternContext and FileCheckString concrete types
from FileCheck API to allow moving it and the other implementation

[FileCheck] Remove implementation types from API

Summary:
Remove use of FileCheckPatternContext and FileCheckString concrete types
from FileCheck API to allow moving it and the other implementation only
only declarations into a private header file.

Reviewers: jhenderson, chandlerc, jdenny, probinson, grimar, arichardson, rnk

Subscribers: hiraditya, llvm-commits

Tags: #llvm

Differential Revision: https://reviews.llvm.org/D68186

llvm-svn: 373211

show more ...

[FileCheck] Document FILECHECK_OPTS in -help

Reviewed By: thopre

Differential Revision: https://reviews.llvm.org/D65707

llvm-svn: 368787

[FileCheck] Move -dump-input diagnostic to first line

Without this patch, `-dump-input` prints a diagnostic at the end of
its marker range. For example:

```
1: Start.
check:1 ^~~~~~

[FileCheck] Move -dump-input diagnostic to first line

Without this patch, `-dump-input` prints a diagnostic at the end of
its marker range. For example:

```
1: Start.
check:1 ^~~~~~
2: Bad.
next:2 X~~~
3: Many lines
next:2 ~~~~~~~~~~
4: of input.
next:2 ~~~~~~~~~
5: End.
next:2 ~~~~ error: no match found
```

This patch moves it to the beginning like this:

```
1: Start.
check:1 ^~~~~~
2: Bad.
next:2 X~~~ error: no match found
3: Many lines
next:2 ~~~~~~~~~~
4: of input.
next:2 ~~~~~~~~~
5: End.
next:2 ~~~~
```

The former somehow looks nicer because the diagnostic doesn't appear
to be somewhere within the marker range. However, the latter is more
practical, especially when the marker range includes the remainder of
a very long dump. First, in the case of an error, this patch enables
me to search the dump for `error:` and usually immediately land where
the detected error began. Second, when trying to follow FileCheck's
logic, it's best to read top down, so this patch enables me to see
each diagnostic as soon as I encounter its marker.

Reviewed By: thopre

Differential Revision: https://reviews.llvm.org/D65702

llvm-svn: 368786

show more ...

Revert r367649: Improve raw_ostream so that you can "write" colors using operator<<

This reverts commit r367649 in an attempt to unbreak Windows bots.

llvm-svn: 367658

Improve raw_ostream so that you can "write" colors using operator<<

1. raw_ostream supports ANSI colors so that you can write messages to
the termina with colors. Previously, in order to change and

Improve raw_ostream so that you can "write" colors using operator<<

1. raw_ostream supports ANSI colors so that you can write messages to
the termina with colors. Previously, in order to change and reset
color, you had to call `changeColor` and `resetColor` functions,
respectively.

So, if you print out "error: " in red, for example, you had to do
something like this:

OS.changeColor(raw_ostream::RED);
OS << "error: ";
OS.resetColor();

With this patch, you can write the same code as follows:

OS << raw_ostream::RED << "error: " << raw_ostream::RESET;

2. Add a boolean flag to raw_ostream so that you can disable colored
output. If you disable colors, changeColor, operator<<(Color),
resetColor and other color-related functions have no effect.

Most LLVM tools automatically prints out messages using colors, and
you can disable it by passing a flag such as `--disable-colors`.
This new flag makes it easy to write code that works that way.

Differential Revision: https://reviews.llvm.org/D65564

llvm-svn: 367649

show more ...

[llvm] [FileCheck] Use FILECHECK_DUMP_INPUT_ON_FAILURE only when non-empty

Enable dumping output only if FILECHECK_DUMP_INPUT_ON_FAILURE is set to
a non-empty value. This is necessary to support di

[llvm] [FileCheck] Use FILECHECK_DUMP_INPUT_ON_FAILURE only when non-empty

Enable dumping output only if FILECHECK_DUMP_INPUT_ON_FAILURE is set to
a non-empty value. This is necessary to support disabling it via
POSIX-compliant env(1) that does not support '-u' argument,
and therefore fix regression caused by r366980.

Differential Revision: https://reviews.llvm.org/D65334

llvm-svn: 367122

show more ...