[lldb] Parse the dotest output to determine the most appropriate result code

Currently we look for keywords in the dotest.py output to determine the
lit result code. This binary approach of a keywor

[lldb] Parse the dotest output to determine the most appropriate result code

Currently we look for keywords in the dotest.py output to determine the
lit result code. This binary approach of a keyword being present works
for PASS and FAIL, where having at least one test pass or fail
respectively results in that exit code. Things are more complicated
for tests that neither passed or failed, but report a combination of
(un)expected failures, skips or unresolved tests.

This patch changes the logic to parse the number of tests with a
particular result from the dotest.py output. For tests that did not PASS
or FAIL, we now report the lit result code for the one that occurred the
most. For example, if we had a test with 3 skips and 4 expected
failures, we report the test as XFAIL.

We're still mapping multiple tests to one result code, so some loss of
information is inevitable.

Differential revision: https://reviews.llvm.org/D127258

show more ...

Remove Python 2 checks from the test suite

We dropped downstream support for Python 2 in the previous release. Now
that we have branched for the next release the window where this kind of
change cou

Remove Python 2 checks from the test suite

We dropped downstream support for Python 2 in the previous release. Now
that we have branched for the next release the window where this kind of
change could introduce conflicts is closing too. Remove Python 2 checks
from the test suite.

Differential revision: https://reviews.llvm.org/D124429

show more ...

[lldb/lua] Supplement Lua bindings for lldb module

Add necessary typemaps for Lua bindings, together with some other files.

Signed-off-by: Siger Yang <[email protected]>

Reviewed By: tammela

D

[lldb/lua] Supplement Lua bindings for lldb module

Add necessary typemaps for Lua bindings, together with some other files.

Signed-off-by: Siger Yang <[email protected]>

Reviewed By: tammela

Differential Revision: https://reviews.llvm.org/D108090

show more ...

[lldb] Remove support for replaying the test suite from a reproducer

This patch removes the infrastructure to replay the test suite from a
reproducer, as well as the modifications made to the indivi

[lldb] Remove support for replaying the test suite from a reproducer

This patch removes the infrastructure to replay the test suite from a
reproducer, as well as the modifications made to the individual tests.

show more ...

[lldb] Make the lit configuration values optional for the API tests

LIT uses a model where the test suite is configurable trough a
lit.site.cfg file. Most of the time we use the lit.site.cfg with va

[lldb] Make the lit configuration values optional for the API tests

LIT uses a model where the test suite is configurable trough a
lit.site.cfg file. Most of the time we use the lit.site.cfg with values
that match the current build configuration, generated by CMake.

Nothing prevents you from running the test suite with a different
configuration, either by overriding some of these values from the
command line, or by passing a different lit.site.cfg.

The latter is currently tedious. Many configuration values are optional
but they still need to be set because lit.cfg.py is accessing them
directly. This patch changes the code to use getattr to return the
attribute if it exists. This makes it possible to specify a minimal
lit.site.cfg with only the mandatory/desired configuration values.

Differential revision: https://reviews.llvm.org/D86821

show more ...

[lldb/Test] Fix ASan/TSan workaround for Xcode Python 3

The Python 3 interpreter in Xcode has a relative RPATH and dyld fails to
load it when we copy it into the build directory.

This patch adds an

[lldb/Test] Fix ASan/TSan workaround for Xcode Python 3

The Python 3 interpreter in Xcode has a relative RPATH and dyld fails to
load it when we copy it into the build directory.

This patch adds an additional check that the copied binary can be
executed. If it doesn't, we assume we're dealing with the Xcode python
interpreter and return the path to the real executable. That is
sufficient for the sanitizers because only system binaries need to be
copied to work around SIP.

This patch also moves all that logic out of LLDBTest and into the lit
configuration so that it's executed only once per test run, instead of
once for every test. Although I didn't benchmark the difference this
should result in a mild speedup.

Differential revision: https://reviews.llvm.org/D81696

show more ...

[lldb/test] Fix for flakiness in TestNSDictionarySynthetic

Summary:
TestNSDictionarySynthetic sets up an NSURL which does not initialize its
_baseURL member. When the test runs and we print out the

[lldb/test] Fix for flakiness in TestNSDictionarySynthetic

Summary:
TestNSDictionarySynthetic sets up an NSURL which does not initialize its
_baseURL member. When the test runs and we print out the NSURL, we print
out some garbage memory pointed-to by the _baseURL member, like:

```
_baseURL = 0x0800010020004029 @"d��qX"
```

and this can cause a python unicode decoding error like:

```
UnicodeDecodeError: 'utf8' codec can't decode byte 0xa0 in position
10309: invalid start byte
```

There's a discrepancy here because lldb's StringPrinter facility tries
to only print out "printable" sequences (see: isprint32()), whereas python
rejects the StringPrinter output as invalid utf8. For the specific error
seen above, lldb's `isprint32(0xa0) = true`, even though 0xa0 is not
really "printable" in the usual sense.

The problem is that lldb and python disagree on what exactly is
"printable". Both have dismayingly hand-rolled utf8 validation code
(c.f. _Py_DecodeUTF8Ex), and I can't really tell which one is more
correct.

I tried replacing lldb's isprint32() with a call to libc's iswprint():
this satisfied python, but broke emoji printing :|.

Now, I believe that lldb (and python too) ought to just call into some
battle-tested utf library, and that we shouldn't aim for compatibility
with python's strict unicode decoding mode until then.

FWIW I ran this test under an ASanified lldb hundreds of times but
didn't turn up any other issues.

rdar://62941711

Reviewers: JDevlieghere, jingham, shafik

Subscribers: lldb-commits

Tags: #lldb

Differential Revision: https://reviews.llvm.org/D79645

show more ...

[lldb/test][Darwin] Ask dyld where the real python is

Summary:
On macOS, we can't do the DYLD_INSERT_LIBRARIES trick with a shim
python binary as the ASan interceptors get loaded too late. Find the

[lldb/test][Darwin] Ask dyld where the real python is

Summary:
On macOS, we can't do the DYLD_INSERT_LIBRARIES trick with a shim
python binary as the ASan interceptors get loaded too late. Find the
"real" python binary, copy it, and invoke it.

Hopefully this makes the GreenDragon and swift-ci sanitizer bots
happy...

I tested this out by running `../llvm-macosx-x86_64/bin/llvm-lit test
--filter TestNSDictionarySynthetic.py` in an ASanified swift-lldb build
directory and it worked (i.e. no more "interceptors loaded too late"
messages).

Reviewers: JDevlieghere

Subscribers: lldb-commits

Tags: #lldb

Differential Revision: https://reviews.llvm.org/D79607

show more ...

[lldb/Test] Run dotest.py with the Python LLDB was built with.

The Python used to run lit can be different from the Python LLDB was
build with. One scenario where this happens is when LLVM can find

[lldb/Test] Run dotest.py with the Python LLDB was built with.

The Python used to run lit can be different from the Python LLDB was
build with. One scenario where this happens is when LLVM can find the
Python 3 interpreter, but not the Python 3 libraries, in which case LLDB
build and links against Python 3. Without this change, you end up with
an ModuleNotFoundError because of the mismatch.

Instead of looking at the Python interpreter that's used to run lit,
lldbtest should use the interpreter that matches the Python version LLDB
was build against.

Differential revision: https://reviews.llvm.org/D79519

show more ...

[lldb/Test] Decode stdout and stderr in case it contains Unicode.

Lit's to_string will just return the string when it's a `str` instance,
which in Python 2 can still contain UTF-8 characters.

Diffe

[lldb/Test] Decode stdout and stderr in case it contains Unicode.

Lit's to_string will just return the string when it's a `str` instance,
which in Python 2 can still contain UTF-8 characters.

Differential revision: https://reviews.llvm.org/D76955

show more ...

[lldb/Reproducers] Capture reproducers from the API test suite.

Make it possible to capture reproducers from the API test suite. Given
the symmetry between capture and replay, this patch also adds t

[lldb/Reproducers] Capture reproducers from the API test suite.

Make it possible to capture reproducers from the API test suite. Given
the symmetry between capture and replay, this patch also adds the
necessary code for replay. For now this is a NO-OP until the
corresponding reproducer instrumentation changes land.

For more info please refer to the RFC on lldb-dev:
http://lists.llvm.org/pipermail/lldb-dev/2020-April/016100.html

Differential revision: https://reviews.llvm.org/D77588

show more ...

[lldb] Fix that a crashing test is marked as unsupported when it prints UNSUPPORTED before crashing

Summary:
I added an `abort()` call to some code and noticed that the test suite was still passing

[lldb] Fix that a crashing test is marked as unsupported when it prints UNSUPPORTED before crashing

Summary:
I added an `abort()` call to some code and noticed that the test suite was still passing and it just marked my test as "UNSUPPORTED".

It seems the reason for that is that we expect failing tests to print "FAIL:" which doesn't happen when we crash. If we then also
have an unsupported because we skipped some debug information in the output, we just mark the test passing because it is unsupported
on the current platform.

This patch marks any test that has a non-zero exit code as failing even if it doesn't print "FAIL:" (e.g., because it crashed).

Reviewers: labath, JDevlieghere

Reviewed By: labath, JDevlieghere

Subscribers: aprantl, lldb-commits

Tags: #lldb

Differential Revision: https://reviews.llvm.org/D75031

show more ...

[lldb/Lit] Fix UnboundLocalError when reaching a timeout.

Fixes the UnboundLocalError for the local variables out, err and
exitCode when a timeout is hit.

[lldb/Lit] Change the lldbtest format to behave more like shell test.

The current lldbtest format has a number of shortcomings, all related to
how we omit information based on why the test fails. Fo

[lldb/Lit] Change the lldbtest format to behave more like shell test.

The current lldbtest format has a number of shortcomings, all related to
how we omit information based on why the test fails. For example, a
successful test would print nothing, even when `-a` is passed to lit.
It's not up to the test format to decide whether to print something or
not, that's handled by lit itself. For other test results we would
sometimes print stdout & stderr, but not always, such as when a timeout
was reached or we couldn't parse the dotest output.

This patch changes the lldbtest format and makes it behave more like
lit. We now always print the dotest invocation, the exit code, the
output to stdout & stderr. If you're used to dealing with ShTests in
lit, this will feel all very familiar.

Differential revision: https://reviews.llvm.org/D73384

show more ...

[lldb/CMake] Rename LLDB_DISABLE_PYTHON to LLDB_ENABLE_PYTHON

This matches the naming scheme used by LLVM and all the other optional
dependencies in LLDB.

Differential revision: https://reviews.llv

[lldb/CMake] Rename LLDB_DISABLE_PYTHON to LLDB_ENABLE_PYTHON

This matches the naming scheme used by LLVM and all the other optional
dependencies in LLDB.

Differential revision: https://reviews.llvm.org/D71482

show more ...

[lldb] Make Asan/SIP workaround work for Python 3

Make the check generic instead of hard-coding the path to Python 2. This
also fixes the print-syntax to be compatible with both versions.

[lldb] Prevent Asan/SIP workaround from affecting Python in /usr/local/bin

The code that works around SIP was unintentionally being triggered for
/usr/local/bin/python as well. That caused trouble o

[lldb] Prevent Asan/SIP workaround from affecting Python in /usr/local/bin

The code that works around SIP was unintentionally being triggered for
/usr/local/bin/python as well. That caused trouble on GreenDragon where
we were swapping out a Python 3 executable with the system's Python 2
executable.

show more ...

Re-land "[test] Split LLDB tests into API, Shell & Unit"

The original patch got reverted because it broke `check-lldb` on a clean
build. This fixes that.

llvm-svn: 374201

[test] Split LLDB tests into API, Shell & Unit

LLDB has three major testing strategies: unit tests, tests that exercise
the SB API though dotest.py and what we currently call lit tests. The
later is

[test] Split LLDB tests into API, Shell & Unit

LLDB has three major testing strategies: unit tests, tests that exercise
the SB API though dotest.py and what we currently call lit tests. The
later is rather confusing as we're now using lit as the driver for all
three types of tests. As most of this grew organically, the directory
structure in the LLDB repository doesn't really make this clear.

The 'lit' tests are part of the root and among these tests there's a
Unit and Suite folder for the unit and dotest-tests. This layout makes
it impossible to run just the lit tests.

This patch changes the directory layout to match the 3 testing
strategies, each with their own directory and their own configuration
file. This means there are now 3 directories under lit with 3
corresponding targets:

- API (check-lldb-api): Test exercising the SB API.
- Shell (check-lldb-shell): Test exercising command line utilities.
- Unit (check-lldb-unit): Unit tests.

Finally, there's still the `check-lldb` target that runs all three test
suites.

Finally, this also renames the lit folder to `test` to match the LLVM
repository layout.

Differential revision: https://reviews.llvm.org/D68606

llvm-svn: 374184

show more ...