[Reland][lit] Use sharding for GoogleTest format

This helps lit unit test performance by a lot, especially on windows. The performance gain comes from launching one gtest executable for many subtest

[Reland][lit] Use sharding for GoogleTest format

This helps lit unit test performance by a lot, especially on windows. The performance gain comes from launching one gtest executable for many subtests instead of one (this is the current situation).

The shards are executed by the test runner and the results are stored in the
json format supported by the GoogleTest. Later in the test reporting stage,
all test results in the json file are retrieved to continue the test results
summary etc.

On my Win10 desktop, before this patch: `check-clang-unit`: 177s, `check-llvm-unit`: 38s; after this patch: `check-clang-unit`: 37s, `check-llvm-unit`: 11s.
On my Linux machine, before this patch: `check-clang-unit`: 46s, `check-llvm-unit`: 8s; after this patch: `check-clang-unit`: 7s, `check-llvm-unit`: 4s.

Reviewed By: yln, rnk, abrachet

Differential Revision: https://reviews.llvm.org/D122251

show more ...

Revert "Reland "[lit] Use sharding for GoogleTest format""

This reverts commit 948f3deca91a66caf4a618f826ff6de8277bed9c.

Reland "[lit] Use sharding for GoogleTest format"

This relands commit a87ba5c86d5d72defdbcdb278baad6515ec99463.

Adjust llvm/utils/lit/tests/googletest-timeout.py for new test output.

Revert "[lit] Use sharding for GoogleTest format"

This reverts commit a87ba5c86d5d72defdbcdb278baad6515ec99463.

Breaks bots:
https://lab.llvm.org/buildbot/#/builders/196/builds/10454

[lit] Use sharding for GoogleTest format

This helps lit unit test performance by a lot, especially on windows. The performance gain comes from launching one gtest executable for many subtests instea

[lit] Use sharding for GoogleTest format

This helps lit unit test performance by a lot, especially on windows. The performance gain comes from launching one gtest executable for many subtests instead of one (this is the current situation).

The shards are executed by the test runner and the results are stored in the
json format supported by the GoogleTest. Later in the test reporting stage,
all test results in the json file are retrieved to continue the test results
summary etc.

On my Win10 desktop, before this patch: `check-clang-unit`: 177s, `check-llvm-unit`: 38s; after this patch: `check-clang-unit`: 37s, `check-llvm-unit`: 11s.
On my Linux machine, before this patch: `check-clang-unit`: 46s, `check-llvm-unit`: 8s; after this patch: `check-clang-unit`: 7s, `check-llvm-unit`: 4s.

Reviewed By: yln, rnk

Differential Revision: https://reviews.llvm.org/D122251

show more ...

[Support] Don't print stacktrace if DbgHelp.dll hasn't been loaded yet

On Windows certain function from `Signals.h` require that `DbgHelp.dll` is loaded. This typically happens when the main program

[Support] Don't print stacktrace if DbgHelp.dll hasn't been loaded yet

On Windows certain function from `Signals.h` require that `DbgHelp.dll` is loaded. This typically happens when the main program calls `llvm::InitLLVM`, however in some cases main program doesn't do that (e.g. when the application is using LLDB via `liblldb.dll`). This patch adds a safe guard to prevent crashes. More discussion in
https://reviews.llvm.org/D119009.

Reviewed By: aganea

Differential Revision: https://reviews.llvm.org/D119181

show more ...

[NFCI][Support] Avoid ASSERT_/EXPECT_TRUE(A <op> B)

The error messages in tests are far better when a test fails if the test
is written using ASSERT_/EXPECT_<operator>(A, B) rather than
ASSERT_/EXPE

[NFCI][Support] Avoid ASSERT_/EXPECT_TRUE(A <op> B)

The error messages in tests are far better when a test fails if the test
is written using ASSERT_/EXPECT_<operator>(A, B) rather than
ASSERT_/EXPECT_TRUE(A <operator> B).

This commit updates all of llvm/unittests/Support to use these macros
where possible.

This change has not been possible in:
- llvm/unittests/Support/FSUniqueIDTest.cpp - due to not overloading
operators beyond ==, != and <.
- llvm/unittests/Support/BranchProbabilityTest.cpp - where the unchanged
tests are of the operator overloads themselves.

There are other possibilities of this conversion not being valid, which
have not applied in these tests, as they do not use NULL (they use
nullptr), and they do not use const char* (they use std::string or
StringRef).

Reviewed By: mubashar_

Differential Revision: https://reviews.llvm.org/D117319

show more ...

[Support] [Windows] Convert paths to the preferred form

This normalizes most paths (except ones input from the user as command
line arguments) into the preferred form, if `real_style()` evaluates to

[Support] [Windows] Convert paths to the preferred form

This normalizes most paths (except ones input from the user as command
line arguments) into the preferred form, if `real_style()` evaluates to
`windows_forward`.

Differential Revision: https://reviews.llvm.org/D111880

show more ...

[RGT] Tweak test so assertion is always executed

Any given Windows system will have only one "system" encoding for
UTF-16 (BE or LE), so the assert for the other one would always
show up as rotten.

[RGT] Tweak test so assertion is always executed

Any given Windows system will have only one "system" encoding for
UTF-16 (BE or LE), so the assert for the other one would always
show up as rotten. Use a common assertion for both paths to avoid
this.

show more ...

[Support] Add file lock/unlock functions

This is recommit of f51bc4fb60fb, reverted in 8577595e03fa, because
the function `flock` is not available on Solaris. In this variant
`flock` was replaced wi

[Support] Add file lock/unlock functions

This is recommit of f51bc4fb60fb, reverted in 8577595e03fa, because
the function `flock` is not available on Solaris. In this variant
`flock` was replaced with `fcntl`, which is a POSIX function.

New functions `lockFile`, `tryLockFile` and `unlockFile` implement
simple file locking. They lock or unlock entire file. This must be
enough to support simulataneous writes to log files in parallel builds.

Differential Revision: https://reviews.llvm.org/D78896

show more ...

[llvm][unittests] Fix ProgramEnvTest.TestExecuteAndWaitStatistics on Solaris

The new `LLVM-Unit :: Support/./SupportTests/ProgramEnvTest.TestExecuteAndWaitStatistics` test currently FAILs on Solaris

[llvm][unittests] Fix ProgramEnvTest.TestExecuteAndWaitStatistics on Solaris

The new `LLVM-Unit :: Support/./SupportTests/ProgramEnvTest.TestExecuteAndWaitStatistics` test currently FAILs on Solaris:

[ RUN ] ProgramEnvTest.TestExecuteAndWaitStatistics
/vol/llvm/src/llvm-project/local/llvm/unittests/Support/ProgramTest.cpp:360: Failure
Expected: (ProcStat->PeakMemory) > (0U), actual: 0 vs 0
[ FAILED ] ProgramEnvTest.TestExecuteAndWaitStatistics (22 ms)

According to `llvm/lib/Support/Unix/Program.inc (llvm::sys::Wait)`, `PeakMemory`
corresponds to `struct rusage.ru_maxrss`.

However, Solaris `getrusage(3C)` documents

NOTES
The ru_maxrss, ru_ixrss, ru_idrss, and ru_isrss members of the rusage
structure are set to 0 in this implementation.

Since changing the test to check for `PeakMemory >= 0` instead is pointless
and would generate a warning on targets where `ru_maxrss` is unsigned, this
patch removes the check.

Tested on `amd64-pc-solaris2.11`.

Differential Revision: https://reviews.llvm.org/D83661

show more ...

Use explicitly unsigned zero to prevent from a warning

[Support] Get process statistics in ExecuteAndWait and Wait

The functions sys::ExcecuteAndWait and sys::Wait now have additional
argument of type pointer to structure, which is filled with process
e

[Support] Get process statistics in ExecuteAndWait and Wait

The functions sys::ExcecuteAndWait and sys::Wait now have additional
argument of type pointer to structure, which is filled with process
execution statistics upon process termination. These are total and user
execution times and peak memory consumption. By default this argument is
nullptr so existing users of these function must not change behavior.

Differential Revision: https://reviews.llvm.org/D78901

show more ...

Revert "raw_ostream: add operator<< overload for std::error_code"

This reverts commit r368849, because it breaks some bots (e.g.
llvm-clang-x86_64-win-fast).

It turns out this is not as NFC as we h

Revert "raw_ostream: add operator<< overload for std::error_code"

This reverts commit r368849, because it breaks some bots (e.g.
llvm-clang-x86_64-win-fast).

It turns out this is not as NFC as we had hoped, because operator== will
consider two std::error_codes to be distinct even though they both hold
"success" values if they have different categories.

llvm-svn: 368854

show more ...

raw_ostream: add operator<< overload for std::error_code

Summary:
The main motivation for this is unit tests, which contain a large macro
for pretty-printing std::error_code, and this macro is dupli

raw_ostream: add operator<< overload for std::error_code

Summary:
The main motivation for this is unit tests, which contain a large macro
for pretty-printing std::error_code, and this macro is duplicated in
every file that needs to do this. However, the functionality may be
useful elsewhere too.

In this patch I have reimplemented the existing ASSERT_NO_ERROR macros
to reuse the new functionality, but I have kept the macro (as a
one-liner) as it is slightly more readable than ASSERT_EQ(...,
std::error_code()).

Reviewers: sammccall, ilya-biryukov

Subscribers: zturner, llvm-commits

Tags: #llvm

Differential Revision: https://reviews.llvm.org/D65643

llvm-svn: 368849

show more ...

Update the file headers across all of the LLVM projects in the monorepo
to reflect the new license.

We understand that people may be surprised that we're moving the header
entirely to discuss the ne

Update the file headers across all of the LLVM projects in the monorepo
to reflect the new license.

We understand that people may be surprised that we're moving the header
entirely to discuss the new license. We checked this carefully with the
Foundation's lawyer and we believe this is the correct approach.

Essentially, all code in the project is now made available by the LLVM
project under our new license, so you will see that the license headers
include that license only. Some of our contributors have contributed
code under our old license, and accordingly, we have retained a copy of
our old license notice in the top-level files in each project and
repository.

llvm-svn: 351636

show more ...

Refactor ExecuteAndWait to take StringRefs.

This simplifies some code which had StringRefs to begin with, and
makes other code more complicated which had const char* to begin
with.

In the end, I th

Refactor ExecuteAndWait to take StringRefs.

This simplifies some code which had StringRefs to begin with, and
makes other code more complicated which had const char* to begin
with.

In the end, I think this makes for a more idiomatic and platform
agnostic API. Not all platforms launch process with null terminated
c-string arrays for the environment pointer and argv, but the api
was designed that way because it allowed easy pass-through for
posix-based platforms. There's a little additional overhead now
since on posix based platforms we'll be takign StringRefs which
were constructed from null terminated strings and then copying
them to null terminate them again, but from a readability and
usability standpoint of the API user, I think this API signature
is strictly better.

llvm-svn: 334518

show more ...

IWYU for llvm-config.h in llvm, additions.

See r331124 for how I made a list of files missing the include.
I then ran this Python script:

for f in open('filelist.txt'):
f = f.strip()

IWYU for llvm-config.h in llvm, additions.

See r331124 for how I made a list of files missing the include.
I then ran this Python script:

for f in open('filelist.txt'):
f = f.strip()
fl = open(f).readlines()

found = False
for i in xrange(len(fl)):
p = '#include "llvm/'
if not fl[i].startswith(p):
continue
if fl[i][len(p):] > 'Config':
fl.insert(i, '#include "llvm/Config/llvm-config.h"\n')
found = True
break
if not found:
print 'not found', f
else:
open(f, 'w').write(''.join(fl))

and then looked through everything with `svn diff | diffstat -l | xargs -n 1000 gvim -p`
and tried to fix include ordering and whatnot.

No intended behavior change.

llvm-svn: 331184

show more ...

s/LLVM_ON_WIN32/_WIN32/, llvm

LLVM_ON_WIN32 is set exactly with MSVC and MinGW (but not Cygwin) in
HandleLLVMOptions.cmake, which is where _WIN32 defined too. Just use the
default macro instead of

s/LLVM_ON_WIN32/_WIN32/, llvm

LLVM_ON_WIN32 is set exactly with MSVC and MinGW (but not Cygwin) in
HandleLLVMOptions.cmake, which is where _WIN32 defined too. Just use the
default macro instead of a reinvented one.

See thread "Replacing LLVM_ON_WIN32 with just _WIN32" on llvm-dev and cfe-dev.
No intended behavior change.

This moves over all uses of the macro, but doesn't remove the definition
of it in (llvm-)config.h yet.

llvm-svn: 331127

show more ...

Convenience/safety fix for llvm::sys::Execute(And|No)Wait

Summary:
Change the type of the Redirects parameter of llvm::sys::ExecuteAndWait,
ExecuteNoWait and other APIs that wrap them from `const St

Convenience/safety fix for llvm::sys::Execute(And|No)Wait

Summary:
Change the type of the Redirects parameter of llvm::sys::ExecuteAndWait,
ExecuteNoWait and other APIs that wrap them from `const StringRef **` to
`ArrayRef<Optional<StringRef>>`, which is safer and simplifies the use of these
APIs (no more local StringRef variables just to get a pointer to).

Corresponding clang changes will be posted as a separate patch.

Reviewers: bkramer

Reviewed By: bkramer

Subscribers: vsk, llvm-commits

Differential Revision: https://reviews.llvm.org/D37563

llvm-svn: 313155

show more ...

Re-sort #include lines for unittests. This uses a slightly modified
clang-format (https://reviews.llvm.org/D33932) to keep primary headers
at the top and handle new utility headers like 'gmock' consi

Re-sort #include lines for unittests. This uses a slightly modified
clang-format (https://reviews.llvm.org/D33932) to keep primary headers
at the top and handle new utility headers like 'gmock' consistently with
other utility headers.

No other change was made. I did no manual edits, all of this is
clang-format.

This should allow other changes to have more clear and focused diffs,
and is especially motivated by moving some headers into more focused
libraries.

llvm-svn: 304786

show more ...

Remove unused lambda captures. NFC

llvm-svn: 291916

Resubmit "Update llvm command line parser to support subcommands."

This fixes an issue where occurrence counts would be unexpectedly
reset when parsing different parts of a command line multiple
tim

Resubmit "Update llvm command line parser to support subcommands."

This fixes an issue where occurrence counts would be unexpectedly
reset when parsing different parts of a command line multiple
times.

**ORIGINAL COMMIT MESSAGE**

This allows command line tools to use syntaxes like the following:

llvm-foo.exe command1 -o1 -o2
llvm-foo.exe command2 -p1 -p2

Where command1 and command2 contain completely different sets of
valid options. This is backwards compatible with previous uses
of llvm cl which did not support subcommands, as any option
which specifies no optional subcommand (e.g. all existing
code) goes into a special "top level" subcommand that expects
dashed options to appear immediately after the program name.
For example, code which is subcommand unaware would generate
a command line such as the following, where no subcommand
is specified:

llvm-foo.exe -q1 -q2

The top level subcommand can co-exist with actual subcommands,
as it is implemented as an actual subcommand which is searched
if no explicit subcommand is specified. So llvm-foo.exe as
specified above could be written so as to support all three
aforementioned command lines simultaneously.

There is one additional "special" subcommand called AllSubCommands,
which can be used to inject an option into every subcommand.
This is useful to support things like help, so that commands
such as:

llvm-foo.exe --help
llvm-foo.exe command1 --help
llvm-foo.exe command2 --help

All work and display the help for the selected subcommand
without having to explicitly go and write code to handle each
one separately.

This patch is submitted without an example of anything actually
using subcommands, but a followup patch will convert the
llvm-pdbdump tool to use subcommands.

Reviewed By: beanz

llvm-svn: 274171

show more ...

Revert r274054 to try to appease the bot

llvm-svn: 274072

Update llvm command line parser to support subcommands.

This allows command line tools to use syntaxes like the following:

llvm-foo.exe command1 -o1 -o2
llvm-foo.exe command2 -p1 -p2

Where com

Update llvm command line parser to support subcommands.

This allows command line tools to use syntaxes like the following:

llvm-foo.exe command1 -o1 -o2
llvm-foo.exe command2 -p1 -p2

Where command1 and command2 contain completely different sets of
valid options. This is backwards compatible with previous uses
of llvm cl which did not support subcommands, as any option
which specifies no optional subcommand (e.g. all existing
code) goes into a special "top level" subcommand that expects
dashed options to appear immediately after the program name.
For example, code which is subcommand unaware would generate
a command line such as the following, where no subcommand
is specified:

llvm-foo.exe -q1 -q2

The top level subcommand can co-exist with actual subcommands,
as it is implemented as an actual subcommand which is searched
if no explicit subcommand is specified. So llvm-foo.exe as
specified above could be written so as to support all three
aforementioned command lines simultaneously.

There is one additional "special" subcommand called AllSubCommands,
which can be used to inject an option into every subcommand.
This is useful to support things like help, so that commands
such as:

llvm-foo.exe --help
llvm-foo.exe command1 --help
llvm-foo.exe command2 --help

All work and display the help for the selected subcommand
without having to explicitly go and write code to handle each
one separately.

This patch is submitted without an example of anything actually
using subcommands, but a followup patch will convert the
llvm-pdbdump tool to use subcommands.

Reviewed By: beanz
Differential Revision: http://reviews.llvm.org/D21485

llvm-svn: 274054

show more ...