compile commands header to source heuristic lower-cases filenames before inferring file types

This leads to ".C" files being rewritten as ".c" files and being inferred to be "c" files as opposed to

compile commands header to source heuristic lower-cases filenames before inferring file types

This leads to ".C" files being rewritten as ".c" files and being inferred to be "c" files as opposed to "c++" files.

Fixes https://github.com/clangd/clangd/issues/1108

Reviewed By: sammccall

Differential Revision: https://reviews.llvm.org/D124262

show more ...

Revert "Rename llvm::array_lengthof into llvm::size to match std::size from C++17"

This reverts commit ef8206320769ad31422a803a0d6de6077fd231d2.

- It conflicts with the existing llvm::size in STLEx

Revert "Rename llvm::array_lengthof into llvm::size to match std::size from C++17"

This reverts commit ef8206320769ad31422a803a0d6de6077fd231d2.

- It conflicts with the existing llvm::size in STLExtras, which will now
never be called.
- Calling it without llvm:: breaks C++17 compat

show more ...

Rename llvm::array_lengthof into llvm::size to match std::size from C++17

As a conquence move llvm::array_lengthof from STLExtras.h to
STLForwardCompat.h (which is included by STLExtras.h so no buil

Rename llvm::array_lengthof into llvm::size to match std::size from C++17

As a conquence move llvm::array_lengthof from STLExtras.h to
STLForwardCompat.h (which is included by STLExtras.h so no build
breakage expected).

show more ...

[Tooling] When transferring compile commands between files, always use '--'

"driver <flags> -- <input>" is a particularly convenient form of the
compile command to manipulate, with fewer special cas

[Tooling] When transferring compile commands between files, always use '--'

"driver <flags> -- <input>" is a particularly convenient form of the
compile command to manipulate, with fewer special cases to handle.

Guaranteeing that the output command is of that form is cheap and makes
it easier to consume the result in some cases.

Differential Revision: https://reviews.llvm.org/D116721

show more ...

[clang] Fix UB when string.front() is used for the empty string

Compilation database might have empty string as a command line argument.
But ExpandResponseFilesDatabase::expand doesn't expect this a

[clang] Fix UB when string.front() is used for the empty string

Compilation database might have empty string as a command line argument.
But ExpandResponseFilesDatabase::expand doesn't expect this and assumes
that string.front() can be used for any argument. It is undefined behaviour if
string is empty. With debug build mode it causes crash in clangd.

Test Plan: check-clang

Differential Revision: https://reviews.llvm.org/D105120

show more ...

[clang] Rename StringRef _lower() method calls to _insensitive()

This is mostly a mechanical change, but a testcase that contains
parts of the StringRef class (clang/test/Analysis/llvm-conventions.c

[clang] Rename StringRef _lower() method calls to _insensitive()

This is mostly a mechanical change, but a testcase that contains
parts of the StringRef class (clang/test/Analysis/llvm-conventions.cpp)
isn't touched.

show more ...

[Tooling] Handle compilation databases containing commands with double dashes

As of CMake commit https://gitlab.kitware.com/cmake/cmake/-/commit/d993ebd4,
which first appeared in CMake 3.19.x series

[Tooling] Handle compilation databases containing commands with double dashes

As of CMake commit https://gitlab.kitware.com/cmake/cmake/-/commit/d993ebd4,
which first appeared in CMake 3.19.x series, in the compile commands for
clang-cl, CMake puts `--` before the input file. When operating on such a
database, the `InterpolatingCompilationDatabase` - specifically, the
`TransferableCommand` constructor - does not recognize that pattern and so, does
not strip the input, or the double dash when 'transferring' the compile command.
This results in a incorrect compile command - with the double dash and old input
file left in, and the language options and new input file appended after them,
where they're all treated as inputs, including the language version option.

Test files for some tests have names similar enough to be matched to commands
from the database, e.g.:

`.../path-mappings.test.tmp/server/bar.cpp`

can be matched to:

`.../Driver/ToolChains/BareMetal.cpp`

etc. When that happens, the tool being tested tries to use the matched, and
incorrectly 'transferred' compile command, and fails, reporting errors similar
to:

`error: no such file or directory: '/std:c++14'; did you mean '/std:c++14'? [clang-diagnostic-error]`

This happens in at least 4 tests:

Clang Tools :: clang-tidy/checkers/performance-trivially-destructible.cpp
Clangd :: check-fail.test
Clangd :: check.test
Clangd :: path-mappings.test

The fix for `TransferableCommand` removes the `--` and everything after it when
determining the arguments that apply to the new file. `--` is inserted in the
'transferred' command if the new file name starts with `-` and when operating in
clang-cl mode, also `/`. Additionally, other places in the code known to do
argument adjustment without accounting for the `--` and causing the tests to
fail are fixed as well.

Differential Revision: https://reviews.llvm.org/D98824

show more ...

[clangd] Use flags from open files when opening headers they include

Currently our strategy for getting header compile flags is something like:

A) look for flags for the header in compile_commands.

[clangd] Use flags from open files when opening headers they include

Currently our strategy for getting header compile flags is something like:

A) look for flags for the header in compile_commands.json
This basically never works, build systems don't generate this info.
B) try to match to an impl file in compile_commands.json and use its flags
This only (mostly) works if the headers are in the same project.
C) give up and use fallback flags
This kind of works for stdlib in the default configuration, and
otherwise doesn't.

Obviously there are big gaps here.

This patch inserts a new attempt between A and B: if the header is
transitively included by any open file (whether same project or not),
then we use its compile command.

This doesn't make any attempt to solve some related problems:
- parsing non-self-contained header files in context (importing PP state)
- using the compile flags of non-opened candidate files found in the index

Fixes https://github.com/clangd/clangd/issues/123
Fixes https://github.com/clangd/clangd/issues/695
See https://github.com/clangd/clangd/issues/519

Differential Revision: https://reviews.llvm.org/D97351

show more ...

Add ability to load a FixedCompilationDatabase from a buffer.

Previously, loading one from a file meant allowing the library to do the IO.
Clangd would prefer to do such IO itself (e.g. to allow cac

Add ability to load a FixedCompilationDatabase from a buffer.

Previously, loading one from a file meant allowing the library to do the IO.
Clangd would prefer to do such IO itself (e.g. to allow caching).

Differential Revision: https://reviews.llvm.org/D92640

show more ...

[Tooling] JSONCompilationDatabase::loadFromBuffer retains the buffer, copy it.

This function doesn't seem to be used in-tree outside tests.
However clangd wants to use it soon, and having the CDB be

[Tooling] JSONCompilationDatabase::loadFromBuffer retains the buffer, copy it.

This function doesn't seem to be used in-tree outside tests.
However clangd wants to use it soon, and having the CDB be self-contained seems
reasonable.

Differential Revision: https://reviews.llvm.org/D92646

show more ...

[libTooling] Recognize sccache as a compiler wrapper in compilation database commands

sccache is a compiler caching tool similar to ccache.

Differential Revision: https://reviews.llvm.org/D88790

[clang][Tooling] Try to avoid file system access if there is no record for the file in compile_commads.json

Summary:
If there is no record in compile_commands.json, we try to find suitable record wi

[clang][Tooling] Try to avoid file system access if there is no record for the file in compile_commads.json

Summary:
If there is no record in compile_commands.json, we try to find suitable record with `MatchTrie.findEquivalent()` call.
This is very expensive operation with a lot of `llvm::sys::fs::equivalent()` calls in some cases.

This patch disables file symlinks for performance reasons.

Example scenario without this patch:
- compile_commands.json generated at clangd build (contains ~3000 files).
- it tooks more than 1 second to get compile command for newly created file in the root folder of LLVM project.
- we wait for 1 second every time when clangd requests compile command for this file (at file change).

Reviewers: sammccall, kadircet, hokein

Reviewed By: sammccall

Subscribers: chandlerc, djasper, klimek, ilya-biryukov, kadircet, usaxena95, cfe-commits

Tags: #clang

Differential Revision: https://reviews.llvm.org/D83621

show more ...

Make llvm::StringRef to std::string conversions explicit.

This is how it should've been and brings it more in line with
std::string_view. There should be no functional change here.

This is mostly m

Make llvm::StringRef to std::string conversions explicit.

This is how it should've been and brings it more in line with
std::string_view. There should be no functional change here.

This is mostly mechanical from a custom clang-tidy check, with a lot of
manual fixups. It uncovers a lot of minor inefficiencies.

This doesn't actually modify StringRef yet, I'll do that in a follow-up.

show more ...

[clang][Tooling] Prefer -x over -std when interpolating

Summary:
Currently interpolation logic prefers -std over -x. But the latter is a
more strong signal, so this patch inverts the order and only

[clang][Tooling] Prefer -x over -std when interpolating

Summary:
Currently interpolation logic prefers -std over -x. But the latter is a
more strong signal, so this patch inverts the order and only makes use of -std
if -x didn't exist.

Fixes https://github.com/clangd/clangd/issues/185

Thanks @sammccall for tracking this down!

Reviewers: sammccall

Subscribers: ilya-biryukov, usaxena95, cfe-commits, sammccall

Tags: #clang

Differential Revision: https://reviews.llvm.org/D71727

show more ...

[clang][Tooling] Add support for .rsp files in compile_commands.json

Summary:
Add support for .rsp files.

Fixes https://github.com/clangd/clangd/issues/81

Patch By: liu hui(@lh123)

Reviewers: sam

[clang][Tooling] Add support for .rsp files in compile_commands.json

Summary:
Add support for .rsp files.

Fixes https://github.com/clangd/clangd/issues/81

Patch By: liu hui(@lh123)

Reviewers: sammccall, ilya-biryukov, hokein, kadircet

Reviewed By: kadircet

Subscribers: merge_guards_bot, mgorny, MaskRay, jkorous, arphaman, kadircet, usaxena95, cfe-commits

Tags: #clang-tools-extra, #clang

Differential Revision: https://reviews.llvm.org/D70222

show more ...

Remove unused variables.

llvm-svn: 369965

[Clang] Migrate llvm::make_unique to std::make_unique

Now that we've moved to C++14, we no longer need the llvm::make_unique
implementation from STLExtras.h. This patch is a mechanical replacement
o

[Clang] Migrate llvm::make_unique to std::make_unique

Now that we've moved to C++14, we no longer need the llvm::make_unique
implementation from STLExtras.h. This patch is a mechanical replacement
of (hopefully) all the llvm::make_unique instances across the monorepo.

Differential revision: https://reviews.llvm.org/D66259

llvm-svn: 368942

show more ...

Re-land [JSONCompilationDatabase] Strip distcc/ccache/gomacc wrappers from parsed commands.

Use //net/dir like other test cases for windows compatibility

llvm-svn: 365975

Revert "[JSONCompilationDatabase] Strip distcc/ccache/gomacc wrappers from parsed commands."

New test is failing on Windows bot

This reverts commit 9c0391b36a76f8e3949588de3f44b7314c2318bf.

llvm-s

Revert "[JSONCompilationDatabase] Strip distcc/ccache/gomacc wrappers from parsed commands."

New test is failing on Windows bot

This reverts commit 9c0391b36a76f8e3949588de3f44b7314c2318bf.

llvm-svn: 365906

show more ...

[JSONCompilationDatabase] Strip distcc/ccache/gomacc wrappers from parsed commands.

Summary:
It's common to use compiler wrappers by setting CC="gomacc clang++".
This results in both args appearing

[JSONCompilationDatabase] Strip distcc/ccache/gomacc wrappers from parsed commands.

Summary:
It's common to use compiler wrappers by setting CC="gomacc clang++".
This results in both args appearing in compile_commands.json, and clang's driver
can't handle this.

This patch attempts to recognize this pattern (by looking for well-known
wrappers) and dropping argv0 in this case.

It conservatively ignores other cases for now:
- wrappers with unknown names
- wrappers that accept -flags
- wrappers where the compiler to use is implied (usually cc or gcc)

This is done at the JSONCompilationDatabase level rather than somewhere more
fundamental, as (hopefully) this isn't a general conceptual problem, but a messy
aspect of the nature of the ecosystem around compile_commands.json.
i.e. compilation databases more tightly tied to the build system should not have
this problem.

Reviewers: phosek, klimek

Subscribers: llvm-commits

Tags: #llvm

Differential Revision: https://reviews.llvm.org/D64297

llvm-svn: 365887

show more ...

[clang][Tooling] Fix windows build-bots after rL364386

llvm-svn: 364396

[clang][Tooling] Infer target and mode from argv[0] when using JSONCompilationDatabase

Summary:
Wraps JSON compilation database with a target and mode adding database
wrapper. So that driver can cor

[clang][Tooling] Infer target and mode from argv[0] when using JSONCompilationDatabase

Summary:
Wraps JSON compilation database with a target and mode adding database
wrapper. So that driver can correctly figure out which toolchain to use.

Note that clients that wants to make use of this target discovery mechanism
needs to link in TargetsInfos and initialize them at startup.

Reviewers: ilya-biryukov

Subscribers: mgorny, cfe-commits

Tags: #clang

Differential Revision: https://reviews.llvm.org/D63755

llvm-svn: 364386

show more ...

[Tooling] Add -x flags when inferring compile commands for files with no/invalid extension.

Summary: We treat them as headers, as the motivating case is C++ standard library.

Reviewers: kadircet

S

[Tooling] Add -x flags when inferring compile commands for files with no/invalid extension.

Summary: We treat them as headers, as the motivating case is C++ standard library.

Reviewers: kadircet

Subscribers: cfe-commits

Tags: #clang

Differential Revision: https://reviews.llvm.org/D61633

llvm-svn: 360153

show more ...

[Tooling] add a Heuristic field indicating that a CompileCommand was guessed.

Summary:
Use cases:
- a tool that dumps the heuristic used for each header in a project can
be used to evaluate chan

[Tooling] add a Heuristic field indicating that a CompileCommand was guessed.

Summary:
Use cases:
- a tool that dumps the heuristic used for each header in a project can
be used to evaluate changes to the heuristic
- we want to expose this information to users in clangd as it affects
accuracy/reliability of editor features
- express interpolation tests more directly

Reviewers: ilya-biryukov, klimek

Subscribers: ioeric, kadircet, cfe-commits

Tags: #clang

Differential Revision: https://reviews.llvm.org/D60194

llvm-svn: 357770

show more ...

[clang][Tooling] Delete dots and dotdots when generating absolute paths

Summary:
GetAllFiles interface returns absolute paths, but keeps dots and dot
dots. This patch makes those paths canonical by

[clang][Tooling] Delete dots and dotdots when generating absolute paths

Summary:
GetAllFiles interface returns absolute paths, but keeps dots and dot
dots. This patch makes those paths canonical by deleting them.

Reviewers: hokein

Subscribers: cfe-commits

Tags: #clang

Differential Revision: https://reviews.llvm.org/D59079

llvm-svn: 355678

show more ...