[include-cleaner] Include-cleaner library structure, and simplistic AST walking.

Include-cleaner is a library that uses the clang AST and preprocessor to
determine which headers are used. It will be

[include-cleaner] Include-cleaner library structure, and simplistic AST walking.

Include-cleaner is a library that uses the clang AST and preprocessor to
determine which headers are used. It will be used in clang-tidy, in
clangd, in a standalone tool at least for testing, and in out-of-tree tools.

Roughly, it walks the AST, finds referenced decls, maps these to
used sourcelocations, then to FileEntrys, then matching these against #includes.
However there are many wrinkles: dealing with macros, standard library
symbols, umbrella headers, IWYU directives etc.

It is not built on the C++20 modules concept of usage, to allow:
- use with existing non-modules codebases
- a flexible API embeddable in clang-tidy, clangd, and other tools
- avoiding a chicken-and-egg problem where include cleanups are needed
before modules can be adopted

This library is based on existing functionality in clangd that provides
an unused-include warning. However it has design changes:
- it accommodates diagnosing missing includes too (this means tracking
where references come from, not just the set of targets)
- it more clearly separates the different mappings
(symbol => location => header => include) for better testing
- it handles special cases like standard library symbols and IWYU directives
more elegantly by adding unified Location and Header types instead of
side-tables
- it will support some customization of policy where necessary (e.g.
for style questions of what constitutes a use, or to allow
both missing-include and unused-include modes to be conservative)

This patch adds the basic directory structure under clang-tools-extra
and a skeleton version of the AST traversal, which will be the central
piece.
A more end-to-end prototype is in https://reviews.llvm.org/D122677

RFC: https://discourse.llvm.org/t/rfc-lifting-include-cleaner-missing-unused-include-detection-out-of-clangd/61228

Differential Revision: https://reviews.llvm.org/D124164

show more ...

[Testing] TestAST, a helper for writing straight-line AST tests

Tests that need ASTs have to deal with the awkward control flow of
FrontendAction in some way. There are a few idioms used:
- don't b

[Testing] TestAST, a helper for writing straight-line AST tests

Tests that need ASTs have to deal with the awkward control flow of
FrontendAction in some way. There are a few idioms used:
- don't bother with unit tests, use clang -dump-ast
- create an ASTConsumer by hand, which is bulky
- use ASTMatchFinder - works pretty well if matchers are actually
needed, very strange if they are not
- use ASTUnit - this yields nice straight-line code, but ASTUnit is a
terrifically complicated library not designed for this purpose

TestAST provides a very simple way to write straight-line tests: specify
the code/flags and it provides an AST that is kept alive until the
object is destroyed.
It's loosely modeled after TestTU in clangd, which we've successfully
used for a variety of tests.

I've updated a couple of clang tests to use this helper, IMO they're clearer.

Differential Revision: https://reviews.llvm.org/D123668

show more ...