[syntax] Some #includes cleanup, NFC.

[syntax] Introduce a TokenManager interface.

TokenManager defines Token interfaces for the clang syntax-tree. This is the level
of abstraction that the syntax-tree should use to operate on Tokens.

[syntax] Introduce a TokenManager interface.

TokenManager defines Token interfaces for the clang syntax-tree. This is the level
of abstraction that the syntax-tree should use to operate on Tokens.

It decouples the syntax-tree from a particular token implementation (TokenBuffer
previously). This enables us to use a different underlying token implementation
for the syntax Leaf node -- in clang pseudoparser, we want to produce a
syntax-tree with its own pseudo::Token rather than syntax::Token.

Differential Revision: https://reviews.llvm.org/D128411

show more ...

[clang][syntax] Replace `std::vector<bool>` use

LLVM Programmer’s Manual strongly discourages the use of `std::vector<bool>` and suggests `llvm::BitVector` as a possible replacement.

This patch rep

[clang][syntax] Replace `std::vector<bool>` use

LLVM Programmer’s Manual strongly discourages the use of `std::vector<bool>` and suggests `llvm::BitVector` as a possible replacement.

This patch replaces `std::vector<bool>` with `llvm::BitVector` in the Syntax library and replaces range-based for loop with regular for loop. This is necessary due to `llvm::BitVector` not having `begin()` and `end()` (D117116).

Reviewed By: dexonsmith, dblaikie

Differential Revision: https://reviews.llvm.org/D118109

show more ...

[NFC][tools] Return underlying strings directly instead of OS.str()

This avoids an unnecessary copy required by 'return OS.str()', allowing
instead for NRVO or implicit move. The .str() call (which

[NFC][tools] Return underlying strings directly instead of OS.str()

This avoids an unnecessary copy required by 'return OS.str()', allowing
instead for NRVO or implicit move. The .str() call (which flushes the
stream) is no longer required since 65b13610a5226b84889b923bae884ba395ad084d,
which made raw_string_ostream unbuffered by default.

Differential Revision: https://reviews.llvm.org/D115374

show more ...

[clang][NFC] Inclusive terms: replace some uses of sanity in clang

Rewording of comments to avoid using `sanity test, sanity check`.

Reviewed By: aaron.ballman, Quuxplusone

Differential Revision:

[clang][NFC] Inclusive terms: replace some uses of sanity in clang

Rewording of comments to avoid using `sanity test, sanity check`.

Reviewed By: aaron.ballman, Quuxplusone

Differential Revision: https://reviews.llvm.org/D114025

show more ...

Reland [Syntax] Add minimal TableGen for syntax nodes. NFC

This reverts commit 09c6259d6d0eb51b282f6c3a28052a8146bc095b.
(Fixed side-effecting code being buried in an assert)

Revert "[Syntax] Add minimal TableGen for syntax nodes. NFC"

This reverts commit 55120f74ca12faea0e90fe552c85c14485f1fd91.

Segfaults during build:
http://lab.llvm.org:8011/#/builders/36/builds/1310

[Syntax] Add minimal TableGen for syntax nodes. NFC

So far, only used to generate Kind and implement classof().

My plan is to have this general-purpose Nodes.inc in the style of AST
DeclNodes.inc e

[Syntax] Add minimal TableGen for syntax nodes. NFC

So far, only used to generate Kind and implement classof().

My plan is to have this general-purpose Nodes.inc in the style of AST
DeclNodes.inc etc, and additionally a special-purpose backend generating
the actual class definitions. But baby steps...

Differential Revision: https://reviews.llvm.org/D90540

show more ...

[SyntaxTree] Add reverse links to syntax Nodes.

Rationale:
Children of a syntax tree had forward links only, because there was no
need for reverse links.

This need appeared when we started mutating

[SyntaxTree] Add reverse links to syntax Nodes.

Rationale:
Children of a syntax tree had forward links only, because there was no
need for reverse links.

This need appeared when we started mutating the syntax tree.
On a forward list, to remove a target node in O(1) we need a pointer to the node before the target. If we don't have this "before" pointer, we have to find it, and that requires O(n).
So in order to remove a syntax node from a tree, we would similarly need to find the node before to then remove. This is both not ergonomic nor does it have a good complexity.

Differential Revision: https://reviews.llvm.org/D90240

show more ...

[Syntax] DeclaratorList is a List

I think this was just an oversight.

Differential Revision: https://reviews.llvm.org/D90541

[Syntax] Add iterators over children of syntax trees.

This gives us slightly nicer syntax (foreach) for idioms currently expressed
as a loop, and the option to use range algorithms where it makes se

[Syntax] Add iterators over children of syntax trees.

This gives us slightly nicer syntax (foreach) for idioms currently expressed
as a loop, and the option to use range algorithms where it makes sense
(e.g. llvm::all_of et al encapsulate the needed flow control in a useful way).

It's also a building block for iteration over filtered views (e.g. iterate over
all Stmt children, with the right type):
for (const Statement &S : filter<Statement>(N.children()))
...

I realize the recent direction has been mostly towards strongly-typed
node-specific facilities, but I think it's important we have convenient
generic facilities too.

Differential Revision: https://reviews.llvm.org/D90023

show more ...

[clangd] Start using SyntaxTrees for folding ranges feature

This is an initial attempt to start using Syntax Trees in clangd while improving state of folding ranges feature and experimenting with Sy

[clangd] Start using SyntaxTrees for folding ranges feature

This is an initial attempt to start using Syntax Trees in clangd while improving state of folding ranges feature and experimenting with Syntax Tree capabilities.

Reviewed By: sammccall

Differential Revision: https://reviews.llvm.org/D88553

show more ...

[SyntaxTree][NFC] Nit on `replaceChildRangeLowLevel`

[SyntaxTree] Improve safety of `replaceChildRangeLowLevel`

* Add assertions for other preconditions.
* If nothing is modified, don't mark it.

Differential Revision: https://reviews.llvm.org/D89303

[SyntaxTree] Test `findFirstLeaf` and `findLastLeaf`

* Introduce `TreeTest.cpp` to unit test `Tree.h`
* Add `generateAllTreesWithShape` to generating test cases
* Add tests for `findFirstLeaf` and `

[SyntaxTree] Test `findFirstLeaf` and `findLastLeaf`

* Introduce `TreeTest.cpp` to unit test `Tree.h`
* Add `generateAllTreesWithShape` to generating test cases
* Add tests for `findFirstLeaf` and `findLastLeaf`
* Fix implementations of `findFirstLeaf` and `findLastLeaf` that had
been broken when empty `Tree` were present.

Differential Revision: https://reviews.llvm.org/D87779

show more ...

[SyntaxTree][NFC] follow naming convention + remove auto on empty vector declaration

Differential Revision: https://reviews.llvm.org/D88004

[SyntaxTree][List] `assertInvariants` for `List`s

Differential Revision: https://reviews.llvm.org/D87600

[SyntaxTree] Provide `List::classof`

Differential Revision: https://reviews.llvm.org/D87599

[SyntaxTree][List] Fix: `ParameterDeclarationList` is the `List` inside `ParametersAndQualifiers`

Differential Revision: https://reviews.llvm.org/D87598

[SyntaxTree] Add const qualifiers, from [llvm-qualified-auto]

Differential Revision: https://reviews.llvm.org/D87522

[SyntaxTree] Rename functions to start with verb

According to LLVM coding standards:
https://llvm.org/docs/CodingStandards.html#name-types-functions-variables-and-enumerators-properly

Differential

[SyntaxTree] Rename functions to start with verb

According to LLVM coding standards:
https://llvm.org/docs/CodingStandards.html#name-types-functions-variables-and-enumerators-properly

Differential Revision: https://reviews.llvm.org/D87498

show more ...

[SyntaxTree] Refactor `NodeRole`s

Previously a NodeRole would generally be prefixed with the `NodeKind`,
we remove this prefix, as it we redundant and made tests more noisy.

Differential Revision:

[SyntaxTree] Refactor `NodeRole`s

Previously a NodeRole would generally be prefixed with the `NodeKind`,
we remove this prefix, as it we redundant and made tests more noisy.

Differential Revision: https://reviews.llvm.org/D86636

show more ...

[SyntaxTree] Migrate `ParamatersAndQualifiers` to use the new List API

Fix: Add missing `List::getTerminationKind()`, `List::canBeEmpty()`,
`List::getDelimiterTokenKind()` for `CallArguments`.

Diff

[SyntaxTree] Migrate `ParamatersAndQualifiers` to use the new List API

Fix: Add missing `List::getTerminationKind()`, `List::canBeEmpty()`,
`List::getDelimiterTokenKind()` for `CallArguments`.

Differential Revision: https://reviews.llvm.org/D86600

show more ...

[SyntaxTree] Extend the syntax tree dump to also cover `NodeRole`

We should see `NodeRole` information in the dump because that exposes how the
accessors will behave.

Functional changes in the dump

[SyntaxTree] Extend the syntax tree dump to also cover `NodeRole`

We should see `NodeRole` information in the dump because that exposes how the
accessors will behave.

Functional changes in the dump:
* Surround Leaf tokens with `'`
* Append `Node` dumps with `NodeRole` information, except for unknown roles
* Append marks to `Node` dumps, instead of prepending

Non-functional changes:
* `::dumpTokens(llvm::raw_ostream, ArrayRef<syntax::Token>, const
SourceManager &SM)` always received as parameter a `syntax::Token *`
pointing to `Leaf::token()`. Changed the function to
`dumpLeaf(llvm::raw_ostream, syntax::Leaf *, const SourceManager&)`
* `dumpTree` acted on a Node, rename to `dumpNode`

Differential Revision: https://reviews.llvm.org/D85330

show more ...

[SyntaxTree] Implement `NestedNameSpecifier` using the `List` base API