[syntax] Introduce a TokenManager interface.

TokenManager defines Token interfaces for the clang syntax-tree. This is the level
of abstraction that the syntax-tree should use to operate on Tokens.

[syntax] Introduce a TokenManager interface.

TokenManager defines Token interfaces for the clang syntax-tree. This is the level
of abstraction that the syntax-tree should use to operate on Tokens.

It decouples the syntax-tree from a particular token implementation (TokenBuffer
previously). This enables us to use a different underlying token implementation
for the syntax Leaf node -- in clang pseudoparser, we want to produce a
syntax-tree with its own pseudo::Token rather than syntax::Token.

Differential Revision: https://reviews.llvm.org/D128411

show more ...

Don't use Optional::hasValue (NFC)

[clang] Use llvm::{count,count_if,find_if,all_of,none_of} (NFC)

[Syntax] No crash on OpaqueValueExpr.

OpaqueValueExpr doesn't correspond to the concrete syntax, it has
invalid source location, ignore them.

Reviewed By: kbobyrev

Differential Revision: https://r

[Syntax] No crash on OpaqueValueExpr.

OpaqueValueExpr doesn't correspond to the concrete syntax, it has
invalid source location, ignore them.

Reviewed By: kbobyrev

Differential Revision: https://reviews.llvm.org/D96112

show more ...

[Syntax] Remove a strict valid source location assertion for TypeLoc.

The EndLoc of a type loc can be invalid for broken code.

Also extend the existing test to support error code with `error-ok`
an

[Syntax] Remove a strict valid source location assertion for TypeLoc.

The EndLoc of a type loc can be invalid for broken code.

Also extend the existing test to support error code with `error-ok`
annotation.

Differential Revision: https://reviews.llvm.org/D96261

show more ...

[Syntax] Support condition for IfStmt.

Differential Revision: https://reviews.llvm.org/D95782

[clang] Simplify buildSyntaxTree API

Follow-up on https://reviews.llvm.org/D88553#inline-837013

Reviewed By: sammccall

Differential Revision: https://reviews.llvm.org/D90672

[SyntaxTree] Add reverse links to syntax Nodes.

Rationale:
Children of a syntax tree had forward links only, because there was no
need for reverse links.

This need appeared when we started mutating

[SyntaxTree] Add reverse links to syntax Nodes.

Rationale:
Children of a syntax tree had forward links only, because there was no
need for reverse links.

This need appeared when we started mutating the syntax tree.
On a forward list, to remove a target node in O(1) we need a pointer to the node before the target. If we don't have this "before" pointer, we have to find it, and that requires O(n).
So in order to remove a syntax node from a tree, we would similarly need to find the node before to then remove. This is both not ergonomic nor does it have a good complexity.

Differential Revision: https://reviews.llvm.org/D90240

show more ...

[clang] Use SourceLocation as key in hash maps, NFCI

The patch adjusts the existing `llvm::DenseMap<unsigned, T>` and
`llvm::DenseSet<unsigned>` objects that store source locations, so
that they use

[clang] Use SourceLocation as key in hash maps, NFCI

The patch adjusts the existing `llvm::DenseMap<unsigned, T>` and
`llvm::DenseSet<unsigned>` objects that store source locations, so
that they use `SourceLocation` directly instead of `unsigned`.

This patch relies on the `DenseMapInfo` trait added in D89719.

It also replaces the construction of `SourceLocation` objects from
the constants -1 and -2 with calls to the trait's methods `getEmptyKey`
and `getTombstoneKey` where appropriate.

Reviewed By: dexonsmith

Differential Revision: https://reviews.llvm.org/D69840

show more ...

Migrate Declarators to use the List API

After this change all nodes that have a delimited-list are using the
`List` API.

Implementation details:
Let's look at a declaration with multiple declarator

Migrate Declarators to use the List API

After this change all nodes that have a delimited-list are using the
`List` API.

Implementation details:
Let's look at a declaration with multiple declarators:
`int a, b;`
To generate a declarator list node we need to have the range of
declarators: `a, b`:
However, the `ClangAST` actually stores them as separate declarations:
`int a ;`
`int b;`
We solve that by appropriately marking the declarators on each separate
declaration in the `ClangAST` and then for the final declarator `int
b`, shrinking its range to fit to the already marked declarators.

Differential Revision: https://reviews.llvm.org/D88403

show more ...

[SyntaxTree][NFC] follow naming convention + remove auto on empty vector declaration

Differential Revision: https://reviews.llvm.org/D88004

Fix clang Wrange-loop-analysis in BuildTree.cpp

Building on Mac OS with clang 12:

```
jhemphill@jhemphill-mbp build % clang --version
Apple clang version 12.0.0 (clang-1200.0.26.2)
Target: x86_64-a

Fix clang Wrange-loop-analysis in BuildTree.cpp

Building on Mac OS with clang 12:

```
jhemphill@jhemphill-mbp build % clang --version
Apple clang version 12.0.0 (clang-1200.0.26.2)
Target: x86_64-apple-darwin19.6.0
Thread model: posix
InstalledDir: /Library/Developer/CommandLineTools/usr/bin
```

yields one warning:

```
/Users/jhemphill/oss/llvm-project/clang/lib/Tooling/Syntax/BuildTree.cpp:1126:22: warning: loop variable 'Arg' is always a copy because the range of type 'llvm::iterator_range<clang::Stmt::CastIterator<clang::Expr, clang::Expr *, clang::Stmt *> >' does not return a reference [-Wrange-loop-analysis]
for (const auto &Arg : Args) {
^
/Users/jhemphill/oss/llvm-project/clang/lib/Tooling/Syntax/BuildTree.cpp:1126:10: note: use non-reference type 'clang::Expr *'
for (const auto &Arg : Args) {
```

It appears that `Arg` is an `Expr*`, passed by value rather than by const reference.

Reviewed By: eduucaldas, gribozavr2

Differential Revision: https://reviews.llvm.org/D87482

show more ...

[SyntaxTree] Add const qualifiers, from [llvm-qualified-auto]

Differential Revision: https://reviews.llvm.org/D87522

[SyntaxTree] Rename functions to start with verb

According to LLVM coding standards:
https://llvm.org/docs/CodingStandards.html#name-types-functions-variables-and-enumerators-properly

Differential

[SyntaxTree] Rename functions to start with verb

According to LLVM coding standards:
https://llvm.org/docs/CodingStandards.html#name-types-functions-variables-and-enumerators-properly

Differential Revision: https://reviews.llvm.org/D87498

show more ...

[SyntaxTree] Fix crash on functions with default arguments.

* Do not visit `CXXDefaultArgExpr`
* To build `CallArguments` nodes, just go through non-default arguments

Differential Revision: https:/

[SyntaxTree] Fix crash on functions with default arguments.

* Do not visit `CXXDefaultArgExpr`
* To build `CallArguments` nodes, just go through non-default arguments

Differential Revision: https://reviews.llvm.org/D87249

show more ...

[SyntaxTree] Ignore implicit `CXXFunctionalCastExpr` wrapping constructor

Differential Revision: https://reviews.llvm.org/D87229

[SyntaxTree] Ignore implicit leaf `CXXConstructExpr`

Differential Revision: https://reviews.llvm.org/D86700

[SyntaxTree] Ignore implicit non-leaf `CXXConstructExpr`

Differential Revision: https://reviews.llvm.org/D86699

[SyntaxTree] Add coverage for declarators and init-declarators

[SyntaxTree][NFC] Refactor function templates into functions taking base class

The refactored functions were
* `isReponsibleForCreatingDeclaration`
* `getQualifiedNameStart`

Differential Revision:

[SyntaxTree][NFC] Refactor function templates into functions taking base class

The refactored functions were
* `isReponsibleForCreatingDeclaration`
* `getQualifiedNameStart`

Differential Revision: https://reviews.llvm.org/D86719

show more ...

[SyntaxTree][NFC][Style] Functions start with lowercase

Differential Revision: https://reviews.llvm.org/D86682

[SyntaxTree] Refactor `NodeRole`s

Previously a NodeRole would generally be prefixed with the `NodeKind`,
we remove this prefix, as it we redundant and made tests more noisy.

Differential Revision:

[SyntaxTree] Refactor `NodeRole`s

Previously a NodeRole would generally be prefixed with the `NodeKind`,
we remove this prefix, as it we redundant and made tests more noisy.

Differential Revision: https://reviews.llvm.org/D86636

show more ...

[SyntaxTree] Migrate `ParamatersAndQualifiers` to use the new List API

Fix: Add missing `List::getTerminationKind()`, `List::canBeEmpty()`,
`List::getDelimiterTokenKind()` for `CallArguments`.

Diff

[SyntaxTree] Migrate `ParamatersAndQualifiers` to use the new List API

Fix: Add missing `List::getTerminationKind()`, `List::canBeEmpty()`,
`List::getDelimiterTokenKind()` for `CallArguments`.

Differential Revision: https://reviews.llvm.org/D86600

show more ...

[SyntaxTree] Add support for `CallExpression`

* Generate `CallExpression` syntax node for all semantic nodes inheriting from
`CallExpr` with call-expression syntax - except `CUDAKernelCallExpr`.
* I

[SyntaxTree] Add support for `CallExpression`

* Generate `CallExpression` syntax node for all semantic nodes inheriting from
`CallExpr` with call-expression syntax - except `CUDAKernelCallExpr`.
* Implement all the accessors
* Arguments of `CallExpression` have their own syntax node which is based on
the `List` base API

Differential Revision: https://reviews.llvm.org/D86544

show more ...

[SyntaxTree] Extend the syntax tree dump to also cover `NodeRole`

We should see `NodeRole` information in the dump because that exposes how the
accessors will behave.

Functional changes in the dump

[SyntaxTree] Extend the syntax tree dump to also cover `NodeRole`

We should see `NodeRole` information in the dump because that exposes how the
accessors will behave.

Functional changes in the dump:
* Surround Leaf tokens with `'`
* Append `Node` dumps with `NodeRole` information, except for unknown roles
* Append marks to `Node` dumps, instead of prepending

Non-functional changes:
* `::dumpTokens(llvm::raw_ostream, ArrayRef<syntax::Token>, const
SourceManager &SM)` always received as parameter a `syntax::Token *`
pointing to `Leaf::token()`. Changed the function to
`dumpLeaf(llvm::raw_ostream, syntax::Leaf *, const SourceManager&)`
* `dumpTree` acted on a Node, rename to `dumpNode`

Differential Revision: https://reviews.llvm.org/D85330

show more ...