[clang-format] Fix a hang when formatting C# $@ string literals

Fixes #56624.

Differential Revision: https://reviews.llvm.org/D130411

[clang-format] Fix incorrect isspace input (NFC)

This change fixes a clang-format unit test failure introduced by [D124748](https://reviews.llvm.org/D124748). The `countLeadingWhitespace` function w

[clang-format] Fix incorrect isspace input (NFC)

This change fixes a clang-format unit test failure introduced by [D124748](https://reviews.llvm.org/D124748). The `countLeadingWhitespace` function was calling `isspace` with values that could fall outside the valid input range. The valid input range for `isspace` is unsigned 0-255. Values outside this range produce undefined behavior, which on Windows manifests as an assertion being raised in the debug runtime libraries. `countLeadingWhitespace` was calling `isspace` with a signed char that could produce a negative value if the underlying byte's value was 128 or above, which can happen for non-ASCII encodings. The fix is to use `StringRef`'s `bytes_begin` and `bytes_end` iterators to read the values as unsigned chars instead.

This bug can be reproduced by building the `check-clang-unit` target with a DEBUG configuration under Windows. This change is already covered by existing unit tests.

Reviewed By: MyDeveloperDay

Differential Revision: https://reviews.llvm.org/D128786

show more ...

[clang-format] Fix uninitialized memory problem

The setLength function checks for the token kind which could be
uninitialized in the previous version.

The problem was introduced in 2e32ff106e.

Rev

[clang-format] Fix uninitialized memory problem

The setLength function checks for the token kind which could be
uninitialized in the previous version.

The problem was introduced in 2e32ff106e.

Reviewed By: MyDeveloperDay, owenpan

Differential Revision: https://reviews.llvm.org/D128607

show more ...

[clang-format] Handle Verilog preprocessor directives

Verilog uses the backtick instead of the hash. In this revision
backticks are lexed manually and then get labeled as hashes so the logic
for ha

[clang-format] Handle Verilog preprocessor directives

Verilog uses the backtick instead of the hash. In this revision
backticks are lexed manually and then get labeled as hashes so the logic
for handling C preprocessor stuff don't have to change. Hashes get
labeled as identifiers for Verilog-specific stuff like delays.

Reviewed By: HazardyKnusperkeks

Differential Revision: https://reviews.llvm.org/D124749

show more ...

[clang-format] Fix whitespace counting stuff

The current way of counting whitespace would count backticks as
whitespace. For Verilog stuff we need backticks to be handled
correctly. For JavaScript

[clang-format] Fix whitespace counting stuff

The current way of counting whitespace would count backticks as
whitespace. For Verilog stuff we need backticks to be handled
correctly. For JavaScript the current way is to compare the entire
token text to see if it's a backtick. However, when the backtick is the
first token following an escaped newline, the escaped newline will be
part of the tok::unknown token. Verilog has macros and escaped newlines
unlike JavaScript. So we can't regard an entire tok::unknown token as
whitespace. Previously, the start of every token would be matched for
newlines. Now, it is all whitespace instead of just newlines.

The column counting problem has already been fixed for JavaScript in
e71b4cbdd140f059667f84464bd0ac0ebc348387 by counting columns elsewhere.

Reviewed By: HazardyKnusperkeks

Differential Revision: https://reviews.llvm.org/D124748

show more ...

[clang-format][NFC] Insert/remove braces in clang/lib/Format/

Differential Revision: https://reviews.llvm.org/D126157

Revert "[clang-format] Fix WhitespaceSensitiveMacros not being honoured when macro closing parenthesis is followed by a newline."

This reverts commit 50cd52d9357224cce66a9e00c9a0417c658a5655.

It pr

Revert "[clang-format] Fix WhitespaceSensitiveMacros not being honoured when macro closing parenthesis is followed by a newline."

This reverts commit 50cd52d9357224cce66a9e00c9a0417c658a5655.

It provoked regressions in C++ and ObjectiveC as described in https://reviews.llvm.org/D123676#3515949.

Reproducers:
```
MACRO_BEGIN
#if A
int f();
#else
int f();
#endif
```

```
NS_SWIFT_NAME(A)
@interface B : C
@property(readonly) D value;
@end
```

show more ...

[clang-format] Fix WhitespaceSensitiveMacros not being honoured when macro closing parenthesis is followed by a newline.

Fixes https://github.com/llvm/llvm-project/issues/54522.

This fixes regressi

[clang-format] Fix WhitespaceSensitiveMacros not being honoured when macro closing parenthesis is followed by a newline.

Fixes https://github.com/llvm/llvm-project/issues/54522.

This fixes regression introduced in https://github.com/llvm/llvm-project/commit/5e5efd8a91f2e340e79a73bedbc6ab66ad4a4281.

Before the culprit commit, macros in WhitespaceSensitiveMacros were correctly formatted even if their closing parenthesis weren't followed by semicolon (or, to be precise, when they were followed by a newline).
That commit changed the type of the macro token type from TT_UntouchableMacroFunc to TT_FunctionLikeOrFreestandingMacro.

Correct formatting (with `WhitespaceSensitiveMacros = ['FOO']`):
```
FOO(1+2)
FOO(1+2);
```

Regressed formatting:
```
FOO(1 + 2)
FOO(1+2);
```

Reviewed By: HazardyKnusperkeks, owenpan, ksyx

Differential Revision: https://reviews.llvm.org/D123676

show more ...

[NFC][Lexer] Make Lexer::LangOpts const reference

This change can be seen as code cleanup but motivation is more performance related.
While browsing perf reports captured during Linux build we can n

[NFC][Lexer] Make Lexer::LangOpts const reference

This change can be seen as code cleanup but motivation is more performance related.
While browsing perf reports captured during Linux build we can notice unusual portion of instructions executed in std::vector<std::string> copy constructor like:

0.59% 0.58% clang-14 clang-14 [.] std::vector<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >,
std::allocator<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > > >::vector

or even:

1.42% 0.26% clang clang-14 [.] clang::LangOptions::LangOptions
|
--1.16%--clang::LangOptions::LangOptions
|
--0.74%--std::vector<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >,
std::allocator<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > > >::vector

After more digging we can see that relevant LangOptions std::vector members (*Files, ModuleFeatures and NoBuiltinFuncs)
are constructed when Lexer::LangOpts field is initialized on list:

Lexer::Lexer(..., const LangOptions &langOpts, ...)
: ..., LangOpts(langOpts),

Since LangOptions copy constructor is called by Lexer(..., const LangOptions &LangOpts,...) and local Lexer objects are created thousands times
(in Lexer::getRawToken, Preprocessor::EnterSourceFile and more) during single module processing in frontend it makes std::vector copy constructors surprisingly hot.

Unfortunately even though in current Lexer implementation mentioned std::vector members are unused and most of time empty,
no compiler is smart enough to optimize their std::vector copy constructors out (take a look at test assembly): https://godbolt.org/z/hdoxPfMYY even with LTO enabled.
However there is simple way to fix this. Since Lexer doesn't access *Files, ModuleFeatures, NoBuiltinFuncs and any other LangOptions fields (but only LangOptionsBase)
we can simply get rid of redundant copy constructor assembly by changing LangOpts type to more appropriate const LangOptions reference: https://godbolt.org/z/fP7de9176

Additionally we need to store LineComment outside LangOpts because it's written in SkipLineComment function.
Also FormatTokenLexer need to be adjusted a bit to avoid lifetime issues related to passing local LangOpts reference to Lexer.

After this change I can see more than 1% speedup in some of my microbenchmarks when using Clang release binary built with LTO.
For Linux build gains are not so significant but still nice at the level of -0.4%/-0.5% instructions drop.

Differential Revision: https://reviews.llvm.org/D120334

show more ...

[clang-format] Use FormatToken::is* functions without passing through `Tok`. NFC.

[clang-format] Remove unnecessary parentheses in return statements. NFC.

[clang-format] Elide unnecessary braces. NFC.

[clang-format] Simplify use of StringRef::substr(). NFC.

[clang-format][NFC] Clean up tryMergeLessLess()

Differential Revision: https://reviews.llvm.org/D117759

[clang-format] Refactor: add FormatToken::hasWhitespaceBefore(). NFC.

This factors out a pattern that comes up from time to time.

Reviewed By: MyDeveloperDay, HazardyKnusperkeks, owenpan

Different

[clang-format] Refactor: add FormatToken::hasWhitespaceBefore(). NFC.

This factors out a pattern that comes up from time to time.

Reviewed By: MyDeveloperDay, HazardyKnusperkeks, owenpan

Differential Revision: https://reviews.llvm.org/D117769

show more ...

[clang-format] Fix bug in parsing `operator<` with template

Fixes https://github.com/llvm/llvm-project/issues/44601.

This patch handles a bug when parsing a below example code :

```
template <clas

[clang-format] Fix bug in parsing `operator<` with template

Fixes https://github.com/llvm/llvm-project/issues/44601.

This patch handles a bug when parsing a below example code :

```
template <class> class S;

template <class T> bool operator<(S<T> const &x, S<T> const &y) {
return x.i < y.i;
}

template <class T> class S {
int i = 42;
friend bool operator< <>(S const &, S const &);
};

int main() { return S<int>{} < S<int>{}; }
```
which parse `< <>` as `<< >`, not `< <>` in terms of tokens as discussed in discord.

1. Add a condition in `tryMergeLessLess()` considering `operator` keyword and `>`
2. Force to leave a whitespace between `tok::less` and a template opener
3. Add unit test

Reviewed By: MyDeveloperDay, curdeius

Differential Revision: https://reviews.llvm.org/D117398

show more ...

[clang-format] NFC use recently added Style.isJavaScript()

Improve the readability of these if(Style==FormatStyle::LK_JavsScript) clauses

Fix segfault in clang-format.

Fix bug where we'd read past the end of the tokens after merging _T
macro strings.

[clang-format] [NFC] build clang-format with -Wall

When building clang-format with -Wall on Visual Studio 20119 we see the following, prevent this the only -Wall error
```
..FormatTokenLexer.cpp(45)

[clang-format] [NFC] build clang-format with -Wall

When building clang-format with -Wall on Visual Studio 20119 we see the following, prevent this the only -Wall error
```
..FormatTokenLexer.cpp(45) : warning C4868: compiler may not enforce left-to-right evaluation order in braced initializer list
```
Reviewed By: HazardyKnusperkeks
Differential Revision: https://reviews.llvm.org/D113844

show more ...

[Format, Sema] Use range-based for loops with llvm::reverse (NFC)

[clang-format] Add IfMacros option

https://bugs.llvm.org/show_bug.cgi?id=49354

Differential Revision: https://reviews.llvm.org/D102730

[clang-format] Rename common types between C#/JS

Reviewed By: curdeius

Differential Revision: https://reviews.llvm.org/D101862

[clang-format] Add more support for C# 8 nullables

This adds support for the null-coalescing assignment and null-forgiving
operators.

https://docs.microsoft.com/en-us/dotnet/csharp/language-referen

[clang-format] Add more support for C# 8 nullables

This adds support for the null-coalescing assignment and null-forgiving
operators.

https://docs.microsoft.com/en-us/dotnet/csharp/language-reference/operators/null-coalescing-operator

https://docs.microsoft.com/en-us/dotnet/csharp/language-reference/operators/null-forgiving

Reviewed By: krasimir, curdeius

Differential Revision: https://reviews.llvm.org/D101702

show more ...

[clang-format] Add StatementAttributeLikeMacros option

This allows to ignore for example Qts emit when
AlignConsecutiveDeclarations is set, otherwise it is parsed as a type
and it results in some mi

[clang-format] Add StatementAttributeLikeMacros option

This allows to ignore for example Qts emit when
AlignConsecutiveDeclarations is set, otherwise it is parsed as a type
and it results in some misformating:

unsigned char MyChar = 'x';
emit signal(MyChar);

Differential Revision: https://reviews.llvm.org/D93776

show more ...

clang-format: [JS] support new assignment operators.

Before:

a && = b;

After:

a &&= b;

These operators are new additions in ES2021.

Differential Revision: https://reviews.llvm.org/D91132