[Basic] SourceManager docs: macro expansion SLocs aren't a single token. NFC

And haven't been since 2011: https://github.com/llvm/llvm-project/commit/eeca36fe9ad767380b2eab76a6fe5ba410a47393

[clang][NFC] Inclusive terms: replace some uses of sanity in clang

Rewording of comments to avoid using `sanity test, sanity check`.

Reviewed By: aaron.ballman, Quuxplusone

Differential Revision:

[clang][NFC] Inclusive terms: replace some uses of sanity in clang

Rewording of comments to avoid using `sanity test, sanity check`.

Reviewed By: aaron.ballman, Quuxplusone

Differential Revision: https://reviews.llvm.org/D114025

show more ...

[clang] GCC directive extension extension: Hash NNN lines

Some time back I extended GCC's '# NNN' line marker semantics.
Specifically popping to a blank filename will restore the filename to
that of

[clang] GCC directive extension extension: Hash NNN lines

Some time back I extended GCC's '# NNN' line marker semantics.
Specifically popping to a blank filename will restore the filename to
that of the popped-to include. Restore to line 5 of including file
(escaped BOL #'s to avoid git eliding them):

\# 5 "" 2

Added documentation for this line control extension.

This was useful in developing modules tests, but turned out to also be
useful with machine-generated source code. Specifically, a generated
include file that itself includes fragments from elsewhere. The
ability to pop to the generated include file -- with its full path
prefix -- is useful for diagnostic & debug purposes. For instance
something like:

// Machine generated -- DO NOT EDIT
Type Var = {
\# 7 "encoded.dsl" 1 // push to snippet-container
{snippet, of, code}
\# 6 " 2 // Restore to machined-generated source
,
};

// user-code
...
\#include "dsl.h"
...

That pop to "" will restore the filename to '..includepath../dsl.h',
which is better than restoring to plain "dsl.h".

Differential Revision: https://reviews.llvm.org/D113425

show more ...

[clang] Introduce SourceLocation::[U]IntTy typedefs.

This is part of a patch series working towards the ability to make
SourceLocation into a 64-bit type to handle larger translation units.

NFC: th

[clang] Introduce SourceLocation::[U]IntTy typedefs.

This is part of a patch series working towards the ability to make
SourceLocation into a 64-bit type to handle larger translation units.

NFC: this patch introduces typedefs for the integer type used by
SourceLocation and makes all the boring changes to use the typedefs
everywhere, but for the moment, they are unconditionally defined to
uint32_t.

Patch originally by Mikhail Maltsev.

Reviewed By: tmatheson

Differential Revision: https://reviews.llvm.org/D105492

show more ...

[clang][SourceManager] Fix -Wparentheses warning (NFC)

GCC warning:
```
/llvm-project/clang/lib/Basic/SourceManager.cpp: In instantiation of ‘constexpr T likelyhasbetween(T, unsigned char, unsigned

[clang][SourceManager] Fix -Wparentheses warning (NFC)

GCC warning:
```
/llvm-project/clang/lib/Basic/SourceManager.cpp: In instantiation of ‘constexpr T likelyhasbetween(T, unsigned char, unsigned char) [with T = long unsigned int]’:
/llvm-project/clang/lib/Basic/SourceManager.cpp:1292:52: required from here
/llvm-project/clang/lib/Basic/SourceManager.cpp:1264:48: warning: suggest parentheses around ‘+’ in operand of ‘&’ [-Wparentheses]
1264 | (x & ~static_cast<T>(0) / 255 * 127) +
| ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^
1265 | (~static_cast<T>(0) / 255 * (127 - (m - 1)))) &
| ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
```

show more ...

[NFC] Fix warning introduced in 20105b6b4874a85813f7a4a3d8ad2a0f023dda14

[clang] Speedup line offset mapping computation

Clang spends a decent amount of time in the LineOffsetMapping::get(...)
function. This function used to be vectorized (through SSE2) then the
optimiza

[clang] Speedup line offset mapping computation

Clang spends a decent amount of time in the LineOffsetMapping::get(...)
function. This function used to be vectorized (through SSE2) then the
optimization got dropped because the sequential version was on-par performance
wise.

This provides an optimization of the sequential version that works on a word at
a time, using (documented) bithacks to provide a portable vectorization.

When preprocessing the sqlite amalgamation, this yields a sweet 3% speedup.

This is a recommit of 6951b72334bbe4c189c71751edc1e361d7b5632c with endianness
and unsigned long vs uint64_t issues fixed (hopefully).

Differential Revision: https://reviews.llvm.org/D99409

show more ...

Revert "[clang] Speedup line offset mapping computation"

This reverts commit 6951b72334bbe4c189c71751edc1e361d7b5632c.
Breaks several bots, see comments on https://reviews.llvm.org/D99409

[clang] Speedup line offset mapping computation

Clang spends a decent amount of time in the LineOffsetMapping::get(...)
function. This function used to be vectorized (through SSE2) then the
optimiza

[clang] Speedup line offset mapping computation

Clang spends a decent amount of time in the LineOffsetMapping::get(...)
function. This function used to be vectorized (through SSE2) then the
optimization got dropped because the sequential version was on-par performance
wise.

This provides an optimization of the sequential version that works on a word at
a time, using (documented) bithacks to provide a portable vectorization.

When preprocessing the sqlite amalgamation, this yields a sweet 3% speedup.

Differential Revision: https://reviews.llvm.org/D99409

show more ...

Use a fast path when initializing LineOffsetMapping

Use the fact that the number of line break is lower than printable characters to
guide the optimization process. Also use a fuzzy test that catche

Use a fast path when initializing LineOffsetMapping

Use the fact that the number of line break is lower than printable characters to
guide the optimization process. Also use a fuzzy test that catches both \n and
\r in a single check to speedup the computation.

Differential Revision: https://reviews.llvm.org/D97320

show more ...

SourceManager: Migrate to FileEntryRef in getOrCreateContentCache, NFC

Change `SourceManager::getOrCreateContentCache` to take a `FileEntryRef`
and update call sites (mostly internal to SourceManage

SourceManager: Migrate to FileEntryRef in getOrCreateContentCache, NFC

Change `SourceManager::getOrCreateContentCache` to take a `FileEntryRef`
and update call sites (mostly internal to SourceManager.cpp). In a
couple of cases this temporarily relies on `FileEntry::getLastRef`, but
those can be cleaned up once other APIs switch over.

The one change outside of SourceManager.cpp is in ASTReader.cpp, which
stops relying on the auto-degrade-to-`FileEntry*` behaviour from
`InputFile::getFile` since it now needs a `FileEntryRef`.

No functionality change here.

Differential Revision: https://reviews.llvm.org/D92983

show more ...

SourceManager: Unify FileEntry/FileEntryRef versions of createFileID

Change `SourceManager::createFileID(const FileEntry*)` to defer to
`SourceManager::createFileID(FileEntryRef)`. This fixes an une

SourceManager: Unify FileEntry/FileEntryRef versions of createFileID

Change `SourceManager::createFileID(const FileEntry*)` to defer to
`SourceManager::createFileID(FileEntryRef)`. This fixes an unexercised
bug where the latter gained support for named pipes and the former
didn't, but since we're trying to remove all calls to the former it
doesn't really make sense to test this explicitly now that the
implementation is hollowed out.

This is a belated follow-up to 245218bb355599771ba43a0fe1449d1670f2666c,
which sunk named pipe support into FileManager and SourceManager. The
original version of that patch was based on top of
https://reviews.llvm.org/D92984, which removed the `FileEntry` overload
of `createFileID()`, and I missed the subtle difference when it was
rebased.

show more ...

Basic: Add native support for stdin to SourceManager and FileManager

Add support for stdin to SourceManager and FileManager. Adds
FileManager::getSTDIN, which adds a FileEntryRef for `<stdin>` and r

Basic: Add native support for stdin to SourceManager and FileManager

Add support for stdin to SourceManager and FileManager. Adds
FileManager::getSTDIN, which adds a FileEntryRef for `<stdin>` and reads
the MemoryBuffer, which is stored as `FileEntry::Content`.

Eventually the other buffers in `ContentCache` will sink to here as well
-- we probably usually want to load/save a MemoryBuffer eagerly -- but
it's happening early for stdin to get rid of
CompilerInstance::InitializeSourceManager's final call to
`SourceManager::overrideFileContents`.

clang/test/CXX/modules-ts/dcl.dcl/dcl.module/dcl.module.export/p1.cpp
relies on building a module from stdin; supporting that requires setting
ContentCache::BufferOverridden.

Differential Revision: https://reviews.llvm.org/D93148

show more ...

Basic: Support named pipes natively in SourceManager and FileManager

Handle named pipes natively in SourceManager and FileManager, removing a
call to `SourceManager::overrideFileContents` in
`Compil

Basic: Support named pipes natively in SourceManager and FileManager

Handle named pipes natively in SourceManager and FileManager, removing a
call to `SourceManager::overrideFileContents` in
`CompilerInstance::InitializeSourceManager` (removing a blocker for
sinking the content cache to FileManager (which will incidently sink
this new named pipe logic with it)).

SourceManager usually checks if the file entry's size matches the
eventually loaded buffer, but that's now skipped for named pipes since
the `stat` won't reflect the full size. Since we can't trust
`ContentsEntry->getSize()`, we also need shift the check for files that
are too large until after the buffer is loaded... and load the buffer
immediately in `createFileID` so that no client gets a bad value from
`ContentCache::getSize`. `FileManager::getBufferForFile` also needs to
treat these files as volatile when loading the buffer.

Native support in SourceManager / FileManager means that named pipes can
also be `#include`d, and clang/test/Misc/dev-fd-fs.c was expanded to
check for that.

This is a new version of 3b18a594c7717a328c33b9c1eba675e9f4bd367c, which
was reverted in b34632201987eed369bb7ef4646f341b901c95b8 since it was
missing the `SourceManager` changes.

Differential Revision: https://reviews.llvm.org/D92531

show more ...

Serialization: Change InputFile to use FileEntryRef and add getVirtualFileRef, NFC

Change the `InputFile` class to store `Optional<FileEntryRef>` instead
of `FileEntry*`. This paged in a few API cha

Serialization: Change InputFile to use FileEntryRef and add getVirtualFileRef, NFC

Change the `InputFile` class to store `Optional<FileEntryRef>` instead
of `FileEntry*`. This paged in a few API changes:

- Added `FileManager::getVirtualFileRef`, and converted `getVirtualFile`
to a wrapper of it.
- Updated `SourceManager::bypassFileContentsOverride` to take
`FileEntryRef` and return `Optional<FileEntryRef>`
(`ASTReader::getInputFile` is the only caller).

Differential Revision: https://reviews.llvm.org/D90053

show more ...

FileManager: Shrink FileEntryRef to the size of a pointer

Shrink `FileEntryRef` to the size of a pointer, by having it directly
reference the `StringMapEntry` the same way that `DirectoryEntryRef`
d

FileManager: Shrink FileEntryRef to the size of a pointer

Shrink `FileEntryRef` to the size of a pointer, by having it directly
reference the `StringMapEntry` the same way that `DirectoryEntryRef`
does. This makes `FileEntryRef::FileEntryRef` private as a side effect
(`FileManager` is a friend!).

There are two helper types added within `FileEntryRef`:

- `FileEntryRef::MapValue` is the type stored in
`FileManager::SeenFileEntries`. It's a replacement for
`SeenFileEntryOrRedirect`, where the second pointer type has been
changed from `StringRef*` to `MapEntry*` (see next bullet).
- `FileEntryRef::MapEntry` is the instantiation of `StringMapEntry<>`
where `MapValue` is stored. This is what `FileEntryRef` has a pointer
to, in order to grab the name in addition to the value.

Differential Revision: https://reviews.llvm.org/D89488

show more ...

SourceManager: Use the same fake SLocEntry whenever it fails to load

Instead of putting a fake `SLocEntry` at `LoadedSLocEntryTable[Index]`
when it fails to load in `SourceManager::loadSLocEntry`, a

SourceManager: Use the same fake SLocEntry whenever it fails to load

Instead of putting a fake `SLocEntry` at `LoadedSLocEntryTable[Index]`
when it fails to load in `SourceManager::loadSLocEntry`, allocate a fake
one. Unless someone is sniffing the address of the returned `SLocEntry`
(doubtful), this won't be a functionality change. Note that
`SLocEntryLoaded[Index]` wasn't being set to `true` either before or
after this change so no accessor is every going to look at
`LoadedSLocEntryTable[Index]`.

As a side effect, drop the `mutable` from `LoadedSLocEntryTable`.

Differential Revision: https://reviews.llvm.org/D89748

show more ...

SourceManager: Fix an SLocEntry memory regression introduced with FileEntryRef

4dc5573acc0d2e7c59d8bac2543eb25cb4b32984 added `FileEntryRef` in order to
help enable sharing of a `FileManager` betwee

SourceManager: Fix an SLocEntry memory regression introduced with FileEntryRef

4dc5573acc0d2e7c59d8bac2543eb25cb4b32984 added `FileEntryRef` in order to
help enable sharing of a `FileManager` between `CompilerInstance`s.

It also added a `StringRef` with the filename on `FileInfo`. This
doubled `sizeof(FileInfo)`, bloating `sizeof(SLocEntry)`, of which we
have one for each (loaded and unloaded) file and macro expansion. This
causes a memory regression in modules builds.

Move the filename down into the `ContentCache`, which is a side data
structure for `FileInfo` that does not impact `sizeof(SLocEntry)`. Once
`FileEntryRef` is used for `ContentCache::OrigEntry` this can go away.

Differential Revision: https://reviews.llvm.org/D89580
Radar-Id: rdar://59908826

show more ...

SourceManager: Return non-const references in getOrCreateContentCache and related, NFC

Update a few APIs to return non-const references instead of pointers,
and remove associated `const_cast`s and n

SourceManager: Return non-const references in getOrCreateContentCache and related, NFC

Update a few APIs to return non-const references instead of pointers,
and remove associated `const_cast`s and non-null assertions.

Differential Revision: https://reviews.llvm.org/D90067

show more ...

SourceManager: Simplify by inlining what remains of ComputeLineNumbers, NFC

Use `LineOffsetMapping:get` directly and remove/inline the helper
`ComputeLineNumbers`, simplifying the callers.

Differen

SourceManager: Simplify by inlining what remains of ComputeLineNumbers, NFC

Use `LineOffsetMapping:get` directly and remove/inline the helper
`ComputeLineNumbers`, simplifying the callers.

Differential Revision: https://reviews.llvm.org/D89922

show more ...

SourceManager: Make LastLineNoContentCache and ContentCache::SourceLineCache mutable, NFC

Avoid some noisy `const_cast`s by making `ContentCache::SourceLineCache`
and `SourceManager::LastLineNoConte

SourceManager: Make LastLineNoContentCache and ContentCache::SourceLineCache mutable, NFC

Avoid some noisy `const_cast`s by making `ContentCache::SourceLineCache`
and `SourceManager::LastLineNoContentCache` both mutable.

Differential Revision: https://reviews.llvm.org/D89914

show more ...

SourceManager: Encapsulate line number mapping into SrcMgr::LineOffsetMapping

Put the guts of `ComputeLineNumbers` into `LineOffsetMapping::get` and
`LineOffsetMapping::LineOffsetMapping`. As a dri

SourceManager: Encapsulate line number mapping into SrcMgr::LineOffsetMapping

Put the guts of `ComputeLineNumbers` into `LineOffsetMapping::get` and
`LineOffsetMapping::LineOffsetMapping`. As a drive-by, store the number
of lines directly in the bump-ptr-allocated array.

Differential Revision: https://reviews.llvm.org/D89913

show more ...

SourceManager: Clarify that FileInfo always has a ContentCache, NFC

It turns out that `FileInfo` *always* has a ContentCache. Clarify that
in the code:
- Update the private version of `SourceManager

SourceManager: Clarify that FileInfo always has a ContentCache, NFC

It turns out that `FileInfo` *always* has a ContentCache. Clarify that
in the code:
- Update the private version of `SourceManager::createFileID` to take a
`ContentCache&` instead of `ContentCache*`, and rename it to
`createFileIDImpl` for clarity.
- Change `FileInfo::getContentCache` to return a reference.

Differential Revision: https://reviews.llvm.org/D89554

show more ...

SourceManager: Remove a redundant nullptr check in getNonBuiltinFilenameForID, NFC

SourceManager: getFileEntryRefForID => getNonBuiltinFilenameForID, NFC

`SourceManager::getFileEntryRefForID`'s remaining callers just want the
filename component, which is coming from the `FileInfo`

SourceManager: getFileEntryRefForID => getNonBuiltinFilenameForID, NFC

`SourceManager::getFileEntryRefForID`'s remaining callers just want the
filename component, which is coming from the `FileInfo`. Replace the API
with `getNonBuiltinFilenameForID`, which also removes another use of
`FileEntryRef::FileEntryRef` outside of `FileManager`.

Both callers are collecting file dependencies, and one of them relied on
this API to filter out built-ins (as exposed by
clang/test/ClangScanDeps/modules-full.cpp). It seems nice to continue
providing that service.

Differential Revision: https://reviews.llvm.org/D89508

show more ...