[Support] [VirtualFileSystem] Detect the windows_slash path style

This fixes the following clang VFS tests, if `windows_slash` is the
default style:

Clang :: VFS/implicit-include.c
Clang :: VFS

[Support] [VirtualFileSystem] Detect the windows_slash path style

This fixes the following clang VFS tests, if `windows_slash` is the
default style:

Clang :: VFS/implicit-include.c
Clang :: VFS/relative-path.c
Clang-Unit :: Frontend/./FrontendTests.exe/CompilerInstance.DefaultVFSOverlayFromInvocation

Also clarify a couple references to `Style::windows` into
`Style::windows_backslash`, to make it clearer that each of them are
opinionated in different directions (even if it doesn't matter for
calls to e.g. `is_absolute`).

Differential Revision: https://reviews.llvm.org/D113272

show more ...

Put implementation details into anonymous namespaces. NFCI.

[VFS] InMemoryFilesystem's UniqueIDs are a function of path and content.

This ensures that re-creating "the same" FS results in the same UIDs for files.
In turn, this means that creating a clang mod

[VFS] InMemoryFilesystem's UniqueIDs are a function of path and content.

This ensures that re-creating "the same" FS results in the same UIDs for files.
In turn, this means that creating a clang module (preamble) using one in-memory
filesystem and consuming it using another doesn't create duplicate FileEntrys
for files that are the same in both FSes.

It's tempting to give the creator control over the UIDs instead. However that
requires fiddly API changes, e.g. what should the UIDs of intermediate
directories be?
This change is more "magic" but seems safe given:
- InMemoryFilesystem is used in testing more than production
- comparing UIDs across filesystems is unusual
- files with the same path and content are usually logically equivalent

(The usual reason for re-creating virtual filesystems rather than reusing them
is that typical use involves mutating their CWD and so is not threadsafe).

Differential Revision: https://reviews.llvm.org/D110711

show more ...

[Support] Qualify auto (NFC)

Identified with readability-qualified-auto.

PR51018: Remove explicit conversions from SmallString to StringRef to future-proof against C++23

C++23 will make these conversions ambiguous - so fix them to make the
codebase forward-compatible wit

PR51018: Remove explicit conversions from SmallString to StringRef to future-proof against C++23

C++23 will make these conversions ambiguous - so fix them to make the
codebase forward-compatible with C++23 (& a follow-up change I've made
will make this ambiguous/invalid even in <C++23 so we don't regress
this & it generally improves the code anyway)

show more ...

[llvm] Rename StringRef _lower() method calls to _insensitive()

This is a mechanical change. This actually also renames the
similarly named methods in the SmallString class, however these
methods do

[llvm] Rename StringRef _lower() method calls to _insensitive()

This is a mechanical change. This actually also renames the
similarly named methods in the SmallString class, however these
methods don't seem to be used outside of the llvm subproject, so
this doesn't break building of the rest of the monorepo.

show more ...

[clang] Don't crash when loading invalid VFS for the module dep collector

The VFS is null when it's invalid so return early in collectVFSFromYAML.

[VFS] Add support to RedirectingFileSystem for mapping a virtual directory to one in the external FS.

Previously file entries in the -ivfsoverlay yaml could map to a file in the
external file system

[VFS] Add support to RedirectingFileSystem for mapping a virtual directory to one in the external FS.

Previously file entries in the -ivfsoverlay yaml could map to a file in the
external file system, but directories had to list their contents in the form of
other file entries or directories. Allowing directory entries to map to a
directory in the external file system makes it possible to present an external
directory's contents in a different location and (in combination with the
'fallthrough' option) overlay one directory's contents on top of another.

rdar://problem/72485443
Differential Revision: https://reviews.llvm.org/D94844

show more ...

[NFC][VFS] Fix a build warning due to an extra semicolon

[VFS] Combine VFSFromYamlDirIterImpl and OverlayFSDirIterImpl into a single implementation (NFC)

As a fixme notes, both of these directory iterator implementations are
conceptually similar and dupli

[VFS] Combine VFSFromYamlDirIterImpl and OverlayFSDirIterImpl into a single implementation (NFC)

As a fixme notes, both of these directory iterator implementations are
conceptually similar and duplicate the functionality of returning and uniquing
entries across two or more directories. This patch combines them into a single
class 'CombiningDirIterImpl'.

This also drops the 'Redirecting' prefix from RedirectingDirEntry and
RedirectingFileEntry to save horizontal space. There's no loss of clarity as
they already have to be prefixed with 'RedirectingFileSystem::' whenever
they're referenced anyway.

rdar://problem/72485443
Differential Revision: https://reviews.llvm.org/D94857

show more ...

[VFS] Fix inconsistencies between relative paths and fallthrough.

This patch addresses inconsistencies in the way fallthrough is handled
in the RedirectingFileSystem. Rather than trying to change th

[VFS] Fix inconsistencies between relative paths and fallthrough.

This patch addresses inconsistencies in the way fallthrough is handled
in the RedirectingFileSystem. Rather than trying to change the working
directory of the external filesystem, the RedirectingFileSystem will
canonicalize every path before handing it down. This guarantees that
relative paths are resolved relative to the RedirectingFileSystem's
working directory.

This allows us to have a strictly virtual working directory, and still
fallthrough for absolute paths, but not for relative paths that would
get resolved incorrectly at the lower layer (for example, in case of the
RealFileSystem, because the strictly virtual path does not exist).

Differential revision: https://reviews.llvm.org/D95188

show more ...

Support: Add RedirectingFileSystem::create from simple list of redirections

Add an overload of `RedirectingFileSystem::create` that builds a
redirecting filesystem off of a simple vector of string p

Support: Add RedirectingFileSystem::create from simple list of redirections

Add an overload of `RedirectingFileSystem::create` that builds a
redirecting filesystem off of a simple vector of string pairs. This is
intended to be used to support `clang::arcmt::FileRemapper` and
`clang::PreprocessorOptions::RemappedFiles`.

Differential Revision: https://reviews.llvm.org/D91317

show more ...

VFS: Return new file systems as uniquely owned when possible, almost NFC

Uniformly return uniquely-owned filesystems from VFS creation APIs. The
one exception is `getRealFileSystem`, which has a sin

VFS: Return new file systems as uniquely owned when possible, almost NFC

Uniformly return uniquely-owned filesystems from VFS creation APIs. The
one exception is `getRealFileSystem`, which has a single instance and
needs to be shared.

This is almost NFC, except that it fixes a memory leak in
`vfs::collectVFSFromYAML()`.

Depends on https://reviews.llvm.org/D92888

Differential Revision: https://reviews.llvm.org/D92890

show more ...

Support: Change InMemoryFileSystem::addFileNoOwn to take a MemoryBufferRef, NFC

Found this by chance when looking at the InMemoryFileSystem API, seems
like an easy cleanup.

Differential Revision: h

Support: Change InMemoryFileSystem::addFileNoOwn to take a MemoryBufferRef, NFC

Found this by chance when looking at the InMemoryFileSystem API, seems
like an easy cleanup.

Differential Revision: https://reviews.llvm.org/D90893

show more ...

[lldb] Add reproducer verifier

Add a reproducer verifier that catches:

- Missing or invalid home directory
- Missing or invalid working directory
- Missing or invalid module/symbol paths
- Miss

[lldb] Add reproducer verifier

Add a reproducer verifier that catches:

- Missing or invalid home directory
- Missing or invalid working directory
- Missing or invalid module/symbol paths
- Missing files from the VFS

The verifier is enabled by default during replay, but can be skipped by
passing --reproducer-no-verify.

Differential revision: https://reviews.llvm.org/D86497

show more ...

Revert "[lldb] Add reproducer verifier"

This reverts commit 297f69afac58fc9dc13897857a5e70131c5adc85. It broke
the Fedora 33 x86-64 bot. See the review for more info.

[lldb] Add reproducer verifier

Add a reproducer verifier that catches:

- Missing or invalid home directory
- Missing or invalid working directory
- Missing or invalid module/symbol paths
- Miss

[lldb] Add reproducer verifier

Add a reproducer verifier that catches:

- Missing or invalid home directory
- Missing or invalid working directory
- Missing or invalid module/symbol paths
- Missing files from the VFS

The verifier is enabled by default during replay, but can be skipped by
passing --reproducer-no-verify.

Differential revision: https://reviews.llvm.org/D86497

show more ...

[YAMLVFSWriter] Fix for delimiters

Differential Revision: https://reviews.llvm.org/D79809

[YAMLVFSWriter] Fix directory handling

For empty directories (except the first one) we've been adding a file
with the same name as the directory to the result VFS mapping.

Differential Revision: ht

[YAMLVFSWriter] Fix directory handling

For empty directories (except the first one) we've been adding a file
with the same name as the directory to the result VFS mapping.

Differential Revision: https://reviews.llvm.org/D79551

show more ...

[VirtualFileSystem] Support directory entries in the YAMLVFSWriter

The current implementation of the JSONWriter does not support writing
out directory entries. Earlier today I added a unit test to i

[VirtualFileSystem] Support directory entries in the YAMLVFSWriter

The current implementation of the JSONWriter does not support writing
out directory entries. Earlier today I added a unit test to illustrate
the problem. When an entry is added to the YAMLVFSWriter and the path is
a directory, it will incorrectly emit the directory as a file, and any
files inside that directory will not be found by the VFS.

It's possible to partially work around the issue by only adding "leaf
nodes" (files) to the YAMLVFSWriter. However, this doesn't work for
representing empty directories. This is a problem for clients of the VFS
that want to iterate over a directory. The directory not being there is
not the same as the directory being empty.

This is not just a hypothetical problem. The FileCollector for example
does not differentiate between file and directory paths. I temporarily
worked around the issue for LLDB by ignoring directories, but I suspect
this will prove problematic sooner rather than later.

This patch fixes the issue by extending the JSONWriter to support
writing out directory entries. We store whether an entry should be
emitted as a file or directory.

Differential revision: https://reviews.llvm.org/D76670

show more ...

[VFS] More consistent support for Windows

Removed some #ifdefs specific to Windows handling of VFS paths. This
eliminates most of the differences between the Windows and non-Windows
code paths.

Ma

[VFS] More consistent support for Windows

Removed some #ifdefs specific to Windows handling of VFS paths. This
eliminates most of the differences between the Windows and non-Windows
code paths.

Making this work required some changes to account for the fact that VFS
file paths can be Posix style or Windows style, so you cannot just assume
that they use the host's native path style. In one case, this means
implementing our own version of make_absolute, since the filesystem code
in Support doesn't have styles in the sense that the path code does.

Differential Review: https://reviews.llvm.org/D71092

show more ...

Make llvm::StringRef to std::string conversions explicit.

This is how it should've been and brings it more in line with
std::string_view. There should be no functional change here.

This is mostly m

Make llvm::StringRef to std::string conversions explicit.

This is how it should've been and brings it more in line with
std::string_view. There should be no functional change here.

This is mostly mechanical from a custom clang-tidy check, with a lot of
manual fixups. It uncovers a lot of minor inefficiencies.

This doesn't actually modify StringRef yet, I'll do that in a follow-up.

show more ...

Fix more VFS tests on Windows

Since VFS paths can be in either Posix or Windows style, we have to use
a more flexible definition of "absolute" path.

The key here is that FileSystem::makeAbsolute is

Fix more VFS tests on Windows

Since VFS paths can be in either Posix or Windows style, we have to use
a more flexible definition of "absolute" path.

The key here is that FileSystem::makeAbsolute is now virtual, and the
RedirectingFileSystem override checks for either concept of absolute
before trying to make the path absolute by combining it with the current
directory.

Differential Revision: https://reviews.llvm.org/D70701

show more ...

Improve VFS compatibility on Windows

Keys in a virtual file system can be in Posix or Windows form or even
a combination of the two. Many VFS tests (and a few Clang tests) were
XFAILed on Windows b

Improve VFS compatibility on Windows

Keys in a virtual file system can be in Posix or Windows form or even
a combination of the two. Many VFS tests (and a few Clang tests) were
XFAILed on Windows because of false negatives when comparing paths.

First, we default CaseSenstive to false on Windows. This allows
drive letters like "D:" to match "d:". Windows filesystems are, by
default, case insensitive, so this makes sense even beyond the drive
letter.

Second, we allow slashes to match backslashes when they're used as the
root component of a path.

Both of these changes are limited to RedirectingFileSystems, so there's
little chance of affecting other path handling.

These changes allow eleven of the VFS tests to pass on Windows as well
as three other Clang tests, so they have re-enabled.

This solves the majority of PR43272. Additional VFS test failures will
be fixed in separate patches.

Differential Revision: https://reviews.llvm.org/D69958

show more ...

VirtualFileSystem - fix uninitialized variable warnings. NFCI.