[llvm] Use value instead of getValue (NFC)

[llvm] Don't use Optional::hasValue (NFC)

This patch replaces Optional::hasValue with the implicit cast to bool
in conditionals only.

Revert "Don't use Optional::hasValue (NFC)"

This reverts commit aa8feeefd3ac6c78ee8f67bf033976fc7d68bc6d.

Don't use Optional::hasValue (NFC)

[llvm][vfs] Implement in-memory symlinks

This patch implements symlinks for the in-memory VFS. Original author: @erik.pilkington.

Depends on D117648 & D117649.

Reviewed By: sammccall

Differential

[llvm][vfs] Implement in-memory symlinks

This patch implements symlinks for the in-memory VFS. Original author: @erik.pilkington.

Depends on D117648 & D117649.

Reviewed By: sammccall

Differential Revision: https://reviews.llvm.org/D117650

show more ...

[llvm][vfs] NFC: Promote `InMemoryDirIterator` to nested class

[llvm][vfs] NFC: Promote `lookupInMemoryNode()` to member function

[llvm][vfs] NFC: Rename `InMemoryFileSystem::addHardLink()` arguments

[llvm] Use value_or instead of getValueOr (NFC)

[VFS] RedirectingFileSystem only replace path if not already mapped

If the `ExternalFS` has already remapped to an external path then
`RedirectingFileSystem` should not change it to the originally p

[VFS] RedirectingFileSystem only replace path if not already mapped

If the `ExternalFS` has already remapped to an external path then
`RedirectingFileSystem` should not change it to the originally provided
path. This fixes the original path always being used if multiple VFS
overlays were provided and the path wasn't found in the highest (ie.
first in the chain).

For now this is accomplished through the use of a new
`ExposesExternalVFSPath` field on `vfs::Status`. This flag is true when
the `Status` has an external path that's different from its virtual
path, ie. the contained path is the external path. See the plan in
`FileManager::getFileRef` for where this is going - eventually we won't
need `IsVFSMapped` any more and all returned paths should be virtual.

Resolves rdar://90578880 and llvm-project#53306.

Reviewed By: dexonsmith

Differential Revision: https://reviews.llvm.org/D123398

show more ...

Revert "[VFS] RedirectingFileSystem only replace path if not already mapped"

This reverts commit 3fda0edc51fd68192a30e302d45db081bb02d7f9, which
breaks crash reproducers in very specific circumstanc

Revert "[VFS] RedirectingFileSystem only replace path if not already mapped"

This reverts commit 3fda0edc51fd68192a30e302d45db081bb02d7f9, which
breaks crash reproducers in very specific circumstances. Specifically,
since crash reproducers have `UseExternalNames` set to false, the
`File->getFileEntry().getDir()->getName()` call in `DoFrameworkLookup`
would use the *cached* directory name instead of the directory of the
looked-up file.

The plan is to re-commit this patch but to *add*
`ExposesExternalVFSPath` rather than replace `IsVFSMapped`.

Differential Revision: https://reviews.llvm.org/D123103

show more ...

[VFS] RedirectingFileSystem only replace path if not already mapped

If the `ExternalFS` has already remapped a path then the
`RedirectingFileSystem` should not change it to the originally provided
p

[VFS] RedirectingFileSystem only replace path if not already mapped

If the `ExternalFS` has already remapped a path then the
`RedirectingFileSystem` should not change it to the originally provided
path. This fixes the original path always being used if multiple VFS
overlays were provided and the path wasn't found in the highest (ie.
first in the chain).

This also renames `IsVFSMapped` to `ExposesExternalVFSPath` and only
sets it if `UseExternalName` is true. This flag then represents that the
`Status` has an external path that's different from its virtual path.
Right now the contained path is still the external path, but further PRs
will change this to *always* be the virtual path. Clients that need the
external can then request it specifically.

Note that even though `ExposesExternalVFSPath` isn't set for all
VFS-mapped paths, `IsVFSMapped` was only being used by a hack in
`FileManager` that was specific to module searching. In that case
`UseExternalNames` is always `true` and so that hack still applies.

Resolves rdar://90578880 and llvm-project#53306.

Differential Revision: https://reviews.llvm.org/D122549

show more ...

[VFS] Add print/dump to the whole FileSystem hierarchy

For now most are implemented by printing out the name of the filesystem,
but this can be expanded in the future. Only `OverlayFileSystem` and
`

[VFS] Add print/dump to the whole FileSystem hierarchy

For now most are implemented by printing out the name of the filesystem,
but this can be expanded in the future. Only `OverlayFileSystem` and
`RedirectingFileSystem` are properly implemented in this patch.
- `OverlayFileSystem`: Prints each filesystem in the order that any
operations are actually run on them. Optionally prints recursively.
- `RedirectingFileSystem`: Prints out all mappings, as well as the
`ExternalFS`. Most of this was already implemented other than the
handling for the `DirectoryRemap` case and to actually print out the
mapping.

Each FS should implement `printImpl` rather than `print`, where the
latter just fowards to the former. This is to avoid spreading the
default arguments through to the subclasses (where we may miss updating
in the future).

Differential Revision: https://reviews.llvm.org/D121421

show more ...

[VFS] Rename `RedirectingFileSystem::dump` to `print`

The rest of LLVM uses `print` for the method taking the `raw_ostream`
and `dump` only for the method with no parameters. Use the same for
`Redir

[VFS] Rename `RedirectingFileSystem::dump` to `print`

The rest of LLVM uses `print` for the method taking the `raw_ostream`
and `dump` only for the method with no parameters. Use the same for
`RedirectingFileSystem`.

Differential Revision: https://reviews.llvm.org/D121494

show more ...

[VFS] Use generic_category for errors generated from the VFS

Errors are generally checked in clients by comparing to the portable
error condition in `std::errc`, which will have the `generic_categor

[VFS] Use generic_category for errors generated from the VFS

Errors are generally checked in clients by comparing to the portable
error condition in `std::errc`, which will have the `generic_category`
(eg. `std::errc::no_such_file_or_directory`). While in practice these
are usually equivalent for the standard errno's, they are not in *all*
implementations. One such example is CentOS 7.

Differential Revision: https://reviews.llvm.org/D120299

show more ...

[VFS] Add back setFallthrough for downstream users

This fixes lldb's build. We can remove this in the future if we want but
for now this will be nicer to existing consumers.

[VFS] Add a "redirecting-with" field to overlays

Extend "fallthrough" to allow a third option: "fallback". Fallthrough
allows the original path to used if the redirected (or mapped) path
fails. Fall

[VFS] Add a "redirecting-with" field to overlays

Extend "fallthrough" to allow a third option: "fallback". Fallthrough
allows the original path to used if the redirected (or mapped) path
fails. Fallback is the reverse of this, ie. use the original path and
fallback to the mapped path otherwise.

While this result *can* be achieved today using multiple overlays, this
adds a much more intuitive option. As an example, take two directories
"A" and "B". We would like files from "A" to be used, unless they don't
exist, in which case the VFS should fallback to those in "B".

With the current fallthrough option this is possible by adding two
overlays: one mapping from A -> B and another mapping from B -> A. Since
the frontend *nests* the two RedirectingFileSystems, the result will
be that "A" is mapped to "B" and back to "A", unless it isn't in "A" in
which case it fallsthrough to "B" (or fails if it exists in neither).

Using "fallback" semantics allows a single overlay instead: one mapping
from "A" to "B" but only using that mapping if the operation in "A"
fails first.

"redirect-only" is used to represent the current "fallthrough: false"
case.

Differential Revision: https://reviews.llvm.org/D117937

show more ...

[llvm] Cleanup header dependencies in ADT and Support

The cleanup was manual, but assisted by "include-what-you-use". It consists in

1. Removing unused forward declaration. No impact expected.
2. R

[llvm] Cleanup header dependencies in ADT and Support

The cleanup was manual, but assisted by "include-what-you-use". It consists in

1. Removing unused forward declaration. No impact expected.
2. Removing unused headers in .cpp files. No impact expected.
3. Removing unused headers in .h files. This removes implicit dependencies and
is generally considered a good thing, but this may break downstream builds.
I've updated llvm, clang, lld, lldb and mlir deps, and included a list of the
modification in the second part of the commit.
4. Replacing header inclusion by forward declaration. This has the same impact
as 3.

Notable changes:

- llvm/Support/TargetParser.h no longer includes llvm/Support/AArch64TargetParser.h nor llvm/Support/ARMTargetParser.h
- llvm/Support/TypeSize.h no longer includes llvm/Support/WithColor.h
- llvm/Support/YAMLTraits.h no longer includes llvm/Support/Regex.h
- llvm/ADT/SmallVector.h no longer includes llvm/Support/MemAlloc.h nor llvm/Support/ErrorHandling.h

You may need to add some of these headers in your compilation units, if needs be.

As an hint to the impact of the cleanup, running

clang++ -E -Iinclude -I../llvm/include ../llvm/lib/Support/*.cpp -std=c++14 -fno-rtti -fno-exceptions | wc -l

before: 8000919 lines
after: 7917500 lines

Reduced dependencies also helps incremental rebuilds and is more ccache
friendly, something not shown by the above metric :-)

Discourse thread on the topic: https://llvm.discourse.group/t/include-what-you-use-include-cleanup/5831

show more ...

[llvm][vfs] Abstract in-memory node creation

The creation of in-memory VFS nodes happens in a single function that deduces what kind of node to create from the arguments. This leads to complicated i

[llvm][vfs] Abstract in-memory node creation

The creation of in-memory VFS nodes happens in a single function that deduces what kind of node to create from the arguments. This leads to complicated if-then-else logic that's difficult to cleanly extend.

This patch abstracts away in-memory node creation via a type-erased factory function that's passed instead.

Reviewed By: dexonsmith

Differential Revision: https://reviews.llvm.org/D117648

show more ...

[llvm][vfs] NFC: Virtualize in-memory `getStatus`

This patch virtualizes the `getStatus` function on `InMemoryNode` in LLVM VFS. Currently, this is implemented via top-level function `getNodeStatus`

[llvm][vfs] NFC: Virtualize in-memory `getStatus`

This patch virtualizes the `getStatus` function on `InMemoryNode` in LLVM VFS. Currently, this is implemented via top-level function `getNodeStatus` that tries to cast `InMemoryNode *` into each subtype. Virtual functions seem to be the simpler solution here.

Reviewed By: dexonsmith

Differential Revision: https://reviews.llvm.org/D117649

show more ...

[clang] support relative roots to vfs overlays

This diff adds support for relative roots to VFS overlays. The directory root
will be made absolute from the current working directory and will be used

[clang] support relative roots to vfs overlays

This diff adds support for relative roots to VFS overlays. The directory root
will be made absolute from the current working directory and will be used to
determine the path style to use. This supports the use of VFS overlays with
remote build systems that might use a different working directory for each
compilation.

Reviewed By: benlangmuir

Differential Revision: https://reviews.llvm.org/D116174

show more ...

[clang][deps] Squash caches for original and minimized files

The minimizing and caching filesystem used by the dependency scanner keeps minimized and original files in separate caches.

This setup i

[clang][deps] Squash caches for original and minimized files

The minimizing and caching filesystem used by the dependency scanner keeps minimized and original files in separate caches.

This setup is not well suited for dealing with files that are sometimes minimized and sometimes not. Such files are being stat-ed and read twice, which is wasteful and also means the two versions of the file can get "out of sync".

This patch squashes the two caches together. When a file is stat-ed or read, its original contents are populated. If a file needs to be minimized, we give the minimizer the already loaded contents instead of reading the file again.

Reviewed By: dexonsmith

Differential Revision: https://reviews.llvm.org/D115346

show more ...

reland: [VFS] Use original path when falling back to external FS

This reverts commit f0cf544d6f6fe6cbca4c07772998272d6bb433d8.

Just a small change to fix:

```
/home/buildbot/as-builder-4/llvm-clan

reland: [VFS] Use original path when falling back to external FS

This reverts commit f0cf544d6f6fe6cbca4c07772998272d6bb433d8.

Just a small change to fix:

```
/home/buildbot/as-builder-4/llvm-clang-x86_64-expensive-checks-ubuntu/llvm-project/llvm/lib/Support/VirtualFileSystem.cpp: In static member function ‘static llvm::ErrorOr<std::unique_ptr<llvm::vfs::File> > llvm::vfs::File::getWithPath(llvm::ErrorOr<std::unique_ptr<llvm::vfs::File> >, const llvm::Twine&)’:
/home/buildbot/as-builder-4/llvm-clang-x86_64-expensive-checks-ubuntu/llvm-project/llvm/lib/Support/VirtualFileSystem.cpp:2084:10: error: could not convert ‘F’ from ‘std::unique_ptr<llvm::vfs::File>’ to ‘llvm::ErrorOr<std::unique_ptr<llvm::vfs::File> >’
return F;
^
```

Differential Revision: https://reviews.llvm.org/D113832

show more ...

Revert "[VFS] Use original path when falling back to external FS"

```
/work/omp-vega20-0/openmp-offload-amdgpu-runtime/llvm.src/llvm/lib/Support/VirtualFileSystem.cpp: In static member function 'sta

Revert "[VFS] Use original path when falling back to external FS"

```
/work/omp-vega20-0/openmp-offload-amdgpu-runtime/llvm.src/llvm/lib/Support/VirtualFileSystem.cpp: In static member function 'static llvm::ErrorOr<std::unique_ptr<llvm::vfs::File> > llvm::vfs::File::getWithPath(llvm::ErrorOr<std::unique_ptr<llvm::vfs::File> >, const llvm::Twine&)':
/work/omp-vega20-0/openmp-offload-amdgpu-runtime/llvm.src/llvm/lib/Support/VirtualFileSystem.cpp:2084:10: error: could not convert 'F' from 'std::unique_ptr<llvm::vfs::File>' to 'llvm::ErrorOr<std::unique_ptr<llvm::vfs::File> >'
return F;
^
```

This reverts commit c972175649f4bb50d40d911659a04d5620ce6fe0.

show more ...

[VFS] Use original path when falling back to external FS

This is a follow up to 0be9ca7c0f9a733f846bb6bc4e8e36d46b518162 to make
paths in the case of falling back to the external file system use the

[VFS] Use original path when falling back to external FS

This is a follow up to 0be9ca7c0f9a733f846bb6bc4e8e36d46b518162 to make
paths in the case of falling back to the external file system use the
original format, preserving relative paths, and allow the external
filesystem to canonicalize them if needed.

Reviewed By: dexonsmith

Differential Revision: https://reviews.llvm.org/D109128

show more ...