[SystemZ][z/OS][Windows] Add new OF_TextWithCRLF flag and use this flag instead of OF_Text

Problem:
On SystemZ we need to open text files in text mode. On Windows, files opened in text mode adds a C

[SystemZ][z/OS][Windows] Add new OF_TextWithCRLF flag and use this flag instead of OF_Text

Problem:
On SystemZ we need to open text files in text mode. On Windows, files opened in text mode adds a CRLF '\r\n' which may not be desirable.

Solution:
This patch adds two new flags

- OF_CRLF which indicates that CRLF translation is used.
- OF_TextWithCRLF = OF_Text | OF_CRLF indicates that the file is text and uses CRLF translation.

Developers should now use either the OF_Text or OF_TextWithCRLF for text files and OF_None for binary files. If the developer doesn't want carriage returns on Windows, they should use OF_Text, if they do want carriage returns on Windows, they should use OF_TextWithCRLF.

So this is the behaviour per platform with my patch:

z/OS:
OF_None: open in binary mode
OF_Text : open in text mode
OF_TextWithCRLF: open in text mode

Windows:
OF_None: open file with no carriage return
OF_Text: open file with no carriage return
OF_TextWithCRLF: open file with carriage return

The Major change is in llvm/lib/Support/Windows/Path.inc to only set text mode if the OF_CRLF is set.
```
if (Flags & OF_CRLF)
CrtOpenFlags |= _O_TEXT;
```

These following files are the ones that still use OF_Text which I left unchanged. I modified all these except raw_ostream.cpp in recent patches so I know these were previously in Binary mode on Windows.
./llvm/lib/Support/raw_ostream.cpp
./llvm/lib/TableGen/Main.cpp
./llvm/tools/dsymutil/DwarfLinkerForBinary.cpp
./llvm/unittests/Support/Path.cpp
./clang/lib/StaticAnalyzer/Core/HTMLDiagnostics.cpp
./clang/lib/Frontend/CompilerInstance.cpp
./clang/lib/Driver/Driver.cpp
./clang/lib/Driver/ToolChains/Clang.cpp

Reviewed By: MaskRay

Differential Revision: https://reviews.llvm.org/D99426

show more ...

[FileCollector] Fix that the file system case-sensitivity check was inverted

real_path returns an `std::error_code` which evaluates to `true` in case an
error happens and `false` if not. This code w

[FileCollector] Fix that the file system case-sensitivity check was inverted

real_path returns an `std::error_code` which evaluates to `true` in case an
error happens and `false` if not. This code was checking the inverse, so
case-insensitive file systems ended up being detected as case sensitive.

Tested using an LLDB reproducer test as we anyway need a real file system and
also some matching logic to detect whether the respective file system is
case-sensitive (which the test is doing via some Python checks that we can't
really emulate with the usual FileCheck logic).

Fixes rdar://67003004

Reviewed By: JDevlieghere

Differential Revision: https://reviews.llvm.org/D96795

show more ...

Support: Remove duplicated code in {File,clang::ModulesDependency}Collector, NFC

Refactor the duplicated canonicalize-path logic in `FileCollector` and
`ModulesDependencyCollector` into a new utilit

Support: Remove duplicated code in {File,clang::ModulesDependency}Collector, NFC

Refactor the duplicated canonicalize-path logic in `FileCollector` and
`ModulesDependencyCollector` into a new utility called
`PathCanonicalizer` that's shared. This popped up when tracking down a
bug common to both in https://reviews.llvm.org/D95202.

As drive-bys, update a few names and comments to better reflect the
effect of the code, delay removal of `..`s to avoid an unnecessary extra
string copy, and leave behind a couple of FIXMEs for future
consideration.

Differential Revision: https://reviews.llvm.org/D95279

show more ...

ADT: Use 'using' to inherit assign and append in SmallString

Rather than reimplement, use a `using` declaration to bring in
`SmallVectorImpl<char>`'s assign and append implementations in
`SmallStrin

ADT: Use 'using' to inherit assign and append in SmallString

Rather than reimplement, use a `using` declaration to bring in
`SmallVectorImpl<char>`'s assign and append implementations in
`SmallString`.

The `SmallString` versions were missing reference invalidation
assertions from `SmallVector`. This patch also fixes a bug in
`llvm::FileCollector::addFileImpl`, which was a copy/paste from
`clang::ModuleDependencyCollector::copyToRoot`, both caught by the
no-longer-skipped assertions.

As a drive-by, this also sinks the `const SmallVectorImpl&` versions of
these methods down into `SmallVectorImpl`, since I imagine they'd be
useful elsewhere.

Differential Revision: https://reviews.llvm.org/D95202

show more ...

[FileCollector] Move interface into FileCollectorBase (NFC)

For the reproducers in LLDB we want to switch to an "immediate mode"
FileCollector that writes every file encountered straight to disk so

[FileCollector] Move interface into FileCollectorBase (NFC)

For the reproducers in LLDB we want to switch to an "immediate mode"
FileCollector that writes every file encountered straight to disk so we
can generate the actual mapping out-of-process. This patch moves the
interface into a separate base class.

Differential revision: https://reviews.llvm.org/D89742

show more ...

[llvm] Don't create the directory hierarchy in the FileCollector...

... if the collected file doesn't exists.

This fixes the situation where LLDB can't create a file when capturing a
reproducer bec

[llvm] Don't create the directory hierarchy in the FileCollector...

... if the collected file doesn't exists.

This fixes the situation where LLDB can't create a file when capturing a
reproducer because the parent path doesn't exist, but can during replay
because the file collector created the directory hierarchy even though
the file doesn't exist.

This is covered by the lldb reproducer test suite.

show more ...

FileCollector.h - reduce Twine.h include to forward declaration. NFC.

[FileCollector] move Root creation

If we don't handle the errors we can't rely on the directory being created early anyway.

Differential Revision: https://reviews.llvm.org/D78959

[FileCollector] Lock Mutex in copyFiles

We should synchronize reading of VFSWriter's data with the rest of the methods.

Differential revision: https://reviews.llvm.org/D78956

Re-land "[FileCollector] Add a method to add a whole directory and it contents."

Extend the FileCollector's API with addDirectory which adds a directory
and its contents to the VFS mapping.

Differe

Re-land "[FileCollector] Add a method to add a whole directory and it contents."

Extend the FileCollector's API with addDirectory which adds a directory
and its contents to the VFS mapping.

Differential revision: https://reviews.llvm.org/D76671

show more ...

Revert "[FileCollector] Add a method to add a whole directory and it contents."

This reverts commit 8913769e353a171ba01fa8ce9d598e979b620be9 because the
unit test is failing on the Windows bot.

[FileCollector] Add a method to add a whole directory and it contents.

Extend the FileCollector's API with addDirectory which adds a directory
and its contents to the VFS mapping.

Differential revi

[FileCollector] Add a method to add a whole directory and it contents.

Extend the FileCollector's API with addDirectory which adds a directory
and its contents to the VFS mapping.

Differential revision: https://reviews.llvm.org/D76671

show more ...

Make llvm::StringRef to std::string conversions explicit.

This is how it should've been and brings it more in line with
std::string_view. There should be no functional change here.

This is mostly m

Make llvm::StringRef to std::string conversions explicit.

This is how it should've been and brings it more in line with
std::string_view. There should be no functional change here.

This is mostly mechanical from a custom clang-tidy check, with a lot of
manual fixups. It uncovers a lot of minor inefficiencies.

This doesn't actually modify StringRef yet, I'll do that in a follow-up.

show more ...

Rename F_{None,Text,Append} to OF_{None,Text,Append}. NFC

F_{None,Text,Append} are kept for compatibility since r334221.

llvm-svn: 367800

[FileCollector] Add a VFS that records FS accesses using the FileCollector

This patch adds a VFS that can be overlaid on top of another VFS
to record file system accesses using the FileCollector.
Th

[FileCollector] Add a VFS that records FS accesses using the FileCollector

This patch adds a VFS that can be overlaid on top of another VFS
to record file system accesses using the FileCollector.
This can help to gather files that are needed for reproducers.

Differential Revision: https://reviews.llvm.org/D65411

llvm-svn: 367278

show more ...

[FileCollector] add support for recording empty directories

The file collector class is useful for constructing reproducers by
creating a snapshot of the files that are accessed. Sometimes it might

[FileCollector] add support for recording empty directories

The file collector class is useful for constructing reproducers by
creating a snapshot of the files that are accessed. Sometimes it might
also be important to construct directories that don't necessarily have files,
but are still accessed by some tool that we want to make a reproducer for.
This is useful for instance for modeling the behavior of Clang's header search,
which scans through a number of directories it doesn't actually access when
looking for framework headers. This commit extends the file collector to allow
it to work with paths that are just directories, by constructing them as the
files are copied over.

Differential Revision: https://reviews.llvm.org/D65297

llvm-svn: 367061

show more ...

[FileCollector] Change coding style from LLDB to LLVM (NFC)

This patch changes the coding style of the FileCollector from the LLDB
to the LLVM coding style. Alex recently lifted it into LLVM and I
v

[FileCollector] Change coding style from LLDB to LLVM (NFC)

This patch changes the coding style of the FileCollector from the LLDB
to the LLVM coding style. Alex recently lifted it into LLVM and I
volunteered to do the conversion.

llvm-svn: 366966

show more ...

[Support] move FileCollector from LLDB to llvm/Support

The file collector class is useful for creating reproducers,
not just for LLDB, but for other tools as well in LLVM/Clang.

Differential Revisi

[Support] move FileCollector from LLDB to llvm/Support

The file collector class is useful for creating reproducers,
not just for LLDB, but for other tools as well in LLVM/Clang.

Differential Revision: https://reviews.llvm.org/D65237

llvm-svn: 366956

show more ...