[Lex] Introduce `PPCallbacks::LexedFileChanged()` preprocessor callback

This is a preprocessor callback focused on the lexed file changing, without conflating effects of line number directives and o

[Lex] Introduce `PPCallbacks::LexedFileChanged()` preprocessor callback

This is a preprocessor callback focused on the lexed file changing, without conflating effects of line number directives and other pragmas.
A client that only cares about what files the lexer processes, like dependency generation, can use this more straightforward
callback instead of `PPCallbacks::FileChanged()`. Clients that want the pragma directive effects as well can keep using `FileChanged()`.

A use case where `PPCallbacks::LexedFileChanged()` is particularly simpler to use than `FileChanged()` is in a situation
where a client wants to keep track of lexed file changes that include changes from/to the predefines buffer, where it becomes
unnecessary complicated trying to use `FileChanged()` while filtering out the pragma directives effects callbacks.

Also take the opportunity to provide information about the prior `FileID` the `Lexer` moved from, even when entering a new file.

Differential Revision: https://reviews.llvm.org/D128947

show more ...

[clang][lex] NFCI: Use FileEntryRef in PPCallbacks::InclusionDirective()

This patch changes type of the `File` parameter in `PPCallbacks::InclusionDirective()` from `const FileEntry *` to `Optional<

[clang][lex] NFCI: Use FileEntryRef in PPCallbacks::InclusionDirective()

This patch changes type of the `File` parameter in `PPCallbacks::InclusionDirective()` from `const FileEntry *` to `Optional<FileEntryRef>`.

With the API change in place, this patch then removes some uses of the deprecated `FileEntry::getName()` (e.g. in `DependencyGraph.cpp` and `ModuleDependencyCollector.cpp`).

Reviewed By: dexonsmith, bnbarham

Differential Revision: https://reviews.llvm.org/D123574

show more ...

[clang] In DependencyCollector on Windows, ignore case and separators when discarding duplicate dependency file paths.

This patch removes duplicates also encountered in the output of clang-scan-deps

[clang] In DependencyCollector on Windows, ignore case and separators when discarding duplicate dependency file paths.

This patch removes duplicates also encountered in the output of clang-scan-deps when one same header file is encountered with different casing and/or different separators ('/' vs '\').

The case of separators can appear when the same file is included externally by
`#include <folder/file.h>`

whereas a file from the same folder does
`#include "file.h"`

Under Windows, clang computes the paths using '/' from the include directive, the `\` from the -I options, and the concatenations use the native `\`, leading to internal paths containing a mix of both separators.

Differential Revision: https://reviews.llvm.org/D102339

show more ...

[SystemZ][z/OS][Windows] Add new OF_TextWithCRLF flag and use this flag instead of OF_Text

Problem:
On SystemZ we need to open text files in text mode. On Windows, files opened in text mode adds a C

[SystemZ][z/OS][Windows] Add new OF_TextWithCRLF flag and use this flag instead of OF_Text

Problem:
On SystemZ we need to open text files in text mode. On Windows, files opened in text mode adds a CRLF '\r\n' which may not be desirable.

Solution:
This patch adds two new flags

- OF_CRLF which indicates that CRLF translation is used.
- OF_TextWithCRLF = OF_Text | OF_CRLF indicates that the file is text and uses CRLF translation.

Developers should now use either the OF_Text or OF_TextWithCRLF for text files and OF_None for binary files. If the developer doesn't want carriage returns on Windows, they should use OF_Text, if they do want carriage returns on Windows, they should use OF_TextWithCRLF.

So this is the behaviour per platform with my patch:

z/OS:
OF_None: open in binary mode
OF_Text : open in text mode
OF_TextWithCRLF: open in text mode

Windows:
OF_None: open file with no carriage return
OF_Text: open file with no carriage return
OF_TextWithCRLF: open file with carriage return

The Major change is in llvm/lib/Support/Windows/Path.inc to only set text mode if the OF_CRLF is set.
```
if (Flags & OF_CRLF)
CrtOpenFlags |= _O_TEXT;
```

These following files are the ones that still use OF_Text which I left unchanged. I modified all these except raw_ostream.cpp in recent patches so I know these were previously in Binary mode on Windows.
./llvm/lib/Support/raw_ostream.cpp
./llvm/lib/TableGen/Main.cpp
./llvm/tools/dsymutil/DwarfLinkerForBinary.cpp
./llvm/unittests/Support/Path.cpp
./clang/lib/StaticAnalyzer/Core/HTMLDiagnostics.cpp
./clang/lib/Frontend/CompilerInstance.cpp
./clang/lib/Driver/Driver.cpp
./clang/lib/Driver/ToolChains/Clang.cpp

Reviewed By: MaskRay

Differential Revision: https://reviews.llvm.org/D99426

show more ...

[clang][cli] Generate and round-trip DependencyOutput options

This patch implements generation of remaining dependency output options and tests it by performing parse-generate-parse round trip.

Rev

[clang][cli] Generate and round-trip DependencyOutput options

This patch implements generation of remaining dependency output options and tests it by performing parse-generate-parse round trip.

Reviewed By: dexonsmith

Differential Revision: https://reviews.llvm.org/D96273

show more ...

SourceManager: getFileEntryRefForID => getNonBuiltinFilenameForID, NFC

`SourceManager::getFileEntryRefForID`'s remaining callers just want the
filename component, which is coming from the `FileInfo`

SourceManager: getFileEntryRefForID => getNonBuiltinFilenameForID, NFC

`SourceManager::getFileEntryRefForID`'s remaining callers just want the
filename component, which is coming from the `FileInfo`. Replace the API
with `getNonBuiltinFilenameForID`, which also removes another use of
`FileEntryRef::FileEntryRef` outside of `FileManager`.

Both callers are collecting file dependencies, and one of them relied on
this API to filter out built-ins (as exposed by
clang/test/ClangScanDeps/modules-full.cpp). It seems nice to continue
providing that service.

Differential Revision: https://reviews.llvm.org/D89508

show more ...

clang-cl: Add a `/showIncludes:user` flag.

This flag is like /showIncludes, but it only includes user headers and
omits system headers (similar to MD and MMD). The motivation is that
projects that a

clang-cl: Add a `/showIncludes:user` flag.

This flag is like /showIncludes, but it only includes user headers and
omits system headers (similar to MD and MMD). The motivation is that
projects that already track system includes though other means can use
this flag to get consistent behavior on Windows and non-Windows, and it
saves tools that output /showIncludes output (e.g. ninja) some work.

implementation-wise, this makes `HeaderIncludesCallback` honor the
existing `IncludeSystemHeaders` bit, and changes the three clients of
`HeaderIncludesCallback` (`/showIncludes`, `-H`, `CC_PRINT_HEADERS=1`)
to pass `-sys-header-deps` to set that bit -- except for
`/showIncludes:user`, which doesn't pass it.

Differential Revision: https://reviews.llvm.org/D75093

show more ...

Make llvm::StringRef to std::string conversions explicit.

This is how it should've been and brings it more in line with
std::string_view. There should be no functional change here.

This is mostly m

Make llvm::StringRef to std::string conversions explicit.

This is how it should've been and brings it more in line with
std::string_view. There should be no functional change here.

This is mostly mechanical from a custom clang-tidy check, with a lot of
manual fixups. It uncovers a lot of minor inefficiencies.

This doesn't actually modify StringRef yet, I'll do that in a follow-up.

show more ...

[clang][DependencyFileGenerator] Fix missing -MT option handling

Targets in DependencyFileGenerator don't necessarily come from -MT option.

Differential Revision: https://reviews.llvm.org/D67308

l

[clang][DependencyFileGenerator] Fix missing -MT option handling

Targets in DependencyFileGenerator don't necessarily come from -MT option.

Differential Revision: https://reviews.llvm.org/D67308

llvm-svn: 371279

show more ...

Use FileEntryRef for PPCallbacks::HasInclude

This fixes the issue where a filename dependendency was missing if the file that
was referenced with __has_include() was accessed through a symlink in an

Use FileEntryRef for PPCallbacks::HasInclude

This fixes the issue where a filename dependendency was missing if the file that
was referenced with __has_include() was accessed through a symlink in an earlier run,
if the file manager was reused between runs.

llvm-svn: 370081

show more ...

Use FileEntryRef for PPCallbacks::FileSkipped

This fixes the issue where a filename dependendency was missing if the file that
was skipped was included through a symlink in an earlier run, if the fi

Use FileEntryRef for PPCallbacks::FileSkipped

This fixes the issue where a filename dependendency was missing if the file that
was skipped was included through a symlink in an earlier run, if the file
manager was reused between runs.

llvm-svn: 369998

show more ...

Introduce FileEntryRef and use it when handling includes to report correct dependencies
when the FileManager is reused across invocations

This commit introduces a parallel API to FileManager's getFi

Introduce FileEntryRef and use it when handling includes to report correct dependencies
when the FileManager is reused across invocations

This commit introduces a parallel API to FileManager's getFile: getFileEntryRef, which returns
a reference to the FileEntry, and the name that was used to access the file. In the case of
a VFS with 'use-external-names', the FileEntyRef contains the external name of the file,
not the filename that was used to access it.

The new API is adopted only in the HeaderSearch and Preprocessor for include file lookup, so that the
accessed path can be propagated to SourceManager's FileInfo. SourceManager's FileInfo now can report this accessed path, using
the new getName method. This API is then adopted in the dependency collector, which now correctly reports dependencies when a file
is included both using a symlink and a real path in the case when the FileManager is reused across multiple Preprocessor invocations.

Note that this patch does not fix all dependency collector issues, as the same problem is still present in other cases when dependencies
are obtained using FileSkipped, InclusionDirective, and HasInclude. This will be fixed in follow-up commits.

Differential Revision: https://reviews.llvm.org/D65907

llvm-svn: 369680

show more ...

[Clang] Migrate llvm::make_unique to std::make_unique

Now that we've moved to C++14, we no longer need the llvm::make_unique
implementation from STLExtras.h. This patch is a mechanical replacement
o

[Clang] Migrate llvm::make_unique to std::make_unique

Now that we've moved to C++14, we no longer need the llvm::make_unique
implementation from STLExtras.h. This patch is a mechanical replacement
of (hopefully) all the llvm::make_unique instances across the monorepo.

Differential revision: https://reviews.llvm.org/D66259

llvm-svn: 368942

show more ...

Rename F_{None,Text,Append} to OF_{None,Text,Append}. NFC

F_{None,Text,Append} are kept for compatibility since r334221.

llvm-svn: 367800

[clang-scan-deps] print the dependencies to stdout
and remove the need to use -MD options in the CDB

Differential Revision: https://reviews.llvm.org/D63579

llvm-svn: 364088

Unify DependencyFileGenerator class and DependencyCollector interface (NFCI)

Make DependencyFileGenerator a DependencyCollector as it was intended when
DependencyCollector was introduced. The missin

Unify DependencyFileGenerator class and DependencyCollector interface (NFCI)

Make DependencyFileGenerator a DependencyCollector as it was intended when
DependencyCollector was introduced. The missing PPCallbacks overrides are added to
the DependencyCollector as well.

This change will allow clang-scan-deps to access the produced dependencies without
writing them out to .d files to disk, so that it will be able collate them and
report them to the user.

Differential Revision: https://reviews.llvm.org/D63290

llvm-svn: 363840

show more ...

Update the file headers across all of the LLVM projects in the monorepo
to reflect the new license.

We understand that people may be surprised that we're moving the header
entirely to discuss the ne

Update the file headers across all of the LLVM projects in the monorepo
to reflect the new license.

We understand that people may be surprised that we're moving the header
entirely to discuss the new license. We checked this carefully with the
Foundation's lawyer and we believe this is the correct approach.

Essentially, all code in the project is now made available by the LLVM
project under our new license, so you will see that the license headers
include that license only. Some of our contributors have contributed
code under our old license, and accordingly, we have retained a copy of
our old license notice in the top-level files in each project and
repository.

llvm-svn: 351636

show more ...

Fix up diagnostics.

Move some diagnostics around between Diagnostic*Kinds.td files. Diagnostics
used in multiple places were moved to DiagnosticCommonKinds.td. Diagnostics
listed in the wrong plac

Fix up diagnostics.

Move some diagnostics around between Diagnostic*Kinds.td files. Diagnostics
used in multiple places were moved to DiagnosticCommonKinds.td. Diagnostics
listed in the wrong place (ie, Sema diagnostics listed in
DiagnosticsParseKinds.td) were moved to the correct places. One diagnostic
split into two so that the diagnostic string is in the .td file instead of in
code. Cleaned up the diagnostic includes after all the changes.

llvm-svn: 349125

show more ...

Add a callback for `__has_include` and use it for dependency scanning.

This adds a preprocessor callback for the `__has_include` and
`__has_include_next` directives.

Successful checking for the pre

Add a callback for `__has_include` and use it for dependency scanning.

This adds a preprocessor callback for the `__has_include` and
`__has_include_next` directives.

Successful checking for the presence of a header should add it to the list of
header dependencies so this overrides the callback in the dependency scanner.

Patch by Pete Cooper with some additions by me.

rdar://problem/39545636

Differential Revision: https://reviews.llvm.org/D30882

llvm-svn: 342517

show more ...

Print correctly dependency paths on Windows

Summary:
Before:
main.o: main.c ../include/lib\test.h

After:
main.o: main.c ../include/lib/test.h

Fixes PR38877

Reviewers: zturner

Subscribers: xbolva

Print correctly dependency paths on Windows

Summary:
Before:
main.o: main.c ../include/lib\test.h

After:
main.o: main.c ../include/lib/test.h

Fixes PR38877

Reviewers: zturner

Subscribers: xbolva00, cfe-commits

Differential Revision: https://reviews.llvm.org/D51847

llvm-svn: 342139

show more ...

Remove trailing space

sed -Ei 's/[[:space:]]+$//' include/**/*.{def,h,td} lib/**/*.{cpp,h}

llvm-svn: 338291

Fix emission of phony dependency targets when adding extra deps

Summary:
This commit fixes a bug where passing extra dependency entries
(using -fdepfile-entry) would result in -MP incorrectly emitti

Fix emission of phony dependency targets when adding extra deps

Summary:
This commit fixes a bug where passing extra dependency entries
(using -fdepfile-entry) would result in -MP incorrectly emitting
a phony target for the input file, and no phony target for the
first extra dependency.

The extra dependencies are added first to the filename vector in
DFGImpl. That clashed with the emission of the phony targets, as
the code assumed that the first index always correspond to the
input file.

Reviewers: rsmith, pcc, krasin, bruno, vsapsai

Reviewed By: vsapsai

Subscribers: vsapsai, bruno, cfe-commits

Differential Revision: https://reviews.llvm.org/D44568

llvm-svn: 333413

show more ...

Reland '[clang] Adding CharacteristicKind to PPCallbacks::InclusionDirective'

This commit relands r331904.

Adding a SrcMgr::CharacteristicKind parameter to the InclusionDirective
in PPCallbacks, an

Reland '[clang] Adding CharacteristicKind to PPCallbacks::InclusionDirective'

This commit relands r331904.

Adding a SrcMgr::CharacteristicKind parameter to the InclusionDirective
in PPCallbacks, and updating calls to that function. This will be useful
in https://reviews.llvm.org/D43778 to determine which includes are
system
headers.

Differential Revision: https://reviews.llvm.org/D46614

llvm-svn: 332021

show more ...

Revert "[clang] Adding CharacteristicKind to PPCallbacks::InclusionDirective"

This reverts commit r331904 because of a memory leak.

llvm-svn: 331932

[clang] Adding CharacteristicKind to PPCallbacks::InclusionDirective

Adding a SrcMgr::CharacteristicKind parameter to the InclusionDirective
in PPCallbacks, and updating calls to that function. This

[clang] Adding CharacteristicKind to PPCallbacks::InclusionDirective

Adding a SrcMgr::CharacteristicKind parameter to the InclusionDirective
in PPCallbacks, and updating calls to that function. This will be useful
in https://reviews.llvm.org/D43778 to determine which includes are system
headers.

Differential Revision: https://reviews.llvm.org/D46614

llvm-svn: 331904

show more ...