[C++20][Modules][HU 1/5] Introduce header units as a module type.

This is the first in a series of patches that introduce C++20 importable
header units.

These differ from clang header modules in th

[C++20][Modules][HU 1/5] Introduce header units as a module type.

This is the first in a series of patches that introduce C++20 importable
header units.

These differ from clang header modules in that:
(a) they are identifiable by an internal name
(b) they represent the top level source for a single header - although
that might include or import other headers.

We name importable header units with the path by which they are specified
(although that need not be the absolute path for the file).

So "foo/bar.h" would have a name "foo/bar.h". Header units are made a
separate module type so that we can deal with diagnosing places where they
are permitted but a named module is not.

Differential Revision: https://reviews.llvm.org/D121095

show more ...

[clang][extract-api] Refactor ExtractAPI and improve docs

- The name SymbolGraph is inappropriate and confusing for the new library
for clang-extract-api. Refactor and rename things to make it cle

[clang][extract-api] Refactor ExtractAPI and improve docs

- The name SymbolGraph is inappropriate and confusing for the new library
for clang-extract-api. Refactor and rename things to make it clear that
ExtractAPI is the core functionality and SymbolGraph is one serializer
for the API information.
- Add documentation comments to ExtractAPI classes and methods to improve
readability and clearness of the ExtractAPI work.

Differential Revision: https://reviews.llvm.org/D122160

show more ...

[clang][extract-api] Add global record support

Add facilities for extract-api:
- Structs/classes to hold collected API information: `APIRecord`, `API`
- Structs/classes for API information:
- `Ava

[clang][extract-api] Add global record support

Add facilities for extract-api:
- Structs/classes to hold collected API information: `APIRecord`, `API`
- Structs/classes for API information:
- `AvailabilityInfo`: aggregated availbility information
- `DeclarationFragments`: declaration fragments
- `DeclarationFragmentsBuilder`: helper class to build declaration
fragments for various types/declarations
- `FunctionSignature`: function signature
- Serialization: `Serializer`
- Add output file for `ExtractAPIAction`
- Refactor `clang::RawComment::getFormattedText` to provide an
additional `getFormattedLines` for a more detailed view of comment lines
used for the SymbolGraph format

Add support for global records (global variables and functions)
- Add `GlobalRecord` based on `APIRecord` to store global records'
information
- Implement `VisitVarDecl` and `VisitFunctionDecl` in `ExtractAPIVisitor` to
collect information
- Implement serialization for global records
- Add test case for global records

Differential Revision: https://reviews.llvm.org/D119479

show more ...

[clang] Add an extract-api driver option

This is the initial commit for the clang-extract-api RFC
<https://lists.llvm.org/pipermail/cfe-dev/2021-September/068768.html>
Add a new driver option `-extr

[clang] Add an extract-api driver option

This is the initial commit for the clang-extract-api RFC
<https://lists.llvm.org/pipermail/cfe-dev/2021-September/068768.html>
Add a new driver option `-extract-api` and associate it with a dummy
(for now) frontend action to set up the initial structure for
incremental works.

Differential Revision: https://reviews.llvm.org/D117809

show more ...

[clang] Add plugin ActionType to run command line plugin before main action

Currently we have a way to run a plugin if specified on the command line
after the main action, and ways to unconditionall

[clang] Add plugin ActionType to run command line plugin before main action

Currently we have a way to run a plugin if specified on the command line
after the main action, and ways to unconditionally run the plugin before
or after the main action, but no way to run a plugin if specified on the
command line before the main action.

This introduces the missing option.

This is helpful because -clear-ast-before-backend clears the AST before
codegen, while some plugins may want access to the AST.

Reviewed By: dblaikie

Differential Revision: https://reviews.llvm.org/D112096

show more ...

Reland "[clang-repl] Allow loading of plugins in clang-repl."

Differential revision: https://reviews.llvm.org/D110484

Revert "[clang-repl] Allow loading of plugins in clang-repl."

This reverts commit 81fb640f83b6a5d099f9124739ab3049be79ea56 due to bot failures:
https://lab.llvm.org/buildbot#builders/57/builds/10807

[clang-repl] Allow loading of plugins in clang-repl.

Differential revision: https://reviews.llvm.org/D110484

[OptTable] Rename PrintHelp to printHelp

To be consistent with other member functions and match the coding standard.

[NFC] Refactor Registry loops to range for

[llvm][clang][IFS] Enhancing the llvm-ifs yaml format for symbol lists.

Prior to this change the clang interface stubs format resembled
something ending with a symbol list like this:

Symbols:
a

[llvm][clang][IFS] Enhancing the llvm-ifs yaml format for symbol lists.

Prior to this change the clang interface stubs format resembled
something ending with a symbol list like this:

Symbols:
a: { Type: Func }

This was problematic because we didn't actually want a map format and
also because we didn't like that an empty symbol list required
"Symbols: {}". That is to say without the empty {} llvm-ifs would crash
on an empty list.

With this new format it is much more clear which field is the symbol
name, and instead the [] that is used to express an empty symbol vector
is optional, ie:

Symbols:
- { Name: a, Type: Func }

or

Symbols: []

or

Symbols:

This further diverges the format from existing llvm-elftapi. This is a
good thing because although the format originally came from the same
place, they are not the same in any way.

Differential Revision: https://reviews.llvm.org/D76979

show more ...

Reland "[analyzer][NFC] Tie CheckerRegistry to CheckerManager, allow CheckerManager to be constructed for non-analysis purposes"

Originally commited in rG57b8a407493c34c3680e7e1e4cb82e097f43744a, bu

Reland "[analyzer][NFC] Tie CheckerRegistry to CheckerManager, allow CheckerManager to be constructed for non-analysis purposes"

Originally commited in rG57b8a407493c34c3680e7e1e4cb82e097f43744a, but
it broke the modules bot. This is solved by putting the contructors of
the CheckerManager class to the Frontend library.

Differential Revision: https://reviews.llvm.org/D75360

show more ...

Revert "[analyzer][NFC] Tie CheckerRegistry to CheckerManager, allow CheckerManager to be constructed for non-analysis purposes"

Temporarily reverting this patch because it breaks the modules build.

[analyzer][NFC] Tie CheckerRegistry to CheckerManager, allow CheckerManager to be constructed for non-analysis purposes

Its been a while since my CheckerRegistry related patches landed, allow me to

[analyzer][NFC] Tie CheckerRegistry to CheckerManager, allow CheckerManager to be constructed for non-analysis purposes

Its been a while since my CheckerRegistry related patches landed, allow me to
refresh your memory:

During compilation, TblGen turns
clang/include/clang/StaticAnalyzer/Checkers/Checkers.td into
(build directory)/tools/clang/include/clang/StaticAnalyzer/Checkers/Checkers.inc.
This is a file that contains the full name of the checkers, their options, etc.

The class that is responsible for parsing this file is CheckerRegistry. The job
of this class is to establish what checkers are available for the analyzer (even
from plugins and statically linked but non-tblgen generated files!), and
calculate which ones should be turned on according to the analyzer's invocation.

CheckerManager is the class that is responsible for the construction and storage
of checkers. This process works by first creating a CheckerRegistry object, and
passing itself to CheckerRegistry::initializeManager(CheckerManager&), which
will call the checker registry functions (for example registerMallocChecker) on
it.

The big problem here is that these two classes lie in two different libraries,
so their interaction is pretty awkward. This used to be far worse, but I
refactored much of it, which made things better but nowhere near perfect.

---

This patch changes how the above mentioned two classes interact. CheckerRegistry
is mainly used by CheckerManager, and they are so intertwined, it makes a lot of
sense to turn in into a field, instead of a one-time local variable. This has
additional benefits: much of the information that CheckerRegistry conveniently
holds is no longer thrown away right after the analyzer's initialization, and
opens the possibility to pass CheckerManager in the shouldRegister* function
rather then LangOptions (D75271).

There are a few problems with this. CheckerManager isn't the only user, when we
honor help flags like -analyzer-checker-help, we only have access to a
CompilerInstance class, that is before the point of parsing the AST.
CheckerManager makes little sense without ASTContext, so I made some changes and
added new constructors to make it constructible for the use of help flags.

Differential Revision: https://reviews.llvm.org/D75360

show more ...

Make llvm::StringRef to std::string conversions explicit.

This is how it should've been and brings it more in line with
std::string_view. There should be no functional change here.

This is mostly m

Make llvm::StringRef to std::string conversions explicit.

This is how it should've been and brings it more in line with
std::string_view. There should be no functional change here.

This is mostly mechanical from a custom clang-tidy check, with a lot of
manual fixups. It uncovers a lot of minor inefficiencies.

This doesn't actually modify StringRef yet, I'll do that in a follow-up.

show more ...

[analyzer] Don't run the analyzer for -analyzer-list-enabled-checkers

Short and sweet. Whenever I use -analyzer-list-enabled-checkers, I'm only
interested about the configuration, not about the anal

[analyzer] Don't run the analyzer for -analyzer-list-enabled-checkers

Short and sweet. Whenever I use -analyzer-list-enabled-checkers, I'm only
interested about the configuration, not about the analysis.

Differential Revision: https://reviews.llvm.org/D66714

llvm-svn: 371756

show more ...

[Driver] Use shared singleton instance of DriverOptTable

Summary:
This significantly reduces the time required to run clangd tests, by
~10%.

Should also have an effect on other tests that run comma

[Driver] Use shared singleton instance of DriverOptTable

Summary:
This significantly reduces the time required to run clangd tests, by
~10%.

Should also have an effect on other tests that run command-line parsing
multiple times inside a single invocation.

Reviewers: gribozavr, sammccall

Reviewed By: sammccall

Subscribers: kadircet, cfe-commits

Tags: #clang

Differential Revision: https://reviews.llvm.org/D67163

llvm-svn: 370908

show more ...

[clang][ifs] Dropping older experimental interface stub formats.

I've been working on a new tool, llvm-ifs, for merging interface stub files
generated by clang and I've iterated on my derivative for

[clang][ifs] Dropping older experimental interface stub formats.

I've been working on a new tool, llvm-ifs, for merging interface stub files
generated by clang and I've iterated on my derivative format of TBE to a newer
format. llvm-ifs will only support the new format, so I am going to drop the
older experimental interface stubs formats in this commit to make things
simpler.

Differential Revision: https://reviews.llvm.org/D66573

llvm-svn: 369719

show more ...

[clang][ifs] New interface stubs format (llvm triple based).

After posting llvm-ifs on phabricator, I made some progress in hardening up how
I think the format for Interface Stubs should look. There

[clang][ifs] New interface stubs format (llvm triple based).

After posting llvm-ifs on phabricator, I made some progress in hardening up how
I think the format for Interface Stubs should look. There are a number of
things I think the TBE format was missing (no endianness, no info about the
Object Format because it assumes ELF), so I have added those and broken off
from being as similar to the TBE schema. In a subsequent commit I can drop the
other formats.

An example of how The format will look is as follows:

--- !experimental-ifs-v1
IfsVersion: 1.0
Triple: x86_64-unknown-linux-gnu
ObjectFileFormat: ELF
Symbols:
_Z9nothiddenv: { Type: Func }
_Z10cmdVisiblev: { Type: Func }
...

The format is still marked experimental.

Differential Revision: https://reviews.llvm.org/D66446

llvm-svn: 369715

show more ...

[Clang] Migrate llvm::make_unique to std::make_unique

Now that we've moved to C++14, we no longer need the llvm::make_unique
implementation from STLExtras.h. This patch is a mechanical replacement
o

[Clang] Migrate llvm::make_unique to std::make_unique

Now that we've moved to C++14, we no longer need the llvm::make_unique
implementation from STLExtras.h. This patch is a mechanical replacement
of (hopefully) all the llvm::make_unique instances across the monorepo.

Differential revision: https://reviews.llvm.org/D66259

llvm-svn: 368942

show more ...

[clang-ifs] Clang Interface Stubs, first version (second landing attempt).

This change reverts r363649; effectively re-landing r363626. At this point
clang::Index::CodegenNameGeneratorImpl has been

[clang-ifs] Clang Interface Stubs, first version (second landing attempt).

This change reverts r363649; effectively re-landing r363626. At this point
clang::Index::CodegenNameGeneratorImpl has been refactored into
clang::AST::ASTNameGenerator. This makes it so that the previous circular link
dependency no longer exists, fixing the previous share lib
(-DBUILD_SHARED_LIBS=ON) build issue which was the reason for r363649.

Clang interface stubs (previously referred to as clang-ifsos) is a new frontend
action in clang that allows the generation of stub files that contain mangled
name info that can be used to produce a stub library. These stub libraries can
be useful for breaking up build dependencies and controlling access to a
library's internal symbols. Generation of these stubs can be invoked by:

clang -fvisibility=<visibility> -emit-interface-stubs \
-interface-stub-version=<interface format>

Notice that -fvisibility (along with use of visibility attributes) can be used
to control what symbols get generated. Currently the interface format is
experimental but there are a wide range of possibilities here.

Currently clang-ifs produces .ifs files that can be thought of as analogous to
object (.o) files, but just for the mangled symbol info. In a subsequent patch
I intend to add support for merging the .ifs files into one .ifs/.ifso file
that can be the input to something like llvm-elfabi to produce something like a
.so file or .dll (but without any of the code, just symbols).

Differential Revision: https://reviews.llvm.org/D60974

llvm-svn: 363948

show more ...

Revert D60974 "[clang-ifs] Clang Interface Stubs, first version."

This reverts commit rC363626.

clangIndex depends on clangFrontend. r363626 adds a dependency from
clangFrontend to clangIndex, whic

Revert D60974 "[clang-ifs] Clang Interface Stubs, first version."

This reverts commit rC363626.

clangIndex depends on clangFrontend. r363626 adds a dependency from
clangFrontend to clangIndex, which creates a circular dependency.

This is disallowed by -DBUILD_SHARED_LIBS=on builds:

CMake Error: The inter-target dependency graph contains the following strongly connected component (cycle):
"clangFrontend" of type SHARED_LIBRARY
depends on "clangIndex" (weak)
"clangIndex" of type SHARED_LIBRARY
depends on "clangFrontend" (weak)
At least one of these targets is not a STATIC_LIBRARY. Cyclic dependencies are allowed only among static libraries.

Note, the dependency on clangIndex cannot be removed because
libclangFrontend.so is linked with -Wl,-z,defs: a shared object must
have its full direct dependencies specified on the linker command line.

In -DBUILD_SHARED_LIBS=off builds, this appears to work when linking
`bin/clang-9`. However, it can cause trouble to downstream clang library
users. The llvm build system links libraries this way:

clang main_program_object_file ... lib/libclangIndex.a ... lib/libclangFrontend.a -o exe

libclangIndex.a etc are not wrapped in --start-group.

If the downstream application depends on libclangFrontend.a but not any
other clang libraries that depend on libclangIndex.a, this can cause undefined
reference errors when the linker is ld.bfd or gold.

The proper fix is to not include clangIndex files in clangFrontend.

llvm-svn: 363649

show more ...

[clang-ifs] Clang Interface Stubs, first version.

Clang interface stubs (previously referred to as clang-ifsos) is a new frontend
action in clang that allows the generation of stub files that contai

[clang-ifs] Clang Interface Stubs, first version.

Clang interface stubs (previously referred to as clang-ifsos) is a new frontend
action in clang that allows the generation of stub files that contain mangled
name info that can be used to produce a stub library. These stub libraries can
be useful for breaking up build dependencies and controlling access to a
library's internal symbols. Generation of these stubs can be invoked by:

clang -fvisibility=<visibility> -emit-interface-stubs \
-interface-stub-version=<interface format>

Notice that -fvisibility (along with use of visibility attributes) can be used
to control what symbols get generated. Currently the interface format is
experimental but there are a wide range of possibilities here.

Differential Revision: https://reviews.llvm.org/D60974

llvm-svn: 363626

show more ...

Add clang source minimizer that reduces source to directives
that might affect the dependency list for a compilation

This commit introduces a dependency directives source minimizer to clang
that min

Add clang source minimizer that reduces source to directives
that might affect the dependency list for a compilation

This commit introduces a dependency directives source minimizer to clang
that minimizes header and source files to the minimum necessary preprocessor
directives for evaluating includes. It reduces the source down to #define, #include,

The source minimizer works by lexing the input with a custom fast lexer that recognizes
the preprocessor directives it cares about, and emitting those directives in the minimized source.
It ignores source code, comments, and normalizes whitespace. It gives up and fails if seems
any directives that it doesn't recognize as valid (e.g. #define 0).

In addition to the source minimizer this patch adds a
-print-dependency-directives-minimized-source CC1 option that allows you to invoke the minimizer
from clang directly.

Differential Revision: https://reviews.llvm.org/D55463

llvm-svn: 362459

show more ...

[analyzer] List checker/plugin options in 3 categories: released, alpha, developer

Same patch as D62093, but for checker/plugin options, the only
difference being that options for alpha checkers are

[analyzer] List checker/plugin options in 3 categories: released, alpha, developer

Same patch as D62093, but for checker/plugin options, the only
difference being that options for alpha checkers are implicitly marked
as alpha.

Differential Revision: https://reviews.llvm.org/D62093

llvm-svn: 361566

show more ...