Change Metadata Index emission in the bitcode to use 2x32 bits for the placeholder

The Bitstream reader and writer are limited to handle a "size_t" at
most, which means that we can't backpatch and r

Change Metadata Index emission in the bitcode to use 2x32 bits for the placeholder

The Bitstream reader and writer are limited to handle a "size_t" at
most, which means that we can't backpatch and read back a 64bits
value on 32 bits platform.

llvm-svn: 290693

show more ...

Add an index for Module Metadata record in the bitcode

This index record the position for each metadata record in
the bitcode, so that the reader will be able to lazy-load
on demand each individual

Add an index for Module Metadata record in the bitcode

This index record the position for each metadata record in
the bitcode, so that the reader will be able to lazy-load
on demand each individual record.

We also make sure that every abbrev is emitted upfront so
that the block can be skipped while reading.

I don't plan to commit this before having the reader
counterpart, but I figured this can be reviewed mostly
independently.

Recommit r290684 (was reverted in r290686 because a test
was broken) after adding a threshold to avoid emitting
the index when unnecessary (little amount of metadata).
This optimization "hides" a limitation of the ability
to backpatch in the bitstream: we can only backpatch
safely when the position has been flushed. So if we emit
an index for one metadata, it is possible that (part of)
the offset placeholder hasn't been flushed and the backpatch
will fail.

Differential Revision: https://reviews.llvm.org/D28083

llvm-svn: 290690

show more ...

Revert "Add an index for Module Metadata record in the bitcode"

This reverts commit a0ca6ae2d38339e4ede0dfa588086fc23d87e836. Revert at
Mehdi's request as it is breaking bots.

llvm-svn: 290686

Add an index for Module Metadata record in the bitcode

Summary:
This index record the position for each metadata record in
the bitcode, so that the reader will be able to lazy-load
on demand each in

Add an index for Module Metadata record in the bitcode

Summary:
This index record the position for each metadata record in
the bitcode, so that the reader will be able to lazy-load
on demand each individual record.

We also make sure that every abbrev is emitted upfront so
that the block can be skipped while reading.

I don't plan to commit this before having the reader
counterpart, but I figured this can be reviewed mostly
independently.

Reviewers: pcc, tejohnson

Subscribers: llvm-commits

Differential Revision: https://reviews.llvm.org/D28083

llvm-svn: 290684

show more ...

[DebugInfo] Added support for Checksum debug info feature.

Differential Revision: https://reviews.llvm.org/D27642

llvm-svn: 290514

IR: Function summary representation for type tests.

Each function summary has an attached list of type identifier GUIDs. The
idea is that during the regular LTO phase we would match these GUIDs to t

IR: Function summary representation for type tests.

Each function summary has an attached list of type identifier GUIDs. The
idea is that during the regular LTO phase we would match these GUIDs to type
identifiers defined by the regular LTO module and store the resolutions in
a top-level "type identifier summary" (which will be implemented separately).

Differential Revision: https://reviews.llvm.org/D27967

llvm-svn: 290280

show more ...

IR: Eliminate non-determinism in the module summary analysis.

Also make the summary ref and call graph vectors immutable. This means
a smaller API surface and fewer places to audit for non-determini

IR: Eliminate non-determinism in the module summary analysis.

Also make the summary ref and call graph vectors immutable. This means
a smaller API surface and fewer places to audit for non-determinism.

Differential Revision: https://reviews.llvm.org/D27875

llvm-svn: 290200

show more ...

[IR] Remove the DIExpression field from DIGlobalVariable.

This patch implements PR31013 by introducing a
DIGlobalVariableExpression that holds a pair of DIGlobalVariable and
DIExpression.

Currently

[IR] Remove the DIExpression field from DIGlobalVariable.

This patch implements PR31013 by introducing a
DIGlobalVariableExpression that holds a pair of DIGlobalVariable and
DIExpression.

Currently, DIGlobalVariables holds a DIExpression. This is not the
best way to model this:

(1) The DIGlobalVariable should describe the source level variable,
not how to get to its location.

(2) It makes it unsafe/hard to update the expressions when we call
replaceExpression on the DIGLobalVariable.

(3) It makes it impossible to represent a global variable that is in
more than one location (e.g., a variable with multiple
DW_OP_LLVM_fragment-s). We also moved away from attaching the
DIExpression to DILocalVariable for the same reasons.

This reapplies r289902 with additional testcase upgrades and a change
to the Bitcode record for DIGlobalVariable, that makes upgrading the
old format unambiguous also for variables without DIExpressions.

<rdar://problem/29250149>
https://llvm.org/bugs/show_bug.cgi?id=31013
Differential Revision: https://reviews.llvm.org/D26769

llvm-svn: 290153

show more ...

Revert "[IR] Remove the DIExpression field from DIGlobalVariable."

This reverts commit 289920 (again).
I forgot to implement a Bitcode upgrade for the case where a DIGlobalVariable
has not DIExpress

Revert "[IR] Remove the DIExpression field from DIGlobalVariable."

This reverts commit 289920 (again).
I forgot to implement a Bitcode upgrade for the case where a DIGlobalVariable
has not DIExpression. Unfortunately it is not possible to safely upgrade
these variables without adding a flag to the bitcode record indicating which
version they are.
My plan of record is to roll the planned follow-up patch that adds a
unit: field to DIGlobalVariable into this patch before recomitting.
This way we only need one Bitcode upgrade for both changes (with a
version flag in the bitcode record to safely distinguish the record
formats).

Sorry for the churn!

llvm-svn: 289982

show more ...

[IR] Remove the DIExpression field from DIGlobalVariable.

This patch implements PR31013 by introducing a
DIGlobalVariableExpression that holds a pair of DIGlobalVariable and
DIExpression.

Currently

[IR] Remove the DIExpression field from DIGlobalVariable.

This patch implements PR31013 by introducing a
DIGlobalVariableExpression that holds a pair of DIGlobalVariable and
DIExpression.

Currently, DIGlobalVariables holds a DIExpression. This is not the
best way to model this:

(1) The DIGlobalVariable should describe the source level variable,
not how to get to its location.

(2) It makes it unsafe/hard to update the expressions when we call
replaceExpression on the DIGLobalVariable.

(3) It makes it impossible to represent a global variable that is in
more than one location (e.g., a variable with multiple
DW_OP_LLVM_fragment-s). We also moved away from attaching the
DIExpression to DILocalVariable for the same reasons.

This reapplies r289902 with additional testcase upgrades.

<rdar://problem/29250149>
https://llvm.org/bugs/show_bug.cgi?id=31013
Differential Revision: https://reviews.llvm.org/D26769

llvm-svn: 289920

show more ...

Revert "[IR] Remove the DIExpression field from DIGlobalVariable."

This reverts commit 289902 while investigating bot berakage.

llvm-svn: 289906

[IR] Remove the DIExpression field from DIGlobalVariable.

This patch implements PR31013 by introducing a
DIGlobalVariableExpression that holds a pair of DIGlobalVariable and
DIExpression.

Currently

[IR] Remove the DIExpression field from DIGlobalVariable.

This patch implements PR31013 by introducing a
DIGlobalVariableExpression that holds a pair of DIGlobalVariable and
DIExpression.

Currently, DIGlobalVariables holds a DIExpression. This is not the
best way to model this:

(1) The DIGlobalVariable should describe the source level variable,
not how to get to its location.

(2) It makes it unsafe/hard to update the expressions when we call
replaceExpression on the DIGLobalVariable.

(3) It makes it impossible to represent a global variable that is in
more than one location (e.g., a variable with multiple
DW_OP_LLVM_fragment-s). We also moved away from attaching the
DIExpression to DILocalVariable for the same reasons.

<rdar://problem/29250149>
https://llvm.org/bugs/show_bug.cgi?id=31013
Differential Revision: https://reviews.llvm.org/D26769

llvm-svn: 289902

show more ...

Fix MSVC bool to uint64_t promotion warning

llvm-svn: 288796

[DIExpression] Introduce a dedicated DW_OP_LLVM_fragment operation
so we can stop using DW_OP_bit_piece with the wrong semantics.

The entire back story can be found here:
http://lists.llvm.org/piper

[DIExpression] Introduce a dedicated DW_OP_LLVM_fragment operation
so we can stop using DW_OP_bit_piece with the wrong semantics.

The entire back story can be found here:
http://lists.llvm.org/pipermail/llvm-commits/Week-of-Mon-20161114/405934.html

The gist is that in LLVM we've been misinterpreting DW_OP_bit_piece's
offset field to mean the offset into the source variable rather than
the offset into the location at the top the DWARF expression stack. In
order to be able to fix this in a subsequent patch, this patch
introduces a dedicated DW_OP_LLVM_fragment operation with the
semantics that we used to apply to DW_OP_bit_piece, which is what we
actually need while inside of LLVM. This patch is complete with a
bitcode upgrade for expressions using the old format. It does not yet
fix the DWARF backend to use DW_OP_bit_piece correctly.

Implementation note: We discussed several options for implementing
this, including reserving a dedicated field in DIExpression for the
fragment size and offset, but using an custom operator at the end of
the expression works just fine and is more efficient because we then
only pay for it when we need it.

Differential Revision: https://reviews.llvm.org/D27361
rdar://problem/29335809

llvm-svn: 288683

show more ...

Bitcode: Introduce BitcodeWriter interface.

This interface allows clients to write multiple modules to a single
bitcode file. Also introduce the llvm-cat utility which can be used
to create a bitcod

Bitcode: Introduce BitcodeWriter interface.

This interface allows clients to write multiple modules to a single
bitcode file. Also introduce the llvm-cat utility which can be used
to create a bitcode file containing multiple modules.

Differential Revision: https://reviews.llvm.org/D26179

llvm-svn: 288195

show more ...

Restore "[ThinLTO] Prevent exporting of locals used/defined in module level asm"

This restores the rest of r286297 (part was restored in r286475).
Specifically, it restores the part requiring adding

Restore "[ThinLTO] Prevent exporting of locals used/defined in module level asm"

This restores the rest of r286297 (part was restored in r286475).
Specifically, it restores the part requiring adding a dependency from
the Analysis to Object library (downstream use changed to correctly
model split BitReader vs BitWriter libraries).

Original description of this part of patch follows:

Module level asm may also contain defs of values. We need to prevent
export of any refs to local values defined in module level asm (e.g. a
ref in normal IR), since that also requires renaming/promotion of the
local. To do that, the summary index builder looks at all values in the
module level asm string that are not marked Weak or Global, which is
exactly the set of locals that are defined. A summary is created for
each of these local defs and flagged as NoRename.

This required adding handling to the BitcodeWriter to look at GV
declarations to see if they have a summary (rather than skipping them
all).

Finally, added an assert to IRObjectFile::CollectAsmUndefinedRefs to
ensure that an MCAsmParser is available, otherwise the module asm parse
would silently fail. Initialized the asm parser in the opt tool for use
in testing this fix.

Fixes PR30610.

llvm-svn: 286844

show more ...

[ThinLTO] Make inline assembly handling more efficient in summary

Summary:
The change in r285513 to prevent exporting of locals used in
inline asm added all locals in the llvm.used set to the refere

[ThinLTO] Make inline assembly handling more efficient in summary

Summary:
The change in r285513 to prevent exporting of locals used in
inline asm added all locals in the llvm.used set to the reference
set of functions containing inline asm. Since these locals were marked
NoRename, this automatically prevented importing of the function.

Unfortunately, this caused an explosion in the summary reference lists
in some cases. In my particular example, it happened for a large protocol
buffer generated C++ file, where many of the generated functions
contained an inline asm call. It was exacerbated when doing a ThinLTO
PGO instrumentation build, where the PGO instrumentation included
thousands of private __profd_* values that were added to llvm.used.

We really only need to include a single llvm.used local (NoRename) value
in the reference list of a function containing inline asm to block it
being imported. However, it seems cleaner to add a flag to the summary
that explicitly describes this situation, which is what this patch does.

Reviewers: mehdi_amini

Subscribers: llvm-commits

Differential Revision: https://reviews.llvm.org/D26402

llvm-svn: 286840

show more ...

Split Bitcode/ReaderWriter.h into separate reader and writer headers

Summary:
Split ReaderWriter.h which contains the APIs into both the BitReader and
BitWriter libraries into BitcodeReader.h and Bi

Split Bitcode/ReaderWriter.h into separate reader and writer headers

Summary:
Split ReaderWriter.h which contains the APIs into both the BitReader and
BitWriter libraries into BitcodeReader.h and BitcodeWriter.h.

This is to address Chandler's concern about sharing the same API header
between multiple libraries (BitReader and BitWriter). That concern is
why we create a single bitcode library in our downstream build of clang,
which led to r286297 being reverted as it added a dependency that
created a cycle only when there is a single bitcode library (not two as
in upstream).

Reviewers: mehdi_amini

Subscribers: dlj, mehdi_amini, llvm-commits

Differential Revision: https://reviews.llvm.org/D26502

llvm-svn: 286566

show more ...

IR: Introduce inrange attribute on getelementptr indices.

If the inrange keyword is present before any index, loading from or
storing to any pointer derived from the getelementptr has undefined
beha

IR: Introduce inrange attribute on getelementptr indices.

If the inrange keyword is present before any index, loading from or
storing to any pointer derived from the getelementptr has undefined
behavior if the load or store would access memory outside of the bounds of
the element selected by the index marked as inrange.

This can be used, e.g. for alias analysis or to split globals at element
boundaries where beneficial.

As previously proposed on llvm-dev:
http://lists.llvm.org/pipermail/llvm-dev/2016-July/102472.html

Differential Revision: https://reviews.llvm.org/D22793

llvm-svn: 286514

show more ...

Revert "[ThinLTO] Prevent exporting of locals used/defined in module level asm"

This reverts commit r286297.
Introduces a dependency from libAnalysis to libObject, which I missed
during the review.

Revert "[ThinLTO] Prevent exporting of locals used/defined in module level asm"

This reverts commit r286297.
Introduces a dependency from libAnalysis to libObject, which I missed
during the review.

llvm-svn: 286329

show more ...

[ThinLTO] Prevent exporting of locals used/defined in module level asm

Summary:
This patch uses the same approach added for inline asm in r285513 to
similarly prevent promotion/renaming of locals us

[ThinLTO] Prevent exporting of locals used/defined in module level asm

Summary:
This patch uses the same approach added for inline asm in r285513 to
similarly prevent promotion/renaming of locals used or defined in module
level asm.

All static global values defined in normal IR and used in module level asm
should be included on either the llvm.used or llvm.compiler.used global.
The former were already being flagged as NoRename in the summary, and
I've simply added llvm.compiler.used values to this handling.

Module level asm may also contain defs of values. We need to prevent
export of any refs to local values defined in module level asm (e.g. a
ref in normal IR), since that also requires renaming/promotion of the
local. To do that, the summary index builder looks at all values in the
module level asm string that are not marked Weak or Global, which is
exactly the set of locals that are defined. A summary is created for
each of these local defs and flagged as NoRename.

This required adding handling to the BitcodeWriter to look at GV
declarations to see if they have a summary (rather than skipping them
all).

Finally, added an assert to IRObjectFile::CollectAsmUndefinedRefs to
ensure that an MCAsmParser is available, otherwise the module asm parse
would silently fail. Initialized the asm parser in the opt tool for use
in testing this fix.

Fixes PR30610.

Reviewers: mehdi_amini

Subscribers: johanengelen, krasin, llvm-commits

Differential Revision: https://reviews.llvm.org/D26146

llvm-svn: 286297

show more ...

[BitcodeWriter] Replace a manual byteswap with read32be.

No functional change intended.

llvm-svn: 286066

Add DWARF debug info support for C++11 inline namespaces.
This implements the DWARF 5 DW_AT_export_symbols feature:
http://dwarfstd.org/ShowIssue.php?issue=141212.1

<rdar://problem/18616046>

llvm-s

Add DWARF debug info support for C++11 inline namespaces.
This implements the DWARF 5 DW_AT_export_symbols feature:
http://dwarfstd.org/ShowIssue.php?issue=141212.1

<rdar://problem/18616046>

llvm-svn: 285959

show more ...

Bitcode: Simplify BitstreamWriter::EnterBlockInfoBlock() interface.

No block info block should need to define local abbreviations, so we can
always use a code width of 2.

Also change all block info

Bitcode: Simplify BitstreamWriter::EnterBlockInfoBlock() interface.

No block info block should need to define local abbreviations, so we can
always use a code width of 2.

Also change all block info block writers to use EnterBlockInfoBlock.

Differential Revision: https://reviews.llvm.org/D26168

llvm-svn: 285660

show more ...

[ThinLTO] Use flags from summary when writing variable summary (NFC)

We already read the flags out of the summary when writing the summary
records for functions and aliases, do the same for variable

[ThinLTO] Use flags from summary when writing variable summary (NFC)

We already read the flags out of the summary when writing the summary
records for functions and aliases, do the same for variables.

This is an NFC change for now since the flags computed on the fly from
the GlobalValue currently will always match those in the summary
already, but once I send a follow-on patch to set the NoRename flag for
locals in the llvm.used set this becomes a necessary change.

llvm-svn: 285433

show more ...