[SVE] Add support to vectorize_width loop pragma for scalable vectors

This patch adds support for two new variants of the vectorize_width
pragma:

1. vectorize_width(X[, fixed|scalable]) where an op

[SVE] Add support to vectorize_width loop pragma for scalable vectors

This patch adds support for two new variants of the vectorize_width
pragma:

1. vectorize_width(X[, fixed|scalable]) where an optional second
parameter is passed to the vectorize_width pragma, which indicates if
the user wishes to use fixed width or scalable vectorization. For
example the user can now write something like:

#pragma clang loop vectorize_width(4, fixed)
or
#pragma clang loop vectorize_width(4, scalable)

In the absence of a second parameter it is assumed the user wants
fixed width vectorization, in order to maintain compatibility with
existing code.
2. vectorize_width(fixed|scalable) where the width is left unspecified,
but the user hints what type of vectorization they prefer, either
fixed width or scalable.

I have implemented this by making use of the LLVM loop hint attribute:

llvm.loop.vectorize.scalable.enable

Tests were added to

clang/test/CodeGenCXX/pragma-loop.cpp

for both the 'fixed' and 'scalable' optional parameter.

See this thread for context: http://lists.llvm.org/pipermail/cfe-dev/2020-November/067262.html

Differential Revision: https://reviews.llvm.org/D89031

show more ...

[Clang] New loop pragma vectorize_predicate

This adds a new vectorize predication loop hint:

#pragma clang loop vectorize_predicate(enable)

that can be used to indicate to the vectoriser that al

[Clang] New loop pragma vectorize_predicate

This adds a new vectorize predication loop hint:

#pragma clang loop vectorize_predicate(enable)

that can be used to indicate to the vectoriser that all (load/store)
instructions should be predicated (masked). This allows, for example, folding
of the remainder loop into the main loop.

This patch will be followed up with D64916 and D65197. The former is a
refactoring in the loopvectorizer and the groundwork to make tail loop folding
a more general concept, and in the latter the actual tail loop folding
transformation will be implemented.

Differential Revision: https://reviews.llvm.org/D64744

llvm-svn: 366989

show more ...

Add two new pragmas for controlling software pipelining optimizations.

This patch adds #pragma clang loop pipeline and #pragma clang loop pipeline_initiation_interval for debugging or reducing compi

Add two new pragmas for controlling software pipelining optimizations.

This patch adds #pragma clang loop pipeline and #pragma clang loop pipeline_initiation_interval for debugging or reducing compile time purposes. It is possible to disable SWP for concrete loops to save compilation time or to find bugs by not doing SWP to certain loops. It is possible to set value of initiation interval to concrete number to save compilation time by not doing extra pipeliner passes or to check created schedule for specific initiation interval.

Patch by Alexey Lapshin.

llvm-svn: 350414

show more ...

Append new attributes to the end of an AttributeList.

Recommit of r335084 after revert in r335516.

... instead of prepending it at the beginning (the original behavior
since implemented in r122535

Append new attributes to the end of an AttributeList.

Recommit of r335084 after revert in r335516.

... instead of prepending it at the beginning (the original behavior
since implemented in r122535 2010-12-23). This builds up an
AttributeList in the the order in which the attributes appear in the
source.

The reverse order caused nodes for attributes in the AST (e.g. LoopHint)
to be in the reverse order, and therefore printed in the wrong order in
-ast-dump. Some TODO comments mention this. The order was explicitly
reversed for enable_if attribute overload resolution and name mangling,
which is not necessary anymore with this patch.

The change unfortunately has some secondary effect, especially on
diagnostic output. In the simplest cases, the CHECK lines or expected
diagnostic were changed to the the new output. If the kind of
error/warning changed, the attributes' order was changed instead.

This unfortunately causes some 'previous occurrence here' hints to be
textually after the main marker. This typically happens when attributes
are merged, but are incompatible to each other. Interchanging the role
of the the main and note SourceLocation will also cause the case where
two different declaration's attributes (in contrast to multiple
attributes of the same declaration) are merged to be reverse. There is
no easy fix because sometimes previous attributes are merged into a new
declaration's attribute list, sometimes new attributes are added to a
previous declaration's attribute list. Since 'previous occurrence here'
pointing to locations after the main marker is not rare, I left the
markers as-is; it is only relevant when the attributes are declared in
the same declaration anyway.

Differential Revision: https://reviews.llvm.org/D48100

llvm-svn: 338800

show more ...

Revert "Append new attributes to the end of an AttributeList."

This reverts commit r335084 as requested by David Jones and
Eric Christopher because of differences of emitted warnings.

llvm-svn: 335

Revert "Append new attributes to the end of an AttributeList."

This reverts commit r335084 as requested by David Jones and
Eric Christopher because of differences of emitted warnings.

llvm-svn: 335516

show more ...

Append new attributes to the end of an AttributeList.

... instead of prepending it at the beginning (the original behavior
since implemented in r122535 2010-12-23). This builds up an
AttributeList i

Append new attributes to the end of an AttributeList.

... instead of prepending it at the beginning (the original behavior
since implemented in r122535 2010-12-23). This builds up an
AttributeList in the the order in which the attributes appear in the
source.

The reverse order caused nodes for attributes in the AST (e.g. LoopHint)
to be in the reverse, and therefore printed in the wrong order by
-ast-dump. Some TODO comments mention this. The order was explicitly
reversed for enable_if attribute overload resolution and name mangling,
which is not necessary anymore with this patch.

The change unfortunately has some secondary effects, especially for
diagnostic output. In the simplest cases, the CHECK lines or expected
diagnostic were changed to the the new output. If the kind of
error/warning changed, the attribute's order was changed instead.

It also causes some 'previous occurrence here' hints to be textually
after the main marker. This typically happens when attributes are
merged, but are incompatible. Interchanging the role of the the main
and note SourceLocation will also cause the case where two different
declaration's attributes (in contrast to multiple attributes of the
same declaration) are merged to be reversed. There is no easy fix
because sometimes previous attributes are merged into a new
declaration's attribute list, sometimes new attributes are added to a
previous declaration's attribute list. Since 'previous occurrence here'
pointing to locations after the main marker is not rare, I left the
markers as-is; it is only relevant when the attributes are declared in
the same declaration anyway, which often is on the same line.

Differential Revision: https://reviews.llvm.org/D48100

llvm-svn: 335084

show more ...

Add loop pragma for Loop Distribution

Summary:
This is similar to other loop pragmas like 'vectorize'. Currently it
only has state values: distribute(enable) and distribute(disable). When
one of t

Add loop pragma for Loop Distribution

Summary:
This is similar to other loop pragmas like 'vectorize'. Currently it
only has state values: distribute(enable) and distribute(disable). When
one of these is specified the corresponding loop metadata is generated:

!{!"llvm.loop.distribute.enable", i1 true/false}

As a result, loop distribution will be attempted on the loop even if
Loop Distribution in not enabled globally. Analogously, with 'disable'
distribution can be turned off for an individual loop even when the pass
is otherwise enabled.

There are some slight differences compared to the existing loop pragmas.

1. There is no 'assume_safety' variant which makes its handling slightly
different from 'vectorize'/'interleave'.

2. Unlike the existing loop pragmas, it does not have a corresponding
numeric pragma like 'vectorize' -> 'vectorize_width'. So for the
consistency checks in CheckForIncompatibleAttributes we don't need to
check it against other pragmas. We just need to check for duplicates of
the same pragma.

Reviewers: rsmith, dexonsmith, aaron.ballman

Subscribers: bob.wilson, cfe-commits, hfinkel

Differential Revision: http://reviews.llvm.org/D19403

llvm-svn: 272656

show more ...

Add new llvm.loop.unroll.enable metadata for use with "#pragma unroll".

This change adds the new unroll metadata "llvm.loop.unroll.enable" which directs
the optimizer to unroll a loop fully if the t

Add new llvm.loop.unroll.enable metadata for use with "#pragma unroll".

This change adds the new unroll metadata "llvm.loop.unroll.enable" which directs
the optimizer to unroll a loop fully if the trip count is known at compile time, and
unroll partially if the trip count is not known at compile time. This differs from
"llvm.loop.unroll.full" which explicitly does not unroll a loop if the trip count is not
known at compile time

With this change "#pragma unroll" generates "llvm.loop.unroll.enable" rather than
"llvm.loop.unroll.full" metadata. This changes the semantics of "#pragma unroll" slightly
to mean "unroll aggressively (fully or partially)" rather than "unroll fully or not at all".

The motivating example for this change was some internal code with a loop marked
with "#pragma unroll" which only sometimes had a compile-time trip count depending
on template magic. When the trip count was a compile-time constant, everything works
as expected and the loop is fully unrolled. However, when the trip count was not a
compile-time constant the "#pragma unroll" explicitly disabled unrolling of the loop(!).
Removing "#pragma unroll" caused the loop to be unrolled partially which was desirable
from a performance perspective.

llvm-svn: 244467

show more ...

Add assume_safety option for pragma loop vectorize and interleave.

Specifying #pragma clang loop vectorize(assume_safety) on a loop adds the
mem.parallel_loop_access metadata to each load/store oper

Add assume_safety option for pragma loop vectorize and interleave.

Specifying #pragma clang loop vectorize(assume_safety) on a loop adds the
mem.parallel_loop_access metadata to each load/store operation in the loop. This
metadata tells loop access analysis (LAA) to skip memory dependency checking.

llvm-svn: 239572

show more ...

Correct Loop Hint Diagnostic Message

When pragma clang loop unroll() is specified without an argument the diagnostic message should inform that user that 'full' and 'disable' are valid arguments (no

Correct Loop Hint Diagnostic Message

When pragma clang loop unroll() is specified without an argument the diagnostic message should inform that user that 'full' and 'disable' are valid arguments (not 'enable').

llvm-svn: 239363

show more ...

Allow constant expressions in pragma loop hints.

Previously loop hints such as #pragma loop vectorize_width(#) required a constant. This patch allows a constant expression to be used as well. Such a

Allow constant expressions in pragma loop hints.

Previously loop hints such as #pragma loop vectorize_width(#) required a constant. This patch allows a constant expression to be used as well. Such as a non-type template parameter or an expression (2 * c + 1).

Reviewed by Richard Smith

llvm-svn: 219589

show more ...

Add a state variable to the loop hint attribute.

This patch is necessary to support constant expressions which replaces the integer value in the loop hint attribute with an expression. The integer v

Add a state variable to the loop hint attribute.

This patch is necessary to support constant expressions which replaces the integer value in the loop hint attribute with an expression. The integer value was also storing the pragma’s state for options like vectorize(enable/disable) and the pragma unroll and nounroll directive. The state variable is introduced to hold the state of those options/pragmas. This moves the validation of the state (keywords) from SemaStmtAttr handler to the loop hint annotation token handler.

Resubmit with changes to try to fix the build-bot issue.

Reviewed by Aaron Ballman

llvm-svn: 214432

show more ...

Revert r214333, "Add a state variable to the loop hint attribute."

It brought undefined behavior.

llvm-svn: 214376

Add a state variable to the loop hint attribute.

This patch is necessary to support constant expressions which replaces the integer value in the loop hint attribute with an expression. The integer v

Add a state variable to the loop hint attribute.

This patch is necessary to support constant expressions which replaces the integer value in the loop hint attribute with an expression. The integer value was also storing the pragma’s state for options like vectorize(enable/disable) and the pragma unroll and nounroll directive. The state variable is introduced to hold the state of those options/pragmas. This moves the validation of the state (keywords) from SemaStmtAttr handler to the loop hint annotation token handler.

Reviewed by Aaron Ballman

llvm-svn: 214333

show more ...

In unroll pragma syntax and loop hint metadata, change "enable" forms to a new form using the string "full".

llvm-svn: 213771

Add support for '#pragma unroll'.

llvm-svn: 213574

Fix PR20069: bad loop pragma arguments crash FE

This patch fixes a crash when handling malformed arguments to loop pragmas such
as: "#pragma clang loop vectorize(()". Essentially any argument which

Fix PR20069: bad loop pragma arguments crash FE

This patch fixes a crash when handling malformed arguments to loop pragmas such
as: "#pragma clang loop vectorize(()". Essentially any argument which is not an
identifier or constant resulted in a crash. This patch also changes a couple of
the error messages which weren't quite correct. New behavior with this patch vs
old behavior:

#pragma clang loop vectorize(1)
OLD: error: missing keyword; expected 'enable' or 'disable'
NEW: error: invalid argument; expected 'enable' or 'disable'

#pragma clang loop vectorize()
OLD: error: expected ')'
NEW: error: missing argument to loop pragma 'vectorize'

#pragma clang loop vectorize_width(bad)
OLD: error: missing value; expected a positive integer value
NEW: error: invalid argument; expected a positive integer value

#pragma clang loop vectorize(bad)
OLD: invalid keyword 'bad'; expected 'enable' or 'disable'
NEW: error: invalid argument; expected 'enable' or 'disable'

http://reviews.llvm.org/D4197

Patch by Mark Heffernan

llvm-svn: 211292

show more ...

Add loop unroll pragma support

http://reviews.llvm.org/D4089

Patch by Mark Heffernan.

llvm-svn: 210667

*Really* fix DOS newlines introduced in r210330

r210369 didn't quite catch all of them.

llvm-svn: 210593

Adding a new #pragma for the vectorize and interleave optimization hints.

Patch thanks to Tyler Nowicki!

llvm-svn: 210330