[mlir] Refactor the Parser library in preparation for an MLIR binary format

The current Parser library is solely focused on providing API for
the textual MLIR format, but MLIR will soon also provide

[mlir] Refactor the Parser library in preparation for an MLIR binary format

The current Parser library is solely focused on providing API for
the textual MLIR format, but MLIR will soon also provide a binary
format. This commit renames the current Parser library to AsmParser to
better correspond to what the library is actually intended for. A new
Parser library is added which will act as a unified parser interface
between both text and binary formats. Most parser clients are
unaffected, given that the unified interface is essentially the same as
the current interface. Only clients that rely on utilizing the
AsmParserState, or those that want to parse Attributes/Types need to be
updated to point to the AsmParser library.

Differential Revision: https://reviews.llvm.org/D129605

show more ...

[mlir] Use value instead of getValue (NFC)

[mlir] Use has_value instead of hasValue (NFC)

Read/write external resource alignment tag in little-endian

https://reviews.llvm.org/D126446 added support for encoding
binary blobs in MLIR assembly. To enable cross-architecture
compatibility, th

Read/write external resource alignment tag in little-endian

https://reviews.llvm.org/D126446 added support for encoding
binary blobs in MLIR assembly. To enable cross-architecture
compatibility, these need to be encoded in little-endian format.

This patch is a first step in that direction by reading and
writing the alignment tag that those blobs are prefixed by
in little-endian format. This fixes assertion failures in
several test cases on big-endian platforms.

The actual content of the blob is not yet handled here.

Differential Revision: https://reviews.llvm.org/D129483

show more ...

[mlir:LSP] Add support for code completing attributes and types

This required changing a bit of how attributes/types are parsed. A new
`KeywordSwitch` class was added to AsmParser that provides a St

[mlir:LSP] Add support for code completing attributes and types

This required changing a bit of how attributes/types are parsed. A new
`KeywordSwitch` class was added to AsmParser that provides a StringSwitch
like API for parsing keywords with a set of potential matches. It intends to
both provide a cleaner API, and enable injection for code completion. This
required changing the API of `generated(Attr|Type)Parser` to handle the
parsing of the keyword, instead of having the user do it. Most upstream
dialects use the autogenerated handling and didn't require a direct update.

Differential Revision: https://reviews.llvm.org/D129267

show more ...

[mlir:LSP] Add support for keyword code completions

This commit adds code completion results to the MLIR LSP when
parsing keywords. Keyword support is currently limited to the
case where the expecte

[mlir:LSP] Add support for keyword code completions

This commit adds code completion results to the MLIR LSP when
parsing keywords. Keyword support is currently limited to the
case where the expected keyword is provided, but a followup will
work on expanding the set of keyword cases we handle (e.g. to
allow capturing attribute/type mnemonics).

Differential Revision: https://reviews.llvm.org/D129184

show more ...

[mlir:LSP] Add support for MLIR code completions

This commit adds code completion results to the MLIR LSP using
a new code completion context in the MLIR parser. This commit
adds initial completion

[mlir:LSP] Add support for MLIR code completions

This commit adds code completion results to the MLIR LSP using
a new code completion context in the MLIR parser. This commit
adds initial completion for dialect, operation, SSA value, and
block names.

Differential Revision: https://reviews.llvm.org/D129183

show more ...

[mlir] Allow for attaching external resources to .mlir files

This commit enables support for providing and processing external
resources within MLIR assembly formats. This is a mechanism with which

[mlir] Allow for attaching external resources to .mlir files

This commit enables support for providing and processing external
resources within MLIR assembly formats. This is a mechanism with which
dialects, and external clients, may attach additional information when
printing IR without that information being encoded in the IR itself.
External resources are not uniqued within the MLIR context, are not
attached directly to any operation, and are solely intended to live and be
processed outside of the immediate IR. There are many potential uses of this
functionality, for example MLIR's pass crash reproducer could utilize this to
attach the pass resource executing when a crash occurs. Other types of
uses may be embedding large amounts of binary data, such as weights in ML
applications, that shouldn't be copied directly into the MLIR context, but
need to be kept adjacent to the IR.

External resources are encoded using a key-value pair nested within a
dictionary anchored by name either on a dialect, or an externally registered
entity. The key is an identifier used to disambiguate the data. The value
may be stored in various limited forms, but general encodings use a string
(human readable) or blob format (binary). Within the textual format, an
example may be of the form:

```mlir
{-#
// The `dialect_resources` section within the file-level metadata
// dictionary is used to contain any dialect resource entries.
dialect_resources: {
// Here is a dictionary anchored on "foo_dialect", which is a dialect
// namespace.
foo_dialect: {
// `some_dialect_resource` is a key to be interpreted by the dialect,
// and used to initialize/configure/etc.
some_dialect_resource: "Some important resource value"
}
},
// The `external_resources` section within the file-level metadata
// dictionary is used to contain any non-dialect resource entries.
external_resources: {
// Here is a dictionary anchored on "mlir_reproducer", which is an
// external entity representing MLIR's crash reproducer functionality.
mlir_reproducer: {
// `pipeline` is an entry that holds a crash reproducer pipeline
// resource.
pipeline: "func.func(canonicalize,cse)"
}
}
```

Differential Revision: https://reviews.llvm.org/D126446

show more ...

Revert "Don't use Optional::hasValue (NFC)"

This reverts commit aa8feeefd3ac6c78ee8f67bf033976fc7d68bc6d.

Don't use Optional::hasValue (NFC)

[mlir] Don't use Optional::getValue (NFC)

[mlir] Don't use Optional::hasValue (NFC)

Apply clang-tidy fixes for modernize-use-bool-literals in Parser.cpp (NFC)

[mlir] Remove the `type` keyword from type alias definitions

This was carry over from LLVM IR where the alias definition can
be ambiguous, but MLIR type aliases have no such problems.
Having the `ty

[mlir] Remove the `type` keyword from type alias definitions

This was carry over from LLVM IR where the alias definition can
be ambiguous, but MLIR type aliases have no such problems.
Having the `type` keyword is superfluous and doesn't add anything.
This commit drops it, which also nicely aligns with the syntax for
attribute aliases (which doesn't have a keyword).

Differential Revision: https://reviews.llvm.org/D125501

show more ...

[mlir:Parser] Emit a better diagnostic when a custom operation is unknown

When a custom operation is unknown and does not have a dialect prefix, we currently
emit an error using the name of the oper

[mlir:Parser] Emit a better diagnostic when a custom operation is unknown

When a custom operation is unknown and does not have a dialect prefix, we currently
emit an error using the name of the operation with the default dialect prefix. This
leads to a confusing error message, especially when operations get moved between dialects.
For example, `func` was recently moved out of `builtin` and to the `func` dialect. The current
error message we get is:

```
func @foo()
^ custom op 'builtin.func' is unknown
```

This could lead users to believe that there is supposed to be a `builtin.func`,
because there used to be. This commit adds a better error message that does
not assume that the operation is supposed to be in the default dialect:

```
func @foo()
^ custom op 'func' is unknown (tried 'builtin.func' as well)
```

Differential Revision: https://reviews.llvm.org/D125351

show more ...

[AsmParser] Improve error recovery again.

Change the parsing logic to use StringRef instead of lower level
char* logic. Also, if emitting a diagnostic on the first token
in the file, we make sure t

[AsmParser] Improve error recovery again.

Change the parsing logic to use StringRef instead of lower level
char* logic. Also, if emitting a diagnostic on the first token
in the file, we make sure to use that position instead of the
very start of the file.

Differential Revision: https://reviews.llvm.org/D125353

show more ...

[MLIR Parser] Improve QoI for "expected token" errors

A typical problem with missing a token is that the missing
token is at the end of a line. The problem with this is that
the error message gets

[MLIR Parser] Improve QoI for "expected token" errors

A typical problem with missing a token is that the missing
token is at the end of a line. The problem with this is that
the error message gets reported on the start of the following
line (which is where the next / invalid token is) which can
be confusing.

Handle this by noticing this case and backing up to the end of
the previous line.

Differential Revision: https://reviews.llvm.org/D125295

show more ...

[mlir] Remove special case parsing/printing of `func` operations

This was leftover from when the standard dialect was destroyed, and
when FuncOp moved to the func dialect. Now that these transitions

[mlir] Remove special case parsing/printing of `func` operations

This was leftover from when the standard dialect was destroyed, and
when FuncOp moved to the func dialect. Now that these transitions
have settled a bit we can drop these.

Most updates were handled using a simple regex: replace `^( *)func` with `$1func.func`

Differential Revision: https://reviews.llvm.org/D124146

show more ...

[AsmParser] Introduce a new "Argument" abstraction + supporting logic

MLIR has a common pattern for "arguments" that uses syntax
like `%x : i32 {attrs} loc("sourceloc")` which is implemented
in adho

[AsmParser] Introduce a new "Argument" abstraction + supporting logic

MLIR has a common pattern for "arguments" that uses syntax
like `%x : i32 {attrs} loc("sourceloc")` which is implemented
in adhoc ways throughout the codebase. The approach this uses
is verbose (because it is implemented with parallel arrays) and
inconsistent (e.g. lots of things drop source location info).

Solve this by introducing OpAsmParser::Argument and make addRegion
(which sets up BlockArguments for the region) take it. Convert the
world to propagating this down. This means that we correctly
capture and propagate source location information in a lot more
cases (e.g. see the affine.for testcase example), and it also
simplifies much code.

Differential Revision: https://reviews.llvm.org/D124649

show more ...

[OpAsmParser] Simplify logic for requiredOperandCount in parseOperandList.

I would ideally like to eliminate 'requiredOperandCount' as a bit of
verification that should be in the client side, but it

[OpAsmParser] Simplify logic for requiredOperandCount in parseOperandList.

I would ideally like to eliminate 'requiredOperandCount' as a bit of
verification that should be in the client side, but it is much more
widely used than I expected. Just tidy some pieces up around it given
we can't drop it immediately.

NFC.

Differential Revision: https://reviews.llvm.org/D124629

show more ...

[AsmParser] Rework logic around "region argument parsing"

The asm parser had a notional distinction between parsing an
operand (like "%foo" or "%4#3") and parsing a region argument
(which isn't supp

[AsmParser] Rework logic around "region argument parsing"

The asm parser had a notional distinction between parsing an
operand (like "%foo" or "%4#3") and parsing a region argument
(which isn't supposed to allow a result number like #3).

Unfortunately the implementation has two problems:

1) It didn't actually check for the result number and reject
it. parseRegionArgument and parseOperand were identical.
2) It had a lot of machinery built up around it that paralleled
operand parsing. This also was functionally identical, but
also had some subtle differences (e.g. the parseOptional
stuff had a different result type).

I thought about just removing all of this, but decided that the
missing error checking was important, so I reimplemented it with
a `allowResultNumber` flag on parseOperand. This keeps the
codepaths unified and adds the missing error checks.

Differential Revision: https://reviews.llvm.org/D124470

show more ...

[AsmParser/Printer] Rework sourceloc support for function arguments.

When Location tracking support for block arguments was added, we
discussed various approaches to threading support for this throu

[AsmParser/Printer] Rework sourceloc support for function arguments.

When Location tracking support for block arguments was added, we
discussed various approaches to threading support for this through
function-like argument parsing. At the time, we added a parallel array
of locations that could hold this. It turns out that that approach was
verbose and error prone, roughly no one adopted it.

This patch takes a different approach, adding an optional source
locator to the UnresolvedOperand class. This fits much more naturally
into the standard structure we use for representing locators, and gives
all the function like dialects locator support for free (e.g. see the
test adding an example for the LLVM dialect).

Differential Revision: https://reviews.llvm.org/D124188

show more ...

[mlir:Parser][NFC] Replace SSAUseInfo with OpAsmParser::UnresolvedOperand

These are functionally identical, and merging the two removes the number of
redundant conversions within the parser.

[mlir] Rework the implementation of TypeID

This commit restructures how TypeID is implemented to ideally avoid
the current problems related to shared libraries. This is done by changing
the "implici

[mlir] Rework the implementation of TypeID

This commit restructures how TypeID is implemented to ideally avoid
the current problems related to shared libraries. This is done by changing
the "implicit" fallback path to use the name of the type, instead of using
a static template variable (which breaks shared libraries). The major downside to this
is that it adds some additional initialization costs for the implicit path. Given the
use of type names for uniqueness in the fallback, we also no longer allow types
defined in anonymous namespaces to have an implicit TypeID. To simplify defining
an ID for these classes, a new `MLIR_DEFINE_EXPLICIT_INTERNAL_INLINE_TYPE_ID` macro
was added to allow for explicitly defining a TypeID directly on an internal class.

To help identify when types are using the fallback, `-debug-only=typeid` can be
used to log which types are using implicit ids.

This change generally only requires changes to the test passes, which are all defined
in anonymous namespaces, and thus can't use the fallback any longer.

Differential Revision: https://reviews.llvm.org/D122775

show more ...

[mlir] Fix leak in case of failed parse

A Block is optionally allocated & leaks in case of failed parse. Inline the
function and ensure Block gets freed unless parse is successful.

Differential Rev

[mlir] Fix leak in case of failed parse

A Block is optionally allocated & leaks in case of failed parse. Inline the
function and ensure Block gets freed unless parse is successful.

Differential Revision: https://reviews.llvm.org/D122112

show more ...