[mlir][sparse] Add F16 and BF16.

This is the first PR to add `F16` and `BF16` support to the sparse codegen. There are still problems in supporting these two data types, such as `BF16` is not quite

[mlir][sparse] Add F16 and BF16.

This is the first PR to add `F16` and `BF16` support to the sparse codegen. There are still problems in supporting these two data types, such as `BF16` is not quite working yet.

Add tests cases.

Reviewed By: aartbik

Differential Revision: https://reviews.llvm.org/D127010

show more ...

[mlir][sparse] Using non-empty function name suffix for OverheadType::kIndex

The trick of using an empty token in the `FOREVERY_O` x-macro relies on preprocessor behavior which is only standard sinc

[mlir][sparse] Using non-empty function name suffix for OverheadType::kIndex

The trick of using an empty token in the `FOREVERY_O` x-macro relies on preprocessor behavior which is only standard since C99 6.10.3/4 and C++11 N3290 16.3/4 (whereas it was undefined behavior up through C++03 16.3/10). Since the `ExecutionEngine/SparseTensorUtils.cpp` file is required to be compile-able under C++98 compatibility mode (unlike the C++11 used elsewhere in MLIR), we shouldn't rely on that behavior.

Also, using a non-empty suffix helps improve uniformity of the API, since all other primary/overhead suffixes are also non-empty. I'm using the suffix `0` since that's the value used by the `SparseTensorEncoding` attribute for indicating the index overhead-type.

Depends On D126720

Reviewed By: aartbik

Differential Revision: https://reviews.llvm.org/D126724

show more ...

[mlir][sparse] Using x-macros in the function-suffix functions

By defining the `{primary,overhead}TypeFunctionSuffix` functions via the same x-macros used to generate the runtime library's functions

[mlir][sparse] Using x-macros in the function-suffix functions

By defining the `{primary,overhead}TypeFunctionSuffix` functions via the same x-macros used to generate the runtime library's functions themselves, this helps avoid bugs from typos or things getting out of sync.

Reviewed By: bixia

Differential Revision: https://reviews.llvm.org/D126720

show more ...

[mlir][sparse] add support for complex zero/one building

Reviewed By: bixia

Differential Revision: https://reviews.llvm.org/D126039

[mlir][sparse] introduce complex type to sparse tensor support

This is the first implementation of complex (f64 and f32) support
in the sparse compiler, with complex add/mul as first operations.
Not

[mlir][sparse] introduce complex type to sparse tensor support

This is the first implementation of complex (f64 and f32) support
in the sparse compiler, with complex add/mul as first operations.
Note that various features are still TBD, such as other ops, and
reading in complex values from file. Also, note that the
std::complex<float> had a bit of an ABI issue when passed as
single argument. It is still TBD if better solutions are possible.

Reviewed By: bixia

Differential Revision: https://reviews.llvm.org/D125596

show more ...

[mlir][sparse] Adding {pointer,index}OverheadTypeEncoding

Work towards: https://github.com/llvm/llvm-project/issues/51652

The new functions fill the gap between `overheadTypeEncoding` and `get{Poin

[mlir][sparse] Adding {pointer,index}OverheadTypeEncoding

Work towards: https://github.com/llvm/llvm-project/issues/51652

The new functions fill the gap between `overheadTypeEncoding` and `get{Pointer,Index}OverheadType`.

Reviewed By: aartbik

Differential Revision: https://reviews.llvm.org/D122056

show more ...

[mlir][sparse] Factoring out type-based function-name suffixes

Depends On D115010

This changes a couple of places that used to `return failure();` to now use `llvm_unreachable()` instead. However,

[mlir][sparse] Factoring out type-based function-name suffixes

Depends On D115010

This changes a couple of places that used to `return failure();` to now use `llvm_unreachable()` instead. However, `Transforms/Sparsification.cpp` should be doing the necessary type checks to ensure that those cases are in fact unreachable.

Reviewed By: aartbik

Differential Revision: https://reviews.llvm.org/D115012

show more ...

[mlir][sparse] adding OverheadType::kIndex

Depends On D115008

This change opens the way for D115012, and removes some corner cases in `CodegenUtils.cpp`. The `SparseTensorAttrDefs.td` already speci

[mlir][sparse] adding OverheadType::kIndex

Depends On D115008

This change opens the way for D115012, and removes some corner cases in `CodegenUtils.cpp`. The `SparseTensorAttrDefs.td` already specifies that we allow `0` bitwidth for the two overhead types and that it is interpreted to mean the architecture's native width.

Reviewed By: aartbik

Differential Revision: https://reviews.llvm.org/D115010

show more ...

[mlir][sparse] Factoring out Transforms/CodegenUtils.{cpp,h}

This moves a bunch of helper functions from `Transforms/SparseTensorConversion.cpp` into `Transforms/CodegenUtils.{cpp,h}` so that they c

[mlir][sparse] Factoring out Transforms/CodegenUtils.{cpp,h}

This moves a bunch of helper functions from `Transforms/SparseTensorConversion.cpp` into `Transforms/CodegenUtils.{cpp,h}` so that they can be reused by `Transforms/Sparsification.cpp`, etc.

See also the dependent D115010 which cleans up some corner cases in this change.

Reviewed By: aartbik, rriddle

Differential Revision: https://reviews.llvm.org/D115008

show more ...