[mlir] Fix the names of exported functionsThe names of the functions that are supposed to be exported do not match the implementations. This is due in part to https://github.com/llvm/llvm-project/c
[mlir] Fix the names of exported functionsThe names of the functions that are supposed to be exported do not match the implementations. This is due in part to https://github.com/llvm/llvm-project/commit/cac7aabbd8236bef2909bfc0dbba17644f7aaade.This change makes the implementations and declarations match and adds a couple missing declarations.The new names follow the pattern of the existing `verify` functions where the prefix is maintained as `_mlir_ciface_` but the suffix follows the new naming convention.Reviewed By: rriddleDifferential Revision: https://reviews.llvm.org/D124891
show more ...
[mlir][NFC] Update remaining textual references of un-namespaced `func` operationsThe special case parsing of operations in the `func` dialect is being removed, andoperations will require the dial
[mlir][NFC] Update remaining textual references of un-namespaced `func` operationsThe special case parsing of operations in the `func` dialect is being removed, andoperations will require the dialect namespace prefix.
[mlir][NFC] Update textual references of `func` to `func.func` in Integration testsThe special case parsing of `func` operations is being removed.
[mlir][linalg] Replace linalg.fill by OpDSL variant.The revision removes the linalg.fill operation and renames the OpDSL generated linalg.fill_tensor operation to replace it. After the change, all
[mlir][linalg] Replace linalg.fill by OpDSL variant.The revision removes the linalg.fill operation and renames the OpDSL generated linalg.fill_tensor operation to replace it. After the change, all named structured operations are defined via OpDSL and there are no handwritten operations left.A side-effect of the change is that the pretty printed form changes from:```%1 = linalg.fill(%cst, %0) : f32, tensor<?x?xf32> -> tensor<?x?xf32>```changes to```%1 = linalg.fill ins(%cst : f32) outs(%0 : tensor<?x?xf32>) -> tensor<?x?xf32>```Additionally, the builder signature now takes input and output value ranges as it is the case for all other OpDSL operations:```rewriter.create<linalg::FillOp>(loc, val, output)```changes to```rewriter.create<linalg::FillOp>(loc, ValueRange{val}, ValueRange{output})```All other changes remain minimal. In particular, the canonicalization patterns are the same and the `value()`, `output()`, and `result()` methods are now implemented by the FillOpInterface.Depends On D120726Reviewed By: nicolasvasilacheDifferential Revision: https://reviews.llvm.org/D120728
[mlir][NFC] Rename StandardToLLVM to FuncToLLVMThe current StandardToLLVM conversion patterns only really handlethe Func dialect. The pass itself adds patterns for Arithmetic/CFToLLVM, butthose s
[mlir][NFC] Rename StandardToLLVM to FuncToLLVMThe current StandardToLLVM conversion patterns only really handlethe Func dialect. The pass itself adds patterns for Arithmetic/CFToLLVM, butthose should be/will be split out in a followup. This commit focuses solelyon being an NFC rename.Aside from the directory change, the pattern and pass creation API have been renamed: * populateStdToLLVMFuncOpConversionPattern -> populateFuncToLLVMFuncOpConversionPattern * populateStdToLLVMConversionPatterns -> populateFuncToLLVMConversionPatterns * createLowerToLLVMPass -> createConvertFuncToLLVMPassDifferential Revision: https://reviews.llvm.org/D120778
[mlir] Split out a new ControlFlow dialect from StandardThis dialect is intended to model lower level/branch based control-flow constructs. The initial setof operations are: AssertOp, BranchOp, Co
[mlir] Split out a new ControlFlow dialect from StandardThis dialect is intended to model lower level/branch based control-flow constructs. The initial setof operations are: AssertOp, BranchOp, CondBranchOp, SwitchOp; all split out from the currentstandard dialect.See https://discourse.llvm.org/t/standard-dialect-the-final-chapter/6061Differential Revision: https://reviews.llvm.org/D118966
[MLIR] Fix integration tests broken by D118285[MLIR] Fix integration tests broken by D118285.
[mlir] AsyncParallelFor: align block size to be a multiple of inner loops iterationsDepends On D115263By aligning block size to inner loop iterations parallel_compute_fn LLVM can later unroll and
[mlir] AsyncParallelFor: align block size to be a multiple of inner loops iterationsDepends On D115263By aligning block size to inner loop iterations parallel_compute_fn LLVM can later unroll and vectorize some of the inner loops with small number of trip counts. Up to 2x speedup in multiple benchmarks.Reviewed By: bkramerDifferential Revision: https://reviews.llvm.org/D115436
[mlir] Improve async parallel for tests + fix typosDo load and store to verify that we process each element of the iteration space once.Reviewed By: cotaDifferential Revision: https://reviews.l
[mlir] Improve async parallel for tests + fix typosDo load and store to verify that we process each element of the iteration space once.Reviewed By: cotaDifferential Revision: https://reviews.llvm.org/D115152
Promote readability by factoring out creation of min/max operation. Remove unnecessary divisions.Reviewed By: ezhulenevDifferential Revision: https://reviews.llvm.org/D110680
[mlir] Move min/max ops from Std to Arith.Differential Revision: https://reviews.llvm.org/D113881
[mlir] Reduce the number of iterations in async microbenchmarksDifferential Revision: https://reviews.llvm.org/D112609
[MLIR] Replace std ops with arith dialect opsPrecursor: https://reviews.llvm.org/D110200Removed redundant ops from the standard dialect that were moved to the`arith` or `math` dialects.Renamed
[MLIR] Replace std ops with arith dialect opsPrecursor: https://reviews.llvm.org/D110200Removed redundant ops from the standard dialect that were moved to the`arith` or `math` dialects.Renamed all instances of operations in the codebase and in tests.Reviewed By: rriddle, jpienaarDifferential Revision: https://reviews.llvm.org/D110797
Rename target block size to min task size for clarity.Reviewed By: ezhulenevDifferential Revision: https://reviews.llvm.org/D110604
[mlir] Factor type reconciliation out of Standard-to-LLVM conversionConversion to the LLVM dialect is being refactored to be more progressive andis now performed as a series of independent passes
[mlir] Factor type reconciliation out of Standard-to-LLVM conversionConversion to the LLVM dialect is being refactored to be more progressive andis now performed as a series of independent passes converting differentdialects. These passes may produce `unrealized_conversion_cast` operations thatrepresent pending conversions between built-in and LLVM dialect types.Historically, a more monolithic Standard-to-LLVM conversion pass did not needthese casts as all operations were converted in one shot. Previous refactoringshave led to the requirement of running the Standard-to-LLVM conversion pass toclean up `unrealized_conversion_cast`s even though the IR had no standardoperations in it. The pass must have been also run the last among all to-LLVMpasses, in contradiction with the partial conversion logic. Additionally, theway it was set up could produce invalid operations by removing casts betweenLLVM and built-in types even when the consumer did not accept the uncastedtype, or could lead to cryptic conversion errors (recursive application of therewrite pattern on `unrealized_conversion_cast` as a means to indicate failureto eliminate casts).In fact, the need to eliminate A->B->A `unrealized_conversion_cast`s is notspecific to to-LLVM conversions and can be factored out into a separate typereconciliation pass, which is achieved in this commit. While the cast operationitself has a folder pattern, it is insufficient in most conversion passes asthe folder only applies to the second cast. Without complex legality setup inthe conversion target, the conversion infra will either consider the castoperations valid and not fold them (a separate canonicalization would benecessary to trigger the folding), or consider the first cast invalid upongeneration and stop with error. The pattern provided by the reconciliation passapplies to the first cast operation instead. Furthermore, having a separatepass makes it clear when `unrealized_conversion_cast`s could not have beeneliminated since it is the only reason why this pass can fail.Reviewed By: nicolasvasilacheDifferential Revision: https://reviews.llvm.org/D109507
[mlir] Async: special handling for parallel loops with zero iterationsReviewed By: mehdi_aminiDifferential Revision: https://reviews.llvm.org/D106590
[mlir] factor memref-to-llvm lowering out of std-to-llvmAfter the MemRef has been split out of the Standard dialect, theconversion to the LLVM dialect remained as a huge monolithic pass.This is u
[mlir] factor memref-to-llvm lowering out of std-to-llvmAfter the MemRef has been split out of the Standard dialect, theconversion to the LLVM dialect remained as a huge monolithic pass.This is undesirable for the same complexity management reasons as havinga huge Standard dialect itself, and is even more confusing given theexistence of a separate dialect. Extract the conversion of the MemRefdialect operations to LLVM into a separate library and a separateconversion pass.Reviewed By: herhut, silvasDifferential Revision: https://reviews.llvm.org/D105625
[mlir:Async] Add an async reference counting pass based on the user defined policyDepends On D104999Automatic reference counting based on the liveness analysis can add a lot of reference counting
[mlir:Async] Add an async reference counting pass based on the user defined policyDepends On D104999Automatic reference counting based on the liveness analysis can add a lot of reference counting overhead at runtime. If the IR is known to be constrained to few particular "shapes", it's much more efficient to provide a custom reference counting policy that will specify where it is required to update the async value reference count.Reviewed By: mehdi_aminiDifferential Revision: https://reviews.llvm.org/D105037
[mlir:Async] Fix a bug in automatic refence counting around function callsDepends On D104998Function calls "transfer ownership" to the callee and it puts additional constraints on the reference c
[mlir:Async] Fix a bug in automatic refence counting around function callsDepends On D104998Function calls "transfer ownership" to the callee and it puts additional constraints on the reference counting optimization passReviewed By: mehdi_aminiDifferential Revision: https://reviews.llvm.org/D104999
[mlir:Async] Implement recursive async work splitting for scf.parallel operation (async-parallel-for pass)Depends On D104780Recursive work splitting instead of sequential async tasks submission g
[mlir:Async] Implement recursive async work splitting for scf.parallel operation (async-parallel-for pass)Depends On D104780Recursive work splitting instead of sequential async tasks submission gives ~20%-30% speedup in microbenchmarks.Algorithm outline:1. Collapse scf.parallel dimensions into a single dimension2. Compute the block size for the parallel operations from the 1d problem size3. Launch parallel tasks4. Each parallel task reconstructs its own bounds in the original multi-dimensional iteration space5. Each parallel task computes the original parallel operation body using scf.for loop nestReviewed By: herhutDifferential Revision: https://reviews.llvm.org/D104850
[mlir][linalg] Change the pretty printed FillOp operand order.The patch changes the pretty printed FillOp operand order from output, value to value, output. The change is a follow up to https://rev
[mlir][linalg] Change the pretty printed FillOp operand order.The patch changes the pretty printed FillOp operand order from output, value to value, output. The change is a follow up to https://reviews.llvm.org/D104121 that passes the fill value using a scalar input instead of the former capture semantics.Differential Revision: https://reviews.llvm.org/D104356
[mlir] Async: add automatic reference counting at async.runtime operations levelDepends On D95311Previous automatic-ref-counting pass worked with high level async operations (e.g. async.execute),
[mlir] Async: add automatic reference counting at async.runtime operations levelDepends On D95311Previous automatic-ref-counting pass worked with high level async operations (e.g. async.execute), however async values reference counting is a runtime implementation detail.New pass mostly relies on the save liveness analysis to place drop_ref operations, and does better verification of CFG with different liveIn sets in block successors.This is almost NFC change. No new reference counting ideas, just a cleanup of the previous version.Reviewed By: mehdi_aminiDifferential Revision: https://reviews.llvm.org/D95390
[MLIR] Create memref dialect and move dialect-specific ops from std.Create the memref dialect and move dialect-specific opsfrom std dialect to this dialect.Moved ops:AllocOp -> MemRef_AllocOpA
[MLIR] Create memref dialect and move dialect-specific ops from std.Create the memref dialect and move dialect-specific opsfrom std dialect to this dialect.Moved ops:AllocOp -> MemRef_AllocOpAllocaOp -> MemRef_AllocaOpAssumeAlignmentOp -> MemRef_AssumeAlignmentOpDeallocOp -> MemRef_DeallocOpDimOp -> MemRef_DimOpMemRefCastOp -> MemRef_CastOpMemRefReinterpretCastOp -> MemRef_ReinterpretCastOpGetGlobalMemRefOp -> MemRef_GetGlobalOpGlobalMemRefOp -> MemRef_GlobalOpLoadOp -> MemRef_LoadOpPrefetchOp -> MemRef_PrefetchOpReshapeOp -> MemRef_ReshapeOpStoreOp -> MemRef_StoreOpSubViewOp -> MemRef_SubViewOpTransposeOp -> MemRef_TransposeOpTensorLoadOp -> MemRef_TensorLoadOpTensorStoreOp -> MemRef_TensorStoreOpTensorToMemRefOp -> MemRef_BufferCastOpViewOp -> MemRef_ViewOpThe roadmap to split the memref dialect from std is discussed here:https://llvm.discourse.group/t/rfc-split-the-memref-dialect-from-std/2667Differential Revision: https://reviews.llvm.org/D98041
Move the MLIR integration tests as a subdirectory of test (NFC)This does not change the behavior directly: the tests only run when`-DMLIR_INCLUDE_INTEGRATION_TESTS=ON` is configured. However runni
Move the MLIR integration tests as a subdirectory of test (NFC)This does not change the behavior directly: the tests only run when`-DMLIR_INCLUDE_INTEGRATION_TESTS=ON` is configured. However running`ninja check-mlir` will not run all the tests within a singlelit invocation. The previous behavior would wait for all the integrationtests to complete before starting to run the first regular test. Thetest results were also reported separately. This change is unifying allof this and allow concurrent execution of the integration tests withregular non-regression and unit-tests.Differential Revision: https://reviews.llvm.org/D97241