[mlir][NFC] Update textual references of `func` to `func.func` in Conversion/ testsThe special case parsing of `func` operations is being removed.
[MLIR] Replace std ops with arith dialect opsPrecursor: https://reviews.llvm.org/D110200Removed redundant ops from the standard dialect that were moved to the`arith` or `math` dialects.Renamed
[MLIR] Replace std ops with arith dialect opsPrecursor: https://reviews.llvm.org/D110200Removed redundant ops from the standard dialect that were moved to the`arith` or `math` dialects.Renamed all instances of operations in the codebase and in tests.Reviewed By: rriddle, jpienaarDifferential Revision: https://reviews.llvm.org/D110797
show more ...
[MLIR][GPU] Add GPU launch op support for dynamic shared memoryAdd support for dynamic shared memory for GPU launch ops: add anoptional operand to gpu.launch and gpu.launch_func ops to specify the
[MLIR][GPU] Add GPU launch op support for dynamic shared memoryAdd support for dynamic shared memory for GPU launch ops: add anoptional operand to gpu.launch and gpu.launch_func ops to specify theamount of "dynamic" shared memory to use. Update lowerings to connectthis operand to the GPU runtime.Differential Revision: https://reviews.llvm.org/D110800
[mlir] create gpu memset opCreate a gpu memset op and corresponding CUDA and ROCm wrappers.Reviewed By: herhut, lorenrose1013Differential Revision: https://reviews.llvm.org/D107548
[MLIR] Create memref dialect and move dialect-specific ops from std.Create the memref dialect and move dialect-specific opsfrom std dialect to this dialect.Moved ops:AllocOp -> MemRef_AllocOpA
[MLIR] Create memref dialect and move dialect-specific ops from std.Create the memref dialect and move dialect-specific opsfrom std dialect to this dialect.Moved ops:AllocOp -> MemRef_AllocOpAllocaOp -> MemRef_AllocaOpAssumeAlignmentOp -> MemRef_AssumeAlignmentOpDeallocOp -> MemRef_DeallocOpDimOp -> MemRef_DimOpMemRefCastOp -> MemRef_CastOpMemRefReinterpretCastOp -> MemRef_ReinterpretCastOpGetGlobalMemRefOp -> MemRef_GetGlobalOpGlobalMemRefOp -> MemRef_GlobalOpLoadOp -> MemRef_LoadOpPrefetchOp -> MemRef_PrefetchOpReshapeOp -> MemRef_ReshapeOpStoreOp -> MemRef_StoreOpSubViewOp -> MemRef_SubViewOpTransposeOp -> MemRef_TransposeOpTensorLoadOp -> MemRef_TensorLoadOpTensorStoreOp -> MemRef_TensorStoreOpTensorToMemRefOp -> MemRef_BufferCastOpViewOp -> MemRef_ViewOpThe roadmap to split the memref dialect from std is discussed here:https://llvm.discourse.group/t/rfc-split-the-memref-dialect-from-std/2667Differential Revision: https://reviews.llvm.org/D98041
Pass GPU events instead of streams across async regions.Lower !gpu.async.tokens returned from async.execute regions to events instead of streams.Make !gpu.async.token returned from !async.execute
Pass GPU events instead of streams across async regions.Lower !gpu.async.tokens returned from async.execute regions to events instead of streams.Make !gpu.async.token returned from !async.execute single-use.This allows creating one event per use and destroying them without leaking or ref-counting.Technically we only need this for stream/event-based lowering. I kept the code separatefrom the rest of the gpu-async-region pass so that we can make this optional or moveto a separate pass as needed.Reviewed By: herhutDifferential Revision: https://reviews.llvm.org/D96965
Revert "[MLIR] Create memref dialect and move several dialect-specific ops from std."This commit introduced a cyclic dependency:Memref dialect depends on Standard because it used ConstantIndexOp.
Revert "[MLIR] Create memref dialect and move several dialect-specific ops from std."This commit introduced a cyclic dependency:Memref dialect depends on Standard because it used ConstantIndexOp.Std depends on the MemRef dialect in its EDSC/Intrinsics.hWorking on a fix.This reverts commit 8aa6c3765b924d86f623d452777eb76b83bf2787.
[MLIR] Create memref dialect and move several dialect-specific ops from std.Create the memref dialect and move several dialect-specific ops withoutdependencies to other ops from std dialect to thi
[MLIR] Create memref dialect and move several dialect-specific ops from std.Create the memref dialect and move several dialect-specific ops withoutdependencies to other ops from std dialect to this dialect.Moved ops:AllocOp -> MemRef_AllocOpAllocaOp -> MemRef_AllocaOpDeallocOp -> MemRef_DeallocOpMemRefCastOp -> MemRef_CastOpGetGlobalMemRefOp -> MemRef_GetGlobalOpGlobalMemRefOp -> MemRef_GlobalOpPrefetchOp -> MemRef_PrefetchOpReshapeOp -> MemRef_ReshapeOpStoreOp -> MemRef_StoreOpTransposeOp -> MemRef_TransposeOpViewOp -> MemRef_ViewOpThe roadmap to split the memref dialect from std is discussed here:https://llvm.discourse.group/t/rfc-split-the-memref-dialect-from-std/2667Differential Revision: https://reviews.llvm.org/D96425
[mlir] Fix gpu-to-llvm lowering for gpu.alloc with dynamic sizes.Reviewed By: ftynseDifferential Revision: https://reviews.llvm.org/D94402
[mlir] replace LLVM dialect float types with built-insContinue the convergence between LLVM dialect and built-in types by replacingthe bfloat, half, float and double LLVM dialect types with their
[mlir] replace LLVM dialect float types with built-insContinue the convergence between LLVM dialect and built-in types by replacingthe bfloat, half, float and double LLVM dialect types with their built-incounterparts. At the API level, this is a direct replacement. At the syntaxlevel, we change the keywords to `bf16`, `f16`, `f32` and `f64`, respectively,to be compatible with the built-in type syntax. The old keywords can still beparsed but produce a deprecation warning and will be eventually removed.Depends On D94178Reviewed By: mehdi_amini, silvas, antiagainstDifferential Revision: https://reviews.llvm.org/D94179
[mlir] replace LLVMIntegerType with built-in integer typeThe LLVM dialect type system has been closed until now, i.e. did not supporttypes from other dialects inside containers. While this has had
[mlir] replace LLVMIntegerType with built-in integer typeThe LLVM dialect type system has been closed until now, i.e. did not supporttypes from other dialects inside containers. While this has had obviousbenefits of deriving from a common base class, it has led to some simple typesbeing almost identical with the built-in types, namely integer and floatingpoint types. This in turn has led to a lot of larger-scale complexity: simpletypes must still be converted, numerous operations that correspond to LLVM IRintrinsics are replicated to produce versions operating on either LLVM dialector built-in types leading to quasi-duplicate dialects, lowering to the LLVMdialect is essentially required to be one-shot because of type conversion, etc.In this light, it is reasonable to trade off some local complexity in theinternal implementation of LLVM dialect types for removing larger-scale systemcomplexity. Previous commits to the LLVM dialect type system have adapted theAPI to support types from other dialects.Replace LLVMIntegerType with the built-in IntegerType plus additional checksthat such types are signless (these are isolated in a utility function thatreplaced `isa<LLVMType>` and in the parser). Temporarily keep the possibilityto parse `!llvm.i32` as a synonym for `i32`, but add a deprecation notice.Reviewed By: mehdi_amini, silvas, antiagainstDifferential Revision: https://reviews.llvm.org/D94178
Remove allow-unregistered-dialect from some tests that don't need itDifferential Revision: https://reviews.llvm.org/D93982
[mlir] Lower gpu.memcpy to GPU runtime calls.Reviewed By: herhutDifferential Revision: https://reviews.llvm.org/D93204
[mlir] Add gpu.allocate, gpu.deallocate ops with LLVM lowering to runtime function calls.The ops are very similar to the std variants, but support async GPU execution.gpu.alloc does not currently
[mlir] Add gpu.allocate, gpu.deallocate ops with LLVM lowering to runtime function calls.The ops are very similar to the std variants, but support async GPU execution.gpu.alloc does not currently support an alignment attribute, and the new ops do not havecanonicalizers/folders like their std siblings do.Reviewed By: herhutDifferential Revision: https://reviews.llvm.org/D91698
[mlir][gpu] Fix leaked stream and module when lowering gpu.launch_func to runtime calls.Reviewed By: mehdi_aminiDifferential Revision: https://reviews.llvm.org/D90370
[mlir][gpu] NFC: switch occurrences of gpu.launch_func to custom format.Reviewed By: herhutDifferential Revision: https://reviews.llvm.org/D89929
[mlir][gpu] Add lowering to LLVM for `gpu.wait` and `gpu.wait async`.Reviewed By: herhutDifferential Revision: https://reviews.llvm.org/D89686
[mlir] Stop allowing LLVMType Int arguments for GPULaunchFuncOp.Conversion to LLVM becomes confusing and incorrect if someone tries to lowerSTD -> LLVM and only then GPULaunchFuncOp to LLVM separa
[mlir] Stop allowing LLVMType Int arguments for GPULaunchFuncOp.Conversion to LLVM becomes confusing and incorrect if someone tries to lowerSTD -> LLVM and only then GPULaunchFuncOp to LLVM separately. Although it istechnically allowed now, it works incorrectly because of the argumentpromotion. The correct way to use this conversion pattern is to add to theSTD->LLVM patterns before running the pass.Differential Revision: https://reviews.llvm.org/D88147
[MLIR] Make gpu.launch_func rewrite pattern part of the LLVM lowering pass.Reviewed By: herhutDifferential Revision: https://reviews.llvm.org/D85073
[mlir] switch the modeling of LLVM types to use the new mechanismA new first-party modeling for LLVM IR types in the LLVM dialect has beendeveloped in parallel to the existing modeling based on wr
[mlir] switch the modeling of LLVM types to use the new mechanismA new first-party modeling for LLVM IR types in the LLVM dialect has beendeveloped in parallel to the existing modeling based on wrapping LLVM `Type *`instances. It resolves the long-standing problem of modeling identifiedstructure types, including recursive structures, and enables future removal ofLLVMContext and related locking mechanisms from LLVMDialect.This commit only switches the modeling by (a) renaming LLVMTypeNew to LLVMType,(b) removing the old implementaiton of LLVMType, and (c) updating the tests. Itis intentionally minimal. Separate commits will remove the infrastructure builtfor the transition and update API uses where appropriate.Depends On D85020Reviewed By: rriddleDifferential Revision: https://reviews.llvm.org/D85021
Clean up cuda-runtime-wrappers API.Do not return error code, instead return created resource handles or void. Error reporting is done by the library function.Reviewed By: herhutDifferential Rev
Clean up cuda-runtime-wrappers API.Do not return error code, instead return created resource handles or void. Error reporting is done by the library function.Reviewed By: herhutDifferential Revision: https://reviews.llvm.org/D84660
Enable FileCheck -enable-var-scope by default in MLIR testThis option avoids to accidentally reuse variable across -LABEL match,it can be explicitly opted-in by prefixing the variable name with $
Enable FileCheck -enable-var-scope by default in MLIR testThis option avoids to accidentally reuse variable across -LABEL match,it can be explicitly opted-in by prefixing the variable name with $Differential Revision: https://reviews.llvm.org/D81531
[mlir][gpu] Refactor ConvertGpuLaunchFuncToCudaCalls pass.Due to similar APIs between CUDA and ROCm (HIP),ConvertGpuLaunchFuncToCudaCalls pass could be used on both platforms with somerefactoring
[mlir][gpu] Refactor ConvertGpuLaunchFuncToCudaCalls pass.Due to similar APIs between CUDA and ROCm (HIP),ConvertGpuLaunchFuncToCudaCalls pass could be used on both platforms with somerefactoring.In this commit:- Migrate ConvertLaunchFuncToCudaCalls from GPUToCUDA to GPUCommon, and rename.- Rename runtime wrapper APIs be platform-neutral.- Let GPU binary annotation attribute be specifiable as a PassOption.- Naming changes within the implementation and tests.Subsequent patches would introduce ROCm-specific tests and runtime wrapperAPIs.Differential Revision: https://reviews.llvm.org/D80167
Revert "[mlir][gpu] Refactor ConvertGpuLaunchFuncToCudaCalls pass."This reverts commit cdb6f05e2d5f0132956020e6b4990af0206c066f.The build is broken with: You have called ADD_LIBRARY for librar
Revert "[mlir][gpu] Refactor ConvertGpuLaunchFuncToCudaCalls pass."This reverts commit cdb6f05e2d5f0132956020e6b4990af0206c066f.The build is broken with: You have called ADD_LIBRARY for library obj.MLIRGPUtoCUDATransforms without any source files. This typically indicates a problem with your CMakeLists.txt file
12