1# MLIR : Language Server Protocol 2 3[TOC] 4 5This document describes the tools and utilities related to supporting 6[LSP](https://microsoft.github.io/language-server-protocol/) IDE language 7extensions for various MLIR-related languages. An LSP language extension is 8generally comprised of two components; a language client and a language server. 9A language client is a piece of code that interacts with the IDE that you are 10using, such as VSCode. A language server acts as the backend for queries that 11the client may want to perform, such as "Find Definition", "Find References", 12etc. 13 14## MLIR LSP Language Server : `mlir-lsp-server` 15 16MLIR provides an implementation of an LSP language server for `.mlir` text files 17in the form of the `mlir-lsp-server` tool. This tool interacts with the MLIR C++ 18API to support rich language queries, such as "Find Definition". 19 20### Supporting custom dialects and passes 21 22`mlir-lsp-server`, like many other MLIR based tools, relies on having the 23appropriate dialects registered to be able to parse in the custom assembly 24formats used in the textual .mlir files. The `mlir-lsp-server` found within the 25main MLIR repository provides support for all of the upstream MLIR dialects and 26passes. Downstream and out-of-tree users will need to provide a custom 27`mlir-lsp-server` executable that registers the entities that they are 28interested in. The implementation of `mlir-lsp-server` is provided as a library, 29making it easy for downstream users to register their dialect/passes and simply 30call into the main implementation. A simple example is shown below: 31 32```c++ 33#include "mlir/Tools/mlir-lsp-server/MlirLspServerMain.h" 34 35int main(int argc, char **argv) { 36 mlir::DialectRegistry registry; 37 registerMyDialects(registry); 38 registerMyPasses(); 39 return mlir::failed(mlir::MlirLspServerMain(argc, argv, registry)); 40} 41``` 42 43See the [Editor Plugins](#editor-plugins) section below for details on how to 44setup support in a few known LSP clients, such as vscode. 45 46### Features 47 48This section details a few of the features that the MLIR language server 49provides. The screenshots are shown in [VSCode](https://code.visualstudio.com/), 50but the exact feature set available will depend on your editor client. 51 52[mlir features]: # 53 54#### Diagnostics 55 56The language server actively runs verification on the IR as you type, showing 57any generated diagnostics in-place. 58 59 60 61##### Automatically insert `expected-` diagnostic checks 62 63MLIR provides 64[infrastructure](https://mlir.llvm.org/docs/Diagnostics/#sourcemgr-diagnostic-verifier-handler) 65for checking expected diagnostics, which is heavily utilized when defining IR 66parsing and verification. The language server provides code actions for 67automatically inserting the checks for diagnostics it knows about. 68 69 70 71#### Code completion 72 73The language server provides suggestions as you type, offering completions for 74dialect constructs (such as attributes, operations, and types), block names, SSA 75value names, keywords, and more. 76 77 78 79#### Cross-references 80 81Cross references allow for navigating the use/def chains of SSA values (i.e. 82operation results and block arguments), [Symbols](../SymbolsAndSymbolTables.md), 83and Blocks. 84 85##### Find definition 86 87Jump to the definition of the IR entity under the cursor. A few examples are 88shown below: 89 90- SSA Values 91 92 93 94- Symbol References 95 96 97 98The definition of an operation will also take into account the source location 99attached, allowing for navigating into the source file that generated the 100operation. 101 102 103 104##### Find references 105 106Show all references of the IR entity under the cursor. 107 108 109 110#### Hover 111 112Hover over an IR entity to see more information about it. The exact information 113displayed is dependent on the type of IR entity under the cursor. For example, 114hovering over an `Operation` may show its generic format. 115 116 117 118#### Navigation 119 120The language server will also inform the editor about the structure of symbol 121tables within the IR. This allows for jumping directly to the definition of a 122symbol, such as a `func.func`, within the file. 123 124 125 126## PDLL LSP Language Server : `mlir-pdll-lsp-server` 127 128MLIR provides an implementation of an LSP language server for `.pdll` text files 129in the form of the `mlir-pdll-lsp-server` tool. This tool interacts with the 130PDLL C++ API to support rich language queries, such as code completion and "Find 131Definition". 132 133### Compilation Database 134 135Similarly to 136[`clangd`](https://clang.llvm.org/docs/JSONCompilationDatabase.html), and 137language servers for various other programming languages, the PDLL language 138server relies on a compilation database to provide build-system information for 139`.pdll` files. This information includes, for example, the include directories 140available for that file. This database allows for the server to interact with 141`.pdll` files using the same configuration as when building. 142 143#### Format 144 145A PDLL compilation database is a YAML file, conventionally named 146`pdll_compile_commands.yml`, that contains a set of `FileInfo` documents 147providing information for individiual `.pdll` files. 148 149Example: 150 151```yaml 152--- !FileInfo: 153 filepath: "/home/user/llvm/mlir/lib/Dialect/Arithmetic/IR/ArithmeticCanonicalization.pdll" 154 includes: "/home/user/llvm/mlir/lib/Dialect/Arithmetic/IR;/home/user/llvm/mlir/include" 155``` 156 157- filepath: <string> - Absolute file path of the file. 158- includes: <string> - Semi-colon delimited list of absolute include directories. 159 160#### Build System Integration 161 162Per convention, PDLL compilation databases should be named 163`pdll_compile_commands.yml` and placed at the top of the build directory. When 164using CMake and `mlir_pdll`, a compilation database is generally automatically 165built and placed in the appropriate location. 166 167### Features 168 169This section details a few of the features that the PDLL language server 170provides. The screenshots are shown in [VSCode](https://code.visualstudio.com/), 171but the exact feature set available will depend on your editor client. 172 173[pdll features]: # 174 175#### Diagnostics 176 177The language server actively runs verification as you type, showing any 178generated diagnostics in-place. 179 180 181 182#### Code completion and signature help 183 184The language server provides suggestions as you type based on what constraints, 185rewrites, dialects, operations, etc are available in this context. The server 186also provides information about the structure of constraint and rewrite calls, 187operations, and more as you fill them in. 188 189 190 191#### Cross-references 192 193Cross references allow for navigating the code base. 194 195##### Find definition 196 197Jump to the definition of a symbol under the cursor: 198 199 200 201If ODS information is available, we can also jump to the definition of operation 202names and more: 203 204 205 206##### Find references 207 208Show all references of the symbol under the cursor. 209 210 211 212#### Hover 213 214Hover over a symbol to see more information about it, such as its type, 215documentation, and more. 216 217 218 219If ODS information is available, we can also show information directly from the 220operation definitions: 221 222 223 224#### Navigation 225 226The language server will also inform the editor about the structure of symbols 227within the IR. 228 229 230 231#### View intermediate output 232 233The language server provides support for introspecting various intermediate 234stages of compilation, such as the AST, the `.mlir` containing the generated 235PDL, and the generated C++ glue. This is a custom LSP extension, and is not 236necessarily provided by all IDE clients. 237 238 239 240#### Inlay hints 241 242The language server provides additional information inline with the source code. 243Editors usually render this using read-only virtual text snippets interspersed 244with code. Hints may be shown for: 245 246* types of local variables 247* names of operand and result groups 248* constraint and rewrite arguments 249 250 251 252## TableGen LSP Language Server : `tblgen-lsp-server` 253 254MLIR provides an implementation of an LSP language server for `.td` text files 255in the form of the `tblgen-lsp-server` tool. This tool interacts with the 256TableGen C++ API to support rich language queries, such as "Find Definition". 257 258### Compilation Database 259 260Similarly to 261[`clangd`](https://clang.llvm.org/docs/JSONCompilationDatabase.html), and 262language servers for various other programming languages, the TableGen language 263server relies on a compilation database to provide build-system information for 264`.td` files. This information includes, for example, the include directories 265available for that file. This database allows for the server to interact with 266`.td` files using the same configuration as when building. 267 268#### Format 269 270A TableGen compilation database is a YAML file, conventionally named 271`tablegen_compile_commands.yml`, that contains a set of `FileInfo` documents 272providing information for individiual `.td` files. 273 274Example: 275 276```yaml 277--- !FileInfo: 278 filepath: "/home/user/llvm/mlir/lib/Dialect/Arithmetic/IR/ArithmeticCanonicalization.td" 279 includes: "/home/user/llvm/mlir/lib/Dialect/Arithmetic/IR;/home/user/llvm/mlir/include" 280``` 281 282- filepath: <string> - Absolute file path of the file. 283- includes: <string> - Semi-colon delimited list of absolute include directories. 284 285#### Build System Integration 286 287Per convention, TableGen compilation databases should be named 288`tablegen_compile_commands.yml` and placed at the top of the build directory. 289When using CMake and `mlir_tablegen`, a compilation database is generally 290automatically built and placed in the appropriate location. 291 292### Features 293 294This section details a few of the features that the TableGen language server 295provides. The screenshots are shown in [VSCode](https://code.visualstudio.com/), 296but the exact feature set available will depend on your editor client. 297 298[tablegen features]: # 299 300#### Diagnostics 301 302The language server actively runs verification as you type, showing any 303generated diagnostics in-place. 304 305 306 307#### Cross-references 308 309Cross references allow for navigating the code base. 310 311##### Find definition 312 313Jump to the definition of a symbol under the cursor: 314 315 316 317##### Find references 318 319Show all references of the symbol under the cursor. 320 321 322 323## Language Server Design 324 325The design of the various language servers provided by MLIR are effectively the 326same, and are largely comprised of three different components: 327 328- Communication and Transport (via JSON-RPC) 329- Language Server Protocol 330- Language-Specific Server 331 332 333 334### Communication and Transport 335 336The language server, such as `mlir-lsp-server`, communicates with the language 337client via JSON-RPC over stdin/stdout. In the code, this is the `JSONTransport` 338class. This class knows nothing about the Language Server Protocol, it only 339knows that JSON-RPC messages are coming in and JSON-RPC messages are going out. 340The handling of incoming and outgoing LSP messages is left to the 341`MessageHandler` class. This class routes incoming messages to handlers in the 342`Language Server Protocol` layer for interpretation, and packages outgoing 343messages for transport. This class also has limited knowledge of the LSP, and 344only has information about the three main classes of messages: notifications, 345calls, and replies. 346 347### Language Server Protocol 348 349`LSPServer` handles the interpretation of the finer LSP details. This class 350registers handlers for LSP messages and then forwards to the 351[`Language-Specific Server`](#language-specific-server) for processing. The 352intent of this component is to hold all of the necessary glue when communicating 353from the LSP world to the language-specific world (e.g. MLIR, PDLL, etc.). In 354most cases, the LSP message handlers simply forward directly to the 355`Language-Specific Server`. In some cases, however, the impedance mismatch 356between the two requires more complicated glue code. 357 358### Language-Specific Server 359 360The language specific server, such as `MLIRServer` or `PDLLServer`, provides the 361internal implementation of all of LSP queries for a specific language. These are 362the classes that directly interacts with the C++ API for the language, including 363parsing text files, interpreting definition/reference information, etc. 364 365## Editor Plugins 366 367LSP Language plugins are available for many popular editors, and in principle 368the language servers provided by MLIR should work with any of them, though 369feature sets and interfaces may vary. Below are a set of plugins that are known 370to work: 371 372### Visual Studio Code 373 374Provides language IDE features for [MLIR](https://mlir.llvm.org/) related 375languages: [MLIR](#mlir---mlir-textual-assembly-format), 376[PDLL](#pdll---mlir-pdll-pattern-files), and [TableGen](#td---tablegen-files) 377 378#### `.mlir` - MLIR textual assembly format: 379 380The MLIR extension adds language support for the 381[MLIR textual assembly format](https://mlir.llvm.org/docs/LangRef/): 382 383##### Features 384 385- Syntax highlighting for `.mlir` files and `mlir` markdown blocks 386- go-to-definition and cross references 387- Detailed information when hovering over IR entities 388- Outline and navigation of symbols and symbol tables 389- Code completion 390- Live parser and verifier diagnostics 391 392[mlir-vscode features]: # 393 394##### Setup 395 396###### `mlir-lsp-server` 397 398The various `.mlir` language features require the 399[`mlir-lsp-server` language server](https://mlir.llvm.org/docs/Tools/MLIRLSP/#mlir-lsp-language-server--mlir-lsp-server). 400If `mlir-lsp-server` is not found within your workspace path, you must specify 401the path of the server via the `mlir.server_path` setting. The path of the 402server may be absolute or relative within your workspace. 403 404#### `.pdll` - MLIR PDLL pattern files: 405 406The MLIR extension adds language support for the 407[PDLL pattern language](https://mlir.llvm.org/docs/PDLL/). 408 409##### Features 410 411- Syntax highlighting for `.pdll` files and `pdll` markdown blocks 412- go-to-definition and cross references 413- Types and documentation on hover 414- Code completion and signature help 415- View intermediate AST, MLIR, or C++ output 416 417[pdll-vscode features]: # 418 419##### Setup 420 421###### `mlir-pdll-lsp-server` 422 423The various `.pdll` language features require the 424[`mlir-pdll-lsp-server` language server](https://mlir.llvm.org/docs/Tools/MLIRLSP/#pdll-lsp-language-server--mlir-pdll-lsp-server). 425If `mlir-pdll-lsp-server` is not found within your workspace path, you must 426specify the path of the server via the `mlir.pdll_server_path` setting. The path 427of the server may be absolute or relative within your workspace. 428 429###### Project setup 430 431To properly understand and interact with `.pdll` files, the language server must 432understand how the project is built (compile flags). 433[`pdll_compile_commands.yml` files](https://mlir.llvm.org/docs/Tools/MLIRLSP/#compilation-database) 434related to your project should be provided to ensure files are properly 435processed. These files can usually be generated by the build system, and the 436server will attempt to find them within your `build/` directory. If not 437available in or a unique location, additional `pdll_compile_commands.yml` files 438may be specified via the `mlir.pdll_compilation_databases` setting. The paths of 439these databases may be absolute or relative within your workspace. 440 441#### `.td` - TableGen files: 442 443The MLIR extension adds language support for the 444[TableGen language](https://llvm.org/docs/TableGen/ProgRef.html). 445 446##### Features 447 448- Syntax highlighting for `.td` files and `tablegen` markdown blocks 449- go-to-definition and cross references 450 451[tablegen-vscode features]: # 452 453##### Setup 454 455###### `tblgen-lsp-server` 456 457The various `.td` language features require the 458[`tblgen-lsp-server` language server](https://mlir.llvm.org/docs/Tools/MLIRLSP/#tablegen-lsp-language-server--tblgen-lsp-server). 459If `tblgen-lsp-server` is not found within your workspace path, you must specify 460the path of the server via the `mlir.tablegen_server_path` setting. The path of 461the server may be absolute or relative within your workspace. 462 463###### Project setup 464 465To properly understand and interact with `.td` files, the language server must 466understand how the project is built (compile flags). 467[`tablegen_compile_commands.yml` files](https://mlir.llvm.org/docs/Tools/MLIRLSP/#compilation-database-1) 468related to your project should be provided to ensure files are properly 469processed. These files can usually be generated by the build system, and the 470server will attempt to find them within your `build/` directory. If not 471available in or a unique location, additional `tablegen_compile_commands.yml` 472files may be specified via the `mlir.tablegen_compilation_databases` setting. 473The paths of these databases may be absolute or relative within your workspace. 474 475#### Contributing 476 477This extension is actively developed within the 478[LLVM monorepo](https://github.com/llvm/llvm-project), at 479[`mlir/utils/vscode`](https://github.com/llvm/llvm-project/tree/main/mlir/utils/vscode). 480As such, contributions should follow the 481[normal LLVM guidelines](https://llvm.org/docs/Contributing.html), with code 482reviews sent to 483[phabricator](https://llvm.org/docs/Contributing.html#how-to-submit-a-patch). 484 485When developing or deploying this extension within the LLVM monorepo, a few 486extra setup steps are required: 487 488- Copy `mlir/utils/textmate/mlir.json` to the extension directory and rename to 489 `grammar.json`. 490- Copy `llvm/utils/textmate/tablegen.json` to the extension directory and rename 491 to `tablegen-grammar.json`. 492- Copy 493 `https://mlir.llvm.org//LogoAssets/logo/PNG/full_color/mlir-identity-03.png` 494 to the extension directory and rename to `icon.png`. 495 496Please follow the existing code style when contributing to the extension, we 497recommend to run `npm run format` before sending a patch. 498
