Understanding LSP: What it is, and what you can use it for

A good code editor can make or break a developer’s productivity.

Writing code in a blank text file without hints or warnings slows even experienced developers. Code editors provide features like error checking and suggestions that plain text editors don’t. These features depend on language tooling that parses and compiles code behind the scenes.

Previously, language providers built separate extensions for each editor because editors exposed different APIs for language features. The Language Server Protocol (LSP) enables separating language servers to function independently of code editors. This enables language servers like dbt LSP to provide the same language features for different development tools.

In this article, we'll explore how LSP works, the problems it solves, and how dbt is leveraging LSP capabilities to make writing data transformation pipelines easier than ever before.

What is Language Server Protocol (LSP)?

LSP is an open protocol that standardizes communication between code editors and language tooling. It uses JSON RPC messages between the development tool acting as the client and a language server. The protocol defines a shared set of messages for language features like code completion and syntax highlighting.

Microsoft developed LSP in 2016 for Visual Studio Code to eliminate the need for building separate language services. The goal was to enable a single language implementation to work across multiple development tools. This approach saves both language service providers and development tools from additional effort.

The client sends notifications to the server about file changes or cursor positions. The language server processes these requests and responds with precise diagnostics and information.

How does LSP help software developers?

Developers faced fragmented support and heavy performance loads before LSP adoption. With LSP, they benefit from the following solutions:

• Language providers no longer need to develop separate plugins for each editor.
• Heavy tasks like code parsing and syntax analysis run in the language server instead of the editor. This keeps the editor lightweight and responsive.
• Developers can more easily integrate linters, formatters, and refactoring tools into their editors.
• A language server backend can be written in any programming language. It can then be integrated easily into a variety of tools.
• Developers can experiment with various plugins and editors while keeping the same language features and plugin support.

In short, LSP makes it easier to support special-purpose development languages without creating a new Integrated Development Environment (IDE) from scratch.

How does LSP work?

LSP defines client-server message exchange using JSON-RPC over stdio or socket transports. Each message contains headers specifying Content-Length and Content-Type before the payload. The payload encodes requests, responses, or notifications following JSON RPC 2.0 rules.

Communication between an LSP client and server

Communication start

Communication starts when the editor launches the language server as a separate process.

1. The client sends an initialization request describing supported features and workspace settings.
2. The server replies with an initialization response declaring supported language features and capabilities.
3. After receiving the response, the client confirms readiness through an initialized notification. From this point on, both sides rely on their declared capabilities to exchange feature messages.

Document management

LSP keeps the server synchronized with editor changes in real time as developers edit files.

1. When a file opens, the client sends a didOpen notification with full document text.
2. As edits occur, the client streams incremental changes using didChange notifications.

Closing a file triggers a didClose notification, which the server uses to release the document state.

Feature interactions & results

The client and server communicate through feature interactions triggered by specific actions within the editor. Features such as hover-over tooltips or go-to-definition clicks trigger this exchange. During the exchange, the client requests language intelligence, and the server returns the relevant results.

1. Feature interactions use request-response pairs for tasks such as completion, definition, or references.
2. The request includes the document uniform resource identifier (URI), cursor position, and method-specific params object. Each request has a unique ID, allowing the client to match responses reliably.
3. Servers analyze documents independently and return diagnostics without executing queries remotely.
4. Editors display results immediately, keeping typing latency low and feedback continuously visible.

The protocol supports multiple transports, including standard input output and TCP sockets. This workflow allows a single language server to provide consistent tooling across multiple editors.

The limitations developers face without LSP diagnostics

Real-time feedback and comprehension from a language server help developers address common challenges earlier. Data developers especially face these challenges in general SQL development. They become even more visible when managing complex data transformations or writing intricate SQL logic.

Common challenges include:

1. Runtime error discovery. Errors in SQL queries or models only become apparent after running them against the data warehouse. This often results in additional compute costs.
2. Previewing common table expression (CTE) outputs. Developers comment out parts of queries to preview CTE outputs. This creates a slow and manual debugging process.
3. Subtle changes impact downstream models. Renaming columns or modifying joins may cause dependent models to break. There is no early visibility into these effects.
4. Opaque dynamic SQL. Some SQL queries are generated automatically, hiding the statements that actually run. This makes it difficult to trace changes across different platforms.
5. SQL dialect differences. Platform-specific syntax, like Snowflake versus Databricks, only becomes apparent at runtime. As a result, incompatible functions may go unnoticed during editing.

Without these immediate diagnostics, debugging often becomes slower, more costly, and more manual.

How dbt uses LSP and Fusion to power SQL development

There are many SQL linters that can detect simple syntactical issues with SQL code. Until now, however, there hasn’t been a solid LSP implementation that could dynamically detect errors in your SQL code relative to your environment.

dbt uses LSP to deliver language tooling features to the dbt VS Code extension via the high-performance dbt Fusion engine. Fusion is the next generation of our dbt Core engine for data transformation that natively understands SQL across multiple data warehouse engine dialects.

Fusion supports most of (and, soon, all) of the core features of dbt Core. This enables using dbt models to transform data with a common syntax, no matter where it lives in your enterprise. Unlike dbt Core, Fusion can locally emulate your remote data warehouse environment and run queries against it without calling out to your remote stores. This means Fusion can detect issues with your code before you even check it in to source control.

Here are the features dbt Fusion provides to developers via its use of LSP in VS Code:

• Live error detection. Language server automatically validates your SQL code instantly as you type. LSP detects errors instantly in VS Code, giving developers full project awareness without querying your data warehouse.
• Lightning-fast parse times. The dbt Fusion engine processes even the largest projects quickly. It parses up to 30 times faster than dbt Core, accelerating the developer workflow even at scale.
• Powerful IntelliSense. This feature provides intelligent autocompletion suggestions for SQL functions, model names, and macros. It autocompletes references and source calls, facilitating faster development workflows.
• Instant refactoring. You can easily rename models or columns using the LSP. Downstream references are automatically updated throughout the entire project, ensuring consistency and validity instantly.
• Rich lineage context. Project metadata like column-level and table-level lineage is displayed directly within the editor. There’s no context switching or breaking the flow during development.
• Live CTE previews. Developers can preview a specific CTE’s output directly. This happens right from inside the current dbt model file. This feature enables faster query validation and debugging cycles.
• Hover insights. You can see critical context without ever leaving your code. Contextual details about tables, columns, and functions are visible within the code editor.
• View compiled code. Developers can get a real-time, side-by-side preview of the SQL code. The compiled SQL reflects what dbt models generate and updates automatically as source files change.

Leveraging LSP for easier data pipeline development

LSP has fundamentally transformed the development and delivery of language support across the industry. Now that LSP is widely adopted, the focus is on enhancing and expanding its capabilities.

The limitations of code editors no longer constrain language providers. They can develop sophisticated tooling to help users write more efficient, higher-quality code. dbt uses this standard to deliver a step-function improvement in building data pipelines with dbt Fusion.

Experience faster development, fewer mistakes, and deeper project understanding. Sign up for a free dbt account. Then, get started with dbt Fusion and the dbt VS Code extension today.