What a year of building ADBC drivers taught Columnar about zero-copy's non-zero problems

At a dbt Summit 2026 breakout session titled "ADBC in production: zero-copy, non-zero problems," Matt Topol, co-founder at Columnar and a member of the Apache Arrow, Apache Iceberg, and Apache Magpie project management committees, walked through a year of lessons from building ADBC drivers for production use.

ADBC, short for Arrow Database Connectivity, is a vendor-neutral, Apache-licensed standard that gives applications one API for talking to many database drivers. Every driver hands data back in Apache Arrow's columnar, in-memory format instead of the row-oriented protocols behind ODBC and JDBC.

Topol traced the problem ADBC solves back to a 2017 paper on client protocol design: for decades, the industry optimized database compute and vectorization while row-oriented client protocols quietly became the real bottleneck, even as hardware and networks got much faster.

dbt v2 is one of the applications that adopted ADBC, using it as the core communication layer for its adapters while layering custom hooks on top for individual data systems. Power BI, Apache Airflow, dltHub, and several other tools have adopted ADBC too, alongside vendor drivers for Snowflake, BigQuery, Databricks, ClickHouse, and more.

DuckDB implements the ADBC interface directly, Polars is built on Arrow as its native in-memory format, and both Snowflake and Databricks return Arrow natively, all of which lets ADBC skip serialization overhead entirely once data lands in Arrow format.

Topol was candid that the reality behind that pitch gets complicated fast: many vendors, including Postgres, Trino, and SQL Server, still only return JSON or other row-oriented formats, and driver authors have to handle that conversion themselves.

Type systems turned out to be where most of the pain lives. SQL's numeric type has no standard maximum precision, so implementations diverge widely: SQL Server caps at 38 digits, ClickHouse at 76, MySQL and MariaDB at 65, and Postgres allows more than 1,000 explicit digits, while BigQuery ignores declared precision entirely and always returns decimal128(38,9).

String handling is just as inconsistent. Oracle treats empty strings as null with no way to tell the two apart, Postgres disallows the UTF-8 null byte, SQLite performs no UTF-8 validation at all, SQL Server strings are UTF-16, and ClickHouse has no separate binary type, just byte strings that get returned as Arrow strings regardless of their actual encoding unless a driver-level setting says otherwise.

Timestamps caused their own set of surprises. Snowflake stores a time zone offset per row rather than per column, which pushed the Arrow project to add a new extension type just to preserve that information, while ClickHouse always attaches a time zone to every timestamp and defaults silently to the local system's time zone when none is given.

Spark isn't really one target either. Its ADBC driver has to support three different backend protocols (Hive/Thrift, Apache Livy, and Spark Connect), each with its own quirks, such as Livy's lack of support for timestamps with no time zone and Thrift's tendency to mangle non-UTF-8 binary data.

Topol demonstrated the same kind of inconsistency live with a BigQuery example: an empty list and a null list both come back looking identical in a result set, even though BigQuery's own query engine can tell the difference internally, and BigQuery cannot represent a null element inside a list at all, per its own documentation.

The underlying design constraint, Topol said, is that a driver has to choose its Arrow type mapping before it ever sees the actual data, based only on a database's documented type limits. That's why a driver has to plan for edge cases like Postgres's four-figure numeric precision even though almost nobody uses it.

To make drivers easier to install and configure than the ODBC era, ADBC introduces driver manifests, simple TOML files in documented locations that describe a driver and where to find its shared library across operating systems, and connection profiles, TOML-based and similar to ODBC's DSN files, referenced as profile://name so credentials and per-environment options never have to live in application code.

ADBC still has open gaps, including a fully synchronous API, uneven type support across drivers, and clunky metadata handling, but Topol pointed to an upcoming spec revision that adds multi-result-set support, non-fatal warning handlers, and a way for applications to request specific types from a driver.

His closing advice for anyone adopting ADBC: test your schemas early, pin your driver versions, and always know exactly which driver version you're running.

Watch every keynote and breakout from dbt Summit 2026 on demand. Watch dbt Summit 2026 on demand →