Databricks vs BigQuery

Architecture^21,22,2,3

Databricks: A lakehouse: compute (clusters, serverless jobs and SQL warehouses) is separate from tables stored in cloud object storage, with Unity Catalog as the governance layer across them.

BigQuery: A serverless warehouse: Google runs the compute, measured in slots, and the storage. It can also manage Iceberg tables whose files sit in a Cloud Storage bucket you own.

Storage and open table formats^13,14,3,4,5

Databricks: Delta Lake by default. Unity Catalog managed Iceberg tables, foreign Iceberg tables and Iceberg v3 features became generally available on 21 May 2026, and external engines can read and write managed Iceberg tables through the Iceberg REST Catalog API.

BigQuery: Native BigQuery storage by default. Apache Iceberg managed tables keep data in your Cloud Storage bucket; other engines read them from exported metadata, and Google says to add data only through BigQuery. BigQuery writes to tables in the Lakehouse runtime catalog are in Preview.

Compute model^22,24,2

Databricks: Choice of classic clusters in your cloud account, serverless compute, and SQL warehouses; serverless SQL warehouses typically start in 2 to 6 seconds.

BigQuery: Queries run on slots. You use on-demand capacity, or reservations in the Standard, Enterprise or Enterprise Plus edition that autoscale; Standard reservations are capped at 1,600 slots.

Data engineering and pipelines^20,4

Databricks: Lakeflow covers ingestion connectors (Lakeflow Connect), Spark Declarative Pipelines for batch and streaming in SQL or Python, Lakeflow Jobs for orchestration, and Lakeflow Designer for visual data preparation.

BigQuery: SQL first, with BigQuery pipelines, the Data Transfer Service, Python UDFs (generally available since May 2026) and a generally available Data Engineering Agent. Spark work runs in Managed Service for Apache Spark, the service formerly called Dataproc.

Machine learning and AI^{26,25,21,8,2,6,10}

Databricks: Python-first ML next to the data. MLflow tracks experiments and manages the model lifecycle, Model Serving deploys custom models and LLMs as REST endpoints, and Agent Bricks builds governed agents; its Knowledge Assistant became generally available in January 2026. Models and functions are governed in Unity Catalog alongside tables.

BigQuery: BigQuery ML trains and runs models with SQL and can call models on Gemini Enterprise Agent Platform (formerly Vertex AI). It works with on-demand pricing and with Enterprise or Enterprise Plus reservations, but not Standard edition. Gemini in BigQuery adds data agents, data canvas and SQL generation, with some features still in Preview.

Governance and catalog^21,4,5

Databricks: Unity Catalog governs tables, views, volumes, functions and models with grants, attribute-based policies, row and column filters, and automatic lineage. An open-source implementation also exists.

BigQuery: Google Cloud IAM plus Knowledge Catalog, the name Google gave Dataplex Universal Catalog in April 2026. Open-format tables use the Lakehouse runtime catalog, formerly BigLake metastore.

Streaming^20,27,13,7

Databricks: Structured Streaming and Spark Declarative Pipelines handle streaming in the same code as batch. Real-time mode in Structured Streaming has been generally available since March 2026, and real-time mode in Spark Declarative Pipelines entered Public Preview in May 2026.

BigQuery: Continuous queries process rows as they arrive and write to BigQuery tables, Iceberg managed tables, Pub/Sub, Bigtable or Spanner. They need an Enterprise or Enterprise Plus reservation and don't run on on-demand pricing.

Operational databases (OLTP)^{15,16,18,19,17,4,7}

Databricks: Lakebase is a managed Postgres database inside the platform, with autoscaling, scale to zero, branching and tables synced from Unity Catalog. It is generally available on AWS and Azure but in Beta on Google Cloud.

BigQuery: BigQuery has no transactional database of its own; Google's operational databases, such as Spanner, are separate products. Continuous queries can export results to Spanner in real time, which Google made generally available in March 2026, and they need an Enterprise or Enterprise Plus reservation.

Cloud availability^24,2,4

Databricks: Runs on AWS, Azure and Google Cloud. On Google Cloud, classic compute runs in your own Google Cloud account.

BigQuery: Runs in Google Cloud regions. It can query data held in AWS and Azure through BigQuery Omni, and through newer cross-cloud connections that are in Preview.

Lock-in and portability^14,3,9

Databricks: Tables stay in Delta or Iceberg in your own cloud storage on any of the three clouds, and any Iceberg client can read and write managed Iceberg tables through Unity Catalog's REST endpoint. Lakeflow pipelines, jobs and SQL warehouse features are Databricks-specific.

BigQuery: Native tables live in BigQuery's own storage. Iceberg managed tables and Delta Lake external tables give you open-format options, each with documented limitations.