What Every CTO Needs To Know About Databricks

Databricks is a data platform that runs inside your own AWS, Azure or Google Cloud account. Data engineering, SQL analytics, dashboards, machine learning and AI agents all work on the same tables, which sit as open-format files in storage you own, under one permission model called Unity Catalog.

That's the core of what Databricks is. What a CTO needs to know beyond it is mostly about money, governance and how easily you could leave, and those are the parts the product pages say least about.

Updated by the TechFabric editors in October 2026.

The names changed, a lot

Databricks renamed much of its product between mid-2025 and mid-2026. If your team's architecture diagram is from 2024, some of its boxes now have different labels:

Two things are new rather than renamed. Lakebase, a managed Postgres database inside the platform, reached general availability on 22 January 2026, with branching, scale-to-zero and point-in-time recovery. Databricks Apps runs web applications built in Streamlit, Dash, React and similar frameworks next to the data, so an internal tool doesn't need its own hosting and its own copy of the tables.

The renames don't change what you'd build. They do matter when you read a proposal, because a partner still writing "Delta Live Tables" and "Mosaic AI" in late 2026 is working from old material.

How the bill works, and how to read it

You pay Databricks in DBUs, which the pricing page defines as "a normalized unit of processing power," billed per second. Each kind of compute (jobs, SQL warehouses, serverless notebooks, model serving) has its own DBU rate.

Where the compute runs matters to the bill. In the classic compute plane, the virtual machines run in your own cloud account, so you also get a cloud invoice for them. In the serverless plane, they run in Databricks' account and start in seconds (architecture). Serverless usage can take up to 24 hours to show in billing data.

The defaults are where the money goes. A serverless SQL warehouse stops after 10 minutes idle by default. A pro or classic warehouse waits 45 minutes (SQL warehouse settings). A classic warehouse that's queried every 30 minutes during the working day never stops at all.

You don't need a dashboard to see where spend goes. Every account has billing system tables, and this query, run in a SQL editor, shows DBUs and estimated list cost by product for the last 30 days. The join is the one Databricks uses in its own cost examples:

SELECT
  u.sku_name,
  SUM(u.usage_quantity) AS dbus,
  SUM(u.usage_quantity * p.pricing.default) AS list_cost
FROM system.billing.usage u
JOIN system.billing.list_prices p
  ON u.cloud = p.cloud
  AND u.sku_name = p.sku_name
  AND u.usage_start_time >= p.price_start_time
  AND (u.usage_end_time <= p.price_end_time OR p.price_end_time IS NULL)
WHERE u.usage_date >= CURRENT_DATE() - INTERVAL 30 DAY
GROUP BY ALL
ORDER BY list_cost DESC;

It's list price, so a negotiated discount won't show. system.billing.usage keeps 365 days, and the usage_metadata column carries the job, cluster and warehouse IDs, so the same table answers "which job" as well as "which product." If the top line is all-purpose compute, people are running scheduled work on interactive clusters, which is one of the first things a cost review looks for.

Governance is the decision to make first

Unity Catalog is where permissions, lineage and audit live. Three facts shape how you set it up:

• Every table has a three-level name, catalog.schema.table.
• You get one metastore per region, shared by every workspace in that region. Catalogs, not workspaces, are how you separate departments or environments.
• The older per-workspace Hive metastore is described by Databricks as a legacy feature. A new platform shouldn't create anything in hive_metastore.

Permissions are SQL, and they're readable by an auditor:

GRANT USE CATALOG ON CATALOG main TO `finance`;
GRANT USE SCHEMA ON SCHEMA main.sales TO `finance`;
GRANT SELECT ON TABLE main.sales.orders TO `finance`;

Row-level and column-level rules are SQL functions attached to a table. This one, from the row filter documentation, lets admins see every row and everyone else only US rows:

CREATE FUNCTION us_filter(region STRING)
RETURN IF(IS_ACCOUNT_GROUP_MEMBER('admin'), true, region = 'US');

ALTER TABLE sales SET ROW FILTER us_filter ON (region);

Lineage and audit are tables too. system.access.table_lineage records which tables feed which, and system.access.audit records who did what, each for 365 days. When someone asks who can see the payroll extract, or what breaks if a source table changes, the answer is a query.

Getting this right before the first pipeline is much cheaper than retrofitting it. Our Unity Catalog page covers how we design it.

How easily could you leave?

Lock-in is the question a CTO should ask of any data platform, and Databricks' answer is better than it was in 2024.

Tables are stored as Delta Lake, an open format that's been under the Linux Foundation since 2019, in your own cloud storage. UniForm writes Iceberg metadata alongside Delta, so Iceberg engines can read the same files without a copy (UniForm). Databricks also supports managed Iceberg tables, readable through the Iceberg REST Catalog API. Unity Catalog itself was open-sourced in June 2024.

Snowflake has moved the same way. Its Iceberg tables keep Parquet in your storage, readable by other engines through its Horizon catalog. Snowflake's standard tables, by contrast, live in its own micro-partitions, and you reach them through Snowflake. We've written separately about why the lock-in argument for leaving Snowflake is over, and the same logic applies in the other direction.

Lakehouse Federation lets Databricks query SQL Server, PostgreSQL, Oracle, Snowflake, BigQuery and others in place, read-only, without copying (federation). It's a good way to start without a migration, and a poor substitute for one when the same remote table is queried thousands of times a day.

Databricks or Snowflake

The 2024 version of this post said our data teams were choosing Databricks more often than other platforms. The reasons are less about features than about where each platform started.

Snowflake began as a cloud data warehouse and added Python (Snowpark), Iceberg and AI over time. Databricks began with Spark at UC Berkeley's AMPLab and added SQL warehouses and governance. Each is now competent at the other's original job.

For a team whose work is mostly SQL and dashboards, either will do, and the decision often comes down to which one your engineers already know. Where Databricks pulls ahead is work that mixes engineering, machine learning and applications on the same data: streaming pipelines, models in production, agents that need governed access to tables, and internal apps on Lakebase.

When a lakehouse is overhead

If your data is one ERP and Power BI is the whole of your reporting, a lakehouse is overhead. Power BI on a well-modelled source, or Microsoft Fabric if you already hold the capacity, will be cheaper to run and easier to staff.

If you have no data engineering capacity and no plan to hire or contract it, a platform that assumes engineers will disappoint you.

Databricks earns its cost when there are several source systems that need to agree, data volumes that outgrow a single database, or AI work that has to run against governed data rather than an export someone emailed.

Evaluating it without a contract

Databricks has two ways in. Free Edition, launched in June 2025, is free and permanent but non-commercial, serverless only, with a single 2X-Small SQL warehouse. It's fine for learning, not for your company's data. The trial gives 14 days of credits on a real account.

A useful two-week evaluation takes one real workload, a report that today needs three spreadsheets and a person, and builds it end to end. Ingest the sources, model them in Unity Catalog with grants, put a dashboard on top, then run the billing query above. You'll know what it costs, how long it took and whether your team can run it, which is more than any feature comparison tells you.

TechFabric is a Databricks partner, and Databricks is where we've chosen to go deep. If you'd like a second opinion on a proposal or an existing workspace, our Databricks consulting page explains how we start.

Frequently asked questions

Why can a classic SQL warehouse cost far more than its queries suggest?

A serverless SQL warehouse stops after 10 minutes idle, and a pro or classic one waits 45, so a classic warehouse that gets a query every 30 minutes never stops. If the top line of your billing query is a warehouse, check its auto-stop setting before you look at anything else.

Will the billing query show what we actually pay?

No, it shows list price, so a negotiated discount won't appear in it. Treat the figure as a way to rank products and jobs, not as the invoice total. On classic compute the virtual machines also arrive as a separate cloud invoice, so that cost isn't in this figure either.

How far back can I look when I ask who changed what, or what feeds a table?

A year. The lineage and audit tables each keep 365 days, and so does the billing usage table. If an auditor's question reaches further back than that, the answer has to come from somewhere you've copied it to, so decide that before you need it.