# Databricks cost optimization

> We find the Databricks spend that is not buying anything: all-purpose compute running jobs that should be serverless, clusters sized for a load that never arrived, schedules nobody owns, and warehouses left running overnight. You get findings ranked by severity with the estimated saving and the reasoning behind each one.

For: Data platform leads whose Databricks bill is outgrowing the work
Canonical: https://www.techfabric.com/databricks/cost-optimization

---

Cut the Databricks bill without cutting what it does

The bill grows faster than the work and nobody can say which part is the problem, because a cost dashboard tells you what you spent rather than which of it bought nothing. That judgement is the deliverable.

## What we look at

- All-purpose compute running scheduled work that belongs on jobs or serverless
- Cluster and warehouse sizing against what the workload actually needs
- Job failures and retries, which are paid for twice and rarely counted
- Schedules and compute that outlived whoever created them
- Warehouses with no auto-stop, which is the quietest line on the bill
- Model serving and Unity Gateway spend, where they are in use

## What you get

- Findings ranked by severity, each with an estimated saving and the reasoning shown
- A thirty, sixty and ninety day order, written so your own team can act on it
- The changes that are safe to make immediately, separated from the ones needing a change window
- A fixed-scope proposal for the remediation, if you would rather we did it

## Clients we have done this for

- [Location Services](https://www.techfabric.com/case-studies/vrm) (Automotive / Fintech): Location Services' repossession platform automates assignment intake, so fewer people handle each case, and our engineers built it with the company's own technology team. When it was time to leave Synapse and SQL Server, we moved it onto Databricks in waves that could each be rolled back, and the rules engine that checks every assignment came through unchanged.
- [AmTab](https://www.techfabric.com/case-studies/amtab) (B2B ecommerce): AmTab, a table manufacturer, had orders moving between departments on paper while demand climbed. Senior engineers from TechFabric worked through those manual steps with AmTab's own people first, then connected the amtab.com store, the CRM and Business Central, so an order placed online reaches the ERP that invoices it without anyone retyping it.

## Which clusters the all-purpose spend came from

The Health Check prices the whole bill by product. When all-purpose compute is near the top of that list, this is the narrower question: which clusters spent it. Jobs pinned to an interactive cluster don't show up as job cost at all, so they surface here, as a cluster somebody created for a notebook and a schedule nobody remembers adding.

The usual fix is moving that work to jobs compute or serverless, which costs less per unit and stops when the run does.

```
SELECT usage_metadata.cluster_id,
       SUM(usage_quantity) AS dbus
FROM system.billing.usage
WHERE usage_date >= current_date() - INTERVAL 30 DAYS
  AND sku_name LIKE '%ALL_PURPOSE%'
GROUP BY ALL
ORDER BY dbus DESC
LIMIT 20;
```

Databricks SQL against system.billing.usage. DBUs, not dollars; the Health Check query prices them.

Checked on 12 October 2026 against: [the billing usage system table](https://docs.databricks.com/aws/en/admin/system-tables/billing).

## The storage work Databricks will now do for you

Predictive optimization runs OPTIMIZE, VACUUM and ANALYZE on Unity Catalog managed tables without anyone scheduling them. It's on by default for accounts created from 11 November 2024, and Databricks scheduled the rollout to older accounts to finish by August 2026, so a lot of hand-written maintenance jobs are now paying twice for the same work.

It only applies to managed tables, which is one more reason to finish a Hive metastore migration rather than stop at external tables. For new tables Databricks recommends liquid clustering over partitioning, and CLUSTER BY AUTO lets predictive optimization choose the keys.

Checked on 12 October 2026 against: [predictive optimization](https://docs.databricks.com/aws/en/optimizations/predictive-optimization), [liquid clustering](https://docs.databricks.com/aws/en/delta/clustering).

## Questions

### How do you reduce Databricks costs?

By finding the spend that bought nothing rather than by turning things down. Usually that is scheduled work on all-purpose compute, clusters sized for a peak that never came, warehouses without auto-stop, retries nobody counted, and schedules whose owner left. Each finding comes with an estimated saving and the reasoning, so you can disagree with it.

### Will this slow our workloads down?

That is the constraint the work runs under. Findings are split into what is safe immediately and what needs a change window, and anything with a performance trade-off is written up as a trade-off rather than a saving. A recommendation that makes the platform worse is not a saving.

### What access do you need?

Read access to the workspace and to system tables where you have them. No production data and no write access. If your security process wants a named scope written before anything is granted, we will write one.

### How is this different from our cost dashboard?

A dashboard reports spend. This says which of it bought nothing, why, and what to change, which is judgement rather than a billing export. It comes from engineers who have run these platforms and recognise the patterns.

