Workload observability
Know what your Databricks estate is doing, and who may touch it
Governed mission control for running data and ML workloads on Databricks.
- Role in the family
- Workload observability
- Databricks surfaces
- Databricks JobsLakebaseUnity CatalogTemporal
- Status
- Domain module, policies, state machines, and org-scoped projections are implemented, with a Temporal intervention worker and a Databricks effector.
- Documentation
- radar.techfabric.com
The problem
Monitoring tells you a job failed. It rarely tells you whether that breach matters, who is allowed to intervene, or what was actually done about it last time. The intervention itself usually happens in a terminal, unlogged.
How it works
Monitors carry declared service level objectives. When reality breaches one, Radar triages the anomaly and makes every intervention a governed action with a policy check and an audit record. Telemetry ingest is deliberately append-only and stays outside the governed pipeline; only state transitions pass through it.
What it does
SLOs as declarations
What 'healthy' means is written down per workload.
Governed intervention
Restarting a job or rerouting a pipeline is an action with a policy and an audit trail, not an untracked click.
Durable remediation
Interventions run as Temporal workflows, so a long remediation survives a restart mid-flight.
Anomaly triage
Detection composes the same evaluation logic.
Someone is watching your live workloads, and every intervention is on the record. Declared SLOs replace tribal knowledge about what healthy looks like, and a two in the morning fix becomes a governed action with an audit trail.
The job failed. Nobody knows whether it mattered.
Monitoring answers whether something broke. The questions that actually cost money are the three it does not answer.
An alert with no severity
A job failed. Whether that breach matters, and to whom, is worked out by somebody who happens to know what that job feeds.
Nobody is allowed to act
The person who sees the alert cannot stop the workload, and the person who can is not looking. The gap between those two is where the bill accrues.
The bill arrives without a cause
Spend doubled last quarter. Which jobs did it, and whether any of them produced anything, is a forensic exercise rather than a dashboard.
All three are the same missing thing. Observation is not the hard part. Deciding what a breach means and who may respond to it is.
The shift
Three ways to watch a live estate
Only one of them lets somebody act on what they see.
Platform monitoring
It ran, or it did not
A dashboard somebody built
Accurate, and unread
Governed mission control
Watched, scored, and actionable
What gets installed
Observation, judgement and the authority to act, as one system.
Workload signals
Job state, latency, cost and data quality collected as events rather than as scraped dashboard state.
Severity from policy
Whether a breach matters is evaluated against rules somebody wrote, not inferred by whoever is on call.
Interventions with authority
Stopping a workload is an action with a policy in front of it, so the person who sees the problem can be the person who fixes it.
Cost, attributed
Spend traced to the jobs and the teams that caused it, so a growing bill has a name rather than a mystery.
Budgets that act
A budget that stops a workload rather than emailing about it after the quarter closed.
A record of every response
What fired, who acted, under which policy. Reading the estate's history is a query.
Policy
Decides severity, rather than whoever is on call
Built for estates we were on the hook for
Radar came out of running Databricks workloads where somebody was going to ask us what happened and why. It is free and open source under Apache-2.0 and it runs in your workspace.
FAQ
TechFabric Radar, answered
How is this different from the monitoring we already have?
Monitoring tells you a job failed. Radar carries declared service level objectives per workload, so it can tell you whether the breach matters, who is allowed to intervene, and what was actually done about it last time. If you are not sure the workspace is worth that yet, the Databricks Health Check at /databricks/health-check is the two-week way in.
What does a governed intervention mean in practice?
Restarting a job or rerouting a pipeline runs as a policy-checked action with an audit record, executed as a durable workflow. A fix at two in the morning stops being an untracked click in a terminal.
Does it ingest all our telemetry?
Telemetry ingest is deliberately append-only and stays outside the governed pipeline. Only state transitions pass through it, which keeps the audit trail meaningful instead of drowning in events.