Skip to content
TechFabric

Beyond basic data catalogs: how Unity Catalog solves real enterprise governance challenges

Preetham Reddy3 min read

A catalogue that only records tables won't answer the questions a security team actually asks. Who can see a row, who changed it, and whether an agent is held to the same grant as the analyst sitting next to it.

We considered running our own context store outside Databricks. It would have been quicker, and it would also have meant copying governed data into a second system and then explaining why two permission models existed. Unity Catalog stays the single place access is decided, which is why Fabric runs on it.

What actually breaks

Hive metastore leftovers. Schemas nobody moved because the pipeline already worked. A service principal granted ALL PRIVILEGES for a Thursday demo.

The one that catches people is coverage that looks complete in the catalogue browser and stops halfway when you list every schema. The browser shows you what you clicked on. It doesn't show you what you never clicked on, and the three schemas still sitting in hive_metastore are exactly the ones nobody has opened in a year.

There's a trap in how people go looking, too. The obvious move is system.information_schema, which describes objects across every catalog in the metastore and is the right tool for most questions. It excludes hive_metastore by design. So the one audit you would run to find your leftovers is the one audit guaranteed not to show them, and the report comes back clean. SHOW SCHEMAS IN hive_metastore is the unglamorous answer, and it takes about a minute.

Troy wrote about the lakehouse version of this in Unity Catalog before the first pipeline. The Databricks Health Check is the two-week look at privilege hygiene, orphaned compute, and the jobs that should be serverless. Fabric Radar is what we use if we stay to make the interventions governed.

Why not a second catalogue

Glue, Purview and BigLake each do a job. The job we get hired for is a Databricks programme where the applications and the agents have to inherit the same grants as a person, and Unity Catalog is what makes that possible at all.

A second catalogue next to it is how it fails, and the failure is not dramatic. It's that two systems now disagree about who can read the customer table, and the answer a reviewer gets depends on which one they ask. Nobody notices until someone asks both.

What we do with it

Catalogue and schema design that matches how the teams actually work. Grants a security review can read. Lineage that stays intact as data moves.

Databricks Apps and agents run under their own service principal, so they can't reach what they have no grant for. Fabric Harness resolves tools through Unity Catalog for the same reason. An agent that gets its permissions from a config file has permissions nobody reviewed.

The lineage tables are the part most teams have and don't use. system.access.table_lineage records every read and write with source_table_full_name and target_table_full_name, and since the May 2025 schema change it carries a statement_id you can join to system.query.history. So "who touched this table, from which query, under whose identity" stops being an investigation and becomes a join.

If you're standing the lakehouse up, that's the Launchpad, with Unity Catalog before the first pipeline. If you already have a workspace and you're not sure the grants hold, start with the Health Check.

Used properly, an audit becomes a query. Used as a sticker on a hive_metastore estate, Unity Catalog is furniture.

The Health Check will tell you which one you have.