A governed lakehouse your executives can question

Ingestion

Where most platforms acquire the debt they never pay off. A source connected badly in month one is still being worked around in year three, usually by a job nobody wants to touch.

• Lakeflow Connect for the sources it covers, and a written reason when we build instead
• Change data capture off operational databases without asking the DBA for a nightly window
• Files, APIs and queues landed once, with the schema drift handled rather than discovered
• Backfill and replay treated as normal operations rather than as incidents

Modelling

Bronze, silver and gold is a naming convention rather than a design. What decides whether the platform is usable is which grain the gold layer settles on and whether two teams can agree on it.

• Grain and conformed dimensions agreed with the people who will be questioned about the numbers
• Slowly changing history where the business genuinely reconstructs the past, and nowhere else
• Incremental processing that does not rebuild the estate to add a day
• Tests on the tables that finance reads, running on every load

Semantic layer and Genie

A Genie space is only as good as the definitions under it. Ours are written down, versioned and scored against ground truth, because the failure mode is not an error message. It is a plausible answer that disagrees with the close.

• Metric definitions held in the semantic layer rather than in each dashboard
• Synonyms and business vocabulary captured from how people actually ask
• An evaluation suite with expected answers, run on a schedule
• A named owner for every metric, so a disagreement has somewhere to go

Lakebase

Serverless Postgres inside the lakehouse, which is what lets an application share the governed data rather than copy it. This is the part where a decade of building applications matters more than a decade of building pipelines.

• Transactional state next to the analytical estate, with the sync direction stated
• Branching for environments, so a test does not run against production rows
• Unity Catalog permissions carried through instead of re-implemented in the app
• Applications and agents reading from it directly, under their own service principal

Cost and operations

The bill is an engineering output. Warehouse sizing, cluster policy, table layout and job scheduling account for most of it, and all four are decisions somebody made rather than facts about the platform.

• Serverless where the workload is spiky, classic where it is steady, with the arithmetic shown
• Liquid clustering and file sizing on the tables that are actually scanned
• Budget policies and tagging, so a spike has an owner within the hour
• Alerting on freshness and reconciliation, not only on job failure