How RELY on a Databricks UNIQUE constraint rewrote our duplicate check
With RELY on, our Databricks duplicate check returned 0 on a table with a duplicate key. What we measured, and the checks that still caught it.
Category
Knowledge Share
64 posts
Investigating failed SQL in a completed Genie One response
How to correlate a completed Genie One reply with a failed SQL query, using our synthetic diagnostic and a separate tracing setup check.
What makes a Gold table ready for business in Databricks
Every daily rate in our Gold table was right, yet one supplier's weekly average read 12.0% against 4.8%. The four checks we'd run before certifying the table.
Exporting AI decision history from Lakebase through one event table
Lakebase Change Data Feed exports whole schemas. How we gave it one event table filled by triggers, and built six gold tables in Delta on top.
Embedded analytics on Databricks: tenant isolation in one JOIN
Row-level security for embedded AI/BI dashboards when viewers have no Databricks identity. One scope table, one JOIN on the embed token, fail-closed.
How we balance rows between Bronze, Silver and Gold
Our row counts added up while one receipt was missing and another was counted twice. The SQL count per arrival ID that caught it, and the daily Gold check.
We caught a Genie metric change its own benchmark missed
Genie's built-in benchmark re-runs your ground-truth SQL, so a changed metric still scores Good. How a golden set with a definition hash caught it.
How we block AI releases when MLflow scorers fail
A scorer that crashed on two wrong answers raised our MLflow evaluation score from 0.75 to 1.00. We built a release rule that counts coverage first.
Search that keeps up with a live game and an open loan
We indexed a wine catalog and refinance queues at iLending and Auto Approve. Search can now read a streaming table or a materialized view.
How we test that AUTO CDC keeps the right customer record
A Databricks CDC flow can finish green and still keep a customer's old city or bring a deleted customer back. How we test ordering and deletes before release.
How we catch wrong AI answers before they reach customers
An AI step in a Databricks job can report success while tickets get no category or the wrong one. How we catch both before the results reach the support team.
Real-time loan data with Databricks Zerobus Ingest
Most lenders' fraud checks and examiner answers run on last night's loan data. How Zerobus Ingest gets it there in seconds, and where Kafka still fits.
Adding WebMCP beside our techfabric.com MCP server
WebMCP doesn't replace MCP. Using our own techfabric.com MCP server, we show which agent tasks belong on a server and which belong in an open web page.
Databricks CustomerLake: a customer data platform inside the lakehouse
CustomerLake builds Customer 360 profiles and audiences on data already in Databricks. What's in Beta, and the workloads we'd move to it first.
What our coding agent's green test runs left untested
A coding agent reported 33 of 41 test tasks successful while the three packages it changed never ran. The checks that would have caught it.
Catching a false success in Databricks notebook CI
A test can catch a broken data rule and the release still goes out green. We reproduced that false success and built a runner that stops it.
Backend engineering habits on a Databricks lakehouse
A retried Databricks job can double-count a report without failing anything. The backend habits that prevent it, and how our test kit fails jobs on purpose.
How our engineers turn their work into posts with coding agents
Engineers who solve the problems rarely have time to write them up. We built an MCP server so they publish from their coding agent, under the site's checks.
How we tested data quality rules against twelve defects
A data quality suite can pass every day and still let a duplicate order into the revenue report. We fed our rules twelve defects to find which got through.
How we merged five embedded Databricks dashboards into one
Five client reports as five embedded dashboards meant a new iframe and embed token on every switch. Why we merged them into one AI/BI dashboard, and the cost.
How we designed a release gate for Databricks data quality
A failed Databricks quality check doesn't stop bad data reaching reports on its own. We designed a release gate that does, and tested it on six cases.
Reading Delta Lake tables from DuckDB through Unity Catalog
We tried reading our own Databricks tables from DuckDB and none were readable. What that means for sharing lakehouse data with partners and AI agents.
OpenCode 2.0 and the open harness as a service
OpenCode 2.0 runs as one background service the terminal, web and desktop share. What changes, and the config we'd use to route work to cheaper models.
How we check our drafts with AI decision models
Approving a claim or routing a ticket is a call software repeats all day. Why Decision-1 checks our posts first, and how it compares with Jev, Clef and Kev.
SkillFS: governing agent skills with a file system
Alibaba Cloud's SkillFS controls which skills an AI agent can load. What our own writing agent's two-day stall taught us about the same problem.
BC29: multiple excise duties per item in Business Central
BC29 lets one Business Central item carry several excise duties. What it changes for the item records, period close and reporting in the systems we build.
What Databricks' Funke changes for HL7v2 pipelines
Funke, announced by Databricks on 8 October, parses HL7v2 into Unity Catalog intact. What it changes for clinical pipelines, and what it leaves alone.
How we cut duplicate queries from embedded Databricks dashboards
A client's embedded Databricks reports sent one query to their app database five times per filter click. How we cut that to once, and what the rule costs.
Replit on Databricks: governed apps with Lakebase Postgres
Replit Agent now deploys apps inside Databricks with a Lakebase database behind each. What we'd decide before anyone prompts an internal tool into existence.
Supermemory for per-user AI agent memory
Supermemory remembers each person an AI agent talks to. Where we'd use it beside the reviewed memory our own agents keep, and the defaults we'd change.
How we split our AI decisions across Cloudflare's three Clef models
Cloudflare's three Clef decision models on one table, with size, context, price and inputs. Where each one fits, from our test on 70 live stories.
Databricks Lakebase: managed Postgres next to your lakehouse
Lakebase is managed Postgres 16 to 18 inside Databricks. How synced tables and the change data feed move data both ways, and how Unity Catalog governs it.
BC29 keeps DEV extensions installed through sandbox updates
BC29 keeps extensions published from VS Code installed when a sandbox updates. What that changes for UAT on the Business Central systems we extend.
Business Central BC29: turning indexes on and off from AL
Each Business Central index slows every write to its table, in every company. BC29 lets AL code switch indexes per company. How we would use it, and where not.
Lakebase branches for test runs, previews and coding agents
A shared test database lets one change break another and holds up releases. How we give each test run its own Lakebase copy, and how we'd do it for AI agents.
Lakebase vs Snowflake Postgres: branching is the difference
Lakebase branches a copy of production on request; Snowflake Postgres runs on a dedicated VM with attached disks. Branching, scale to zero and governance.
Starting our SOC 2 journey: what we found before the audit
Customers asking for a SOC 2 report? Preparing for SOC 2 Type II, we found the controls were already running and proving them was the work.
Unity Catalog for AI agents in practice: four FHIR scenarios
An AI agent can show records a person isn't cleared to see with every row filter working. Four FHIR scenarios on Databricks, with the performance tradeoffs.
Business Central reporting: built-in, Power BI or lakehouse
Three ways to report on Dynamics 365 Business Central, what each one is good for, the limits Microsoft documents, and how to tell which one you need.
An AI governance framework is four answers you can query
The EU moved its high-risk AI deadline to December 2027. What NIST AI RMF, ISO 42001 and the EU AI Act ask, mapped to controls you can run on Databricks.
LakeWright.NET carries a SaaS tenant boundary into Databricks
An Apache-2.0 .NET kit that resolves the customer tenant before a request reaches Databricks SQL, Jobs, AI/BI, Genie or Model Serving.
Fabric or Databricks is the wrong question on Azure
Microsoft and Databricks shipped the interop this year. For teams already on Azure the question is which platform holds the tables and which one reads them.
The lock-in argument for leaving Snowflake is over
Snowflake and Databricks now read and write each other's Iceberg tables. If a vendor is still selling you a migration on lock-in, they are selling you 2023.
Export to Data Lake stops on 30 November
Export to Data Lake stops permanently on 30 November 2026. Microsoft names two replacements for Dynamics 365 teams. There is a third answer.
Nobody wanted to run the migration on a Friday
Staging has ten thousand rows, production has forty million, and every migration is a guess. What Lakebase branching does to a team's calendar.
Freshness, volume and schema checks on Databricks
Every job green, nothing failing, and the numbers quietly wrong because a source stopped sending. Freshness, volume and schema checks on system tables.
Conflicting metric definitions in Databricks Genie
Genie fails when two teams mean different things by the same word. Genie Accuracy hardens the semantic layer. TechFabric Experiments keeps the suite running.
Four thousand objects and nobody would name the exclusions
Warehouse migrations slip because the exclusions were never written down. The Migration Readiness Sprint inventories the estate before anyone picks a date.
The cost dashboard said the workspace was fine
A Databricks cost dashboard shows spend. A two-week Health Check finds the spend that bought nothing, from all-purpose clusters to orphaned schedules.
Unity Catalog before the first pipeline
Lakehouse pilots skip grants and then cannot be handed to security. The Lakehouse Launchpad puts Unity Catalog first, with TechFabric Airlift underneath.
Nobody could tell whether the agent was any good
A team shipped an agent in nine days and argued for two months about whether its answers were right. Write down what a good answer is first.
Databricks Repos for QA test automation workflows
Moving QA off ad-hoc notebooks and onto version-controlled test automation with Databricks Repos. Repo structure, modular tests, and how a team works in one.
Using Application Insights as a Sink for logging in ASP.NET Core
ASP.NET Core has very extensible logging interface. I provides an ILogger interface along with few default implementations that can be used to log data.
3 Ways To Run Automated Tests On Azure DevOps
Catch bugs before production by running tests automatically in Azure DevOps. Unit tests on CI, Selenium on release, Test Plans on demand, with the YAML.
Setting Up Production Ready Infrastructure For Microservices
Service Fabric is Microsoft's microservices orchestrator. Service discovery, fault tolerance and containers, set up for production rather than a demo.
Securing microservices on Azure Service Fabric with Traefik
Service Fabric's built-in reverse proxy exposes every microservice in the cluster or none. How Traefik lets you choose which services the world can reach.
Multi-tenant AI infrastructure in the cloud with Temporal
Some tenants want a cheap shared setup, others an isolated one. How we keep LLM costs, vendor swaps and per-tenant deployments manageable with Temporal.
Durable RAG with Temporal and Chainlit
An AI assistant that calls several tools per question can fail halfway and lose the answer. How we made a RAG chatbot durable with Temporal, with the code.
Dynamics 365 CRM: Transforming Your Sales Quote Process
The Dynamics 365 quote process state by state, with the product catalog it needs. Templates and PDFs, approvals in Power Automate, and quote to order.
A wide configuration for running Cypress in GitHub Actions
How to make one test suite check every release in each environment, browser and screen size your customers use, from GitHub Actions, without rewrites.
Hiring Databricks specialists: 12 expert insights
Twelve things to get right when hiring Databricks specialists in-house. Assess the team, set the budget, decide who manages them and screen for fit.
How Temporal Has Transformed My Development Approach
Retries, idempotency and state across services used to take more time than the business logic. A senior developer on what changed after adopting Temporal.
Temporal: Airlines Booking Demo Application
Weighing Temporal for workflows that must survive failures in production? What our engineers learned modelling ticket purchases, returns and delayed flights.
The effects of Cypress component testing on your React app
Cypress component testing catches a broken React form in a test that runs in about a second, long before QA. A working setup on Vite, and when to use it.