From a six-line ticket to embedded analytics on Databricks

A Saturday lunchtime session at MOST-City, on beanbags. Andrii Taran told the story of how a six-line customer ticket turned into Databricks dashboards embedded in a production SaaS app, and was open about the parts that broke.

He gave the talk in Ukrainian. The slides below are his English version, so a few of them don't match what was on the wall.

Andrii started with the ticket. It was six lines long. The customer wanted dashboards inside their own portal that they could change without waiting for a release, with proper filters, good-looking charts and some AI, and without slowing the app down or showing numbers that disagreed with it. Then on a call they added a seventh thing: they wanted to see a row they'd saved a minute earlier.

Live data and no load on the app pull in opposite directions, and most of the hour was about where the team ended up drawing that line.

Starting with federation

The first version didn't copy anything. Lakehouse Federation pointed Databricks straight at the app's Azure SQL database, and the customer had a working demo within days.

It was slow, though, and every chart was a query against the production database. One view was pulling 6.46 million rows out of Azure SQL to show about 800. Andrii also warned people off timing things with a stopwatch around the API, which added one and a half to two seconds to every query in their tests. The real numbers are in query history.

Too many queries

One dashboard was firing 285 queries in the window they measured, because every chart had its own dataset and every filter change re-ran all of them. They moved to one shared dataset per dashboard, and turned five separate reports into pages of a single dashboard so switching between them stopped reloading everything.

The white screen

Embedded viewers don't have Databricks accounts, so Unity Catalog row filters can't keep one customer out of another's data. The tenant id travels in the embed token instead, and the dashboard SQL joins on it. If the id is missing, you see nothing.

That design is also how every dashboard went blank for every customer at once. The tenant key used a pipe as a separator, Databricks rejected the token, and the portal cut off the error so users just saw a white box. The team had checked how long the key could be and never which characters it could contain. It uses a tilde now, and each deploy opens all seven dashboards with a real token before anything goes out.

Moving it into Delta

The big win came from copying all of it into Delta, lookups included. With only the main table in Delta a typical query still took about two seconds, and with everything in Delta it took half a second. A full dashboard load went from around two minutes to 20 seconds.

Copies go stale, so the refresh schedule follows the customer's working day. It runs every 30 minutes on weekdays between 7am and 7pm Eastern, when nearly all their users are in, and less often at night and at weekends. It costs about $43 a month. Keeping a warehouse running all day would have been closer to $970.

The slides cover the rest, including the Refresh button they had to build themselves and an outage caused by nothing more than changing who owned a view. Andrii has also written the project up, starting with Embedded analytics on Databricks, part 1.