A shopper types part of a wine label and expects the bottle. A loan consultant types a last name and expects every refinance still open in the queue, including the stipulation that landed after lunch. Both answers used to come from a second copy of the catalog or the loan file. That copy takes a pipeline to maintain, and the answer is wrong for as long as the pipeline sits behind the product.
We built those copies at TechFabric, in Elasticsearch and in Azure Search. For a California wine retailer we indexed the catalog and turned on fuzzy match and semantic ranking, so part of a product name was enough to reach the bottle. For iLending and Auto Approve we indexed the refinance applications, pulling documents out of the CRM and the other systems a file is spread across. A consultant could search the whole queue from that index.
Databricks AI Search can point a Delta Sync index at a streaming table, a materialized view, or a managed Iceberg table on v3 or later, the same way it already points one at a Delta table (Building live intelligence). On the systems we already run, the first check is small. BZZR already keeps click analytics on Databricks through a live season, so the check there is one game. On a refinance queue the check is one application that is still open. The number to come back with is how long a new row takes to show up in search. That gap decides whether the copied index can be retired for that question.
What the copied index was good at
The wine catalog and the refinance queue had the same job for us. We took documents from the systems that already held them, pushed those documents into a search engine, and let fuzzy match and semantic ranking return a hit from part of a name. Part of the label was enough for the shopper. The consultant could search the queue from what they remembered of the borrower, and find the application without hunting for the right screen.
Consultants used that index to find an application in the queue. Shoppers used it to land on a bottle. Search knew the catalog and the queue as of the last load. A price that changed after the crawl, or a payoff that posted after the last extract, waited until the next one. The shopper saw yesterday's price. The consultant saw a payoff the lender had already replaced.
A game that is on right now needs the row that just landed.
Clicks on a game that is still on
We built BZZR as one platform for live games, creator video and podcasts, and we put its click analytics on Databricks so a club can be told who watched while the season is live. During a broadcast the question gets narrower than the season report. The club wants the audience for this game, counted while the game is still on.
Those clicks can append to a streaming table as they happen, and a Delta Sync index on that table is what the club report queries. The query the case study leaves open, this audience joined to what else they follow, is a materialized view over the same clicks. The view stays current as rows arrive, and the report reads an index on the view. The person asking during the broadcast is reading the table the clicks landed in.
Another club with the same broadcasts would start on one game, events in a streaming table and the audience question in a view. A lag between the click and the index shows up while that game is on, before a second club depends on the same report.
The refinance application still open
iLending's consultants work the Phoenix app, and Auto Approve's consultants work the Approve Engine. We gave both desks search across the applications in the queue. The published work on those platforms is turning a lead into a funded loan: assignment, rate sheets, stipulation documents, and the steps that used to live in the CRM, a lender site and a spreadsheet. The search index sat on documents we had already pulled out of those systems.
The file keeps moving after that load. A stipulation reply comes in, or a payoff figure is replaced. Those updates can append to a streaming table in the order they arrive. The application as it stands is a materialized view: which stipulations are still open, and what the payoff says now. The consultant searches the index on that view and sees the file they would see if they opened it, on the data that has landed today.
Assignment, funding and title stay on the workflow those platforms already run. The consultant who searches at 4pm sees the stipulation that arrived at 3pm.
When the table is already Iceberg
Some of this data already sits in an open table format, and a Delta copy that exists only so search can read it is the pipeline we would stop. A managed Iceberg table on v3 or later can be the Delta Sync source. The index points at that table. An external Iceberg table, or a managed table on an older Iceberg version, still needs a path into a source AI Search can index, and we would settle that from the catalog before anyone schedules a conversion.
One game, then one application
Pick the question someone asks while the data is changing, and point one index at the table that already holds the answer. On BZZR that question is who is on this broadcast, and the table is the clicks for that game. On a refinance queue the question is the application still open, and the table is the updates that landed today. Time the gap once, from the row landing to the same row coming back from search. If that gap is short enough for the person asking, retire the copied index for that question and leave every other table alone.
Frequently asked questions
What did the older search indexes contain?
Documents we extracted from the systems the product already used. The wine index held the catalog. The iLending and Auto Approve indexes held refinance applications drawn from the CRM and the systems around it. Fuzzy match and semantic ranking ran on that copy, in Elasticsearch or Azure Search. The product kept its own record, and search kept a second one, refreshed when the load ran.
How does a club count the audience while the game is on?
Clicks for that broadcast append to a streaming table, and the club report queries a Delta Sync index on that table. The audience joined to what else they follow is a materialized view, with its own index, maintained as the clicks arrive. The figure worth having, before a second club depends on it, is the delay between a click landing in the table and that click coming back from the index, measured on one game.
What can a loan consultant search while a file is still changing?
Updates to the application append to a streaming table as they arrive. A materialized view holds the file as it stands, including stipulations still open and the current payoff, and the consultant searches the index on that view. The result is a read of the file. Funding, title and the rest of the Approve Engine, and the steps iLending's consultants already take from a lead to a funded loan, stay on the workflow they already have.
Does an Iceberg table have to be copied into Delta first?
A managed Iceberg table on v3 or later can be the index source, so the conversion into Delta is the step that drops. An external Iceberg table, or a managed table on an older Iceberg version, still needs its own path into a supported source before an index can point at it.