Author
Sushruth Aeluguri
Databricks Engineer at TechFabric
Builds on Databricks: Unity Catalog governance, Lakeflow pipelines and the lakehouse work underneath the applications.
- Databricks Certified Data Engineer Professional
9 posts
Faster Databricks pipelines by filtering before Spark joins
How filtering input DataFrames before Spark joins took an incremental Azure Databricks pipeline from about 60 minutes to 10–15 on half the nodes.
Sushruth AeluguriReadHow we built policy history that survives backdated endorsements
A backdated endorsement can erase what an insurer's data showed before it. We built a Databricks pipeline that keeps both answers for audits and claims.
Sushruth AeluguriReadHow we'd price Gemini 4 Argon per finished agent task
Gemini 4 Argon's price doubles on December 1. We price a model by what a finished agent task costs, and in our own replay the cheapest model finished none.
Sushruth AeluguriReadUnity Catalog for AI agents in practice: four FHIR scenarios
An AI agent can show records a person isn't cleared to see with every row filter working. Four FHIR scenarios on Databricks, with the performance tradeoffs.
Sushruth AeluguriReadFabric or Databricks is the wrong question on Azure
Microsoft and Databricks shipped the interop this year. For teams already on Azure the question is which platform holds the tables and which one reads them.
Sushruth AeluguriReadThe lock-in argument for leaving Snowflake is over
Snowflake and Databricks now read and write each other's Iceberg tables. If a vendor is still selling you a migration on lock-in, they are selling you 2023.
Sushruth AeluguriReadExport to Data Lake stops on 30 November
Export to Data Lake stops permanently on 30 November 2026. Microsoft names two replacements for Dynamics 365 teams. There is a third answer.
Sushruth AeluguriReadFreshness, volume and schema checks on Databricks
Every job green, nothing failing, and the numbers quietly wrong because a source stopped sending. Freshness, volume and schema checks on system tables.
Sushruth AeluguriReadUnity Catalog before the first pipeline
Lakehouse pilots skip grants and then cannot be handed to security. The Lakehouse Launchpad puts Unity Catalog first, with TechFabric Airlift underneath.
Sushruth AeluguriRead