If your product calls Claude Opus 5.5, Sonnet 5.5 or Haiku 5.5 on Databricks, a request that specifies a response_format on the Chat Completions API returns HTTP 400. At TechFabric our engineers build Fabric, our AI work platform, where each organization picks its own model provider, and on October 1 each failed call of our feature that turns a ticket into a clean specification fell back to saving the ticket text exactly as typed. All 14 of its calls to Claude Sonnet 5.5 that day failed, 13 with a 400 saying structured output is not supported for this model and one on a workspace rate limit.
Databricks documents the proper fix, the Anthropic Messages API, but for us it meant a second client for one provider. So for now we kept the chat route, moved the schema into the prompt, and validate the reply on our side before anything downstream sees it.
A provider table now decides which routes get a strict schema at all, and a strict call that a provider rejects with a 400 or 422 is retried once without strict. If your teams call Claude through Databricks, the first thing to check is which of your features send a response_format on that route.
What Databricks does with a response_format request
Fabric talks to Databricks through the OpenAI-compatible chat completions route, the same way the Vercel AI SDK talks to any provider, and that route is where the failure lives.
Databricks documents structured outputs as a response_format carrying a json_schema, with the provider-specific translation handled on its side. For the Claude 5.5 models it draws a line. Opus 5.5, Sonnet 5.5 and Haiku 5.5 don't support response_format on the Chat Completions API, and a request that sends one returns HTTP 400. The documented alternative is the format field of output_config on the Anthropic Messages API, which is a different route with a different request body.
This is the full error our logs recorded for those 13 calls on October 1:
INVALID_PARAMETER_VALUE: Structured output is not supported for this model because its translation requires forced tool use, which newer Anthropic models reject.
The message gives the reason. On this route Databricks translates a response_format into forced tool use, and the newer Anthropic models reject forced tool use. Two more documented limits apply to Claude models. Only json_schema is accepted, never json_object, and a response_format for Claude can't be combined with tools or tool_choice.
Switching Fabric's Databricks route to the Messages API would have meant a second client for one provider, so we kept the chat route and moved the schema. If your stack can call the Messages API, that's the stronger fix, because output_config.format constrains the output to the schema, where our approach only asks for it.
Prompt-steered JSON for Databricks-served Claude
The path we built puts the schema in the system prompt, sends the request with no response_format, and lets the AI SDK validate the reply against the same schema when it comes back. The middleware is small.
const promptSteered = wrapLanguageModel({
model: databricks('databricks-claude-sonnet-5-5'),
middleware: {
transformParams: async ({ params }) => ({
...params,
responseFormat: undefined,
prompt: [
{ role: 'system', content: `Reply with one JSON object matching this schema and nothing else:\n${JSON.stringify(schema)}` },
...params.prompt,
],
}),
},
});
The first version of it had a quieter bug. On Sonnet 5.5 through Databricks the schema's field descriptions outranked the task prompt, so the drafted specifications followed the wording of the schema rather than the instructions for the feature. The fix was to keep the schema terse and put the task's rules back in the task prompt, and it reached production on October 7. The failures stopped.
Prompt-steered JSON is a weaker guarantee than a native schema, because the model can still answer with something the validator rejects. In practice the AI SDK's validation catches that and the call fails loudly with no half-shaped object behind it, which is the behavior we wanted from the 400 in the first place.
Request shaping for Claude 5.5 on the Databricks route
The 5.5 models also changed what a request may carry. On Databricks, Haiku 5.5 takes thinking of type adaptive or disabled, and Opus 5.5 takes adaptive only, rejecting both disabled thinking and a manual budget_tokens. The effort levels differ between the two as well. Our Databricks route already dropped temperature for every model and never sent a thinking budget, so the 5.5 constraints needed no change there, and the same constraints file that drives the direct Anthropic route now says adaptive thinking and no temperature for all three models.
One more thing we did on that route is worth copying. When we added Haiku 5.5 on October 8, the docs listed the model and not yet its endpoint name, so we mapped system.ai.claude-haiku-5-5 from the pattern of the Opus and Sonnet entries and said so in the pull request. The supported-models page now lists exactly that name, with databricks-claude-haiku-5-5 as the endpoint.
Strict JSON schema and the providers we trust with it
Databricks documents a strict flag too, and its structured-outputs page lists the subset of JSON Schema that Foundation Model APIs support: no pattern, anyOf, oneOf, allOf, prefixItems or $ref, and no type lists beyond a type and null. We wrote one middleware that converts our Zod schemas to the strict form, and 25 of our 26 real schemas convert. The one that doesn't has an unknown-typed field and falls back.
The question was who gets strict mode. A structured-output request enters the model factory and a provider table decides. Strict JSON schema goes to OpenAI direct, Azure AI Foundry and gateway openai models. Native structured output goes to Anthropic direct. Prompt-steered JSON, the schema in the system prompt, goes to Databricks-served Claude, and the AI SDK validates the reply. When a strict call is rejected with a 400 or a 422, the request retries once without strict, and that endpoint, model and schema stay non-strict for an hour.
The first draft of that change removed the request filters that switched strict off for Azure and Databricks, and review caught that Databricks would get strict mode with no evidence its endpoints accept it from our client. Those filters also protect code paths outside the model factory. So Databricks stays non-strict until someone checks it against a live endpoint.
The same fallback for oversized schemas on direct Anthropic
The prompt-steered path turned out to be reusable. On the direct Anthropic route, native structured output has limits on schema size, 24 optional parameters and 16 union-typed parameters in the count our code keeps, and a schema the code can't count at all. A request with no tools whose schema is over those limits now takes the Databricks path, schema in the system prompt and validation on the reply, while requests within the limits keep native structured output and requests that pass their own tools are unchanged.
One fallback covers two providers, and the table has one row per route, with no per-model rows to maintain.
Before you trust any provider with a strict schema, send it one from your own client and see what comes back. Until a route has passed that test, give it a single non-strict retry and remember the result for a short while. For Claude served by Databricks on the chat route, keep the schema in the prompt and validate the reply until you're ready to move that route to the Messages API.
Frequently asked questions
Why does Claude on Databricks reject response_format?
The Databricks docs say Opus 5.5, Sonnet 5.5 and Haiku 5.5 don't support response_format on the Chat Completions API and return a 400 for it. The error our calls got was INVALID_PARAMETER_VALUE: Structured output is not supported for this model because its translation requires forced tool use, which newer Anthropic models reject. The documented alternative is the format field of output_config on the Anthropic Messages API, which is a different route from the OpenAI-compatible one most SDKs use.
Is prompt-steered JSON as reliable as native structured output?
No. The model can still produce something the schema rejects, and the AI SDK's validation then fails the call rather than returning a half-shaped object. The fix for the schema's field descriptions outranking the task prompt was to keep the schema terse and put the task's rules back in the task prompt.
When do we send a strict JSON schema?
To OpenAI direct, Azure AI Foundry and gateway openai models. Databricks stays non-strict until someone checks it against a live endpoint, and a strict call that comes back 400 or 422 retries once without strict and is remembered for an hour.
Does the Databricks route accept a temperature for Claude 5.5?
Our route already dropped temperature for every model on Databricks and never sent a thinking budget, so the 5.5 constraints needed no change there. On Databricks, Haiku 5.5 accepts adaptive or disabled thinking and Opus 5.5 accepts adaptive only.