# Structured output from Claude on Databricks Model Serving

> Claude Opus, Sonnet and Haiku 5.5 on Databricks return HTTP 400 for structured output via response_format. Why, and the prompt-steered JSON we use instead.

Published: 2026-10-23
Author: Raja Vemuri
Tags: Databricks, Applied AI
Canonical: https://www.techfabric.com/blog/structured-output-from-claude-on-databricks-model-serving

---

If your product calls Claude Opus 5.5, Sonnet 5.5 or Haiku 5.5 on Databricks, a request that specifies a `response_format` on the Chat Completions API returns HTTP 400. At TechFabric our engineers build Fabric, our AI work platform, where each organization picks its own model provider, and on October 1 each failed call of our feature that turns a ticket into a clean specification fell back to saving the ticket text exactly as typed. All 14 of its calls to Claude Sonnet 5.5 that day failed, 13 with a 400 saying structured output is not supported for this model and one on a workspace rate limit.

Databricks documents the proper fix, the Anthropic Messages API, but for us it meant a second client for one provider. So for now we kept the chat route, moved the schema into the prompt, and validate the reply on our side before anything downstream sees it.

A provider table now decides which routes get a strict schema at all, and a strict call that a provider rejects with a 400 or 422 is retried once without strict. If your teams call Claude through Databricks, the first thing to check is which of your features send a `response_format` on that route.

## What Databricks does with a response_format request

Fabric talks to Databricks through the OpenAI-compatible chat completions route, the same way the Vercel AI SDK talks to any provider, and that route is where the failure lives.

Databricks documents [structured outputs](https://learn.microsoft.com/en-us/azure/databricks/machine-learning/model-serving/structured-outputs) as a `response_format` carrying a `json_schema`, with the provider-specific translation handled on its side. For the Claude 5.5 models it draws a line. Opus 5.5, Sonnet 5.5 and Haiku 5.5 don't support `response_format` on the Chat Completions API, and a request that sends one returns HTTP 400. The documented alternative is the `format` field of `output_config` on the Anthropic Messages API, which is a different route with a different request body.

This is the full error our logs recorded for those 13 calls on October 1:

```text
INVALID_PARAMETER_VALUE: Structured output is not supported for this model because its translation requires forced tool use, which newer Anthropic models reject.
```

The message gives the reason. On this route Databricks translates a `response_format` into forced tool use, and the newer Anthropic models reject forced tool use. Two more documented limits apply to Claude models. Only `json_schema` is accepted, never `json_object`, and a `response_format` for Claude can't be combined with `tools` or `tool_choice`.

Switching Fabric's Databricks route to the [Messages API](https://learn.microsoft.com/en-us/azure/databricks/machine-learning/model-serving/query-anthropic-messages) would have meant a second client for one provider, so we kept the chat route and moved the schema. If your stack can call the Messages API, that's the stronger fix, because `output_config.format` constrains the output to the schema, where our approach only asks for it.

## Prompt-steered JSON for Databricks-served Claude

The path we built puts the schema in the system prompt, sends the request with no `response_format`, and lets the AI SDK validate the reply against the same schema when it comes back. The middleware is small.

```ts
const promptSteered = wrapLanguageModel({
  model: databricks('databricks-claude-sonnet-5-5'),
  middleware: {
    transformParams: async ({ params }) => ({
      ...params,
      responseFormat: undefined,
      prompt: [
        { role: 'system', content: `Reply with one JSON object matching this schema and nothing else:\n${JSON.stringify(schema)}` },
        ...params.prompt,
      ],
    }),
  },
});
```

The first version of it had a quieter bug. On Sonnet 5.5 through Databricks the schema's field descriptions outranked the task prompt, so the drafted specifications followed the wording of the schema rather than the instructions for the feature. The fix was to keep the schema terse and put the task's rules back in the task prompt, and it reached production on October 7. The failures stopped.

Prompt-steered JSON is a weaker guarantee than a native schema, because the model can still answer with something the validator rejects. In practice the AI SDK's validation catches that and the call fails loudly with no half-shaped object behind it, which is the behavior we wanted from the 400 in the first place.

## Request shaping for Claude 5.5 on the Databricks route

The 5.5 models also changed what a request may carry. On Databricks, [Haiku 5.5 takes `thinking` of type `adaptive` or `disabled`](https://learn.microsoft.com/en-us/azure/databricks/machine-learning/model-serving/query-reason-models), and Opus 5.5 takes `adaptive` only, rejecting both disabled thinking and a manual `budget_tokens`. The effort levels differ between the two as well. Our Databricks route already dropped `temperature` for every model and never sent a thinking budget, so the 5.5 constraints needed no change there, and the same constraints file that drives the direct Anthropic route now says adaptive thinking and no temperature for all three models.

One more thing we did on that route is worth copying. When we added Haiku 5.5 on October 8, the docs listed the model and not yet its endpoint name, so we mapped `system.ai.claude-haiku-5-5` from the pattern of the Opus and Sonnet entries and said so in the pull request. The [supported-models page](https://learn.microsoft.com/en-us/azure/databricks/machine-learning/foundation-model-apis/supported-models) now lists exactly that name, with `databricks-claude-haiku-5-5` as the endpoint.

## Strict JSON schema and the providers we trust with it

Databricks [documents a `strict` flag too](https://learn.microsoft.com/en-us/azure/databricks/machine-learning/foundation-model-apis/api-reference), and its structured-outputs page lists the subset of JSON Schema that Foundation Model APIs support: no `pattern`, `anyOf`, `oneOf`, `allOf`, `prefixItems` or `$ref`, and no type lists beyond a type and null. We wrote one middleware that converts our Zod schemas to the strict form, and 25 of our 26 real schemas convert. The one that doesn't has an unknown-typed field and falls back.

The question was who gets strict mode. A structured-output request enters the model factory and a provider table decides. Strict JSON schema goes to OpenAI direct, Azure AI Foundry and gateway openai models. Native structured output goes to Anthropic direct. Prompt-steered JSON, the schema in the system prompt, goes to Databricks-served Claude, and the AI SDK validates the reply. When a strict call is rejected with a 400 or a 422, the request retries once without strict, and that endpoint, model and schema stay non-strict for an hour.

```mermaid
%% caption: The route, and not the model, decides how the schema travels, and a rejected strict call degrades once rather than failing.
flowchart TD
  A["Structured-output request"] --> B["Provider table"]
  B --> C["Strict JSON schema, OpenAI direct, Azure AI Foundry, gateway openai models"]
  B --> D["Native structured output, Anthropic direct"]
  B --> E["Prompt-steered JSON, Databricks-served Claude, schema in the system prompt"]
  E --> F["AI SDK validates the reply"]
  C --> G["400 or 422 on a strict call, retry once non-strict, remembered for an hour"]
  class E accent
```

The first draft of that change removed the request filters that switched strict off for Azure and Databricks, and review caught that Databricks would get strict mode with no evidence its endpoints accept it from our client. Those filters also protect code paths outside the model factory. So Databricks stays non-strict until someone checks it against a live endpoint.

## The same fallback for oversized schemas on direct Anthropic

The prompt-steered path turned out to be reusable. On the direct Anthropic route, native structured output has limits on schema size, 24 optional parameters and 16 union-typed parameters in the count our code keeps, and a schema the code can't count at all. A request with no tools whose schema is over those limits now takes the Databricks path, schema in the system prompt and validation on the reply, while requests within the limits keep native structured output and requests that pass their own tools are unchanged.

One fallback covers two providers, and the table has one row per route, with no per-model rows to maintain.

Before you trust any provider with a strict schema, send it one from your own client and see what comes back. Until a route has passed that test, give it a single non-strict retry and remember the result for a short while. For Claude served by Databricks on the chat route, keep the schema in the prompt and validate the reply until you're ready to move that route to the Messages API.

## Frequently asked questions

### Why does Claude on Databricks reject response_format?

The Databricks docs say Opus 5.5, Sonnet 5.5 and Haiku 5.5 don't support `response_format` on the Chat Completions API and return a 400 for it. The error our calls got was `INVALID_PARAMETER_VALUE: Structured output is not supported for this model because its translation requires forced tool use, which newer Anthropic models reject.` The documented alternative is the `format` field of `output_config` on the Anthropic Messages API, which is a different route from the OpenAI-compatible one most SDKs use.

### Is prompt-steered JSON as reliable as native structured output?

No. The model can still produce something the schema rejects, and the AI SDK's validation then fails the call rather than returning a half-shaped object. The fix for the schema's field descriptions outranking the task prompt was to keep the schema terse and put the task's rules back in the task prompt.

### When do we send a strict JSON schema?

To OpenAI direct, Azure AI Foundry and gateway openai models. Databricks stays non-strict until someone checks it against a live endpoint, and a strict call that comes back 400 or 422 retries once without strict and is remembered for an hour.

### Does the Databricks route accept a temperature for Claude 5.5?

Our route already dropped `temperature` for every model on Databricks and never sent a thinking budget, so the 5.5 constraints needed no change there. On Databricks, Haiku 5.5 accepts adaptive or disabled thinking and Opus 5.5 accepts adaptive only.