Step failed with "output truncated"

Symptom: a gateway step ends failed with a message like

Step 'generate-rows': output truncated — the model stopped at its output-token
limit (stop_reason=max_tokens) before finishing the JSON. Raise the agent's
max_tokens or ask for less output. Last payload: {"confidence": 0.9, "rows": [...

Cause: the agent’s reply is a JSON object, and the model hit its max_tokens ceiling partway through writing it. The cut-off body can’t be parsed, so the step fails. Long generations (many rows, long documents) are the usual trigger.

What you’ll see

• The error names the output-token limit instead of a generic “no payload contained valid JSON”. Gateways older than the version that added stopReason can’t tell the two apart and still report the generic parse error.
• The step is not retried, even with a retry: block. The same request truncates the same way, so retrying only pays for it again.
• The step row keeps the agent trace, token counts and cost of the failed call, so you can read what the model produced and see what the attempt cost. The same holds for any failed step that reached the model, not only truncations.

Fix

1. Raise max_tokens on the agent (agent.yaml) — for Anthropic models values above 21,333 are fine, as the Gateway streams the response.
2. Or ask for less per call: split the work across a fan-out so each branch produces a smaller piece.
3. A long generation must still finish inside the Gateway’s limits.request_timeout_seconds (default 180s).

Cost of failed attempts

Failed attempts count toward budget.max_usd and budget.max_tokens. Each retry attempt that reached the model is priced against its own model and summed onto the one step row (cost_source_detail lists attempt_1, attempt_2, …). An attempt that never reached the model — a timeout or connection error — has no tokens to count. If any attempt’s model has no price, the step’s cost is shown as unknown rather than a partial sum.