Handle refusals and cut-off answers

How to do it

Each provider reports why output ended. The Anthropic API sets stop_reason, with values such as end_turn, max_tokens, refusal and model_context_window_exceeded. The OpenAI Responses API sets status to incomplete with incomplete_details.reason, and puts a refusal in a content item of type refusal. The Gemini API sets finishReason. Map them in the gateway to a few values of your own, such as complete, cut off, refused and filtered. Tool calls and paused turns, such as tool_use and pause_turn, are not endings, so the tool loop handles them. Map any value you do not know to unknown, log it and treat it as not complete. For a cut-off answer, retry once with a higher limit or show it marked as incomplete. For a refusal, show a fixed message and a way to reach a person. Count each kind per feature.

Example

type Ending = 'complete' | 'cut_off' | 'refused' | 'filtered' | 'unknown';

const ENDINGS: Record<string, Ending> = {
  end_turn: 'complete', stop_sequence: 'complete', max_tokens: 'cut_off', model_context_window_exceeded: 'cut_off', refusal: 'refused',
  completed: 'complete', max_output_tokens: 'cut_off', content_filter: 'filtered',
  STOP: 'complete', MAX_TOKENS: 'cut_off', SAFETY: 'filtered', RECITATION: 'filtered', PROHIBITED_CONTENT: 'filtered', BLOCKLIST: 'filtered', SPII: 'filtered',
};

export function endingOf(providerReason: string): Ending {
  const ending = ENDINGS[providerReason];
  if (!ending) log.warn({ event: 'ai.unknown_ending', providerReason });
  return ending ?? 'unknown';
}

ENDINGS = {
    "end_turn": "complete", "stop_sequence": "complete", "max_tokens": "cut_off",
    "model_context_window_exceeded": "cut_off", "refusal": "refused",
    "completed": "complete", "max_output_tokens": "cut_off", "content_filter": "filtered",
    "STOP": "complete", "MAX_TOKENS": "cut_off", "SAFETY": "filtered", "RECITATION": "filtered",
    "PROHIBITED_CONTENT": "filtered", "BLOCKLIST": "filtered", "SPII": "filtered",
}

def ending_of(provider_reason: str) -> str:
    ending = ENDINGS.get(provider_reason)
    if ending is None:
        log.warning("ai.unknown_ending", provider_reason=provider_reason)
    return ending or "unknown"

How to check

A unit test with a fake provider returns each stop reason in turn, plus one invented value. The gateway maps each one to the right value and the invented one to unknown. The UI test shows no cut-off answer without its incomplete label.