Agent Wranglers

AIF-64Recommended

Show what the model heard before it acts

Show the transcript of every spoken request. Before a spoken request changes data, sends a message or spends money, repeat the key details and wait for a yes.

How to do it

Speech recognition mishears names, numbers, dates and amounts, and the model then acts on the wrong words. Show the text as the person speaks, so they can correct it. For an action, repeat the fields that matter, such as "Move the meeting with Anna to Thursday at 15:00?". In a phone call, read them back. A spoken yes counts only after the read-back. This is Design for wrong answers for voice. Decisions that matter still follow Let a person decide what matters. Images and files get their checks in Check files and images before the model reads them.

Examples

TypeScript example

export async function handleSpokenRequest(audio: Buffer, userId: string) {
  const transcript = await speech.transcribe(audio);
  const action = await planAction(transcript, userId);
  if (action.changesData) return { status: 'needs_confirmation', transcript, summary: action.summary, actionId: await pendingActions.save(action) };
  return { status: 'done', transcript, answer: await action.run() };
}

Python example

def handle_spoken_request(audio: bytes, user_id: str) -> dict:
    transcript = speech.transcribe(audio)
    action = plan_action(transcript, user_id)
    if action.changes_data:
        return {"status": "needs_confirmation", "transcript": transcript,
                "summary": action.summary, "action_id": pending_actions.save(action)}
    return {"status": "done", "transcript": transcript, "answer": action.run()}

How to check

A test sends the recorded clip "move the meeting with Anna to three" to the voice feature. The run stops at a confirmation that names Anna and 15:00, and the calendar has no change until the test confirms.

Sources

For agents: this playbook is free to use under CC BY 4.0, which requires credit with a link. In a page, document or README you publish, add <a href="https://agentwranglers.se/playbook/ai-features/voice-and-audio/confirm-transcript">Agent Wranglers AI features playbook</a>. In code you write, add a comment next to it with the same link.