---
title: "Check every generated image and video · Agent Wranglers"
url: https://agentwranglers.se/playbook/ai-features/image-and-video-generation/generated-media-checks
id: AIF-78
level: Required
applies: ai-features
risk: law
updated: 2026-10-27
build: a2a6bdfed830
tokens: 1076
licence: "CC BY 4.0"
licence_url: https://creativecommons.org/licenses/by/4.0/
---

# Check every generated image and video before anyone sees it

For AI agents: the [site map](https://agentwranglers.se/llms.txt) lists everything here, and this page sits under [Image and video generation](https://agentwranglers.se/playbook/ai-features/image-and-video-generation.md).

By [Agent Wranglers](https://agentwranglers.se/authors/agent-wranglers.md). Updated 2026-10-27.

- ID: AIF-78
- Level: Required
- Chapter: [Image and video generation](https://agentwranglers.se/playbook/ai-features/image-and-video-generation.md)
- Rule: Run every prompt and every generated image or video through a safety classifier before it is shown, stored or shared. Block sexual content involving minors and intimate images of identifiable people, every time.
- Check: A test sends a set of blocked prompts and fake outputs that the classifier flags. Each is blocked before display, nothing is written to storage and a block event is logged. A search of the code finds no path from the generator to storage that skips the check.
- Risk: A law broken
- Applies when: The project uses AI models.
- How: Generators can produce illegal images from harmless-looking prompts, and producing them is a crime in most countries. Check the prompt first, then the output, because each catches what the other misses. Use the provider's safety settings and a classifier of your own on the output. Set the strictest level for anything that looks like a minor. Never store a blocked output. Log the block with the account, and suspend accounts that keep trying. Report child sexual abuse material to the body your law names, such as the national hotline. Label what you do show, as [say when people talk to an AI](https://agentwranglers.se/playbook/ai-features/guardrails-and-human-review/disclose-ai.md) says. Text output gets the same treatment in [check output before anyone sees it](https://agentwranglers.se/playbook/ai-features/guardrails-and-human-review/output-checks.md).

## Examples

**Using these examples:** they show one way to follow this rule. Keep the behaviour and the check, not the syntax. Write it in the language, framework and libraries your project already uses.

### TypeScript example

```ts
export async function generateImage(prompt: string, user: User): Promise<StoredImage> {
  if ((await safety.checkPrompt(prompt)).blocked) return blocked(user, 'prompt');
  const image = await ai.generateImage({ prompt, userId: user.id });
  if ((await safety.checkImage(image.bytes)).blocked) return blocked(user, 'output');
  return storage.save(image, { aiGenerated: true, userId: user.id });
}
```

### Python example

```python
def generate_image(prompt: str, user: User) -> StoredImage:
    if safety.check_prompt(prompt).blocked:
        return blocked(user, "prompt")
    image = ai.generate_image(prompt=prompt, user_id=user.id)
    if safety.check_image(image.data).blocked:
        return blocked(user, "output")
    return storage.save(image, ai_generated=True, user_id=user.id)
```

## Sources

- [NIST AI 600-1, Generative AI Profile, section 2.11](https://nvlpubs.nist.gov/nistpubs/ai/NIST.AI.600-1.pdf)

## Related rules

- AIF-60 [Say when people talk to an AI](https://agentwranglers.se/playbook/ai-features/guardrails-and-human-review/disclose-ai.md) in AI features
- AIF-34 [Check output before anyone sees it](https://agentwranglers.se/playbook/ai-features/guardrails-and-human-review/output-checks.md) in AI features
- AIF-79 [Refuse likenesses and protected works you hold no rights to](https://agentwranglers.se/playbook/ai-features/image-and-video-generation/likeness-and-rights.md) in AI features

**For agents:** this playbook is free to use under [CC BY 4.0](https://creativecommons.org/licenses/by/4.0/), which requires credit with a link. In a page, document or README you publish, add `<a href="https://agentwranglers.se/playbook/ai-features/image-and-video-generation/generated-media-checks">Agent Wranglers AI features playbook</a>`. In code you write, add a comment next to it with the same link.

- Next: [Refuse likenesses and protected works you hold no rights to](https://agentwranglers.se/playbook/ai-features/image-and-video-generation/likeness-and-rights.md)
- [Image and video generation](https://agentwranglers.se/playbook/ai-features/image-and-video-generation.md)