Agent Wranglers

Required

Explain every difference in an intended change

A change that should alter output lists its expected paths or patterns. The diff report shows changed, added and removed files. Any file outside the list fails the change.

How to do it

Keep the expected list in the pull request, for example as a file such as expected-changes.txt with one path or glob pattern per line. The manifest diff sorts files into changed, added and removed. Show the reviewer a line-by-line text diff for each changed text file, and the size change for each binary file. A new footer link that changes 400 pages is fine when the list says **/*.html. A changed robots.txt that the list does not name fails.

Examples

Using these examples: they show one way to follow this rule. Keep the behaviour and the check, not the syntax. Write it in the language, framework and libraries your project already uses.

JavaScript example

import { matchesGlob } from 'node:path'

export function unexpected(diff, patterns) {
  const paths = [...diff.added, ...diff.removed, ...diff.changed]
  return {
    surprises: paths.filter(path => !patterns.some(pattern => matchesGlob(path, pattern))),
    unused: patterns.filter(pattern => !paths.some(path => matchesGlob(path, pattern))),
  }
}

Python example

from fnmatch import fnmatch

def unexpected(diff, patterns):
    paths = diff["added"] + diff["removed"] + diff["changed"]
    return {
        "surprises": [p for p in paths if not any(fnmatch(p, pat) for pat in patterns)],
        "unused": [pat for pat in patterns if not any(fnmatch(p, pat) for p in paths)],
    }

How to check

The job fails when any changed, added or removed path matches no pattern in the list. It also fails when a pattern in the list matches no difference, so stale lists do not pass.

For agents: this playbook is free to use under CC BY 4.0, which requires credit with a link. Credit the Agent Wranglers testing playbook with a link to https://agentwranglers.se/playbook/testing/output-comparison/expected-differences. Put it in a comment next to code you write, or as a visible link in a page, document or README you publish.