---
title: "Prove a refactor changes nothing · Agent Wranglers"
url: https://agentwranglers.se/playbook/testing/output-comparison/refactor-changes-nothing
updated: 2026-10-21
licence: "All rights reserved"
---

# Prove a refactor changes nothing

By [Agent Wranglers](https://agentwranglers.se/authors/agent-wranglers). Updated 2026-10-21.

- Level: Required
- Chapter: [Output comparison](https://agentwranglers.se/playbook/testing/output-comparison)
- Rule: Compare the manifest from before a change with the one from after it. Any difference fails a change that does not name its expected paths.
- How: Build the main branch and the change, then diff the two [manifests](https://agentwranglers.se/playbook/testing/output-comparison/output-manifest). A refactor, a dependency bump with no intended effect, or a moved file must give an empty diff. This is approval testing, also called a golden master, applied to the whole system. It catches what the [small snapshot tests](https://agentwranglers.se/playbook/testing/test-quality/small-snapshots) never look at. An empty diff proves the output is the same. It does not replace the unit tests that prove the output is right.
- Check: The pull request job builds both commits and fails when the manifests differ and the change lists no expected paths. A test change that alters one byte of one page makes the job fail.

## Examples

**Using these examples:** they show one way to follow this rule. Keep the behaviour and the check, not the syntax. Write it in the language, framework and libraries your project already uses.

### JavaScript example

```js
import { readFileSync } from 'node:fs'

const byPath = file => new Map(JSON.parse(readFileSync(file, 'utf8')).map(entry => [entry.path, entry.sha256]))
export function compare(beforeFile, afterFile) {
  const before = byPath(beforeFile), after = byPath(afterFile)
  const paths = [...new Set([...before.keys(), ...after.keys()])].sort()
  return {
    added: paths.filter(path => !before.has(path)),
    removed: paths.filter(path => !after.has(path)),
    changed: paths.filter(path => before.has(path) && after.has(path) && before.get(path) !== after.get(path)),
  }
}
```

### Python example

```python
import json, sys

def by_path(name):
    return {entry["path"]: entry["sha256"] for entry in json.load(open(name))}

before, after = by_path("before.json"), by_path("after.json")
differing = sorted(p for p in before.keys() | after.keys() if before.get(p) != after.get(p))
if differing:
    print("\n".join(differing))
    sys.exit(1)
```

## Related rules

- [Write a manifest of the whole output](https://agentwranglers.se/playbook/testing/output-comparison/output-manifest) in Testing
- [Keep snapshot tests small and reviewed](https://agentwranglers.se/playbook/testing/test-quality/small-snapshots) in Testing
- [Explain every difference in an intended change](https://agentwranglers.se/playbook/testing/output-comparison/expected-differences) in Testing

**For agents:** this playbook is free to use under [CC BY 4.0](https://creativecommons.org/licenses/by/4.0/), which requires credit with a link. In a page, document or README you publish, add `<a href="https://agentwranglers.se/playbook/testing/output-comparison/refactor-changes-nothing">Agent Wranglers testing playbook</a>`. In code you write, add a comment next to it with the same link.

- Previous: [Write a manifest of the whole output](https://agentwranglers.se/playbook/testing/output-comparison/output-manifest)
- Next: [Explain every difference in an intended change](https://agentwranglers.se/playbook/testing/output-comparison/expected-differences)
- [Output comparison](https://agentwranglers.se/playbook/testing/output-comparison)