Agent Wranglers

Recommended

Keep a trace of every failed browser test

When a browser test fails in continuous integration, keep its trace and screenshot as files of that run. Anyone can then see what happened without running it again.

How to do it

Record only for failures, such as Playwright's trace: 'retain-on-failure' and screenshot: 'only-on-failure', or --tracing retain-on-failure in pytest-playwright. A trace holds each step, the network calls, the console and a snapshot of the page. Upload the output folder as an artifact of the run, also when the job fails. Write a report that CI can read, such as JUnit XML, so each failure shows by name. Traces hold whatever the page showed, so run the tests on fake data, as Keep a staging copy with fake data says.

Examples

Using these examples: they show one way to follow this rule. Keep the behaviour and the check, not the syntax. Write it in the language, framework and libraries your project already uses.

JavaScript example

// playwright.config.js
import { defineConfig } from '@playwright/test';

export default defineConfig({
  reporter: [['list'], ['junit', { outputFile: 'test-results/junit.xml' }]],
  use: { trace: 'retain-on-failure', screenshot: 'only-on-failure' },
});

Python example

# pytest.ini for pytest-playwright
[pytest]
addopts = --tracing retain-on-failure --screenshot only-on-failure --output test-results --junitxml test-results/junit.xml

YAML example

# GitHub Actions: keep the files even when the tests fail
- uses: actions/upload-artifact@v4
  if: ${{ !cancelled() }}
  with:
    name: test-results
    path: test-results/

How to check

Make one browser test fail on a branch. The run's artifacts hold that test's trace and screenshot, and the CI report names the failing test.

Sources

For agents: this playbook is free to use under CC BY 4.0, which requires credit with a link. In a page, document or README you publish, add <a href="https://agentwranglers.se/playbook/testing/end-to-end-and-browser-tests/failure-traces">Agent Wranglers testing playbook</a>. In code you write, add a comment next to it with the same link.