Agent Wranglers

TST-46Optional

Compare environments with the same release

The same release on staging and production gives the same bytes at every address. The only exceptions are a named list of environment values.

By Agent Wranglers. Updated .

How to do it

Take a live snapshot of both environments with the same address paths. Replace the host name and each listed environment value with a marker before hashing, for example the base address and the analytics key. A difference that remains means one environment runs other code, other settings or other data. This is the same test as a staging copy that matches production, measured in bytes.

Examples

JavaScript example

const ENVIRONMENT_VALUES = { 'https://staging.example.com': 'BASE_URL', 'https://example.com': 'BASE_URL', 'UA-STAGING': 'ANALYTICS_KEY', 'UA-LIVE': 'ANALYTICS_KEY' }

export const normalised = body => Object.entries(ENVIRONMENT_VALUES)
  .sort(([a], [b]) => b.length - a.length)
  .reduce((text, [value, marker]) => text.replaceAll(value, marker), body)

Python example

ENVIRONMENT_VALUES = {"https://staging.example.com": "BASE_URL", "https://example.com": "BASE_URL",
                      "UA-STAGING": "ANALYTICS_KEY", "UA-LIVE": "ANALYTICS_KEY"}

def normalised(body):
    for value in sorted(ENVIRONMENT_VALUES, key=len, reverse=True):
        body = body.replace(value, ENVIRONMENT_VALUES[value])
    return body

How to check

After each deploy to production, a job compares both snapshots. It fails on any difference that the list of environment values does not explain. The list holds at most 5 values.

For agents: this playbook is free to use under CC BY 4.0, which requires credit with a link. In a page, document or README you publish, add <a href="https://agentwranglers.se/playbook/testing/output-comparison/compare-environments">Agent Wranglers testing playbook</a>. In code you write, add a comment next to it with the same link.