Compare the live system before and after each deploy

How to do it

Build the address list from the sitemap and add the machine files. These include robots.txt, feeds, sitemaps, /.well-known/ files, API endpoints and known redirects. Send the same Accept, Accept-Encoding and User-Agent on every request, and never follow redirects. Record content-type, cache-control, location and the security headers. Apply the masks before hashing bodies. This runs next to the smoke test and the rollback.

Examples

Using these examples: they show one way to follow this rule. Keep the behaviour and the check, not the syntax. Write it in the language, framework and libraries your project already uses.

JavaScript example

const HEADERS = ['content-type', 'cache-control', 'location', 'content-security-policy', 'strict-transport-security']
const REQUEST = { redirect: 'manual', headers: { 'accept': '*/*', 'accept-encoding': 'identity', 'user-agent': 'output-compare/1' } }

async function snapshot(urls) {
  const entries = []
  for (const url of urls) {
    const response = await fetch(url, REQUEST)
    const body = new Uint8Array(await response.arrayBuffer())
    const digest = Buffer.from(await crypto.subtle.digest('SHA-256', body)).toString('hex')
    entries.push({ url, status: response.status, headers: Object.fromEntries(HEADERS.map(name => [name, response.headers.get(name)])), sha256: digest })
  }
  return entries
}

const urls = ['https://example.com/robots.txt', 'https://example.com/feed.xml', 'https://example.com/old-pricing']
console.log(JSON.stringify(await snapshot(urls), null, 2))

Python example

import hashlib, json, urllib.error, urllib.request

HEADERS = ["content-type", "cache-control", "location", "content-security-policy", "strict-transport-security"]
REQUEST_HEADERS = {"Accept": "*/*", "Accept-Encoding": "identity", "User-Agent": "output-compare/1"}

class NoRedirect(urllib.request.HTTPRedirectHandler):
    def redirect_request(self, *args):
        return None

opener = urllib.request.build_opener(NoRedirect)

def fetch(url):
    try:
        response = opener.open(urllib.request.Request(url, headers=REQUEST_HEADERS))
    except urllib.error.HTTPError as error:
        response = error
    body = response.read()
    return {"url": url, "status": response.status,
            "headers": {name: response.headers.get(name) for name in HEADERS},
            "sha256": hashlib.sha256(body).hexdigest()}

urls = ["https://example.com/robots.txt", "https://example.com/feed.xml", "https://example.com/old-pricing"]
print(json.dumps([fetch(u) for u in urls], indent=2))

How to check

After each deploy, a job compares the two snapshots against the expected paths. One address that changed and is not listed raises an alert or rolls the deploy back.

Related rules

• Write a manifest of the whole output in Testing
• Mask volatile parts narrowly in Testing
• Smoke test every deploy in Testing
• Deploy in one step and roll back by itself in Engineering
• Explain every difference in an intended change in Testing
• Compare environments with the same release in Testing