Collect evidence automatically

How to do it

Run every control check on its frequency, in CI or a scheduled job. Save the raw API answer as well as pass or fail. Examples are branch protection settings, the two-factor status of users and bucket encryption settings. A compliance platform such as Vanta, Drata or Secureframe does this, and so does a set of small scripts. When a check fails, open a ticket for the control owner the same day.

Example

import { createHash } from 'node:crypto';
import { mkdirSync, writeFileSync } from 'node:fs';

const response = await fetch('https://api.github.com/repos/example-shop/web/branches/main/protection', {
  headers: { Authorization: `Bearer ${process.env.EVIDENCE_TOKEN}`, Accept: 'application/vnd.github+json' },
});
const body = await response.text();
const collectedAt = new Date().toISOString();
const evidence = { control: 'CTL-CHANGE-01', collectedAt, status: response.status, body, sha256: createHash('sha256').update(body).digest('hex') };
const folder = `evidence/${collectedAt.slice(0, 10)}`;
mkdirSync(folder, { recursive: true });
writeFileSync(`${folder}/CTL-CHANGE-01.json`, JSON.stringify(evidence));

import datetime, hashlib, json, os, pathlib, urllib.request

request = urllib.request.Request(
    "https://api.github.com/repos/example-shop/web/branches/main/protection",
    headers={"Authorization": f"Bearer {os.environ['EVIDENCE_TOKEN']}", "Accept": "application/vnd.github+json"},
)
with urllib.request.urlopen(request) as response:
    body = response.read().decode()
    status = response.status
collected_at = datetime.datetime.now(datetime.timezone.utc).isoformat()
folder = pathlib.Path("evidence", collected_at[:10])
folder.mkdir(parents=True, exist_ok=True)
evidence = {"control": "CTL-CHANGE-01", "collected_at": collected_at, "status": status,
            "body": body, "sha256": hashlib.sha256(body.encode()).hexdigest()}
(folder / "CTL-CHANGE-01.json").write_text(json.dumps(evidence))

How to check

For every automated control, the evidence store holds a result newer than its frequency, with a date and a hash that matches the saved body. Each failed result in the last quarter links to a ticket.