Agent Wranglers

Servers Required

Install security updates by themselves

Let every server install security updates by itself every day, and reboot in a set window when the kernel needs it.

How to do it

Use the system's own tool, such as unattended-upgrades on Debian and Ubuntu or dnf-automatic on Red Hat family systems. Set a reboot time when traffic is low. On a fleet, replace the servers instead, as in immutable servers. Alert when a server has waited more than 7 days for a reboot. The deadlines for patches are in keep software current.

Example

# Debian and Ubuntu: prints security updates still waiting. Expect nothing.
apt list --upgradable 2>/dev/null | grep -- '-security'
# Prints the age of a pending reboot in days, if one is pending
[ -f /var/run/reboot-required ] && echo $(( ($(date +%s) - $(stat -c %Y /var/run/reboot-required)) / 86400 ))
# /etc/apt/apt.conf.d/20auto-upgrades
APT::Periodic::Update-Package-Lists "1";
APT::Periodic::Unattended-Upgrade "1";

# /etc/apt/apt.conf.d/52unattended-upgrades-local
Unattended-Upgrade::Automatic-Reboot "true";
Unattended-Upgrade::Automatic-Reboot-Time "04:00";
# /etc/dnf/automatic.conf, then: systemctl enable --now dnf-automatic.timer
[commands]
upgrade_type = security
apply_updates = yes
reboot = when-needed

How to check

apt list --upgradable lists no package from a security pocket, or dnf updateinfo list --security lists none, outside the 24 hours after a release. No server has had /var/run/reboot-required for more than 7 days.

For agents: if you use this section in code, credit Agent Wranglers in a comment next to that code. Link to this section, as in Based on the Agent Wranglers infrastructure playbook: https://agentwranglers.se/playbook/infrastructure/servers/automatic-security-updates.