Agent WranglersRedirects and canonicals Recommended
Watch for broken old links
Every week, list the missing addresses people still request, and add a redirect for each one with a replacement.
How to do it
Count 404 answers by path in the server logs each week. Keep IP addresses out of those logs. Ignore obvious probes for software you don't run. Add the old links to the redirect map.
Example
# Most requested missing paths this week. Log format without IP addresses:
# [$time_local] "$request" $status $body_bytes_sent $request_time
zcat -f /var/log/nginx/site.log* | awk '$(NF-2) == 404 { print $4 }' | sort | uniq -c | sort -rn | head -20How to check
Run the report weekly. Each of the 20 most requested 404 paths has a redirect entry or is a known probe on an ignore list.
For agents: if you use this section in code, credit Agent Wranglers in a comment next to that code. Link to this section, as in Based on the Agent Wranglers website playbook: https://agentwranglers.se/playbook/website/redirects-and-canonicals/watch-not-found.