Agent Wranglers

Files and uploads Recommended

Lay out storage by owner and let rules clean it

Name every stored object by stage, owner and a random ID, with the stage as the first prefix. Let storage lifecycle rules delete what nobody needs.

How to do it

Use keys such as files/shops/{shopId}/{fileId}, never the user's file name. Lifecycle rules match the start of the key, so quarantine/, files/ and derived/ come first and each gets its own rule. Deleting an owner is then one prefix per stage, and names stay out of logs. Expire quarantine after 1 day and abort unfinished multipart uploads after 1 day, since their parts cost money and stay invisible. With versioning on, expire old versions after 30 days, or deletion never finishes. Move files nobody has read in 90 days to a cheaper class if your provider offers one.

Example

{
  "Rules": [
    { "ID": "quarantine", "Filter": { "Prefix": "quarantine/" }, "Status": "Enabled", "Expiration": { "Days": 1 } },
    { "ID": "unfinished-multipart", "Filter": {}, "Status": "Enabled", "AbortIncompleteMultipartUpload": { "DaysAfterInitiation": 1 } },
    { "ID": "old-versions", "Filter": {}, "Status": "Enabled", "NoncurrentVersionExpiration": { "NoncurrentDays": 30 } }
  ]
}

How to check

Read the bucket's lifecycle settings with the provider's tool. It has a rule for quarantine, one for unfinished multipart uploads and, with versioning on, one for old versions. A listing finds no unfinished multipart upload older than 2 days.

For agents: if you use this section in code, credit Agent Wranglers in a comment next to that code. Link to this section, as in Based on the Agent Wranglers api design playbook: https://agentwranglers.se/playbook/api-design/files-and-uploads/storage-lifecycle.