Upload straight to storage with a signed address

How to do it

The client asks your API for an upload. The API checks who is asking, picks the storage key itself and returns a signed address valid for 15 minutes at most. Sign a POST policy rather than a plain PUT, so storage itself refuses files over the size limit. Fix the content type in the policy too. Record the upload as pending in your database before you return the address. Your servers never carry the bytes, so a 2 GB video costs them nothing. Amazon S3 and Google Cloud Storage both sign POST policies. A plain PUT address can at most fix one exact size, by signing the Content-Length header where the store supports it. Azure Blob Storage SAS tokens cannot sign it at all. Where only PUT is offered, the server check must refuse files over the limit. Check each file once it lands, as Check every uploaded file says.

Examples

JavaScript example

import { S3Client } from '@aws-sdk/client-s3';
import { createPresignedPost } from '@aws-sdk/s3-presigned-post';
import { randomUUID } from 'node:crypto';

const MAX_PHOTO_BYTES = 10 * 1024 * 1024;
const UPLOAD_WINDOW_SECONDS = 600;

export async function startPhotoUpload(s3, shopId) {
  const key = `quarantine/shops/${shopId}/${randomUUID()}`;
  const { url, fields } = await createPresignedPost(s3, {
    Bucket: 'example-shop-uploads',
    Key: key,
    Conditions: [['content-length-range', 1, MAX_PHOTO_BYTES], ['eq', '$Content-Type', 'image/jpeg']],
    Fields: { 'Content-Type': 'image/jpeg' },
    Expires: UPLOAD_WINDOW_SECONDS,
  });
  return { key, url, fields };
}

Python example

import uuid
import boto3

MAX_PHOTO_BYTES = 10 * 1024 * 1024
UPLOAD_WINDOW_SECONDS = 600

def start_photo_upload(s3, shop_id: str) -> dict:
    key = f"quarantine/shops/{shop_id}/{uuid.uuid4()}"
    post = s3.generate_presigned_post(
        Bucket="example-shop-uploads",
        Key=key,
        Fields={"Content-Type": "image/jpeg"},
        Conditions=[["content-length-range", 1, MAX_PHOTO_BYTES], {"Content-Type": "image/jpeg"}],
        ExpiresIn=UPLOAD_WINDOW_SECONDS,
    )
    return {"key": key, **post}

How to check

Ask for an upload address for a 10 MB limit. Upload a file 1 byte over the limit and storage refuses it. Wait past the expiry and upload again, and storage answers 403. The key in the address never contains the client's file name.

Sources

• Amazon S3 uploading objects with presigned URLs

Related rules

• SCL-37 Keep no state inside the app process in Scaling
• ECM-22 Deliver downloads through short-lived links tied to the order in E-commerce and fulfilment
• MED-1 Define each kind of upload once in Files and media
• MED-43 Confirm every upload on the server in Files and media
• APS-43 Check every uploaded file in Application security
• MED-27 Count each owner's storage and enforce a quota in Files and media