Store the same file once per owner

How to do it

Hash the bytes on the server after upload, never trusting a hash the client sends. Look up the hash within the same owner only. Matching across owners tells an attacker that someone else holds a given file. Point each file record at the shared object and count references. Delete the object only when the count reaches zero, inside the same transaction that removes the last record. Count the owner's references down as Delete files when their owner is deleted says.

Examples

JavaScript example

import { createHash } from 'node:crypto';
import { createReadStream } from 'node:fs';

export async function sha256OfFile(path) {
  const hash = createHash('sha256');
  for await (const chunk of createReadStream(path)) hash.update(chunk);
  return hash.digest('hex');
}

export const contentKey = (shopId, sha256) => `files/shops/${shopId}/blobs/${sha256}`;

Python example

import hashlib

def sha256_of_file(path: str) -> str:
    with open(path, "rb") as f:
        return hashlib.file_digest(f, "sha256").hexdigest()

def content_key(shop_id: str, sha256: str) -> str:
    return f"files/shops/{shop_id}/blobs/{sha256}"

How to check

One owner uploads the same file twice, and storage holds one object with a count of 2. Deleting one record keeps the object, and deleting the second removes it. A second owner uploading the same file gets their own object.

Sources

• FIPS 180-4 Secure Hash Standard

Related rules

• MED-49 Delete files when their owner is deleted in Files and media