Limit what the prediction endpoint gives away

How to do it

Many queries that return full scores let an attacker copy the model, or test whether a person was in the training data. Return only what the product needs, such as "approved" or a score rounded to two decimals. Set a quota per caller, and alert on a caller that sends many near-identical inputs, because that is how copying looks in the logs. Keep the full scores in the prediction log for your own monitoring. The privacy test for the model itself is Test that the model does not give away its training records.

Examples

Python example

SCORE_DECIMALS = 2

def public_prediction(probabilities: list[float], threshold: float) -> dict:
    score = round(probabilities[1], SCORE_DECIMALS)
    return {"decision": "approved" if score >= threshold else "declined", "score": score}

TypeScript example

const SCORE_DECIMALS = 2;

export function publicPrediction(probabilities: number[], threshold: number) {
  const score = Number(probabilities[1].toFixed(SCORE_DECIMALS));
  return { decision: score >= threshold ? 'approved' : 'declined', score };
}

How to check

A request to the public endpoint returns only the fields in its response schema, and no score has more than two decimals. A test sends one caller's quota plus one request, and the last one gets 429.

Sources

• OWASP Secure AI Model Ops Cheat Sheet

Related rules

• MLO-25 Log the features and prediction of every request in Machine learning
• MLO-41 Test that the model does not give away its training records in Machine learning