Call models through one module

How to do it

Give the module one function per kind of call, such as generate and embed. Put the model name, timeouts, retries, logging, cost tracking and fallbacks inside it. Feature code passes a prompt ID, the variables and an output schema. Then a provider change, a new log field or a spending cap is one change in one place. A hosted gateway, such as LiteLLM or a cloud provider's AI gateway, can be the module, as long as all calls go through it.

Example

! grep -rnE "from ['\"](openai|@anthropic-ai/sdk|@google/genai)['\"]|^(from|import) (openai|anthropic|google\.genai)\b|^from google import genai" src/ --include=*.ts --include=*.py | grep -v "^src/ai/gateway"

// src/ai/gateway.ts is the only file that knows the provider.
export async function generate<T>(request: { promptId: string; vars: Record<string, string>; schema: ZodType<T>; userId: string }): Promise<T> {
  const prompt = await loadPrompt(request.promptId);
  return withLimits(request.userId, () => callProvider(prompt.render(request.vars), request.schema));
}

# src/ai/gateway.py is the only file that knows the provider.
def generate(prompt_id: str, variables: dict[str, str], schema: type[BaseModel], user_id: str) -> BaseModel:
    prompt = load_prompt(prompt_id)
    return with_limits(user_id, lambda: call_provider(prompt.render(variables), schema))

How to check

A search for provider SDK imports outside the gateway module finds nothing, including the from google import genai form. A lint rule or a test runs that search in CI.