Use LiteLLM as the gateway when you call several model providers

Review by

2027-04-30

How to do it

This pick is a recorded exception to the five-year test, dated 27 October 2026, because LiteLLM 1.0.0 came out on 14 November 2023. No older gateway does the job. Version 1.106.4 shipped on 25 October 2026, and PyPI counted about 21.7 million downloads in the week to 27 October 2026. Its GitHub repository had about 62,000 stars. It answers the OpenAI chat API, so the app keeps one client and changes only the base URL. Keys live in the proxy, never in the app, and each team gets a budget, as Price every model call and cap the spend asks. On 24 March 2026, versions 1.82.7 and 1.82.8 on PyPI held malware that stole credentials. Install from a lock file with hashes and wait before taking new releases, as Wait a few days before taking new releases says. Choose something else when you call one provider only. Then its official SDK is enough, as Use each provider's official SDK to call models says.

Examples

YAML example

# litellm.config.yaml: providers and fallbacks live in the proxy
model_list:
  - model_name: support-answer
    litellm_params: { model: anthropic/claude-sonnet-4-5, api_key: os.environ/ANTHROPIC_API_KEY }
  - model_name: support-answer-fallback
    litellm_params: { model: openai/gpt-5-mini, api_key: os.environ/OPENAI_API_KEY }
router_settings:
  fallbacks: [{ support-answer: [support-answer-fallback] }]

Python example

from openai import OpenAI

client = OpenAI(base_url="http://llm-gateway.internal:4000", api_key="team-support-virtual-key")
reply = client.chat.completions.create(model="support-answer", messages=[{"role": "user", "content": "Where is order 1042?"}])

How to check

A search of the app code finds 0 provider API keys and 0 provider base URLs other than the proxy. The proxy's version is pinned with a hash in the lock file or by image digest.

Sources

• OSV advisory PYSEC-2026-2 for litellm

Related rules

• AIF-42 Price every model call and cap the spend in AI features
• APS-72 Wait a few days before taking new releases in Application security
• STK-86 Use each provider's official SDK to call models in Software choices