The moment a team uses more than one LLM provider, the problems stop being about models and start being about plumbing: three SDKs, three billing dashboards, API keys pasted into too many places, and no answer to "what did we spend on inference last week?". LiteLLM is the standard self-hosted fix - a proxy that puts one OpenAI-compatible endpoint in front of every provider (Anthropic, OpenAI, Gemini, Bedrock, Mistral, your own vLLM), with virtual keys, per-key budgets, spend tracking, fallbacks, and retries.
This guide deploys it with a real database and the two secrets people get wrong. Disclosure, as usual: Miget is our platform and the deployment uses our Compose Stack mechanism - the base compose file is portable and runs the same on your laptop.
The architecture
| Service | Role | Public |
|---|---|---|
litellm | proxy + admin UI at /ui, port 5000 | yes (every request is key-authenticated) |
db | Postgres - models, virtual keys, spend logs, encrypted provider credentials | no |
Two design points worth understanding before you deploy:
- No config file.
STORE_MODEL_IN_DBkeeps models and provider credentials (encrypted) in Postgres, so you manage everything in the admin UI without redeploys. - The container is stateless. All state lives in the database - the proxy can be rebuilt or scaled at will, and backing up the deployment means backing up Postgres.
Deploy it in three steps
The template is in the deployable.sh stacks catalogue.
1. Create a Compose Stack in app.miget.com pointing at https://github.com/deployable-sh/stacks, path litellm. The managed Postgres is provisioned automatically and DATABASE_URL is auto-wired.
2. Set two secrets - and understand the second one:
# admin/master key - must start with sk-
LITELLM_MASTER_KEY=sk-$(openssl rand -hex 24)
# encrypts provider API keys stored in Postgres
LITELLM_SALT_KEY=$(openssl rand -hex 32)
The salt key is this stack's equivalent of n8n's encryption key: set it before adding your first model, and never change it - it cannot be rotated once provider credentials are stored, and losing it orphans them.
3. Deploy, open /ui, log in with the master key. Add your providers and models in the UI, then mint virtual keys - scoped keys per app or per team, each with its own budget and rate limits. Your apps inside the same project call http://litellm:5000/v1; external clients use the public TLS domain. Every request is authenticated, which is what makes the public exposure safe.
Local first? git clone https://github.com/deployable-sh/stacks && cd stacks/litellm && cp .env.example .env && docker compose up -d.
Virtual keys are the whole point
The proxy would be worth running just for the single endpoint, but virtual keys are what change how a team operates:
- One key per app or team member, not one shared provider key in five codebases
- Budgets: cap a key at $50/month and the proxy enforces it - no surprise invoice from an agent loop gone wild
- Spend tracking per key: "what did the staging environment spend on Claude this week" becomes a UI query
- Fallbacks and retries: route
gpt-xfailures to an equivalent model automatically
Rotating a compromised key means revoking one virtual key in the UI - your real provider keys never left the encrypted database.
Pair it with Open WebUI
The natural front-end for this gateway is self-hosted Open WebUI: the team gets a ChatGPT-style workspace, LiteLLM enforces budgets underneath, and every provider is switchable per conversation. The wiring is two variables on the Open WebUI side (OPENAI_API_BASE_URL=http://litellm:5000/v1 plus a virtual key), and both stacks run side by side on the same platform.
What it actually costs
The template sizes the proxy at 2 GiB RAM plus a 1 GiB managed Postgres - it fits Miget's $25/month plan (4 GiB RAM, 2 vCPU) with room to spare, and the database adds $0 because managed Postgres is included in the plan. Running the full self-hosted AI stack - LiteLLM plus Open WebUI - fits an 8 GiB plan at $49/month, still with no per-seat or per-request fees on top of your raw provider token costs.
Production notes
- The salt key is forever. Generate it, store it with your secrets, never touch it again. This is the single most common way people brick a LiteLLM deployment.
- Pin the image. The template tracks
litellm-database:main-stable; for production, pin a specific release and upgrade deliberately. - Back up Postgres, not the container. Keys, budgets, spend history, and encrypted credentials all live in the database. For critical deployments, consider PostgreSQL HA.
- Give every consumer a virtual key - including internal apps. The master key is for the admin UI, not for embedding in services.
- Watch the spend dashboards weekly. Budgets fail loudly, but the spend-per-key view is where you notice the quiet 3x growth before it matters.
Frequently Asked Questions
What does LiteLLM actually do?
It translates the OpenAI API format to every major provider's API, behind one endpoint. Your apps speak one protocol; the proxy handles provider differences, auth, budgets, spend logging, retries, and fallbacks.
Is the LiteLLM proxy free to self-host?
Yes - the proxy is open source. Your costs are infrastructure ($25/month on Miget for this setup) plus whatever you spend on provider tokens. LiteLLM also sells an enterprise tier with extra features, which this template does not require.
Can I connect Open WebUI, Cursor, or other OpenAI-compatible tools?
Yes - anything that accepts a custom OpenAI base URL works. Point the tool at your LiteLLM domain with a virtual key. That includes Open WebUI, IDE assistants with OpenAI-compatible settings, and your own applications using OpenAI SDKs.
How do I set spending limits per team?
Mint a virtual key per team in the admin UI and set a budget on it (monthly cap, rate limits, allowed models). The proxy rejects requests once the budget is exhausted - the cap is enforced at the gateway, not on the honor system.
How do I upgrade LiteLLM safely?
Pin a release tag, back up Postgres, bump the tag, redeploy. The container is stateless so rollback is just reverting the tag - state compatibility lives in the database migrations, which is why the backup comes first.
What to read next
- How to Self-Host Open WebUI - the team chat workspace that pairs with this gateway
- How to Self-Host n8n on Miget - same production treatment for workflow automation
- Free PostgreSQL Hosting with Public Access - the managed Postgres behind this stack