Security hardening guide for LiteLLM proxy server.
Set a strong master key for authentication:
export LITELLM_MASTER_KEY="sk-$(openssl rand -hex 32)"
In config:
general_settings:
master_key: os.environ/LITELLM_MASTER_KEY
Required for hashing sensitive data:
export LITELLM_SALT_KEY="sk-$(openssl rand -hex 32)"
⚠️ Important: Never change the salt key after setup - it will break existing hashed data.
Never hardcode API keys in config files:
# ✅ Correct
litellm_params:
api_key: os.environ/OPENAI_API_KEY
# ❌ Wrong
litellm_params:
api_key: "sk-1234567890" # Never do this!
Create virtual keys with specific permissions:
curl -X POST 'http://localhost:4000/key/generate' \
-H 'Authorization: Bearer sk-master-key' \
-H 'Content-Type: application/json' \
-d '{
"key_alias": "team-a-key",
"models": ["gpt-4o", "claude-sonnet"],
"max_budget": 100,
"budget_duration": "30d"
}'
Restrict key access to specific models:
# Via API
{
"models": ["gpt-4o"],
"max_budget": 50,
"tpm_limit": 10000,
"rpm_limit": 100
}
model_list:
- model_name: gpt-4o
litellm_params:
model: openai/gpt-4o
rpm: 100 # Requests per minute
tpm: 100000 # Tokens per minute
curl -X POST 'http://localhost:4000/key/generate' \
-H 'Authorization: Bearer sk-master' \
-d '{
"rpm_limit": 60,
"tpm_limit": 50000
}'
Generate certificates:
openssl req -x509 -newkey rsa:4096 \
-keyout keyfile.key \
-out certfile.crt \
-days 365 -nodes
Run with SSL:
litellm --config litellm_config.yaml \
--ssl_keyfile_path /path/to/keyfile.key \
--ssl_certfile_path /path/to/certfile.crt
Allow only necessary ports:
# Allow LiteLLM port
ufw allow 4000/tcp
# Allow from specific IPs
ufw allow from 192.168.1.0/24 to any port 4000
version: "3.9"
services:
litellm:
image: docker.litellm.ai/berriai/litellm:main-stable
networks:
- litellm-net
ports:
- "127.0.0.1:4000:4000" # Bind to localhost only
networks:
litellm-net:
driver: bridge
litellm_settings:
set_verbose: true
general_settings:
store_model_in_db: true
docker logs litellm-proxy
# Or with systemd
journalctl -u litellm -f
For production, use managed services:
export DATABASE_URL="postgresql://user:pass@host:5432/dbname?sslmode=require"
router_settings:
redis_host: your-redis-host
redis_password: os.environ/REDIS_PASSWORD
redis_port: 6379
# Bind Redis to localhost only
redis-server --bind 127.0.0.1