LocalAI exposes an OpenAI-compatible API for local models. Most risk comes from broad API exposure, untrusted model artifacts, and over-privileged container/runtime settings.
¶ 1) Restrict API exposure and authentication
- Bind LocalAI to private interfaces unless public access is required (
ADDRESS=127.0.0.1:8080)
- Place API behind a reverse proxy (Nginx, Traefik, Caddy) with TLS and authentication
- Apply request throttling and body-size limits for generation endpoints
- Segment user-facing UI from inference backend where possible
- Use API keys for authentication:
API_KEY=your-secret-key
- Consider OAuth/OIDC integration via reverse proxy for user authentication
- Pull model files only from trusted registries or validated internal mirrors (Hugging Face, official sources)
- Verify checksums/signatures where available (SHA256 hashes from model authors)
- Keep model directories read-only for runtime accounts when possible
- Remove unused models to reduce attack surface and storage risk
- Scan model files with antivirus/malware tools before deployment
- Use model configuration files to pin specific model versions
¶ 3) Harden runtime and observability
- Run service as non-root user in container (
user: 1000:1000 in Docker Compose)
- Drop unnecessary container capabilities (
cap_drop: ALL)
- Restrict GPU device access to only required containers
- Set resource limits (CPU, memory) to prevent denial-of-service
- Log API access and abnormal prompt/usage patterns
- Patch LocalAI images and inference dependencies on a regular cadence
- Use read-only root filesystem where possible (
read_only: true)
- Enable Docker security options (
security_opt: no-new-privileges:true)
- Use internal Docker networks for backend communication
- Implement firewall rules to restrict access to trusted IPs
- Enable mutual TLS (mTLS) for service-to-service communication
- Rate limit API endpoints to prevent abuse
- Consider API gateway for additional security controls
- Never commit API keys or secrets to version control
- Use environment variables or secrets management (Vault, AWS Secrets Manager)
- Rotate API keys and credentials on a regular schedule
- Encrypt sensitive configuration at rest
services:
localai:
image: localai/localai:latest
user: "1000:1000"
read_only: true
security_opt:
- no-new-privileges:true
cap_drop:
- ALL
volumes:
- ./models:/models:ro
- ./config:/config:ro
- /tmp
deploy:
resources:
limits:
cpus: '4'
memory: 8G
- LocalAI installation docs: https://localai.io/installation/
- LocalAI source repository: https://github.com/mudler/LocalAI
- Docker security best practices: https://docs.docker.com/engine/security/
- CIS Docker Benchmark: https://www.cisecurity.org/benchmark/docker
Any questions?
Feel free to contact us. Find all contact information on our contact page.