This guide walks through a self-hosted installation of LocalAI.
For Docker installation, see Docker.
Start the LocalAI container and expose its API port. Mount a volume for persistent model storage.
docker run -p 8080:8080 -v /path/to/models:/models --name local-ai -ti localai/localai:latest
For GPU acceleration, add the appropriate runtime flags:
# NVIDIA GPU
docker run -p 8080:8080 -v /path/to/models:/models --gpus all --name local-ai -ti localai/localai:latest-gpu-nvidia-cuda-12
# AMD GPU
docker run -p 8080:8080 -v /path/to/models:/models --device /dev/kfd --device /dev/dri --name local-ai -ti localai/localai:latest-gpu-hipblas
Prefer automation? See LocalAI Ansible Setup for an example playbook.
Prefer containers? See LocalAI Docker Setup.
Any questions?
Feel free to contact us. Find all contact information on our contact page.