⚠️ Important Note: Jan is primarily a desktop application and does not have an official Docker image. For Docker-based local LLM deployments, consider the alternatives below or use Jan’s API server from a desktop installation.
| Aspect | Status |
|---|---|
| Official Docker Image | ❌ Not available |
| Desktop Application | ✅ Windows, macOS, Linux |
| API Server | ✅ localhost:1337 (from desktop app) |
| Headless Mode | ⚠️ Limited support |
The recommended approach is to install Jan Desktop and use its built-in API server:
Install Jan Desktop (see Jan Setup)
Enable API Server:
Use API from Docker Containers:
services:
my-app:
image: my-app:latest
extra_hosts:
- "host.docker.internal:host-gateway"
environment:
- JAN_API_URL=http://host.docker.internal:1337
For Docker-based local LLM deployments, consider these alternatives:
version: '3.8'
services:
ollama:
image: ollama/ollama:latest
container_name: ollama
ports:
- "11434:11434"
volumes:
- ollama_data:/root/.ollama
deploy:
resources:
reservations:
devices:
- driver: nvidia
count: all
capabilities: [gpu]
volumes:
ollama_data:
Usage:
docker compose up -d
docker exec -it ollama ollama run llama3.2
API Endpoint: http://localhost:11434
version: '3.8'
services:
localai:
image: localai/localai:latest
container_name: localai
ports:
- "8080:8080"
volumes:
- localai_models:/build/models
environment:
- MODELS_PATH=/build/models
deploy:
resources:
reservations:
devices:
- driver: nvidia
count: all
capabilities: [gpu]
volumes:
localai_models:
Usage:
docker compose up -d
# Download models to localai_models volume
API Endpoint: http://localhost:8080 (OpenAI-compatible)
version: '3.8'
services:
vllm:
image: vllm/vllm-openai:latest
container_name: vllm
ports:
- "8000:8000"
volumes:
- vllm_models:/models
environment:
- HUGGING_FACE_HUB_TOKEN=${HF_TOKEN}
command: >
--model meta-llama/Llama-3.1-8B-Instruct
--host 0.0.0.0
--port 8000
deploy:
resources:
reservations:
devices:
- driver: nvidia
count: all
capabilities: [gpu]
volumes:
vllm_models:
API Endpoint: http://localhost:8000 (OpenAI-compatible)
⚠️ Warning: These are community-maintained and not officially supported by Jan HQ.
version: '3.8'
services:
open-webui:
image: ghcr.io/open-webui/open-webui:main
container_name: open-webui
ports:
- "3000:8080"
volumes:
- open-webui_data:/app/backend/data
environment:
- OLLAMA_BASE_URL=http://ollama:11434
depends_on:
- ollama
ollama:
image: ollama/ollama:latest
container_name: ollama
volumes:
- ollama_data:/root/.ollama
deploy:
resources:
reservations:
devices:
- driver: nvidia
count: all
capabilities: [gpu]
volumes:
open-webui_data:
ollama_data:
This provides a ChatGPT-like web interface similar to Jan’s desktop UI.
If you have Jan Desktop running on your host machine:
version: '3.8'
services:
my-llm-app:
image: my-llm-app:latest
extra_hosts:
- "host.docker.internal:host-gateway"
environment:
- OPENAI_API_BASE=http://host.docker.internal:1337/v1
- OPENAI_API_KEY=not-needed
# Your app connects to Jan API at localhost:1337
from openai import OpenAI
# Connect to Jan API
client = OpenAI(
base_url="http://host.docker.internal:1337/v1",
api_key="not-needed" # Jan doesn't require auth by default
)
response = client.chat.completions.create(
model="jan-v1",
messages=[
{"role": "user", "content": "Hello!"}
]
)
print(response.choices[0].message.content)
services:
llm-service:
image: ollama/ollama:latest
deploy:
resources:
reservations:
devices:
- driver: nvidia
count: all
capabilities: [gpu]
Prerequisites:
services:
llm-service:
image: ollama/ollama:latest
devices:
- /dev/kfd:/dev/kfd
- /dev/dri:/dev/dri
group_add:
- video
services:
llm-service:
image: ollama/ollama:latest
devices:
- /dev/dri:/dev/dri
group_add:
- video
| Feature | Jan Desktop | Ollama (Docker) | LocalAI (Docker) | vLLM (Docker) |
|---|---|---|---|---|
| Interface | Desktop GUI | CLI/API | API | API |
| Model Discovery | Built-in Hub | CLI commands | Manual | Manual |
| GPU Support | CUDA, ROCm, Metal, MLX | CUDA, ROCm | CUDA, ROCm | CUDA |
| API | OpenAI-compatible | OpenAI-compatible | OpenAI-compatible | OpenAI-compatible |
| Multi-User | Limited | Limited | Yes | Yes |
| License | Apache-2.0 | MIT | MIT | Apache 2.0 |
| Best For | Desktop users | Simple deployment | Full API clone | Production |
host.docker.internal# Verify Jan is running
jan --version
# Check API server status
curl http://localhost:1337/v1/models
# From Docker container
curl http://host.docker.internal:1337/v1/models
# Verify NVIDIA Container Toolkit
docker run --rm --gpus all nvidia/cuda:11.0-base nvidia-smi
# Check device visibility
ls -la /dev/nvidia*
# Check if port is in use
sudo lsof -i :1337
# Change Jan API port in settings
# Or change Docker port mapping
ports:
- "8080:11434" # Use different host port
Any questions?
Feel free to contact us. Find all contact information on our contact page.