LLM Harbor is a CLI tool and companion application that lets you spin up a complete local LLM stack with a single command. It orchestrates Docker Compose services including LLM backends, frontends, and supporting tools—all pre-wired to work together without manual configuration. Harbor manages 50+ services including Ollama, Open WebUI, vLLM, llama.cpp, SearXNG, Dify, and more. It is designed for local LLM experimentation and development, not production deployment.
- 🚀 One-command setup with
harbor up
- 🔄 Support for 15+ LLM backends (Ollama, vLLM, llama.cpp, TGI, SGLang)
- 🔌 MCP ecosystem integration (MetaMCP, MCPO)
- 🎨 Image generation with ComfyUI + Flux
- 🌐 Local Web RAG with SearXNG + Perplexica
- 📊 LLM workflow platforms (Dify, n8n, LangFlow, Flowise)
- 🎤 Voice chat via Speaches (STT/TTS)
- 📱 QR codes for mobile/LAN access
- 🌍 Built-in tunneling via Traefik
- ⚙️ Configuration profiles for different scenarios
- 📜 Local command history for reproducible setups
- 📦 Export to standalone Docker Compose with
harbor eject
- 🚀 Harbor Boost for LLM workflow scripting
- Local LLM development environment
- Quick experimentation with different model backends
- Testing LLM workflows and RAG pipelines
- Running self-hosted AI tools for privacy
- Prototyping before production deployment
- Python (CLI tool)
- Docker, Docker Compose
- Bash
- Active development (v0.x)
- Current version: 0.3.34 (January 2026)
- Open-source and self-hosted
- Discord community: https://discord.gg/8nDRphrhSF
¶ History and References
Any questions?
Feel free to contact us. Find all contact information on our contact page.