LLM Harbor uses a CLI-based configuration system with profiles for switching between different service setups. Configuration is managed through harbor config commands and saved profiles.
# List all configuration options
harbor config list
# Get a specific config value
harbor config get webui.host.port
# Set a configuration value
harbor config set webui.host.port 8080
# Set custom port for WebUI
harbor config set webui.host.port 8080
# Set custom host for WebUI
harbor config set webui.host.name 0.0.0.0
# Configure model for llama.cpp backend
harbor llamacpp model https://huggingface.co/user/repo/model.gguf
# Set llama.cpp arguments (e.g., GPU layers)
harbor llamacpp args -ngl 32
# Set model for vLLM backend
harbor vllm model google/gemma-2-2b-it
# Set model for TGI backend
harbor tgi model google/gemma-2-2b-it
Profiles allow you to save and switch between different service configurations:
# Save current configuration as a profile
harbor profile save llama4
# List available profiles
harbor config list
# Switch to a saved profile
harbor profile use default
# Import profile from URL
harbor profile use https://example.com/path/to/harbor-profile.env
Harbor provides unified model management across different backends:
# Ollama models
harbor ollama pull llama3.2
harbor ollama list
# HuggingFace models (shared cache)
harbor hf find gguf gemma-2
harbor hf download google/gemma-2-2b-it
harbor hf scan-cache
# Direct model setting for backends
harbor llamacpp model https://huggingface.co/user/repo/model.gguf
harbor vllm model google/gemma-2-2b-it
# Change WebUI port
harbor config set webui.host.port 8080
# Access WebUI URL
harbor url webui
# Configure llama.cpp with GPU layers
harbor llamacpp args -ngl 32
# Set custom model path
harbor llamacpp model /path/to/model.gguf
Export your setup to a standalone Docker Compose file:
# Export current setup to docker-compose file
harbor eject searxng llamacpp > docker-compose.harbor.yml
# Export specific services
harbor eject ollama webui > docker-compose.yml
Back up the following for disaster recovery:
harbor profile save <name>~/.cache/huggingface)~/.ollama/modelsharbor ejectRecovery steps:
harbor profile use <path-to-profile.env>harbor upharbor doctor to check system compatibilityAny questions?
Feel free to contact us. Find all contact information on our contact page.