![Harbor splash](https://raw.githubusercontent.com/av/harbor/main/docs/harbor-2.png)

Firstly, welcome to Harbor! This guide will help you get started with Harbor CLI and manage the main services. It assumes that you already have the Harbor CLI installed and running. See the [Harbor CLI Installation](./1.0.-Installing-Harbor.md#harbor-cli) guide for more information.

## Getting Started

Start Harbor with default services:

```bash
harbor up
```

This will start [Ollama](./2.2.1-Backend&colon-Ollama.md) and [Open WebUI](./2.1.1-Frontend&colon-Open-WebUI.md) by default. These two are the pre-configured as the default [Frontend](./2.-Services.md#frontends) and [Backend](./2.-Services.md#backends) respectively. Being a "default" service means that they will be started automatically when you run `harbor up` (and many other commands) without any additional arguments.

Additionally, these two services are the ones that are tested the most and targeted to be integrated with the rest of the Harbor in the most seamless way.

> [!TIP]
> You can configure default services using [`harbor defaults`](./3.-Harbor-CLI-Reference.md#harbor-defaults)

Here are some sample logs from a successful start:
```bash
user@os:~$ ▼ h u
[+] Running 3/3
 ✔ Network harbor_harbor-network  Created          0.1s
 ✔ Container harbor.ollama        Healthy          0.7s
 ✔ Container harbor.webui         Healthy          5.7s
 ```

`h` and `u` are both aliases. First is for the CLI itself (see [`harbor link --short`](./3.-Harbor-CLI-Reference.md#harbor-link)), the second is for the `up` command. You can typically see available aliases in the CLI Help output as well as on the pages of CLI Reference.

When the `harbor` is running, you can access its default frontend service in the web browser:

```bash
harbor open
```

[`harbor open`](./3.-Harbor-CLI-Reference.md#harbor-open-service) uses [`harbor url`](./3.-Harbor-CLI-Reference.md#harbor-url-service) under the hood. You can also specify a service name to open a specific service:
```bash
# See the URL
harbor url <service handle>
harbor open <service handle>
```

When you are running `harbor up` / `harbor open` / `harbor url` commands without specifying a service handle, it will default to the service that is configured as the default Frontend. You can adjust that using [`harbor config`](./3.-Harbor-CLI-Reference.md#harbor-config) if needed.

Now, let's add some more services to your Harbor setup. To start a new service, that is not a default one, you can use the `harbor up` command, but also add the service handle as an argument:

```bash
harbor up searxng
```

> [!TIP]
> Every service in Harbor will have its own dedicated "handle" that can be used with most compatible commands. You'll find all the handles in the [Services](./2.-Services.md) section. For example, Open WebUI has the handle `webui`. You'll see the same handles used for configuration, logs, files and other service-specific operations.

## Default Services

Harbor allows you to specify the set of services that will be started by default. You can manage the default services using the `harbor defaults` command. It includes frontends, backends and satellite services that may be included in your Harbor setup.

```bash
# Show the list of default services
harbor defaults

# Add a new default service
harbor defaults add searxng

# Swap a default backend
harbor defaults rm ollama
harbor defaults add vllm
```

## Running LLMs

When running Harbor, most of the value comes from running everything at once with a few simple commands. This includes running LLMs, which are the main focus of Harbor. In order to run them, you'll want to run one of the supported [backend services](./2.-Services.md#backends).

### Pulling Models

The primary way to get models into Harbor is using [`harbor pull`](./3.-Harbor-CLI-Reference.md#harbor-pull-servicemodel). This unified command works with both Ollama and llama.cpp backends, automatically detecting the model source and using the appropriate method to download it.

**Ollama models:**
```bash
# Pull from Ollama's registry
harbor pull llama3.2
harbor pull gemma3:8b

# Pull from HuggingFace via Ollama
harbor pull hf.co/bartowski/gemma-2-2b-it-GGUF:Q4_K_M
```

**llama.cpp models from HuggingFace:**
```bash
# Pull a GGUF model repository (with optional tag/file specifier)
harbor pull microsoft/Phi-3.5-mini-instruct-gguf
harbor pull microsoft/Phi-3.5-mini-instruct-gguf:Q4_K_M
```

When you pull a model using `harbor pull`:
- **Ollama models** are pulled using the Ollama registry and stored in Ollama's cache
- **llama.cpp models** (format: `org/repo-name`) are downloaded from HuggingFace to llama.cpp's cache using an ephemeral server
- Models are automatically available to all compatible services once downloaded

> [!TIP]
> By default, `llamacpp` runs in router mode and automatically discovers all models in its cache. After pulling a model, start `llamacpp` and it will be available for use without additional configuration:
> ```bash
> harbor pull microsoft/Phi-3.5-mini-instruct-gguf
> harbor up llamacpp
> ```

> [!TIP]
> You can quickly search for models on HuggingFace:
> ```bash
> harbor hf find gemma gguf
> ```
> See [`harbor hf find`](./3.-Harbor-CLI-Reference.md#harbor-hf-find-query) for more details.

### Ollama

[`ollama`](./2.2.1-Backend&colon-Ollama.md) is the default LLM backend due to its convenience for the end-user. Its main benefit is that it determines system resources and features automatically.

Harbor gives you full access to Ollama's CLI via the dockerized [`harbor ollama`](./3.-Harbor-CLI-Reference.md#harbor-ollama-command) command:

```bash
# List pulled models
harbor ollama list

# Check running models
harbor ollama ps

# Pull a model using Ollama CLI directly
harbor ollama pull <model_name>

# Run any Ollama command
harbor ollama --help
```

### llama.cpp

[`llamacpp`](./2.2.2-Backend&colon-llama.cpp.md) provides direct access to llama.cpp inference without the Ollama layer. It runs in router mode by default, which means:
- All models in the llama.cpp cache are automatically discovered
- Models are loaded on-demand when requested via the API
- No need to manually configure which model to run

```bash
# Pull models (they'll be auto-discovered)
harbor pull microsoft/Phi-3.5-mini-instruct-gguf
harbor pull bartowski/gemma-2-2b-it-GGUF

# Start llamacpp
harbor up llamacpp

# Models are available via OpenAI-compatible API
# The service will load them automatically when requested
```

For advanced configuration options like setting a specific model, using custom model directories, or adjusting server parameters, see the [llama.cpp documentation](./2.2.2-Backend&colon-llama.cpp.md).

### vLLM

[vLLM](./2.2.3-Backend&colon-vLLM.md) ships with a default model configured in the default profile (`Qwen/Qwen3.5-4B`). You can start it as-is or override the model before starting:

```bash
# [Optional] Confirm or change the model to run
# See harbor vllm for more options
harbor vllm model Qwen/Qwen3.5-4B

# Start vLLM
harbor up vllm
```

> [!TIP]
> You can set your HuggingFace Hub token to access gated/private models:
> ```bash
> harbor hf token <your HF token>
> ```
> This token will be pre-configured for services that need it. See [`harbor hf token`](./3.-Harbor-CLI-Reference.md#harbor-hf-token) for more details.

### Other Backends

Please refer to the [Backends](2.-Services.md#backends) section in the Services documentation for even more backends and their configuration.

## Launching Host Tools

[`harbor launch`](./3.-Harbor-CLI-Reference.md#harbor-launch-launch-options---service-servicetool-args) runs installed coding and agent CLIs against Harbor backends from the directory where you invoke Harbor. Use it when you want local tools such as Codex, Claude Code, OpenCode, mi, Pi, or VS Code to use the same Harbor inference stack as your web frontends.

```bash
# Start or select a backend, then run the host tool in this workspace
harbor launch --backend ollama --model qwen3.5:4b codex

# Let Harbor pick a running backend and discovered model
harbor launch codex

# Route through Harbor Boost with web search and URL reading
harbor launch --web --backend ollama --model qwen3.5:4b codex

# Write adapter config without launching the tool
harbor launch --config opencode
```

`--backend` and `--model` are Harbor launch options, so put them before the tool name. Every argument after the tool name is passed to the tool unchanged:

```bash
harbor launch --model qwen3.5:4b codex --sandbox workspace-write
harbor launch --backend ollama --model qwen3.5:4b copilot -p "explain this repo"
```

Some launch targets share names with Harbor services. In those cases, the plain handle runs the installed host adapter, and `--service` forces the containerized service path:

```bash
harbor launch mi -p "say hello"
harbor launch --service opencode --help
```

Supported host tools are listed in the [`harbor launch` CLI reference](./3.-Harbor-CLI-Reference.md#harbor-launch-launch-options---service-servicetool-args). If no compatible backend is already running, Harbor starts `llamacpp` by default. If you pass an explicit stopped backend, Harbor starts it before launching the tool.

## Configuring Services

There are three layers of configuration in Harbor:

### Service CLIs

For the most common/frequent configs, Harbor provides CLI aliases, one such example is configuration of service models:
```bash
# Setting service model via CLI alias
harbor llamacpp model https://huggingface.co/lm-kit/gemma-2-2b-gguf/blob/main/gemma-2-2B-Q8_0.gguf
harbor tgi model google/gemma-2-2b-it
harbor vllm model google/gemma-2-2b-it
harbor aphrodite model google/gemma-2-2b-it
harbor tabbyapi model google/gemma-2-2b-it-exl2
harbor mistralrs model google/gemma-2-2b-it
harbor opint model google/gemma-2-2b-it
harbor cmdh model google/gemma-2-2b-it
harbor fabric model google/gemma-2-2b-it
harbor parler model parler-tts/parler-tts-large-v1
harbor airllm model meta-llama/Meta-Llama-3.1-8B-Instruct
harbor txtai rag model llama3.1:8b-instruct-q6_K
harbor aider model llama3.1:8b-instruct-q6_K
harbor chatui model llama3.1:8b-instruct-q6_K
harbor aichat model llama3.1:8b-instruct-q6_K
```

These aliases are all linked to the [`harbor config`](./3.-Harbor-CLI-Reference.md#harbor-config) under the hood. It means that you can use either the alias or the config option directly - it'll work the same way.

Aliases are typically setup for the "hot" configuration options, like setting the model, version, or other frequently used ones.

> [!NOTE]
> As you can see above, services accept different values for the model specifier, depending on their internal configuration. Refer to [Services](./2.-Services.md) docs for detailed instructions on how to configure each service.

Versions:
```bash
# Setting service version via CLI alias
harbor vllm version 0.5.3
harbor webui version main
harbor mistralrs version 0.3
harbor chatui version latest
harbor comfyui version latest-cuda
```

> [!TIP]
> Most Service CLIs will also have dedicated help entries showing all available options:
> ```bash
> harbor <service> --help
> ```

One example of the option that doesn't typically have an alias are the service ports on the host. But you can still adjust them, see the section below!

### Harbor Config

Service CLIs are aliases for the underlying Harbor Config. It can be accessed directly via [`harbor config`](./3.-Harbor-CLI-Reference.md#harbor-config) command:

```bash
# Usage help
harbor config --help
```

`harbor config` works with the `.env` file as a key/value store, allowing you to set and get configuration values for services.

```bash
# See all configuration options
harbor config ls
# WEBUI_NAME    Harbor
# VLLM_VERSION  v0.5.3
# ... Many more options

# Get a specific value, accepts various aliases
harbor config get VLLM_VERSION
harbor config get vllm_version
harbor config get vllm.version

# Set a value, accepts same aliases as above
harbor config set webui.name v0.5.3
```
See a more detailed overview in the [`harbor config` CLI reference](./3.-Harbor-CLI-Reference.md#harbor-config)

### Using configuration files

Most of the services can be configured via either `.env` or a specific `yaml`, `json`, `toml` configuration (refer to service docs for specifics). These files are all stored in the Harbor workspace on your machine and can be edited directly.

```bash
# Show the path to the Harbor workspace
harbor home

# Open workspace in the file manager
open $(harbor home)

# Shortcut for VS Code users
harbor vscode
```

### Environment Variables

> [!TIP]
> You can add arbitrary environment variables to the `.env` file in the workspace and they will be available to all services. Alternatively, you can use [`harbor env`](./3.-Harbor-CLI-Reference.md#harbor-env) for variables that'll only be visible to that service.

```bash
# Set a service-specific environment variable
harbor env ollama OLLAMA_DEBUG 1

# Show all environment variables for a service
harbor env ollama

# Get env var value
harbor env ollama OLLAMA_DEBUG

# Open the global .env file with default editor
open $(harbor home)/.env

# Open service-specific env overrides
open $(harbor home)/services/ollama/override.env
```

You can find all the external tracked configuration folders by listing related configurations:

```bash
harbor config ls | grep CONFIG_PATH
```

Certain services will also have a dedicated cache folder, which can be found in the same way:
```bash
harbor config ls | grep CACHE

# One-liner to open one of the cache folders
open $(eval echo "$(harbor config get hf.cache)")
```

### Custom Cache Locations

By default, Harbor stores service caches in standard locations (e.g., `~/.ollama`, `~/.cache/huggingface`). You can customize these paths, for example to keep everything relative to your Harbor installation:

```bash
harbor config set hf.cache $(harbor home)/__cache/hf
harbor config set llamacpp.cache $(harbor home)/__cache/llamacpp
harbor config set ollama.cache $(harbor home)/__cache/ollama
harbor config set vllm.cache $(harbor home)/__cache/vllm
harbor config set txtai.cache $(harbor home)/__cache/txtai
harbor config set parllama.cache $(harbor home)/__cache/parllama
harbor config set lmeval.cache $(harbor home)/__cache/lmeval
```

This is useful when you want a self-contained Harbor installation or need to manage disk space on specific volumes.

> [!TIP]
> See all available cache options with `harbor config ls | grep CACHE`

For more details, see [`harbor home`](./3.-Harbor-CLI-Reference.md#harbor-home) and [`harbor config`](./3.-Harbor-CLI-Reference.md#harbor-config) in the CLI Reference.

## Harbor Profiles

When you have multiple configurations for different use-cases, you can save them as profiles for easy switching. Profiles include everything that can be configured via [`harbor config`](./3.-Harbor-CLI-Reference.md#harbor-config) (most of the settings configured via CLI) and are stored in the Harbor workspace. They are just a way to swap between `.env` files using command line.

```bash
# Use a profile
$ ▼ h profile use default
21:49:20 [INFO] Profile 'default' loaded.

# Use a profile from a URL
# 1. Can be a gist
$ ▼ h profile use https://gist.github.com/av/f325107c6f5a974a518f9e9135a76632
# 2. Can be a file in a repo
$ ▼ h profile use https://github.com/org/repo/blob/branch/path/to/file.env
# 3. Also supports: GitLab, Bitbucet, and Pastebin, or any direct file URLs

# Check settings specific to the vLLM service
$ ▼ h config ls | grep VLLM
VLLM_CACHE                     ~/.cache/vllm
VLLM_HOST_PORT                 33911
VLLM_VERSION                   v0.6.0
VLLM_MODEL                     Qwen/Qwen3.5-4B
VLLM_EXTRA_ARGS
VLLM_MODEL_SPECIFIER           --model Qwen/Qwen3.5-4B

# Switch to another profile
$ ▼ h profile use phimoe
21:49:42 [INFO] Profile 'phimoe' loaded.

# vLLM settings are different now
$ ▼ h config ls | grep VLLM
VLLM_CACHE                     ~/.cache/vllm
VLLM_HOST_PORT                 33911
VLLM_VERSION                   v0.6.1.post2
VLLM_MODEL                     microsoft/Phi-3.5-MoE-instruct
VLLM_EXTRA_ARGS                --max-model-len 1024 --trust-remote-code --cpu-offload-gb 56 --enforce-eager --gpu-memory-utilization 0 --device cpu
VLLM_MODEL_SPECIFIER           --model microsoft/Phi-3.5-MoE-instruct
```

There are a few considerations when using profiles:
- When the profile is loaded, modifications are not saved by default and will be lost when switching to another profile (or reloading the current one). Use `harbor profile save <name>` to persist the changes after making them
- Profiles are stored in the Harbor workspace and can be shared between different Harbor instances
- Profiles can be partial, meaning that you can only specify the options you want to change in a profile, without needing to include everything
- Profiles are not versioned and are not guaranteed to work between different Harbor versions
- You can also edit profiles as `.env` files in the workspace, it's not necessary to use the CLI

## Using Satellite Services

Harbor comes with a variety of satellite services intended to be used with LLMs. These services can be started, stopped and configured in the same way as the Frontends and Backends.

### SearXNG (Web Search)

SearXNG is a great example of a satellite service that is exceptionally useful for LLMs and on its own.

```bash
harbor up searxng
```

[Open WebUI](./2.1.1-Frontend&colon-Open-WebUI.md) (and many other built-in services) will automatically use it for Web RAG functionality.

### Tools

Tools are essential for expanding the capabilities of your local LLM setup. Harbor comes with a few services that help discover, setup and run such tools.

```bash
harbor up metamcp mcpo
```

[MetaMCP](./2.3.42-Satellite-metamcp.md) is a service that allows to discover and install MCP-based tools. You can also use [mcpo](./2.3.43-Satellite-mcpo.md) to make such tools compatible with Open WebUI and other OpenAPI-based services.

You'll find more information and examples in the [Tools guide](./1.2-Tools.md).

### Text-to-Speech

Start TTS service:
```bash
harbor up tts
```
Configure voices in ```services/tts/config/voice_to_speaker.yaml```

### Other Satellites

Please refer to the [Satellites](2.-Services.md#satellite-services) section in the Services documentation for even more additional services and their configuration in Harbor.

## Troubleshooting

Harbor combines many services and configurations together, all of which are in the constant motion, receiving updates and changes. This can sometimes lead to issues. Harbor comes with a set of tools to help you diagnose and fix these issues.

### Service State

When something unexpected happens, the very first thing to check is the state of the service.

```bash
# See currently running services
# [-a] - active services only
harbor ls -a

# See "docker ps" specifically for services managed by Harbor
harbor ps
```

See [`harbor ps`](./3.-Harbor-CLI-Reference.md#harbor-ps) for more information.

### Service Logs

If service is not running - it likely didn't start or crashed right after starting. You can check the logs to see what happened.

```bash
# Show logs for all running services
harbor logs

# Show logs for a specific service
harbor logs <service handle>

# Start tailing logs immediately
# after the service is started
harbor up <service handle> --tail
```

See [`harbor logs`](./3.-Harbor-CLI-Reference.md#harbor-logs) for more options.

### Commands in the service container

You can always inspect the service as it runs via Bash shell, Harbor has a few built-in helpers to simplify this process:

```bash
# Open a shell in the service container
harbor shell <service handle>

# Exec a command in a running service
harbor exec <service handle> <command>

# Run a command in a service container starting it if necessary
harbor run <service handle> <command>
```

See [`harbor shell`](./3.-Harbor-CLI-Reference.md#harbor-shell-service), [`harbor exec`](./3.-Harbor-CLI-Reference.md#harbor-exec-service-command), and [`harbor run`](./3.-Harbor-CLI-Reference.md#harbor-run-service-command) for more details.

### File System Permissions

Mounting volumes in Docker often comes with access permission issues, as the Docker mounts your local files into the container with same exact set of permissions as on your host machine, so if on your host, User with ID `1000` owns the files and the container happens to run with the User ID `100` - it won't be able to access the files. Vice versa, when the container's User writes something to a volume - your host User might not be able to access these files due to permissions being set to the container's User.

Harbor have an unsafe utility (Linux only, use at your own discretion) to fix such issues at locations managed by Harbor.

```bash
# Will walk through all folders managed by Harbor,
# including global service caches
# and will apply FS ACLs to match UID/GID of containers
# used by Harbor
harbor fixfs
```

## Stopping Services

```bash
# Stop all running services
harbor down

# 💡 TIP: many Harbor commands have aliases
harbor d

# Stop a specific service
harbor down <service handle>
harbor d <service handle>
```

See [`harbor down`](./3.-Harbor-CLI-Reference.md#harbor-down) for more information.

## Next steps

- [Services](./2.-Services.md) - Detailed documentation on services available in Harbor
- [Harbor CLI Reference](./3.-Harbor-CLI-Reference.md) - Detailed documentation for the Harbor CLI
