Cube AI

Prerequisites

• Docker and Docker Compose
• ~8 GB RAM free (Ollama model + Postgres instances)
• TEE hardware (AMD SEV-SNP / Intel TDX) is only needed for attestation in the prod stack — not for local development

Getting Started

Cube ships two self-contained Docker stacks:

The local stack is intentionally minimal: no attestation (TEE), no audit logs, no guardrails, no reverse proxy. Services are reachable directly on localhost ports.

edit docker/prod/.env first (CUBE_DOMAIN, secrets)

make up-prod # start make logs-prod # follow logs make down-prod # stop ```

Quick Start

1. Clone the repository

   git clone https://github.com/ultravioletrs/cube.git
   cd cube
   

1. Build the Cube images

The local stack runs the proxy, agent, and embedder images built from this repo, so build them first:

   make docker-proxy docker-agent docker-embedder
   

This tags ghcr.io/ultravioletrs/cube/{proxy,agent,embedder}:latest (and :<version>), exactly what the compose files reference. Use make dockers to build every service image (including guardrails, image-embedder, and ui).

1. Start the local stack

   make up        # start
   make logs      # follow logs
   make down      # stop
   make clean-volumes   # stop and wipe databases, models, uploads
   

The first start pulls Ollama and ATOM images and downloads the llama3.2:3b and nomic-embed-text models — give it a few minutes.

Local access (direct ports, plain HTTP):

| Service | URL | | ------------------------ | ----------------------------- | | Cube web UI | http://localhost:5173 | | ATOM identity UI | http://localhost:3005 | | Cube proxy (API gateway) | http://localhost:8900 | | Cube embedder (RAG) | http://localhost:8082 | | ATOM API (GraphQL) | http://localhost:8080/graphql |

1. Open the web UI and sign in

Open http://localhost:5173. The local ATOM has self-signup enabled, so create an account (email + password) from the login screen — email verification is skipped in local dev. After signing in, create a workspace (domain) and start chatting. Uploading documents in the UI exercises the RAG pipeline through the embedder.

For frontend hot-reload development, see the UI development guide.

1. (Optional) Use the API directly

Get a token from ATOM via GraphQL (use the account you created):

   curl -s http://localhost:8080/graphql \
     -H "Content-Type: application/json" \
     -d '{"query":"mutation($i:LoginInput!){login(input:$i){token entityId}}","variables":{"i":{"identity":"you@example.com","secret":"your-password","kind":"password"}}}'
   

The token is your bearer token. Send chat requests through the proxy, with your workspace (domain) ID in the path:

   curl -s http://localhost:8900/YOUR_DOMAIN_ID/v1/chat/completions \
     -H "Content-Type: application/json" \
     -H "Authorization: Bearer YOUR_TOKEN" \
     -d '{"model":"llama3.2:3b","messages":[{"role":"user","content":"Hello!"}]}'
   

Configuration

API Endpoints

Inference and RAG go through the cube-proxy gateway. Every protected request needs an Authorization: Bearer <token> header (token from ATOM — see step 5 of the Quick Start). The workspace (domain) ID goes in the path.

• Local base URL: http://localhost:8900
• Prod base URL: https://<your-domain> (Traefik, under /proxy)

Proxy Endpoints (OpenAI- and Ollama-compatible)

Replace {domainID} with your workspace ID.

Embedder Endpoints (RAG)

Document ingestion, sources, and retrieval are served by the embedder under /{domainID}/api/v1/... (records, sources, conversations, retrieve, chat, models). The web UI drives these; see the embedder runbook for the raw API.

vLLM Integration

Cube AI now supports vLLM, a high-throughput and memory-efficient inference engine for Large Language Models. vLLM provides:

• High Throughput: Optimized for serving multiple concurrent requests with continuous batching
• Memory Efficiency: Advanced memory management techniques for large models
• Fast Inference: Optimized CUDA kernels and efficient attention mechanisms
• Model Compatibility: Supports popular architectures including LLaMA, Mistral, Qwen, and more

Ollama Integration

Cube AI integrates with Ollama for local model deployment, providing:

• Model management and deployment
• Local inference
• Support for various open-source models

Local vs. prod

- Local (docker/local/.env, docker/local/config.json) — Ollama models, ports, and database credentials. No TEE, audit, or guardrails. - Prod (docker/prod/.env, docker/prod/config.json) — set CUBE_DOMAIN, replace every change-me-* secret, and toggle ATTESTED_TLS. Guardrails, the audit pipeline, and the image embedder are always on in prod.