RADIANT_LLM

Prerequisites: API keys

You will typically need at least one LLM provider API key.

Key sources:

- OpenAI API key here, or Gemini API key (optional, including free access for gemini-2.5) here - LangChain (LangSmith) API key (optional, for tracing/logs) here - Google Custom Search API key here - Google Custom Search Engine ID here - Hugging Face API key (HF_API_KEY, required for document parsing in RAG) here ---

Prerequisites

• Docker Desktop (Windows/macOS) or Docker Engine (Linux)
• A .env file with at least one LLM provider key (OPENAI_API_KEY, GEMINI_API_KEY, etc.)
• (Optional, for GPU) NVIDIA GPU + recent drivers + NVIDIA Container Toolkit (docker run --gpus all)

Quick Start: Docker (prebuilt images)

Prebuilt images are published on Docker Hub: zev94/radiant-llm.

Offline install (legacy `.tar` releases)

If you have a release .tar instead of Hub access:

docker load -i .\radiant-llm_0.1.0.tar
docker images   # use the tag printed by Docker

Older images used port 8050, mount path /host_data, and log folder DecodedAI_logs. Current Hub images use 8060:8080, /host, and RADIANT_LLM_Logs to /radiant-llm/RADIANT_LLM_Logs.

---

5) Pull and run visual-parser (optional)

Build a multi-modal JSONL knowledge base before or alongside RADIANT-LLM QA. See visual-parser/README.md for CLI flags.

docker pull zev94/radiant-llm:visual-parser-latest

Windows PowerShell:

docker run --rm --env-file .env `
  -v "C:\path\to\pdfs:/data" `
  zev94/radiant-llm:visual-parser-latest `
  --input-dir /data --output-dir /data

WSL / Linux:

docker run --rm --env-file .env \
  -v "/path/to/pdfs:/data" \
  zev94/radiant-llm:visual-parser-latest \
  --input-dir /data --output-dir /data

Help:

docker run --rm zev94/radiant-llm:visual-parser-latest --help

---

Local Models: Grace HPRC vLLM (optional)

RADIANT-LLM supports self-hosted inference via vLLM on the Texas A&M University (TAMU) Grace High Performance Research Computing (HPRC) cluster (or any compatible Slurm + A100 cluster). When configured, Grace model variants appear alongside cloud models (GPT, Gemini) in the model selector — no cloud API key is needed for inference.

Available variants:

How it works: A Slurm job runs vLLM on a Grace GPU compute node. An SSH tunnel on your local machine (or Docker host) forwards requests from RADIANT-LLM to that node. No model weights or GPU compute leave the cluster.

Setup guide: developer_scripts/README.md — covers Grace account prerequisites, one-time model download, sbatch job templates, tunnel management scripts, and troubleshooting.

Docker users: The SSH tunnel binds to your host machine. RADIANT-LLM running in Docker reaches it automatically via host.docker.internal — the provided docker-compose.yml and .env in this repo already have this configured.

---

Troubleshooting

• pull access denied / repository does not exist
• Log in: docker login
• Use the full image name: zev94/radiant-llm:1.0 (not a local-only name unless you built or loaded it yourself).

• Invalid directory. Default temporary directory is being used
• The UI working-directory path must exist inside the container.
• Match the right-hand side of your volume mount (for example mount to /host, then use /host/... in the UI).

• Skills not loading
• Mount skills at /radiant-llm/radiant_llm_skills, or set Skills directory in Settings to your custom mount path.

• GPU not detected
• Verify GPU support with: docker run --gpus all nvidia/cuda:12.2.0-base-ubuntu22.04 nvidia-smi

---