开源AI工作流

Getting Started

• Installation & Quick Start
• Onboarding Wizard
• OpenCrabs Overview

Latency (criterion, release build)

cargo bench --bench memory, in-memory SQLite: FTS search 2.57 ms at 50 docs, vector search 1.02 ms, hybrid RRF fusion 3.49 ms, indexing 214 µs per file. Full tables in the Development section.

---

🎬 Demo

https://github.com/user-attachments/assets/7f45c5f8-acdf-48d5-b6a4-0e4811a9ee23

config.toml

[image.vision] enabled = true model = "gemini-3.1-flash-image-preview" provider = "openrouter" # Optional: force vision to use a specific provider (bypasses enabled gate)

keys.toml ← the Gemini key MUST live here, NOT in config.toml

[image] api_key = "YOUR_GEMINI_KEY"

> **Gotcha:** `[image.vision] api_key = "..."` in `config.toml` is silently ignored — the field carries `#[serde(skip)]` for security. Use `keys.toml` `[image]` section, or `[providers.image.gemini]` in config.toml + the key in keys.toml.

> **Pinning vision to a provider:** set `[providers.fallback] vision = ["name"]` to try that provider first for `analyze_image` and `analyze_video`, regardless of its `enabled` flag (vision needs only `vision_model` plus a key). Names follow the same rule as every other provider key: the bare section name, so `[providers.custom.myprovider]` is `"myprovider"`. An entry that does not resolve is skipped with a warning and resolution falls through to the normal provider scan. There is no `[image.vision] provider` key; that section configures the Gemini backend only.

**Diagnostic:** when vision is unavailable for any reason, `is_vision_available` logs the exact cause at INFO level in `~/.opencrabs/logs/opencrabs.YYYY-MM-DD` — search for `target=vision`.

#### Context window & auto-compaction (effectively unlimited memory)

OpenCrabs never makes you start a fresh session to "clear context." Instead it auto-compacts: as a session's history approaches the model's context window, it summarizes the older turns in place and keeps going. Two tiers:

- **65% — soft trigger:** spawns a background LLM compaction that summarizes history back down to ~65% of the budget. Non-blocking — the conversation keeps streaming.
- **90% — hard trigger:** synchronous compaction before the next request, so a single turn can never overflow the window.

The triggers are **percentages of the effective window**, so they scale to whatever you set: at the 200k default compaction kicks in around 130k; at a 1M window, around 650k.

It's **transparent** — most of the time you won't notice it happen at all. Occasionally you'll catch a brief inline notice while it summarizes (the agent tends to mention it dynamically, in its own voice), then the conversation carries on with the older turns condensed. You never have to start over or manually clear anything.

**The budget defaults to 200,000 tokens** — the battle-tested sweet spot: large enough for long sessions, small enough to keep each request fast and cheap. Override it **per provider** in `config.toml`:

[providers.anthropic] context_window = 1000000 # native providers too — Anthropic, Gemini, and the CLI providers

This override works for **every provider** — OpenAI-compatible (custom, xiaomi, qwen, openrouter, minimax, …) and the native Anthropic, Gemini, and CLI providers. A provider with no override inherits the 200k default.

> **Sizing guidance.** 200k is the battle-tested sweet spot for essentially **every cloud model**. Going bigger has two real downsides — **cost** and **context loss**:
>
> - **Cost** is the lesser one, and it's softened by caching: OpenCrabs uses prompt caching across every caching-capable provider (currently averaging ~87% efficiency), and a long context is mostly an unchanged prefix served from cache — so a bigger window costs far less than the raw token count suggests.
> - **Context loss** is the one to watch: most models degrade as the window fills — they lose track of the middle and recall less reliably. Only the latest SOTA models hold large context robustly: closed (Opus 4.7 / 4.8, Fable 5, GPT-5.5, Gemini 3.1, …) or open (Qwen 3.7, Kimi K2.7, MiMo V2.5, GLM 5.2, DeepSeek V4, and the newer releases that keep coming from these and other labs). So raise `context_window` mainly on those frontier models; on anything older or smaller, staying near 200k keeps answers sharper.
>
> **Local models want less:** 128k is a good sweet spot — go **lower** if your machine is tight on resources or you start noticing hallucinations/fabrications, and **higher** only if you have more than 32GB of RAM and have tested your model at a larger window. Not sure what fits your setup? Reach out to Adolfo for suggestions/support, or open a [GitHub discussion](https://github.com/adolfousier/opencrabs/discussions).
>
> **Leave auto-compaction on** — it's been battle-tested over months and needs no babysitting. Only run a *manual* compaction (the `/compact` command) if you have a specific, strong reason to summarize early; otherwise let it manage itself.

#### Prompt caching (every caching-capable provider)

A long context is mostly a **stable prefix** — system prompt, brain files, earlier turns rarely change between requests. OpenCrabs caches that prefix wherever the provider supports it, so you pay full price for it once and a fraction on every reuse. Across real usage it's currently averaging **~87% cache efficiency**, which you can watch live in the **Cache Efficiency** card of `/usage`. This is the main reason a larger `context_window` costs far less than its raw token count suggests.

How it turns on depends on the provider:

- **Anthropic** — native prompt caching, on by default: the `cache_control: ephemeral` markers and the caching beta header are added automatically to the system prompt and tools.
- **OpenAI / OpenAI-compatible** — OpenAI caches automatically server-side. **OpenRouter caches by default too** — OpenCrabs enables it automatically, so there's no flag to discover; set `cache_enabled = false` only if you specifically want to opt out.
- **Qwen / Alibaba** — auto-enabled, zero-config (detected by endpoint or a `qwen-` model name; unlocks Alibaba's explicit context cache, ~90% off on hits). See the Qwen note further below.
- **Xiaomi (MiMo)** — caches automatically server-side; nothing to configure.

Reference

• Architecture
• Brain Constitution
• Adding New Providers
• Plan JSON Specification
• Dynamic Workflows Guide — orchestrate agents with scripts: fan-out, pipelines, structured outputs, checkpointing

Messaging Integrations

File & Media Input Support

When users send files, images, or documents across any channel, the agent receives the content automatically — no manual forwarding needed. Example: a user uploads a dashboard screenshot to a Trello card with the comment "I'm seeing this error" — the agent fetches the attachment, passes it through the vision pipeline, and responds with full context.

Images are passed to the active model's vision pipeline if it supports multimodal input, or routed to the analyze_image tool (Google Gemini vision) otherwise. Text files (.txt, .md, .json, .csv, source code, etc.) are extracted as UTF-8 and included inline up to 8 000 characters — in the TUI simply paste or type the file path.

Videos uploaded on any channel (mp4, m4v, mov, webm, mkv, avi, 3gp, flv) auto-route to analyze_video when image.vision.enabled = true with a Gemini API key. The TUI also detects pasted video paths and labels them Video #N in the attachment indicator. Provider-side limits to keep in mind: Gemini's inline-bytes mode caps at ~20 MB (we use ≤18 MB), and the resumable Files API supports up to 2 GB / ~1 hour videos. Channel-side limits are tighter — Telegram's Bot API hard-caps getFile downloads at 20 MB even though chats accept larger uploads, so videos over that size will get a friendly "compress to under 20 MB and resend" reply. Slack file downloads use the bot token (files:read scope) and inherit the workspace's per-file upload cap. Frame-extraction fallback for non-Gemini providers is not yet wired — without a Gemini key, video uploads return an "unsupported" notice.

Telegram rich message formatting

When a Telegram reply carries structured Markdown (tables, headings, lists, - [ ] task lists, fenced code, or math), OpenCrabs can render it natively using Telegram's rich messages (Bot API 10.1) — real tables, real section headings, real checkboxes — instead of plain text or basic HTML.

This is on by default via channels.telegram.rich_messages. The one caveat: native rich messages are unreadable on Telegram Web and older clients — those show a "this message is not supported, update Telegram" placeholder, and the rich API has no text fallback. If your audience runs outdated clients, disable it in the onboarding dialog (the "Rich text experience" checkbox) or ask the agent: /onboard:channels telegram richtext off. With the flag off, the universal HTML rendering is used — tables come out as aligned monospace grids, task items as ☐/☑, with proper paragraph spacing, so structured replies look decent on every client.

When enabled, native rich applies to the agent's reply (sent as a fresh rich message so it renders cleanly) and to proactive telegram_send messages. Plain-prose replies are left untouched, so incidental characters like a stray * or # are never reinterpreted. If the rich send fails for any reason, OpenCrabs falls back silently to HTML, so a message is never dropped.

Flow logs (processing-log messages showing tool calls and intermediate text) also use the rich API when enabled, supporting 32K characters instead of HTML's 4K limit. Long tool chains fit in a single message without splitting. If the rich send fails, flow logs fall back to HTML rendering. The block auto-freezes at 30K characters to stay within limits.

/cowork — Telegram-only workspace creation

The /cowork command creates a team workspace directly from Telegram. It is Telegram-only because it relies on Telegram-specific primitives: group creation via ?startgroup deep links, invite links, QR codes from t.me URLs, and new_chat_members service messages for auto-registration. None of these exist in Discord, Slack, or WhatsApp.

Prerequisite: Telegram must be configured (bot token set via /onboard:channels telegram or manual config.toml setup).

Flow: 1. Owner sends /cowork in DM (owner-only command) → bot replies with an Add to Group inline button 2. Owner taps it → Telegram's native group picker opens. The deep link requests admin rights inline (?startgroup=cowork_<id>&admin=invite_users+delete_messages+pin_messages+manage_chat), so the bot is added already promoted to admin — no manual promotion step. Always keep the bot as admin: an admin bot reads every message regardless of privacy mode and can create invite links. 3. On joining via cowork, the bot sets that group's open = true (persisted) so every member is allowed — existing and new, no per-user step — and posts a short welcome. If it somehow landed without admin, the welcome also nudges the owner to promote it. 4. Members are auto-registered in an open group: joining members are added to the group's own allowlist ([channels.telegram.groups.<chat_id>].allowed_users) on join, and anyone who was already in the group before the bot can send /start to be tracked. This is group-scoped only — members can chat in that group (@mention the bot) but cannot DM it privately unless also on the global allowed_users or bot_owner. Already in the group and want to open it without re-adding the bot? Send /cowork inside the group (owner-only). 5. /start in a DM never auto-registers (DMs are invite-only): the bot just returns the sender's Telegram ID so they can share it with the owner to be added (or add it to config.toml when self-hosting). /start in a non-open group likewise returns the ID and points the user to ask the owner to run /cowork. The owner's own /start in a group is silent — they are already allowed everywhere.

Cross-channel behavior: /cowork works from any surface. In Telegram DMs, the native flow activates directly. From the TUI, Discord, Slack, or WhatsApp, the agent calls the cowork_connect tool which mints a session, registers it with the bot, and returns the t.me deep link plus a scannable QR code PNG. The TUI shows the clickable link; channels deliver the QR as a photo.

Telegram group security model

When allowed_users is configured, the bot enforces a strict allowlist on all incoming messages. The behavior differs between DMs and groups:

In DMs: - Non-allowed users always get a reply: "You are not authorized. Send /start to get your user ID." — so they know what to do.

In groups: - Non-allowed users get silently dropped (no reply, no processing) for normal messages. - If the user explicitly @mentions or replies to the bot, they get the "not authorized" reply — so they know they need to be added. - /start in an open group (open = true, see Per-group access control) registers the sender into that group's allowlist and confirms. In a non-open group it returns the sender's ID and tells them to ask the owner to run /cowork or add them — it never silently self-adds. The owner's /start (which Telegram auto-fires when the bot is added) is silent.

This prevents the bot from spamming "not authorized" in active groups where most members aren't on the allowlist. The bot only engages with non-allowed users when they explicitly reach out.

Config:

[channels.telegram]
allowed_users = ["123456789"]    # Only these users can interact
respond_to = "mention"           # Bot only responds to @mentions in groups
silence_group_start = true       # Silently ignore /start from non-allowed users in groups

Bot owner and owner-only commands

Every channel has a bot_owner field ([channels.telegram], [channels.discord], [channels.slack], [channels.whatsapp], [channels.trello]). It names the user ID(s) (phone for WhatsApp) treated as the bot owner. On first-run setup the owner is seeded automatically from the first entry in your allow list (allowed_users, or allowed_phones for WhatsApp), and existing configs are migrated on load. Set bot_owner explicitly to pin the owner instead of relying on list order.

The owner gets access that other allowlisted users do not. All channel commands except /new are owner-only: /compact, /doctor, /evolve, /help, /models, /rtk, /sessions, /stop, /usage, /profiles, /goal, /mission-control, /rename, /cd, /respond_to, /redact, /restart, /exit. /new stays open for session recovery (bugged/hallucinated sessions). Non-owners who try get a short "owner only" notice.

Deny-by-default access model: if neither allowed_users nor bot_owner is configured, the bot refuses all interactions — unconfigured installs are locked down by default. Set at least one to unlock access. This prevents open-mode footguns on fresh deployments.

```toml [channels.telegram] allowed_users = ["123456789"] # who may interact

OpenCrabs vs Node.js Agent Frameworks