AI Chat: an agent that reads your terminal, not its screenshot

Turning it on

AI Chat lives behind Settings > Experimental Features. Flip Enable experimental features, then AI Chat — that switches on the chat panel, its keyboard shortcut, and a command-palette entry. The related AI Watchers toggle in the same group unlocks the reactive side covered below. It's beta: under active development, occasionally rough, and exactly the part where early feedback is most useful.

The problem with chat sidebars

The standard pattern is a chat panel bolted next to your editor. It can read files. What it usually can't read is your terminal — the build that just failed, the test output, the command you ran three minutes ago. So you copy-paste error text into the chat by hand, stripping it of all the context that made it meaningful. The assistant is reasoning about a screenshot you transcribed, not about what actually happened.

That's backwards for an agent-driven workflow, where the terminal is the workspace. We built AI Chat to start from the terminal and work outward.

The advantage: structured state, not scraped pixels

AI Chat doesn't screen-scrape ANSI bytes. It reads the same structured terminal model the rest of TUICommander is built on — OSC 133 semantic blocks. Because every command, its output, and its exit code are first-class data, the agent can answer questions the raw scrollback can't support:

• Command history with outcomes — what ran, in what order, and which ones failed (exit code is captured per block, not guessed from text).
• "Why did that fail?" — it reads the last failing block directly instead of you pasting it.
• Semantic zones — prompt vs. input vs. output are distinguished per cell, so the model isn't confused about which lines are your command and which are the program's reply.
• Hyperlinks — OSC 8 links in the output (a failing CI URL, a docs link) are enumerable, not lost in the text.
• Scrollback search — it can grep the buffer for a string instead of asking you to find it.

All of it runs through secret redaction before it reaches a model, and read tools cap their output so a chatty buffer can't blow the context budget.

The potential: a copilot that wakes itself up

Reading state on demand is useful. Reacting to it without being asked is where this gets interesting. AI Chat can arm reactive watchers: rules that fire a fresh autonomous conversation when something happens in a terminal — it goes idle, a command finishes, an error appears, a question is asked, or a pattern shows up. "Watch this deploy and tell me if it errors" becomes a thing the agent does while you look elsewhere.

The power here is obvious, so the safety is deliberate. Arming a watch is approval-gated — the model can't silently set up loops that run on their own; you confirm first. Every watcher carries a cooldown, a burst limit, and a maximum number of fires, and a watcher won't fire on top of an active conversation or while you're typing. Model-armed watches default to a low fire cap. The goal is a copilot that can act on its own without ever surprising you with a runaway.

Local-first, with a cost meter

AI Chat is provider-agnostic. It auto-detects a local Ollama install and lists your installed models, so you can run entirely on-device — nothing leaves your machine — or point it at a cloud provider when you want a bigger model. Each conversation picks its own model independently, so a throwaway question and a serious refactor don't have to share one setting.

Two details make this practical rather than just flexible. First, there's a per-token cost estimate in USD for known cloud models, so you can see what a conversation actually costs. Second, tools are split into read and write phases — the cheaper, safe read tools can run on a smaller model while the heavier write actions escalate to a stronger one. You're not paying frontier-model prices to grep your scrollback.

How it's wired into the rest of TUICommander

AI Chat isn't a separate app stapled on — it reuses the machinery TUIC already has:

• Per-session context. The agent's view is scoped to the terminal it's attached to. Its file tools operate in a sandbox rooted at that session's working directory, so "list the files" means this project.
• Persisted conversations. Chats are saved to disk and can be listed, reopened, and deleted — they survive restarts like everything else in TUIC.
• The same panel system. It's a dockable panel with a shortcut and a command-palette entry, alongside the file browser, outline, and triage panels.
• Shared notifications. It speaks the same toast and sound layer the rest of the app uses, so an agent's output can reach you even when its tab isn't focused.

And it's distinct from the terminal agents you already run. Claude Code or Codex live inside a PTY and you drive them as processes; AI Chat is TUIC's own agent looking at those terminals through structured state. The two compose — see cross-agent orchestration for the other half of that story.

What's next

The roadmap is about closing the loop between an event and a response. Next up: a one-click "ask why" when a command fails, a notify_user action so the agent can get your attention (with an OS notification when the window is unfocused), and routing CI failures through the watcher engine so a red build can trigger a heal attempt through the same safe, gated path instead of a raw write to the terminal. The foundation — OSC-native reading and approval-gated reactive watches — is in place; the rest is teaching it when to speak up.

It's beta, the toggle is one click away, and the feedback loop is short. If you try it, tell us where it's blind.