# Coordinator Chat Protocol

## Status: Design (March 2026)

## Overview

This document specifies the IPC extension, file-based response channel, and instant wake-up mechanism for user↔coordinator communication. It covers the complete path from a user typing `wg chat "message"` through IPC dispatch, coordinator processing, response storage, and response retrieval.

## 1. IPC Types

### 1.1 UserChat Request

Add to `IpcRequest` in `src/commands/service/ipc.rs`:

```rust
/// Send a chat message from the user to the coordinator agent.
/// Unlike SendMessage (which targets a specific task's queue), UserChat
/// targets the coordinator directly and expects a conversational response.
UserChat {
    /// The user's message text
    message: String,
    /// Unique request ID for correlating this request with a response.
    /// Generated by the CLI/TUI using UUID v4 or similar.
    request_id: String,
},
```

### 1.2 IPC Response (Immediate)

The IPC response to `UserChat` is an **acknowledgement**, not the chat response itself. The coordinator agent processes the message asynchronously — the actual conversational response arrives via the outbox.

```rust
// On success:
IpcResponse::success(json!({
    "status": "accepted",
    "request_id": request_id,
}))

// On error (e.g., coordinator agent not running):
IpcResponse::error("Coordinator agent not available")
```

### 1.3 Why a Separate IPC Type (Not SendMessage)

`SendMessage` targets a task's message queue (`messages/{task-id}.jsonl`). Chat is fundamentally different:

1. **No task target** — chat messages go to the coordinator agent, which is not a task.
2. **Response expected** — `SendMessage` is fire-and-forget. `UserChat` expects a correlated response.
3. **Instant wake-up** — `SendMessage` goes through the normal settling delay. `UserChat` needs zero delay.
4. **Different storage** — chat messages go to `chat/inbox.jsonl`, not `messages/{task-id}.jsonl`.

## 2. Chat Storage Format

### 2.1 Directory Structure

```
.wg/chat/
├── inbox.jsonl      # User → coordinator messages
├── outbox.jsonl     # Coordinator → user responses
└── .cursor          # CLI/TUI read cursor (last-read outbox message ID)
```

### 2.2 Message Format

Both inbox and outbox use the same `ChatMessage` struct:

```rust
/// A single chat message between user and coordinator.
#[derive(Debug, Clone, Serialize, Deserialize)]
pub struct ChatMessage {
    /// Monotonically increasing ID within the file (inbox and outbox have separate sequences)
    pub id: u64,
    /// ISO 8601 timestamp
    pub timestamp: String,
    /// "user" or "coordinator"
    pub role: String,
    /// Message content (free-form text, may contain markdown)
    pub content: String,
    /// Correlates a user request with the coordinator's response.
    /// The CLI generates this when sending; the coordinator includes it when responding.
    pub request_id: String,
}
```

### 2.3 JSONL Format Examples

**inbox.jsonl** (user → coordinator):
```jsonl
{"id":1,"timestamp":"2026-03-02T10:00:00Z","role":"user","content":"help me plan the auth system","request_id":"a1b2c3d4"}
{"id":2,"timestamp":"2026-03-02T10:05:00Z","role":"user","content":"also add rate limiting","request_id":"e5f6g7h8"}
```

**outbox.jsonl** (coordinator → user):
```jsonl
{"id":1,"timestamp":"2026-03-02T10:00:05Z","role":"coordinator","content":"I'll create a task plan for authentication. Let me check the current graph...","request_id":"a1b2c3d4"}
{"id":2,"timestamp":"2026-03-02T10:05:03Z","role":"coordinator","content":"I've added a rate-limiting task with dependencies on the auth system.","request_id":"e5f6g7h8"}
```

### 2.4 Storage Operations

```rust
/// Append a user message to the inbox.
/// Returns the assigned inbox message ID.
pub fn append_inbox(dir: &Path, content: &str, request_id: &str) -> Result<u64>;

/// Append a coordinator response to the outbox.
/// Returns the assigned outbox message ID.
pub fn append_outbox(dir: &Path, content: &str, request_id: &str) -> Result<u64>;

/// Read outbox messages with ID > cursor.
/// Does not advance the cursor (caller decides when to advance).
pub fn read_outbox_since(dir: &Path, cursor: u64) -> Result<Vec<ChatMessage>>;

/// Block until a response with the given request_id appears in the outbox,
/// or timeout expires. Used by `wg chat` CLI for synchronous send-and-wait.
///
/// Implementation: poll outbox.jsonl every 200ms, checking for a message
/// whose request_id matches. Returns the first matching response.
pub fn wait_for_response(dir: &Path, request_id: &str, timeout: Duration) -> Result<Option<ChatMessage>>;

/// Read and advance the cursor.
/// Returns (cursor_value, messages_since_cursor).
pub fn read_and_advance_cursor(dir: &Path) -> Result<(u64, Vec<ChatMessage>)>;
```

### 2.5 Concurrency Model

Follows the same pattern as `src/messages.rs`:

- **Writers**: `O_APPEND` for atomic single-line writes. `flock()` around the read-max-id + append sequence for safe ID assignment.
- **Readers**: read-only, no locking needed.
- **Cursor**: write-to-temp + rename for atomicity.

### 2.6 Why Separate Inbox/Outbox (Not a Single File)

1. **Writer separation** — The inbox is written by the CLI/TUI (user process); the outbox is written by the coordinator agent (daemon process). Separating them avoids contention between these two writers, who never need to interleave into the same file.
2. **Cursor simplicity** — The CLI/TUI only needs a cursor on the outbox. The coordinator only needs a cursor on the inbox (or reads all on wake-up). A single file would need separate cursor semantics for each direction.
3. **Permissions** — In a future multi-user scenario, inbox and outbox could have different access controls.

## 3. Instant Wake-Up Mechanism

### 3.1 Problem

The daemon loop uses a settling delay (default 2000ms) after `GraphChanged` events to debounce burst graph construction. For chat, this adds unacceptable latency — the user expects sub-second acknowledgement.

### 3.2 Solution: Urgent Wake Flag

Add an `urgent_wake` boolean alongside the existing `settling_deadline` in the daemon loop. When a `UserChat` IPC request arrives, the handler sets `urgent_wake = true`. The tick decision logic checks this flag first and ticks immediately, bypassing both the settling delay and the background poll interval.

### 3.3 Daemon Loop Changes (src/commands/service/mod.rs)

Current flow:
```
IPC arrives → handle_connection() → wake_coordinator = true → settling_deadline = now + delay
```

New flow for UserChat:
```
UserChat arrives → handle_connection() → urgent_wake = true (no settling delay)
```

The daemon loop tick decision becomes:

```rust
let mut should_tick = false;
if !daemon_cfg.paused || urgent_wake {
    // Urgent wake: tick immediately (bypasses paused state for chat)
    if urgent_wake {
        urgent_wake = false;
        should_tick = true;
        logger.info("Urgent wake (UserChat), running coordinator tick immediately");
    }
    // Settled tick: settling deadline expired after GraphChanged events
    else if let Some(deadline) = settling_deadline && Instant::now() >= deadline {
        settling_deadline = None;
        should_tick = true;
        logger.info("Settling delay elapsed, running coordinator tick now");
    }
    // Background safety-net tick
    else if last_coordinator_tick.elapsed() >= daemon_cfg.poll_interval {
        should_tick = true;
    }
}
```

### 3.4 Why Urgent Wake Bypasses Paused State

When the coordinator is paused (`wg service pause`), it stops spawning new agents. But chat is a user-facing interaction channel — the user explicitly asked for attention. Ignoring chat while paused would be confusing. The coordinator tick triggered by urgent wake should handle the chat message but still respect the pause for agent spawning.

This means the coordinator tick itself needs to distinguish between "tick because of chat" and "tick because of graph changes". The simplest approach: the coordinator tick always checks the chat inbox, regardless of why it was triggered. The pause flag only suppresses agent spawning, not chat processing.

### 3.5 IPC Handler Implementation

In `handle_request()`:

```rust
IpcRequest::UserChat { message, request_id } => {
    logger.info(&format!("IPC UserChat: request_id={}", request_id));

    // 1. Append to chat inbox
    match chat::append_inbox(dir, &message, &request_id) {
        Ok(msg_id) => {
            // 2. Signal urgent wake (not settling delay)
            *urgent_wake = true;

            IpcResponse::success(json!({
                "status": "accepted",
                "request_id": request_id,
                "inbox_id": msg_id,
            }))
        }
        Err(e) => IpcResponse::error(&format!("Failed to store chat message: {}", e)),
    }
}
```

Note: `handle_connection()` signature needs a new `urgent_wake: &mut bool` parameter (or a combined wake enum). See section 3.6.

### 3.6 Wake Signal Types

Currently `handle_connection` uses a single `wake_coordinator: &mut bool`. With chat, we need to distinguish wake types. Two options:

**Option A: Separate booleans** (recommended for simplicity)
```rust
pub(crate) fn handle_connection(
    dir: &Path,
    stream: UnixStream,
    running: &mut bool,
    wake_coordinator: &mut bool,   // GraphChanged, AddTask, Resume — use settling delay
    urgent_wake: &mut bool,        // UserChat — bypass settling delay
    daemon_cfg: &mut DaemonConfig,
    logger: &DaemonLogger,
) -> Result<()>
```

**Option B: Wake enum**
```rust
pub(crate) enum WakeSignal {
    None,
    Settling,    // use settling delay
    Immediate,   // bypass settling delay
}
```

Option A is recommended because it's backward-compatible and minimal. The `wake_coordinator` flag continues to work exactly as before; `urgent_wake` is a new parallel path.

## 4. Request-Response Correlation

### 4.1 The Full Path

```
┌──────┐        ┌────────┐        ┌───────────┐        ┌────────────┐
│ User │        │ CLI/TUI│        │  Daemon   │        │Coordinator │
│      │        │        │        │  (IPC)    │        │   Agent    │
└──┬───┘        └───┬────┘        └─────┬─────┘        └─────┬──────┘
   │                │                    │                     │
   │ "plan auth"    │                    │                     │
   │───────────────▶│                    │                     │
   │                │                    │                     │
   │                │ generate           │                     │
   │                │ request_id=a1b2    │                     │
   │                │                    │                     │
   │                │ IPC UserChat       │                     │
   │                │ {message, req_id}  │                     │
   │                │───────────────────▶│                     │
   │                │                    │                     │
   │                │                    │ append inbox.jsonl  │
   │                │                    │ set urgent_wake     │
   │                │                    │                     │
   │                │  IPC Response      │                     │
   │                │  {status:accepted} │                     │
   │                │◀───────────────────│                     │
   │                │                    │                     │
   │                │                    │ coordinator tick    │
   │                │                    │ (immediate)         │
   │                │                    │────────────────────▶│
   │                │                    │                     │
   │                │                    │  reads inbox.jsonl  │
   │                │                    │  processes message  │
   │                │                    │  calls wg tools     │
   │                │                    │                     │
   │                │                    │  appends outbox.jsonl
   │                │                    │  with request_id=a1b2
   │                │                    │◀────────────────────│
   │                │                    │                     │
   │                │ poll outbox.jsonl  │                     │
   │                │ find req_id=a1b2   │                     │
   │                │◀ ─ ─ ─ ─ ─ ─ ─ ─ ─│                     │
   │                │                    │                     │
   │ "I'll create   │                    │                     │
   │  a task plan"  │                    │                     │
   │◀───────────────│                    │                     │
```

### 4.2 Request ID Generation

The CLI/TUI generates the `request_id` before sending. Format: `chat-{unix_millis}-{random_suffix}`.

```rust
fn generate_request_id() -> String {
    let millis = SystemTime::now()
        .duration_since(UNIX_EPOCH)
        .unwrap_or_default()
        .as_millis();
    let suffix: u32 = rand::random();
    format!("chat-{}-{:08x}", millis, suffix)
}
```

The timestamp prefix makes request IDs naturally sortable and debuggable. The random suffix prevents collisions from simultaneous senders.

### 4.3 How the Coordinator Writes Responses

In Phase 1 (before the persistent coordinator agent exists), the coordinator tick checks the chat inbox and responds. The coordinator has access to `request_id` from the inbox message and includes it in the outbox entry:

```rust
// In coordinator tick (or coordinator agent):
let inbox_messages = chat::read_inbox_since(dir, last_inbox_cursor)?;
for msg in &inbox_messages {
    // Process the message (LLM call, tool use, etc.)
    let response_content = process_chat_message(&msg.content)?;

    // Write response with same request_id
    chat::append_outbox(dir, &response_content, &msg.request_id)?;
}
```

In Phase 2 (persistent coordinator agent), the agent is injected with the inbox message via stream-json stdin. The agent's tools include a `chat_respond` tool that calls `append_outbox` with the correct `request_id`. The `request_id` is included in the injected user message so the agent can reference it.

### 4.4 CLI Response Retrieval

`wg chat` sends the IPC request, then polls the outbox for a response with the matching `request_id`:

```rust
// 1. Generate request_id
let request_id = generate_request_id();

// 2. Send IPC UserChat
let ipc_response = send_ipc(dir, IpcRequest::UserChat {
    message: user_input.clone(),
    request_id: request_id.clone(),
})?;

if !ipc_response.ok {
    return Err(anyhow!("Chat failed: {}", ipc_response.error.unwrap_or_default()));
}

// 3. Wait for response (poll outbox)
match chat::wait_for_response(dir, &request_id, Duration::from_secs(120))? {
    Some(response) => {
        println!("{}", response.content);
    }
    None => {
        eprintln!("Timeout: coordinator did not respond within 120s");
    }
}
```

### 4.5 TUI Response Retrieval

The TUI doesn't block on a single request. Instead, it polls the outbox periodically (every 200ms during active chat) and displays all new messages:

```rust
// In TUI event loop:
let (new_cursor, new_messages) = chat::read_and_advance_cursor(dir)?;
for msg in new_messages {
    chat_panel.append_message(msg);
}
```

The TUI tracks request_ids of sent messages to show "waiting..." indicators until the corresponding response arrives.

### 4.6 Multiple Concurrent Requests

Both CLI and TUI can have multiple outstanding requests. The `request_id` correlation handles this naturally — each sender waits for (or tracks) its own `request_id`. The coordinator processes inbox messages in order and writes responses with the corresponding `request_id`.

If the coordinator agent is processing message A when message B arrives, message B queues in the inbox. After responding to A, the coordinator reads B from the inbox on its next cycle (or immediately if it's a persistent session monitoring the inbox).

## 5. Coordinator Tick Integration (Phase 1)

In Phase 1, there is no persistent coordinator agent. The coordinator tick (pure Rust code) handles chat messages alongside its existing duties. This section specifies how.

### 5.1 Chat Processing in Coordinator Tick

After the coordinator tick runs its normal cycle (agent lifecycle, ready task dispatch), it checks the chat inbox:

```rust
// In coordinator::coordinator_tick():
// ... existing agent lifecycle and task dispatch logic ...

// Check for chat messages
let chat_dir = dir.join("chat");
if chat_dir.exists() {
    let inbox_cursor = chat::read_coordinator_cursor(dir)?;
    let new_messages = chat::read_inbox_since(dir, inbox_cursor)?;

    for msg in &new_messages {
        // Phase 1: simple response acknowledging the message.
        // Phase 2: the persistent coordinator agent handles this.
        let response = format!(
            "Message received. The coordinator agent (Phase 2) will provide \
             intelligent responses. For now, your message has been logged: \"{}\"",
            msg.content
        );
        chat::append_outbox(dir, &response, &msg.request_id)?;
    }

    if let Some(last) = new_messages.last() {
        chat::write_coordinator_cursor(dir, last.id)?;
    }
}
```

This is intentionally minimal for Phase 1. The goal is to wire the full path (CLI → IPC → inbox → coordinator tick → outbox → CLI) so that Phase 2 can replace the response generation with the persistent LLM agent while everything else stays the same.

### 5.2 Coordinator Cursor

The coordinator maintains its own cursor on the inbox, separate from the CLI/TUI cursor on the outbox:

```
.wg/chat/
├── inbox.jsonl
├── outbox.jsonl
├── .cursor              # CLI/TUI cursor on outbox (last-read outbox msg ID)
└── .coordinator-cursor  # Coordinator cursor on inbox (last-processed inbox msg ID)
```

## 6. Edge Cases

### 6.1 Service Not Running

`wg chat` connects to the daemon via IPC. If the socket doesn't exist or the connection fails, it should print a clear error:

```
Error: Service not running. Start it with: wg service start
```

### 6.2 Coordinator Agent Not Available (Phase 2)

In Phase 2, if the persistent coordinator agent has crashed and not yet restarted, the IPC handler detects this and responds with:

```json
{"ok": false, "error": "Coordinator agent is restarting. Try again in a few seconds."}
```

The CLI retries once after 3 seconds before giving up.

### 6.3 Response Timeout

`wait_for_response` has a configurable timeout (default 120s). If no response arrives:

```
Timeout: coordinator did not respond within 120s.
Your message was stored in the inbox. The response will appear when the coordinator processes it.
Use 'wg chat --history' to view past messages.
```

### 6.4 Large Messages

Chat messages should be limited to a reasonable size. The CLI truncates at 100KB with a warning. This prevents accidental pipe-of-entire-file scenarios.

### 6.5 Chat History

Both inbox and outbox are append-only JSONL files. `wg chat --history` displays them interleaved by timestamp:

```
[10:00:00] user: help me plan the auth system
[10:00:05] coordinator: I'll create a task plan for authentication...
[10:05:00] user: also add rate limiting
[10:05:03] coordinator: I've added a rate-limiting task...
```

### 6.6 Garbage Collection

Chat files grow over time. Options:

- **Manual**: `wg chat --clear` truncates both files and resets cursors.
- **Automatic**: When inbox.jsonl exceeds 10MB, rotate to inbox.jsonl.1 and start fresh. The coordinator cursor resets.
- **Session-based**: Add an optional `session_id` field. `wg chat --new-session` starts a fresh context.

For Phase 1, only manual clear is needed. Automatic rotation can be added later.

### 6.7 Concurrent TUI and CLI

Both TUI and CLI can send chat messages simultaneously. Each generates a unique `request_id`. The coordinator processes them in inbox order. Both consumers poll the outbox — TUI with periodic polling, CLI with `wait_for_response`.

The single `.cursor` file works for a single consumer (either TUI or CLI at a time). For concurrent consumers, each needs its own cursor. This can be handled by:

- TUI maintains cursor in memory (resets on restart, reads full outbox)
- CLI uses the file cursor only for `--history` display, and uses `request_id` matching for active waits

This avoids cursor conflicts between the two.

## 7. Summary of Changes by File

### New files
| File | Purpose |
|------|---------|
| `src/chat.rs` | ChatMessage struct, inbox/outbox storage ops, cursor management |
| `docs/design/coordinator-chat-protocol.md` | This document |

### Modified files
| File | Change |
|------|--------|
| `src/lib.rs` | Add `pub mod chat;` |
| `src/commands/service/ipc.rs` | Add `UserChat` variant to `IpcRequest`; add handler; add `urgent_wake` parameter to `handle_connection` and `handle_request` |
| `src/commands/service/mod.rs` | Add `urgent_wake` flag to daemon loop; add urgent wake tick path; pass `urgent_wake` to `handle_connection` |
| `src/commands/service/coordinator.rs` | Add chat inbox processing to coordinator tick |
| `src/commands/chat.rs` | New `wg chat` CLI command (created by sh-impl-chat-cli) |
| `src/commands/mod.rs` | Register chat subcommand |
| `src/cli.rs` | Wire up chat CLI args |

### Downstream task mapping

| Task | What it implements from this design |
|------|-------------------------------------|
| `sh-chat-storage` | Section 2 (ChatMessage, inbox/outbox, cursors, storage ops) |
| `sh-instant-wakeup` | Section 3 (urgent_wake flag, daemon loop changes, IPC handler) |
| `sh-impl-chat-cli` | Section 4.4 (`wg chat` command, request_id generation, response waiting) |

## 8. Design Decisions Log

| Decision | Choice | Rationale |
|----------|--------|-----------|
| Separate inbox/outbox vs single file | Separate | Different writers (user vs coordinator), different cursors, cleaner separation |
| IPC ack + file polling vs IPC streaming | IPC ack + file polling | Simpler; avoids holding IPC connection open; works across process restarts |
| Urgent wake vs zero settling delay | Urgent wake (separate flag) | Preserves settling semantics for GraphChanged; clean separation of concerns |
| request_id generation | Client-side (CLI/TUI) | Decouples client from server; client knows ID immediately for correlation |
| Chat bypasses pause | Yes | Chat is user-facing; ignoring the user while paused is confusing; pause only suppresses agent spawning |
| Phase 1 coordinator response | Stub acknowledging receipt | Wires the full path for testing; Phase 2 replaces with LLM agent |
