# Voice Interaction

Voice is the primary — and now the **only** — way you start a turn with Rocky. Text remains available inside the chat detail screen, but the home surface is voice-first by design.

## Single-Track Architecture

Earlier versions of Rocky shipped multiple voice tracks (Classic STT + Chat + TTS, plus GLM Realtime). All of that has been removed. Today there is exactly one path:

```
Microphone → OpenAI Realtime → ROS Runtime ┬─ direct iOS tools (timers, weather, …)
                                            └─ delegate-task → Chat sub-agent → tools → result
                                                                          ↓
                                            Speakers + Visual UI ←────────┘
```

The realtime voice model carries low-latency turns directly. Anything multi-step or research-heavy — code, web research, batch operations — is offloaded to a back-end **Chat sub-agent** through the `delegate-task` tool. The sub-agent uses whichever Chat provider you configured (OpenAI, Anthropic, Gemini, …), runs to completion, and returns a single summary that the voice model speaks back.

That separation keeps the voice loop snappy without giving up agent depth.

## Voice Session Lifecycle

1. **Start** — user taps the orb on the home screen. Mic permission is gated by `OpenRockyPermissionCenter`; first-time users see a priming sheet, deniers a settings deep-link.
2. **Connecting** — `OpenRockyRealtimeVoiceBridge` opens a WebSocket session to OpenAI Realtime.
3. **Listening** — audio streams up; the live transcript appears as a bubble.
4. **Thinking / Responding** — the model replies with audio + transcript; concurrent tool calls execute against `OpenRockyToolbox`.
5. **Delegate (optional)** — for heavier tasks the model calls `delegate-task`; status updates from the sub-agent flow back into the timeline.
6. **Ready** — turn completes. Session stays open for follow-ups; tap the orb again to stop.

## The Home Canvas

The voice home view is intentionally minimal:

- **One hero orb.** It's the visual identity *and* the action. No duplicate PTT pill underneath.
- **Animated pulse rings.** Three TimelineView-driven concentric rings expand outward, calmer at idle, stronger when the mic is hot.
- **Top-bar provider chip.** A glanceable "OpenAI · gpt-realtime ●" capsule shows the active model and a connection dot — tap it to jump to settings.
- **Status pill.** `Tap to talk` / `Connecting…` / `Listening` / `Thinking` / `Responding` / `Ready` — short, glanceable.
- **Live waveform.** While listening, seven sine-driven bars render under the orb to confirm the mic is hot.
- **Rotating tip.** When idle and there's no transcript, a quiet tip cycles through example phrasings ("Try 'Set a 25-minute focus timer'").
- **Chat-bubble transcript.** Recent turns float above the orb iMessage-style: user-right tinted, Rocky-left card. Older bubbles fade; the live turn is full bright with auto-scroll.

On iPad (regular size class) the orb takes the left half of the canvas; the transcript, when present, takes the right.

## Provider

| | OpenAI Realtime |
|---|---|
| Models | `gpt-realtime`, `gpt-realtime-mini` |
| Protocol | WebSocket realtime API |
| Tool calls | Yes — restricted surface, plus `delegate-task` for the rest |
| Mic suspension during TTS | Yes (handled by `OpenRockyRealtimeVoiceBridge`) |

Configured via **Settings → Model → Realtime Voice**. The realtime config is a single instance (not a list) — there is one active voice configuration at a time.

## Voice + Character Integration

The active character's personality and voice preferences are propagated into every realtime session:

- **System prompt** — character personality is appended (shortened for realtime brevity)
- **Voice selection** — character maps to OpenAI realtime voices: `alloy`, `sage`, `ash`, `echo`, `shimmer`
- **Greeting** — optional opening line spoken on session start

## Session State Machine

```
ready → planning (connecting) → listening ⇄ planning (thinking) → executing → ready
```

Each state drives:

- The orb icon (`mic.fill` / `waveform` / `ellipsis` / `bolt.fill`)
- The status pill (`Tap to talk` / `Connecting…` / `Listening` / `Thinking` / `Responding` / `Ready`)
- The status dot color (muted / warning / success / mode tint)

## Audio Pipeline

- **Input** — device microphone → raw PCM buffer → streamed to OpenAI Realtime
- **Output** — provider audio response → device speakers
- **Echo control** — the bridge suspends the mic during TTS playback to prevent feedback loops
