# RFD D18: Plugin Event Subscriptions and Query Delegation

- **Status**: Draft
- **Category**: Design
- **Authors**: Jean Mertz <git@jeanmertz.com>
- **Date**: 2026-04-06
- **Extends**: [RFD 072][RFD 072-2]

## Summary

This RFD extends the command plugin protocol ([RFD 072]) with event
subscriptions and agent loop delegation.
Plugins can subscribe to live conversation events, respond to interactive
prompts (tool approval, inquiries), and trigger LLM queries that JP executes on
their behalf.
Together, these capabilities enable plugins to act as alternative frontends to
JP's agent loop — the key building block for a web-based chat interface.

## Motivation

[RFD 072] defines a request/response protocol that covers workspace queries and
mutations.
A plugin can list conversations, read events, lock, write, and produce formatted
output.
This is sufficient for read-only tools (web viewer, exporters) and simple write
tools (importers, bulk editors).

It is not sufficient for a chat interface.
A chat plugin needs to:

1. **Stream LLM responses in real time** — tokens must arrive at the plugin as
   they are generated, not after the full response is complete.
2. **Handle tool approval prompts** — when the agent loop wants to run a tool,
   the plugin must present the approval UI and send the decision back.
3. **Handle inquiry questions** — tools may ask questions that need user input.
4. **Observe concurrent activity** — if another `jp query` session writes to
   the same conversation, the plugin should see those events too.

All four require JP to *push* messages to the plugin, which the request/response
model from [RFD 072] does not support.
This RFD adds that capability.

## Design

### Subscriptions

A plugin subscribes to a conversation to receive events as they occur.

**Subscribe:**

```json
{
  "type": "subscribe",
  "sub_id": "chat-main",
  "conversation": "17127583920",
  "events": [
    "chat_response",
    "tool_call_request",
    "tool_call_response"
  ]
}
```

Response:

```json
{
  "type": "subscribed",
  "sub_id": "chat-main"
}
```

The `sub_id` is chosen by the plugin and must be unique among its active
subscriptions.
JP echoes it on every pushed event, allowing the plugin to maintain multiple
independent subscriptions — e.g., two chat windows side-by-side, or one
subscription for rendering and another for logging.

The `events` filter is optional.
When omitted, all event types are delivered.

Once subscribed, JP pushes events as they occur on the conversation (from any
source — a concurrent `jp query` session, another plugin, or this plugin's own
`query` operation):

```json
{
  "type": "event",
  "sub_id": "chat-main",
  "data": {
    "timestamp": "...",
    "type": "chat_response",
    "message": "Here's what I found..."
  }
}
```

**Unsubscribe:**

```json
{
  "type": "unsubscribe",
  "sub_id": "chat-main"
}
```

Response:

```json
{
  "type": "unsubscribed",
  "sub_id": "chat-main"
}
```

Subscriptions are automatically removed when the plugin exits.

### Interactive Events

Some pushed events require a response from the plugin.
These carry a `respond` field indicating what kind of response JP expects.

**Tool call approval:**

```json
{
  "type": "event",
  "sub_id": "chat-main",
  "respond": "tool_approval",
  "data": {
    "type": "tool_call_request",
    "id": "tc_1",
    "name": "cargo_check",
    "arguments": {}
  }
}
```

The plugin responds:

```json
{
  "type": "respond",
  "sub_id": "chat-main",
  "respond": "tool_approval",
  "tool_call_id": "tc_1",
  "action": "approve"
}
```

Valid actions: `approve`, `reject`, `modify` (with an `arguments` field
containing the modified arguments).

**Inquiry question:**

```json
{
  "type": "event",
  "sub_id": "chat-main",
  "respond": "inquiry",
  "data": {
    "type": "inquiry_request",
    "id": "inq_1",
    "question": "Which file?",
    "options": [
      "a.rs",
      "b.rs"
    ]
  }
}
```

The plugin responds:

```json
{
  "type": "respond",
  "sub_id": "chat-main",
  "respond": "inquiry",
  "inquiry_id": "inq_1",
  "answer": "a.rs"
}
```

If the plugin does not respond within a configurable timeout, JP falls back to
the default behavior configured for non-interactive mode (auto-approve, reject,
etc.).

Events without a `respond` field are informational — the plugin observes them
but does not need to reply.

### Query (Agent Loop Delegation)

The `query` operation triggers JP's agent loop on behalf of the plugin.
The plugin sends a user message, JP runs the full turn loop (streaming, tool
calls, retries), and events flow back to the plugin via its subscription.

```json
{
  "type": "query",
  "conversation": "17127583920",
  "content": "What files changed?"
}
```

The `query` payload supports optional fields for the full range of
`run_turn_loop` inputs:

- `attachments` (array, optional): Attachments to include in the turn,
  equivalent to `jp query -a`.
  Each entry follows the attachment serialization format (inline content with a
  `type` discriminator).
- `schema` (object, optional): A JSON Schema constraining the assistant's
  response format, triggering structured output.
  Equivalent to the `schema` field on `ChatRequest`.

Example with both:

```json
{
  "type": "query",
  "conversation": "17127583920",
  "content": "Summarize this file",
  "attachments": [
    {
      "type": "file_content",
      "path": "src/main.rs",
      "content": "fn main() {}"
    }
  ],
  "schema": {
    "type": "object",
    "properties": {
      "summary": {
        "type": "string"
      }
    },
    "required": [
      "summary"
    ]
  }
}
```

When omitted, `attachments` defaults to an empty list and `schema` defaults to
`null` (free-form response).

Response (acknowledges the query has started):

```json
{
  "type": "query_started",
  "conversation": "17127583920"
}
```

JP locks the conversation, starts a turn, and runs the agent loop.
Events stream to the plugin via its active subscription on that conversation.
When the turn completes:

```json
{
  "type": "event",
  "sub_id": "chat-main",
  "data": {
    "type": "query_complete"
  }
}
```

During the turn, the plugin receives all intermediate events: `chat_response`
chunks (for streaming tokens to a browser), `tool_call_request` events (which
may require approval via `respond`), `inquiry_request` events, and
`tool_call_response` results.
The plugin is effectively an alternative frontend to the same agent loop that
powers `jp query` in the terminal.

The conversation must either be already locked by the plugin or unlocked (in
which case JP acquires the lock for the duration of the query and releases it on
completion).
If another process holds the lock, the request returns an error.

If the plugin sends `query` without an active subscription on that conversation,
JP returns an error — there would be no way to deliver the streaming events or
interactive prompts.

### Integration with the Agent Loop

The agent loop is currently implemented in `jp_cli::cmd::query::turn_loop`.
Some of its external dependencies are trait-based today:

| Dependency      | Current abstraction    | Status          |
| --------------- | ---------------------- | --------------- |
| Prompt backend  | `PromptBackend` trait  | Exists          |
| Tool execution  | `ExecutorSource` trait | Exists          |
| Inquiry backend | `InquiryBackend` trait | Exists          |
| Response output | `ChatResponseRenderer` | Concrete struct |

`ChatResponseRenderer` is a concrete struct instantiated inside
`TurnCoordinator::new()`.
There is no `ResponseRenderer` trait in the current codebase —
`TurnCoordinator` owns the renderer directly and is itself created inside
`run_turn_loop`.
This means a protocol-backed renderer cannot be injected without first
extracting rendering behind a trait.

For `query` delegation, JP provides protocol-backed implementations of the
required traits:

- **`ProtocolPromptBackend`**: When the turn loop asks for tool approval, this
  implementation sends a `respond: "tool_approval"` event to the plugin and
  blocks until the plugin sends `respond` back.
- **`ProtocolResponseRenderer`**: When the turn loop emits `ChatResponse`
  chunks, this implementation pushes them as `event` messages on the plugin's
  subscription.
  Requires the `ResponseRenderer` trait extraction described above.
- **`ProtocolInquiryBackend`**: When a tool asks a question, this implementation
  sends a `respond: "inquiry"` event and awaits the answer.

[RFD 026] proposes extracting the turn loop into a standalone `jp_agent` crate
with explicit trait boundaries, which would introduce a `ResponseRenderer` trait
as part of that work.
If RFD 026 has landed by the time Phase 3 begins, the protocol bridges plug into
`jp_agent::run_turn_loop()` directly.
If not, Phase 3 of this RFD must extract `TurnCoordinator`'s rendering into a
trait as a prerequisite step before the protocol-backed implementations can be
wired in.

### Web Chat Example

A web server plugin (`jp-serve`) with chat support:

1. When a browser opens a conversation, `subscribe` to it with a unique `sub_id`
   per browser session.
2. When the user sends a message, issue a `query` with the message content.
3. Receive streaming `chat_response` events via the subscription and forward
   them to the browser over SSE or WebSocket.
4. Receive `tool_call_request` events with `respond: "tool_approval"` and
   present an approval UI in the browser.
   Send the user's decision back via `respond`.
5. Receive `query_complete` and signal the browser that the turn is done.
6. `unsubscribe` when the browser disconnects.

The plugin never calls the LLM directly, manages conversation locks, or persists
events — JP handles all of that.
The plugin is purely a presentation layer.

## Drawbacks

- **Bidirectional complexity**: The protocol shifts from pure request/response
  to a bidirectional event stream.
  Plugins must handle unsolicited messages (pushed events) arriving at any time,
  interleaved with responses to their own requests.
  Multi-threaded plugins handle this naturally; single-threaded plugins (shell
  scripts) cannot easily participate in subscriptions.

- **Timeout semantics**: Interactive events require the plugin to respond within
  a timeout.
  The right default timeout and fallback behavior may vary by use case (a web UI
  might want a long timeout to let the human think; a batch script might want to
  auto-approve immediately).

- **Agent loop coupling**: The `query` operation exposes JP's internal agent
  loop behavior as a protocol surface.
  Changes to how the turn loop handles tool calls, retries, or interrupts become
  visible to plugins.
  The trait boundaries from [RFD 026] help, but the protocol-level contract
  (which event types arrive and in what order) is an additional compatibility
  surface.

## Alternatives

### Plugin calls the LLM directly

The plugin could bypass JP's agent loop entirely: call the LLM API itself,
manage tool execution, and use JP only for conversation storage via
`push_events`.

Rejected because:

- Duplicates the agent loop (streaming, retries, tool coordination, inquiry
  handling) in every chat-capable plugin.
- The plugin would need LLM API keys and provider configuration, breaking the
  principle that JP manages credentials.
- Tool execution requires access to JP's tool definitions and MCP servers, which
  are not exposed through the current protocol.

### Polling instead of subscriptions

The plugin could repeatedly call `read_events` to check for new events.

Rejected as the primary mechanism because polling adds latency and wastes
resources for streaming use cases.
However, polling remains available as a fallback for simple plugins that don't
need real-time updates.

## Non-Goals

- **Multi-conversation queries**: A single `query` operates on one conversation.
  Orchestrating parallel queries across conversations is the plugin's
  responsibility.
- **Plugin-side tool execution**: Tools are executed by JP, not by the plugin.
  The plugin can approve, reject, or modify tool calls, but cannot provide its
  own tool implementations through this protocol.
  (Wasm plugins via [RFD 016] serve that purpose.)
- **Streaming protocol optimization**: The JSON-lines format is kept for
  consistency with [RFD 072].
  A binary framing format (msgpack, protobuf) is future work if latency becomes
  measurable.

## Risks and Open Questions

- **Cross-process event delivery**: Subscriptions deliver events from any
  source.
  For events generated by the same plugin's `query` operation, delivery is
  straightforward (JP controls the turn loop).
  For events from a concurrent `jp query` session writing to the same
  conversation, JP needs a notification mechanism.
  The current storage layer uses append-only files with flock coordination but
  has no change notification.
  Polling the events file periodically is the simplest approach;
  inotify/kqueue-based notification is an optimization for later.

- **Interactive event ordering**: When multiple tool calls arrive in a batch,
  the plugin receives multiple `respond: "tool_approval"` events.
  The protocol does not currently define whether the plugin must respond in
  order or can respond out of order.
  The simplest rule: responses can arrive in any order, matched by
  `tool_call_id`.

- **Query cancellation**: If the plugin wants to cancel a running `query` (e.g.,
  the user navigated away in the browser), there is no cancellation message
  defined.
  A `cancel_query` message type would be a natural addition but is deferred
  until the need is validated.

- **Backpressure**: If the LLM streams tokens faster than the plugin can consume
  them (e.g., the browser connection is slow), pushed events queue up in JP's
  pipe buffer.
  For human-speed interactions this is unlikely to be a problem.
  For high-throughput scenarios, a flow control mechanism may be needed.

## Implementation Plan

### Phase 1: Subscriptions

- Add `subscribe`, `unsubscribe` message types to `jp_plugin`.
- Implement the event push mechanism in JP's dispatcher.
- For events from the same process (e.g., `push_events` by the same plugin),
  push to active subscriptions immediately.
- For cross-process events, implement file-based polling as the initial
  notification mechanism.
- Test with a plugin that watches a conversation while `jp query` writes to it.
- Depends on [RFD 072] Phase 1 (protocol core).

### Phase 2: Interactive events

- Add `respond` field to pushed events and `respond` message type.
- Implement timeout and fallback behavior.
- Test with a shell script that auto-approves tool calls.
- Depends on Phase 1.

### Phase 3: Query delegation

- If [RFD 026] has not landed: extract `TurnCoordinator`'s rendering into a
  `ResponseRenderer` trait so protocol-backed and terminal-backed renderers can
  be swapped.
- Implement `query`, `query_started`, `query_complete` message types.
- Build protocol-backed implementations of `PromptBackend`, `ResponseRenderer`,
  and `InquiryBackend`.
- Support `attachments` and `schema` fields in the `query` payload.
- Bridge into `run_turn_loop` (via `jp_agent` if [RFD 026] has landed, otherwise
  via `jp_cli` internals with the trait extraction above).
- Test with `jp-serve` serving a chat interface.
- Depends on Phases 1 and 2, and [RFD 072] Phase 4 (write operations).

## References

- [RFD 072: Command Plugin System][RFD 072]
- [RFD 016: Wasm Plugin Architecture][RFD 016]
- [RFD 026: Agent Loop Extraction][RFD 026]
- [RFD 027: Client-Server Query Architecture][RFD 027]

[RFD 016]: 016-wasm-plugin-architecture.md
[RFD 026]: 026-agent-loop-extraction.md
[RFD 027]: 027-client-server-query-architecture.md
[RFD 072]: 072-command-plugin-system.md
[RFD 072-2]: ../072-command-plugin-system.md
