# RFD D23: JSON Input and Schema for CLI

- **Status**: Draft
- **Category**: Design
- **Authors**: Jean Mertz <git@jeanmertz.com>
- **Date**: 2026-04-13
- **Requires**: [RFD D19]

## Summary

This RFD adds a machine-readable JSON interface to JP's CLI.
A new `--args-schema` flag prints a JSON Schema describing the full CLI surface,
and a new `--args-json` flag accepts a JSON object conforming to that schema to
drive any JP command.
The CLI's existing clap argument structs are the single source of truth — they
gain `serde::Deserialize` and `schemars::JsonSchema` derives alongside their
existing `clap` derives, so the schema and JSON input/output format fall
directly out of the Rust types.

## Motivation

JP is a CLI tool designed for interactive use by humans.
But two categories of callers need programmatic access:

1. **LLMs.** An LLM that wants to invoke JP (e.g. as a tool in an agent loop)
   can be better served with a machine-readable description of JP's interface
   and a structured way to call it.
   Today the LLM must construct shell command strings, which is fragile and
   error-prone.

2. **Scripts and automation.** CI pipelines, editor integrations, and wrapper
   tools benefit from JSON input over string-splicing shell arguments.

Without this change, programmatic callers must reverse-engineer the CLI's flag
names and value formats from help text, and construct argv arrays by hand.
This is both fragile and limits JP's usefulness as a building block.

## Design

### Guiding Principle

The CLI argument structs (`Cli`, `Globals`, `Query`, `Conversation`, etc.) are
the single source of truth.
They already define the interface via `clap` derives.
This RFD adds `serde::Deserialize` and `schemars::JsonSchema` to the same
structs so that the JSON schema and deserialization behavior are derived from
the same types — not maintained as a parallel hierarchy.

Where a field type works naturally with both clap and serde/schemars, no changes
are needed beyond adding derives.
Where a type is clap-specific (manual `FromArgMatches` impls,
`Option<Option<T>>` triple-state patterns), the type is replaced or annotated to
support all three trait families.

Fields that use custom `value_parser` functions do not need new bridging types.
Clap's `value_parser` translates CLI string input into the field's target type
(e.g.
`schemars::Schema`, `DateTime<Utc>`, `String`).
Serde deserializes JSON input into the same target type directly.
The `value_parser` functions remain CLI-only conveniences — they are invisible
to serde.

### User-Facing Interface

Two new global flags provide the two sides of the JSON interface:

**Schema output:**

```shell
jp --args-schema                    # full schema
jp --args-schema query              # schema for `jp query`
jp --args-schema conversation.edit  # schema for `jp conversation edit`
```

Prints the JSON Schema for the `Cli` struct (or a subcommand) to stdout and
exits.
The full schema is generated by calling `schemars::schema_for!(Cli)`.

The optional argument is a dot-delimited command path that prunes the command
tree to only the path from root to the target subcommand.
Globals, root options, and metadata are always included.
At each nesting level, sibling variants are dropped but the structural nesting
is preserved — the output is always a valid `--args-json` schema for invoking
that specific subcommand.

For example, `jp --args-schema query` emits the full top-level schema but with
`command` containing only the `query` variant (not `conversation`, `config`,
etc.).
`jp --args-schema conversation.edit` keeps the `command.conversation.edit`
nesting but drops `conversation`'s sibling commands (`query`, `config`, ...) and
`edit`'s sibling subcommands (`ls`, `show`, `fork`, ...).
When omitted, all variants at all levels are included.

Because `schemars` generates schemas using `$ref` and `definitions`, pruning
requires walking the generated schema AST, filtering `oneOf` arrays at each
command level, and removing orphaned definitions.
This is non-trivial but mechanical.

The output includes JP metadata:

```json
{
  "$schema": "https://json-schema.org/draft/2020-12/schema",
  "title": "jp",
  "$jp": {
    "schema_version": 1,
    "jp_version": "0.42.0"
  },
  "type": "object",
  "properties": {
    "...": "..."
  }
}
```

`schema_version` is a monotonic integer that bumps only on breaking changes to
the JSON input contract.
`jp_version` is informational.

**JSON input:**

```shell
jp --args-json '{ ... }'
echo '{ ... }' | jp --args-json -
```

Accepts a JSON object conforming to the schema.
The JSON is first deserialized into a thin envelope struct that extracts and
validates the optional `schema_version` metadata, then the inner payload is
deserialized into the `Cli` struct and passed into the existing `run_inner`
execution path.
The rest of the pipeline (config loading, workspace resolution, command
dispatch) is unchanged.

Both flags are handled before clap parsing — they are detected by scanning raw
`argv`.
`--args-schema` prints and exits.
`--args-json` deserializes and dispatches.

Errors are returned as structured JSON when stdout is not a TTY (consistent with
existing `--format json` behavior).

`--args-json` does not imply non-interactive mode.
TTY detection continues to govern interactive behavior as usual.
A user invoking `--args-json` from an interactive terminal gets the same
interactive experience (editor prompts, confirmation dialogs) as normal CLI
usage.

**Stdin interaction.** When `--args-json -` is used, the pre-parse logic
consumes stdin to read the JSON payload.
This means commands that also read stdin (e.g.
`Query` piping input) will find it empty.
JSON callers must embed all input in the JSON payload (e.g. the `query` field)
rather than relying on stdin piping.
This should be documented in the CLI help text for `--args-json`.

### Bridging Types

An audit of all ~30 command structs identifies the following categories of field
types and the work required for each.

#### Types that already work

Primitives (`bool`, `String`, `Option<String>`, `u8`, `usize`, `Option<usize>`,
`Option<NonZeroUsize>`), `Option<Utf8PathBuf>`, and `schemars::Schema` already
have or trivially gain `Serialize`, `Deserialize`, and `JsonSchema`.
No changes needed beyond adding derives to the containing struct.

#### Fields with custom `value_parser` functions

Four fields use custom `value_parser` functions to translate CLI string input
into typed values: `string_or_path` (→ `String`), `parse_schema` (→
`schemars::Schema`), `parse_fork_turns` (→ `Option<usize>`), and
`parse_duration` (→ `DateTime<Utc>`).

These do not need bridging types.
The target types already implement `Deserialize` and `JsonSchema`.
Clap uses the `value_parser` to convert CLI strings; serde deserializes from
JSON directly into the target type.
The `value_parser` functions remain CLI-only and are invisible to serde.

For example, `Query::schema` is `Option<schemars::Schema>`.
On the CLI path, clap applies `string_or_path.try_map(parse_schema)` to convert
the DSL string.
On the JSON path, serde deserializes a JSON Schema object natively.

#### Simple enums

`CliFormat`, `LogFormat`, and the two `Sort` enums (in `conversation/ls` and
`conversation/grep`) are `clap::ValueEnum` types.
They need `Serialize`, `Deserialize`, and `JsonSchema` added — one derive line
each.

`ReasoningConfig` already has `Serialize`/`Deserialize` via schematic's `Config`
derive and needs only `JsonSchema`.

#### FromStr types

Seven types (`Editor`, `AttachmentUrlOrPath`, `WorkspaceIdOrPath`,
`ConversationTarget`, `KvAssignment`, `ExpirationDuration`, `KeyValueOrPath`)
already implement `FromStr`.
They need `Deserialize` (delegating to `FromStr` for string inputs) and
`JsonSchema` (emitting `{"type": "string"}` with appropriate descriptions or
enum values).
A shared helper trait or macro can reduce boilerplate across these.

#### Manual clap types

Three types have hand-written `clap::Args` / `FromArgMatches` impls:

- **`ToolDirectives`**: An ordered list of tool enable/disable operations.
  Needs manual `Deserialize` and `JsonSchema` impls representing the same
  concept as a JSON array of directive objects.
  Because these types represent lists (not maps), they must not use
  `#[serde(flatten)]` — they are regular named fields in the serde
  representation.

- **`FlagIds<S, M>` and `PositionalIds<S, M>`**: Lists of conversation targets.
  For JSON, both collapse to `Vec<ConversationTarget>` — the positional-vs-flag
  distinction is a CLI-only concern.
  Need manual `Deserialize` and `JsonSchema` impls.
  Same caveat about not using `#[serde(flatten)]`.

#### Triple-state fields (`Option<Option<T>>`)

Nine fields across `Query`, `Edit`, `Fork`, and `Print` use clap's `num_args =
0..=1` with `default_missing_value` to represent three states: absent,
flag-present-without-value, flag-present-with-value.

For the JSON path, the three states map to: field absent or `undefined`, `null`
or `true`, and a concrete value.
These fields need custom serde handling to distinguish absent from null.

The exact mechanism needs prototyping.
Options include a `TriState<T>` newtype that works as a clap field type,
per-field `#[serde(default, deserialize_with)]` annotations, or a combination.
Clap's derive macro has special handling for `Option<Option<T>>` tied to
`num_args`, so a replacement type must either replicate that behavior or keep
`Option<Option<T>>` for clap and use serde attributes for the JSON path.
This will be resolved during Phase 1 implementation.

#### `Commands::External`

The `#[command(external_subcommand)]` variant currently holds `Vec<String>`.
Clap's `external_subcommand` requires a type implementing
`FromIterator<String>`.
For JSON, the raw vector is not ergonomic — a structured object with `name` and
`args` fields is better.

The variant becomes `External(ExternalCommand)`, where `ExternalCommand` is a
struct that implements `FromIterator<String>` (for clap), `Serialize`,
`Deserialize`, and `JsonSchema` (for the JSON path).

For `--args-schema`, installed plugins that implement [RFD D19]'s `Help`
protocol contribute their structured arg definitions to the schema as `oneOf`
variants under the `External` discriminator.
Plugins that don't implement `Help` get an opaque `{"name": "string", "args":
["string"]}` fallback.

[RFD D19]'s `PluginArg` type needs a small extension for this: an optional
`value_type` field (`"string" | "integer" | "number" | "boolean"`, defaulting to
`"string"`) so the schema can emit proper JSON Schema types for plugin
arguments.

### Schema Versioning

The schema version (`$jp.schema_version`) tracks breaking changes to the JSON
input contract — field renames, removed fields, structural changes to command
nesting.
It is independent of JP's release version.

Adding new optional fields is not a breaking change and does not bump the schema
version.
Removing or renaming fields is breaking.
Changing a field's type is breaking.

The `schema_version` field in the input JSON is optional.
When present, JP validates it matches the expected version and returns an error
on mismatch.
Validation happens in the envelope struct before the inner payload is
deserialized into `Cli`.

## Drawbacks

**Annotation overhead.** Every command struct gains three additional derives and
some fields need `#[serde(...)]` / `#[schemars(...)]` attributes.
This adds visual noise to the arg definitions.
In practice, most structs need only the derive line — the handful of fields
needing attributes are concentrated in `Query` and the conversation targeting
types.

**Dual maintenance of manual impls.** The three types with manual `clap::Args`
impls (`ToolDirectives`, `FlagIds`, `PositionalIds`) now also need manual
`Deserialize` and `JsonSchema` impls.
These must be kept in sync with the clap behavior.
Tests that round-trip CLI args through JSON can catch drift.

**clap constraint gaps.** Clap's `conflicts_with` and `requires` constraints are
not automatically enforced on the serde deserialization path.
Invalid combinations (e.g. both `edit` and `no_edit` set to true) are caught by
the execution pipeline rather than at parse time.
This is acceptable for the initial implementation; a `validate()` pass can be
added incrementally.

**Binary size.** `schemars` adds code generation for schema derivation.
The impact is expected to be modest since `schemars` is already a dependency
(used by `Query::schema`), but should be measured.

## Alternatives

### Generate schema from clap introspection at runtime

Walk `clap::Command` at runtime using its introspection API (`get_subcommands`,
`get_arguments`, etc.) to produce a JSON Schema.

Rejected because clap's introspection is lossy — value parsers are opaque, so
the schema would emit `"string"` for nearly every argument.
The core motivation for the schema is to give LLMs precise type information,
which requires the richer type data that `schemars::JsonSchema` provides.

### Dedicated parallel schema structs

Maintain a separate set of structs annotated only with `schemars` that mirror
the CLI shape, with a conversion layer between them.

Rejected because two sources of truth means every CLI change requires a
corresponding schema struct update.
This maintenance burden grows with every new command and flag.

### JSON → synthetic argv → clap parsing

For `--args-json`, convert the JSON object into a `Vec<String>` argv and feed it
through clap's normal parsing path.

This is simpler to implement but lossy in the other direction — some JSON
structures (nested objects for `--cfg`, ordered arrays for `ToolDirectives`)
have no natural argv representation.
It also means every JSON input is subject to clap's string-based parsing quirks.
Direct deserialization into the struct is more robust and preserves types
end-to-end.

## Non-Goals

- **Full clap constraint enforcement in JSON mode.** Constraints like
  `conflicts_with` and `requires` are not replicated in the `Deserialize` path.
  The execution pipeline catches real conflicts at runtime.

- **Shell completions from the schema.** The JSON Schema could in theory
  generate shell completions, but that is out of scope.
  See [RFD 059].

- **Non-interactive mode.** `--args-json` does not imply non-interactive
  behavior.
  See [RFD 049] for the `--non-interactive` flag.

- **`--cfg` JSON objects.** Accepting JSON objects as `--cfg` values is a useful
  extension to the config pipeline but orthogonal to the JSON CLI interface.
  It is tracked separately.

- **Long-running JSON API server.** The schema defined here could serve as the
  request format for a persistent JSON-over-HTTP (or JSON-over-stdio) server
  that accepts JP commands without process-per-invocation overhead.
  That is a separate design concern, but this RFD's schema is intended to be
  reusable as the request contract for such a server.

## Risks and Open Questions

1. **schemars compatibility with schematic types.** Several CLI fields use types
   from `jp_config` that are derived via schematic's `Config` macro.
   These types have `Serialize`/`Deserialize` but may not have `JsonSchema`.
   Each needs to be checked — adding `JsonSchema` to schematic-derived types
   may require `#[schemars(with = "...")]` annotations or wrapper types.

2. **Schema stability.** Once external consumers depend on the schema, breaking
   changes become costly.
   The `schema_version` mechanism helps, but the initial schema should be
   designed carefully.
   Starting with `Query` as the first fully-schema'd command (the primary use
   case for LLM callers) allows iteration before committing to the full surface
   area.

3. **schemars version.** The project uses `schemars` 1.0.0-alpha.17.
   If the alpha API changes before 1.0 stable, derives and custom impls may need
   updating.

4. **Enum representation for `Commands`.** Serde's default externally-tagged
   enum produces `{"variant_name": {fields}}`.
   This is clean but the exact tag name and nesting for subcommands (e.g.
   `{"conversation": {"ls": {fields}}}`) needs to be validated against what
   schemars generates and whether LLMs handle nested discriminators well.

5. **Negative flags.** CLI has `--no-edit`, `--no-reasoning`, `--no-tool-use`,
   etc. In JSON, these are better expressed as the positive field set to `false`
   (e.g.
   `"edit": false` instead of `"no_edit": true`).
   The serde representation needs to decide whether to mirror the CLI naming or
   optimize for JSON ergonomics.
   Mirroring CLI naming is the default (since the structs are the source of
   truth), but `#[serde(rename)]` can alias where it matters.

6. **Triple-state clap interaction.** Clap's `Option<Option<T>>` has special
   derive macro support that a replacement type may not receive automatically.
   The exact bridging strategy (new type vs serde attributes vs hybrid) needs
   prototyping.

## Implementation Plan

### Phase 1: Bridging Types

Create the types and annotations that command structs depend on:

- Triple-state handling for `Option<Option<T>>` fields (prototyping needed to
  determine: `TriState<T>` newtype, per-field `#[serde(deserialize_with)]`, or
  hybrid approach)
- `ExternalCommand` struct with `FromIterator<String>`, `Serialize`,
  `Deserialize`, `JsonSchema`
- `Deserialize` + `JsonSchema` impls for `FromStr`-based types
  (`ConversationTarget`, `AttachmentUrlOrPath`, `WorkspaceIdOrPath`, `Editor`,
  `KvAssignment`, `ExpirationDuration`)
- `Deserialize` + `JsonSchema` impls for manual-clap types (`ToolDirectives`,
  `FlagIds`, `PositionalIds`)

Can be reviewed and merged independently.
No user-visible behavior change.

### Phase 2: Derive Additions — `Query` and `Globals`

Add `Serialize`, `Deserialize`, `JsonSchema` to `Cli`, `Globals`, `RootOpts`,
`Commands`, and `Query`.
This covers the primary use case (LLM callers invoking `jp query`).

Validate that `schemars::schema_for!(Cli)` produces a usable schema.
Write round-trip tests: construct a `Query` from JSON, verify it produces the
same `PartialAppConfig` as the equivalent CLI args.

### Phase 3: `--args-schema` and `--args-json`

Add the pre-parse detection for both flags.
`--args-schema` generates and prints the schema with embedded `schema_version`
and `jp_version` metadata, with optional command-path pruning of the schema AST.
`--args-json` deserializes the input via the envelope struct, validates schema
version if present, and dispatches into `run_inner`.

These two flags are shipped together — the schema defines the contract, the
input consumes it.

### Phase 4: Remaining Commands

Add derives to the remaining command structs incrementally: `Conversation` (and
all subcommands), `Config`, `Attachment`, `Plugin`, `Init`.

Each subcommand tree can be a separate PR.

### Phase 5: External Plugin Schema (depends on [RFD D19])

Extend `PluginArg` with `value_type`.
On `--args-schema`, query installed plugins via the `Help` protocol and inject
their schemas into the `External` variant.

## References

- [RFD 049: Non-Interactive Mode and Detached Prompt Policy][RFD 049]
- [RFD 059: Shell Completions and Man Pages][RFD 059]
- [RFD 072: Command Plugin System][RFD 072]
- [RFD D19: Structured Plugin Help Protocol][RFD D19]

[RFD 049]: 049-non-interactive-mode-and-detached-prompt-policy.md
[RFD 059]: 059-shell-completions-and-man-pages.md
[RFD 072]: 072-command-plugin-system.md
[RFD D19]: D19-structured-plugin-help-protocol.md
