# v0.3 — Plugin API Skeleton (Track C)

> Status: design-only (no code yet for the four non-backend plugin types)
> Existing partial implementation: `core/reasoning/backends/_load_plugins`
> (Track 1 PR-C, #326) covers the **backend** plugin path; this document
> generalizes the same pattern to the four pack-level plugin types.
> Tracks: `docs/handovers/session-2026-05-09-license-infrastructure.md`
> Track C (sections C-1 through C-11).
> Companion documents:
>   - `docs/PLATFORM_READINESS.md` Gate v0.3
>   - `docs/ARCHITECTURE.md` §5.7 (cognitive middleware) — plugins layer
>     on top of cognitive primitives, not inside them
>   - `docs/LICENSE_PLAN.md` §5.2 (`license:` field in manifest)
>   - `core/reasoning/backends/__init__.py` (the reference loader)

## Why this exists

JAMES is approaching the **v0.3 → v0.4 gate** of
`docs/PLATFORM_READINESS.md`. The gate's done-when conditions (1 and 2)
require:

1. A new contributor can build a no-op pack in under a day, load it
   via `JAMES_PLUGINS=…`, and observe its effect.
2. `packs/general/` exists, runs `STEP 7` byte-identical to v0.2 main,
   and **fails cleanly** when removed (the dogfood gate).

Neither is currently met. PR #326 shipped `_load_plugins` for the
LLM-backend slot only — that's the contract the Track 1 Provider work
needed. The four other plugin slots (ontology, prompts, UI panels,
scoring) still have **no extension surface**: every default behavior
lives directly inside `core/` and is hardcoded.

This document fixes the surface so a future v0.4 domain pack (legal,
food, retail, …) can land without touching `core/`.

## Position in the architecture

```
┌──────────────────────────────────────────────────────────────┐
│  packs/                    user-installed plugins            │  ← NEW
│  ├─ general/              dogfood pack (always loaded)        │
│  └─ <domain>/             optional domain pack (v0.4+)        │
├──────────────────────────────────────────────────────────────┤
│  core/plugins/             plugin contract + loader           │  ← NEW
│  ├─ base.py               4 Protocol types                    │
│  ├─ loader.py             JAMES_PLUGINS env, dogfood gate     │
│  ├─ manifest.py           pack.yaml schema + license: field   │
│  ├─ registry.py           in-memory registry                  │
│  └─ errors.py             PluginLoadError, PluginVersionError │
├──────────────────────────────────────────────────────────────┤
│  core/reasoning/, core/retrieval/, core/graph/, …             │
│  the cognitive + retrieval + security stack (unchanged)       │
└──────────────────────────────────────────────────────────────┘
```

**The plugin contract layer is above cognitive middleware, not inside
it.** A pack supplies content (ontology terms, prompts, panels,
scoring weights) — it does not rewrite the reasoning loop. That
discipline is what keeps the v0.3 contract from leaking domain-pack
complexity back into `core/`.

## The four plugin types

Pinned to `docs/handovers/session-2026-05-09-license-infrastructure.md`
§C-2 with the typed shape JAMES's existing module conventions need:

```python
# core/plugins/base.py
from typing import Protocol, runtime_checkable

@runtime_checkable
class OntologyPack(Protocol):
    """Entity types, relation types, hierarchies. Read at startup, used
    by retrieval / graph / reasoning. Lookup cost is O(1) on dict.
    """
    pack_id: str
    entity_types: tuple[str, ...]
    relation_types: tuple[str, ...]
    hierarchies: dict[str, tuple[str, ...]]   # parent → children

@runtime_checkable
class PromptPack(Protocol):
    """System prompts + few-shot examples per task. Returned strings
    are concatenated into the existing prompt builder in
    core/reasoning/modes/.
    """
    pack_id: str

    def system_prompt(self, mode: str) -> str:
        """Return the system prompt for one of the 5 modes
        ('chat', 'meta', 'wiki_edit', 'self_evolve', 'coding').
        Unknown mode returns the empty string (graceful)."""
        ...

    def few_shot(self, task: str) -> list[dict]:
        """Optional few-shot examples for a named task. Empty list
        means 'no examples'; the existing prompt path stays unchanged.
        """
        ...

@runtime_checkable
class UIPanel(Protocol):
    """Server-rendered admin/user widgets. The HTML returned by
    render() is sandboxed at the response-filter layer
    (core/security_layer) — packs cannot bypass that.
    """
    pack_id: str
    panel_id: str        # used in path: /panels/{pack_id}/{panel_id}

    def render(self, ctx: "PanelContext") -> str:
        """Return server-rendered HTML. The ctx exposes the current
        user_role + session_id; render() must respect the ABAC
        decisions in core/policy_engine.
        """
        ...

@runtime_checkable
class Scorer(Protocol):
    """Custom retrieval / answer scoring overrides. Plugged into the
    existing reranker / hybrid search slot.
    """
    pack_id: str

    def score(self, query: str, candidate: dict) -> float:
        """Return a [0.0, 1.0] score for the candidate's relevance to
        the query. Higher = more relevant. The existing scorer in
        core/retrieval/ stays the default — Scorer entries override
        only the slot they're registered against.
        """
        ...
```

A pack supplies **zero or more** of these. A pure ontology pack has
no PromptPack / UIPanel / Scorer. A pure UI pack has no ontology. The
loader registers each slot independently.

## Manifest — `pack.yaml`

Every pack has a top-level `pack.yaml` that the loader reads at startup.
The schema is pinned in `core/plugins/manifest.py`:

```yaml
# pack.yaml — schema v1
name: general                                # SPDX-safe slug; matches dir
version: 1.0.0                                # SemVer
james_api: ">=0.3,<0.4"                       # SemVer range against JAMES core
description: "Default JAMES behavior pack"
author: "Hashevolution"

# license: field per docs/LICENSE_PLAN.md §5.2. Allowed values are an
# explicit closed enum during v0.3 — the v1.0 SDK widens this.
license: "MIT"                                # MIT | Apache-2.0 | AGPL-3.0 | proprietary
                                              # proprietary warns at load
                                              # time + requires the
                                              # commercial-license token
                                              # block (not enforced in
                                              # v0.3; logged only)

# Each declared slot maps "slot:" → "import.path:ClassName".
# All four slots are optional. A pack with zero slots is legal but
# useless and the loader warns once.
plugins:
  ontology: ontology:GeneralOntology
  prompts: prompts:GeneralPrompts
  ui:
    - ui.search:SearchPanel
    - ui.graph:GraphPanel
  scorers:
    - scoring.recency:RecencyScorer
```

**Why YAML over TOML/JSON**: pyyaml is already a transitive dep, and
the keys map naturally to a Python dict. JSON-with-comments would
work but adds a custom parser.

**Why a closed `license:` enum during v0.3**: the validation surface
should be testable. Once the v1.0 SDK ships with the plugin marketplace
in mind, the enum is widened to "any SPDX identifier" with an allowlist
in `docs/LICENSE_PLAN.md`.

## Loader semantics

`core/plugins/loader.py` orchestrates startup. Pseudocode of the happy
path:

```python
def load_packs_from_env() -> dict[str, Pack]:
    raw = os.environ.get("JAMES_PLUGINS", "general").strip()
    if not raw:
        raise PluginLoadError(
            "no pack loaded; set JAMES_PLUGINS=general (or another pack name)"
        )
    pack_names = [p.strip() for p in raw.split(",") if p.strip()]
    loaded = {}
    for name in pack_names:
        pack_dir = ROOT / "packs" / name
        if not pack_dir.is_dir():
            raise PluginLoadError(
                f"pack {name!r} not found at packs/{name}/"
            )
        manifest = read_manifest(pack_dir / "pack.yaml")
        check_semver(manifest.james_api)   # raises PluginVersionError
        pack = import_pack(pack_dir, manifest)
        register_slots(pack, manifest.plugins)
        loaded[name] = pack
    return loaded
```

Three failure modes — all **fatal at startup**, no silent fallback:

1. `JAMES_PLUGINS=` (empty) → `PluginLoadError("no pack loaded …")`.
2. `JAMES_PLUGINS=general` but `packs/general/` missing → as above.
3. `pack.yaml` declares `james_api: ">=0.4"` against a v0.3 core →
   `PluginVersionError` with the exact mismatch.

The dogfood gate (§C-10) verifies these three lines.

**Env default**: if `JAMES_PLUGINS` is *unset*, the loader treats it as
`general` (the dogfood default). Empty string vs unset is meaningful —
unset means "operator made no choice, load the default"; empty means
"operator explicitly asked for no packs", which is an error.

## Relationship to PR #326's `_load_plugins`

PR #326 already ships a working plugin loader for one slot type:
**backend** (LLM provider). It reads `JAMES_PLUGINS` as a list of
import paths, imports each, and lets the module register itself via
`register_backend(...)`. Failures are fatal.

This design generalizes the same pattern to the four pack-level slots:

- PR #326 reads `JAMES_PLUGINS=my_plugins.gemini_api_provider,…`
  (Python module paths).
- This design reads `JAMES_PLUGINS=general,legal,…` (pack names that
  map to `packs/<name>/`).

The two env-var semantics **need to reconcile** — see Open Question 1
below. The most likely resolution is a separate env name
(`JAMES_BACKEND_PLUGINS` for backends, `JAMES_PLUGINS` for packs).
PR #326's existing surface is in the field already, so the rename
costs more than introducing a new env name for the new layer.

## Dogfood gate

The §C-5 contract: `packs/general/` extracted from current JAMES
behavior such that **removing it breaks the server cleanly**.

`scripts/dogfood_check.py` runs three positive + three negative cases:

| Scenario | Expected outcome |
|---|---|
| `unset JAMES_PLUGINS && server` | Normal boot (general auto-loaded) |
| `JAMES_PLUGINS=general && server` | Normal boot (explicit, same) |
| `JAMES_PLUGINS= && server` | RefusedToStart with empty-env error |
| `JAMES_PLUGINS=general && rm -rf packs/general && server` | PluginLoadError, exit non-zero |
| `JAMES_PLUGINS=nonexistent && server` | PluginLoadError, exit non-zero |
| `JAMES_PLUGINS=general && server && pytest STEP_7` | Byte-identical to v0.2 main numbers |

The last one is the **dogfood gate**: `packs/general/` must produce
exactly the same retrieval/answer behavior as v0.2 main did. Drift
here means the extraction was lossy, and the v0.2 → v0.3 promise
breaks.

## Eval contract (§C-9)

When a `packs/*/` file changes in a PR, CI runs:

1. `scripts/bench.py --suite=step7 --check` — STEP 7 regression
2. RAGAS retrieval / faithfulness / answer-relevance against the
   pack's declared eval suite (`packs/<name>/eval/`)

Both must stay within tolerance against the committed baseline (the
`step7_baseline.json` v5 from #330; RAGAS thresholds named in the
pack's own `pack.yaml::eval` block). A regression blocks merge —
same `--check` exit-code pattern the rest of `scripts/bench.py`
already uses.

**The Eval Contract is the structural reason domain packs do not
land before v0.3** — without enforcement, a pack can ship that
silently degrades the mother's numbers, and the regression appears
in v0.4 customer feedback rather than at merge.

## `JAMES_WORKSPACE=` env (§C-6) — landed in PR-C6 + PR-C6.b

Multi-instance hosting (one process per workspace) is a v0.4
operator option per `docs/ARCHITECTURE.md` §5.7.7. The env var
(`JAMES_WORKSPACE=`) shipped in two PRs:

- **PR-C6 (#410, 2026-05-23)** — resolver only
  (`core/plugins/workspace.py::get_workspace_root`). Defines the
  semantics (unset / empty → BASE_DIR; absolute / relative;
  non-existent → `PluginLoadError`).
- **PR-C6.b (#421, 2026-05-23)** — `config.py` consumes the resolver
  for `RAW_DIR` / `WIKI_DIR` / `UPLOAD_DIR` / `CHROMA_DIR`. With the
  env unset the data dirs anchor to the project root, byte-identical
  to the pre-v0.3 codepath. With `JAMES_WORKSPACE=/some/dir` the
  four data dirs follow the workspace while `BASE_DIR` (project
  root, used for code/asset lookups) stays anchored.

## Test plan

For PR-C2 (`base.py` + tests), no integration; just shape checks:

1. Each Protocol is `@runtime_checkable` and `isinstance(obj, P)` works
2. A minimal class implementing each Protocol satisfies it
3. A class missing one required field/method fails the isinstance check

For PR-C3 (`loader.py` + tests), with fixture packs under
`tests/fixtures/plugins/`:

4. `JAMES_PLUGINS` unset → `general` auto-loaded
5. Empty string → `PluginLoadError`
6. Missing pack dir → `PluginLoadError` with the exact pack name
7. Manifest with `james_api: ">=0.4"` → `PluginVersionError`
8. Manifest with two packs → both loaded in order
9. Pack import that raises → `PluginLoadError` wraps + names the pack

For PR-C5 (`packs/general/` + dogfood):

10. `pytest tests/test_dogfood_step7.py` runs `step7_queries.json`
    against the general pack and locks the diff to zero.

For PR-C10 (`scripts/dogfood_check.py` automation):

11. CI runs `dogfood_check.py` on every PR touching `packs/*/`.

## PR sequence

| PR | Scope |
|---|---|
| **PR-C2** | `core/plugins/base.py` (4 Protocols) + `core/plugins/errors.py` + Protocol-shape tests. **No loader yet.** Wiring: zero. |
| **PR-C3** | `core/plugins/loader.py` + `core/plugins/manifest.py` + `core/plugins/registry.py`. Loader runs at startup, but with **only fixture packs** (`tests/fixtures/plugins/`). Production server still uses hardcoded defaults. |
| **PR-C5** | Extract current JAMES default behavior into `packs/general/`. Server startup now loads from `JAMES_PLUGINS=general`. Dogfood gate active. **Bench-required** PR (STEP 7 byte-identical confirmation). |
| **PR-C7** | `docs/PLUGIN_AUTHORING.md` — Quickstart, 4 type examples, manifest authoring, local test, eval requirements. |
| **PR-C8** | `docs/VERSIONING.md` — SemVer + 12-month deprecation policy for the Plugin API. |
| **PR-C9** | CI workflow that runs the eval contract on `packs/*/` PRs. |
| **PR-C10** | `scripts/dogfood_check.py` + CI hook. |

PR-C1 / C-4 / C-6 / C-11 from the handover map onto this sequence:
- C-1 (directory design) folds into PR-C2 as the package layout
- C-4 (manifest schema) lands in PR-C3
- C-6 (`JAMES_WORKSPACE=`) is a separate PR after C-5, not on this
  track
- C-11 (Knowledge Cascade Phase A parallel) — decided No: Knowledge
  Cascade already closed (#266-#274), no parallelism needed

## Open questions

1. **`JAMES_PLUGINS` collision** between the existing backend loader
   (PR #326, reads dotted module paths) and the new pack loader
   (reads pack names). Proposed resolution: rename the new env to
   keep #326's behavior backward-compatible.
   - Option A: backend env stays `JAMES_PLUGINS`, pack env is
     `JAMES_PACKS`.
   - Option B: backend env renamed to `JAMES_BACKEND_PLUGINS`, pack
     env is `JAMES_PLUGINS`.
   - **Recommendation**: Option A — #326 has been in the field for
     hours; renaming an env requires operator coordination. Document
     the resulting two-env surface in `docs/PLUGIN_AUTHORING.md`.
   - Decided in: PR-C3.

2. **Should `packs/general/` ship in this repo or a separate one?**
   Same-repo dogfooding (current handover assumption) means JAMES core
   PRs and pack PRs live together. v1.0 marketplace would want
   separation. Recommendation: same repo for v0.3, split at v1.0.

3. **Manifest signature hash** (§C-4 handover note). The current
   pack-author trust model is "you wrote it, you sign for it" — no
   hash verification. v1.0 marketplace would need it. Defer to v1.0
   (the SDK PR that ships the marketplace handshake).

4. **Plugin Eval suites — RAGAS thresholds per pack?** Each pack
   declares its own RAGAS thresholds in `pack.yaml::eval`. STEP 7 is
   the universal floor (every pack must pass mother numbers). RAGAS
   is the per-domain ceiling. Decided in: PR-C9.

## What this PR does NOT do

- Any code under `core/plugins/` — that lands in PR-C2 onward.
- Manifest yaml parsing — PR-C3.
- `packs/general/` extraction — PR-C5.
- `JAMES_WORKSPACE=` env wiring — separate PR after this track.
- Marketplace / signing — v1.0.

## Diff log

| Date | Author | Change | PR |
|---|---|---|---|
| 2026-05-19 | Jiwon | Initial design (Track C of v0.3 platform skeleton) | (this PR) |
