# v0.3.x — LLM Provider Contract

> Status: contract-only (no implementation wiring yet)
> Pinned to: `core/reasoning/backends/__init__.py` (L0 — registry exists, L1 wiring pending)
> Version: `v0.3.x-1` (SemVer; breaking changes bump the leading number)
> Implementations on `main` at time of publish: `OllamaLocalBackend`, `ClaudeCodeCLIBackend`
> Companion documents:
>   - `docs/handovers/v0.3.x-ali-collaboration-track.md` (Track 1 of the collaboration)
>   - `docs/design/v0.3-gemma4-variant-3x3-eval-plan.md` (the experiment that consumes this contract)
>   - `docs/ARCHITECTURE.md` §5.7.2 (the architectural rule this contract formalizes)

## Why this contract exists

JAMES is moving from a single hardcoded LLM path (`engine.llm.call_gemma(...)` everywhere) to a **named-adapter pattern** where every LLM call goes through a registered backend. The architectural rule, from `docs/ARCHITECTURE.md` §5.7.2:

> _LLM 호출은 명명된 어댑터를 통해서만 발생, 미들웨어는 모델 SDK 직접 import 금지. 새 백엔드는 registry 만 확장, schema (core/reasoning/trace_schema.TraceStep) 는 확장 안 함._

Translated: **call sites use names, not SDKs**. The middleware layer doesn't import OpenAI or Google or Anthropic SDKs; it asks the registry for "ollama_local" or "claude_code_cli" or "gemini_api" and gets back a uniform object.

This contract document is what an external contributor (e.g. someone wiring up Gemini API, Claude API, OpenAI, Mistral, etc.) reads to ship a working backend against `main`. The contract is intentionally narrow — once your backend honors it, your code can drop into JAMES with no core-side changes.

## The minimum surface

A backend is anything that satisfies this Protocol (already in `core/reasoning/backends/__init__.py`):

```python
@runtime_checkable
class Backend(Protocol):
    backend_id: str

    def complete(
        self,
        prompt: str,
        *,
        system: str = "",
        max_tokens: int = 1024,
        timeout: float = 60.0,
        **opts,
    ) -> CompletionResult: ...
```

That's it. One attribute (`backend_id`) and one method (`complete`).

The `CompletionResult` dataclass is the response shape:

```python
@dataclass(frozen=True)
class CompletionResult:
    text: str
    backend_id: str
    model: str = ""
    latency_ms: int = 0
    error: str = ""
```

## Required behavior

These are the rules a conforming backend follows. They are tested by the conformance suite (see "Conformance tests" below).

### R1 — Never raise on the user-facing path

If the upstream model returns an error, a quota cap fires, the network drops, or anything else goes wrong, the backend **returns a `CompletionResult` with `error="..."` populated and `text=""`**. The caller in `core/reasoning/pipeline.py` is responsible for deciding what to surface to the user.

Why: matches the existing `RouterWrapper.call_gemma` pattern in `core/reasoning/engine.py`, which uses `_LLM_ERROR_PREFIXES` to detect error strings inline. JAMES's user-facing path never sees an uncaught exception from the LLM layer.

### R2 — `backend_id` matches the registry name

If the operator registers an instance as `gemini_api`, the instance's `backend_id` is the string `"gemini_api"`. This is checked at registration time and again at the conformance test.

Why: trace records (`core/reasoning/trace_schema.TraceStep`) include `backend_id` for each stage, and the audit pipeline ingests it directly. Mismatched names create unreplayable traces.

### R3 — `latency_ms` measures end-to-end wall clock

From the moment `.complete(...)` is entered to the moment the `CompletionResult` is returned, including any retries the backend does internally. Measured in milliseconds, integer.

### R4 — Reserved keyword arguments

The following kwargs are reserved by the contract. If a backend's underlying model doesn't support a given argument, the backend either translates it (e.g. ollama's `num_predict` from `max_tokens`) or ignores it silently — it does not raise:

| Reserved kwarg | Type | Meaning | Backend obligation |
|---|---|---|---|
| `system` | `str` | System prompt content | Either prepend to user prompt with `"\n\n"` separator (Ollama pattern) or pass to the model's system slot (OpenAI chat-completions pattern) |
| `max_tokens` | `int` | Max tokens to emit | Cap at this or the model's hard ceiling, whichever is smaller |
| `timeout` | `float` | Seconds to wait | Hard wall — if exceeded, return `error="timeout"`, do not raise |
| `model` | `str` (optional) | Model variant inside the backend (e.g. `"gemma4:e4b"` for ollama) | If supported, switch; if not, ignore and use the backend's default |
| `temperature` | `float` (optional) | Sampling temperature | If supported, apply; if not, ignore. **This is the Gemma 4 3×3 experiment's swept variable** — backends used in that experiment must accept it |
| `use_cache` | `bool` (optional) | Whether the backend's own cache layer should be consulted | If the backend has no cache, ignore |

Backends are free to accept *additional* kwargs via `**opts`; they just can't refuse the reserved ones.

### R5 — No SDK leakage upward

The backend module is the only place in the JAMES codebase that imports the underlying model SDK. `core/reasoning/`, `core/retrieval/`, `core/graph/`, `core/policy_engine.py`, `core/security_layer.py` — none of these contain `import openai` or `import google.generativeai` or equivalent. Architectural test enforces this in `tests/test_no_sdk_leakage.py` (added when the conformance suite lands).

### R6 — Stateless or explicitly-stateful

A backend is either **stateless** (every call is independent) or **explicitly stateful** with documented invariants (e.g. cache between calls, retry counter, sticky session ID). If stateful, the state must be documented in the backend module's docstring and observable via a `reset()` method (the conformance test calls `reset()` between runs).

## Registration

There are two ways to register:

### 1. Direct registration (current)

In any module that's imported at server startup, call `register_backend(name, instance)`:

```python
# Example: my_plugins/gemini_api_provider.py
from core.reasoning.backends import register_backend, CompletionResult

class GeminiApiBackend:
    backend_id = "gemini_api"
    def __init__(self, api_key: str):
        self._api_key = api_key

    def complete(self, prompt, *, system="", max_tokens=1024, timeout=60.0,
                 model="gemini-1.5-pro", temperature=None, **opts):
        # ... call Gemini API here ...
        return CompletionResult(text="...", backend_id=self.backend_id,
                                model=model, latency_ms=...)

# At import time (idempotent re-registration is OK with the same instance):
register_backend("gemini_api", GeminiApiBackend(api_key=os.environ["GEMINI_API_KEY"]))
```

Then ensure the module is imported during server startup — either by adding it to `server_llmwiki.py`'s import block, or by listing it in the `JAMES_PLUGINS` env var (see below).

### 2. `JAMES_PLUGINS` env var (v0.3.x deliverable)

The plugin loader reads a comma-separated list of module paths from `JAMES_PLUGINS` and imports each at startup. This lets an operator wire a custom backend without editing JAMES source:

```bash
# Hypothetical Gemini-API-enabled deployment
export JAMES_PLUGINS="my_plugins.gemini_api_provider,my_plugins.local_cache"
python server_llmwiki.py
```

Each module in the list is imported normally; if it has a top-level `register_backend(...)` call, the backend is now available. Failures during plugin import are **fatal** — a plugin that doesn't load shouldn't silently degrade the server.

Reserved env-var names for cooperation with backend modules:

- `JAMES_LLM_MODEL` — default model identifier (already exists, `core/config.py`)
- `JAMES_LLM_TEMPERATURE` — default temperature (to be added in the wiring PR; backends should accept it via the `temperature` kwarg above)
- `JAMES_REASONING_BACKEND` — global backend selection (already exists, PR #305)
- `JAMES_PLUGINS` — plugin discovery (to be added in this v0.3.x cycle)

## Selection — which backend gets called

At the call sites in `core/reasoning/pipeline.py` and `core/reasoning/pipeline_synth.py`, the L1 wiring replaces `engine.llm.call_gemma(...)` with `get_backend(name).complete(...)`. The `name` is resolved per stage:

1. **Per-stage env override**: `JAMES_BACKEND_SYNTH=gemini_api` overrides `JAMES_REASONING_BACKEND` for the synthesis stage only. Per-stage names map to call-site stages (`auth`, `retrieve`, `graph`, `synth`, `verify`, `reflect`, `plan`, `rewrite`).
2. **Global default**: `JAMES_REASONING_BACKEND` from PR #305. Used at any stage that doesn't have a per-stage override.
3. **Hardcoded default**: `"ollama_local"`. The L0 default; always registered.

The 3×3 Gemma 4 experiment runs by setting `JAMES_BACKEND_SYNTH=ollama_local` for the synthesis layer specifically (with `JAMES_LLM_MODEL` cycling through `gemma4:e4b`, `gemma-4-26b-a4b-it`, `gemma-4-31b-it`), while the conditioning stages stay on whatever the operator's default is.

## Conformance tests (the backend's job before merge)

External backends must pass a small conformance suite before being eligible for inclusion in the JAMES handbook of registered backends. The suite — to be added at `tests/test_backend_conformance.py` in this v0.3.x cycle — checks:

1. **Type**: `isinstance(backend, Backend)` is `True`
2. **R1**: Calling `.complete(...)` against an intentionally-broken upstream (e.g. invalid API key) does not raise; it returns `CompletionResult` with `error` set
3. **R2**: `backend.backend_id == name_registered_under`
4. **R3**: `result.latency_ms >= 0` (and is plausibly large for a real network call)
5. **R4**: All reserved kwargs are accepted without raising — at minimum, `system`, `max_tokens`, `timeout`. `temperature` is required for backends used in the 3×3 experiment
6. **R5**: `import tests.test_no_sdk_leakage` does not flag the backend module unless the import is inside the module itself (the contract permits SDK use *inside* the backend module; what it forbids is SDK leakage *upward* into middleware)
7. **R6**: Stateless backends produce the same result for the same inputs (modulo the model's own stochasticity); stateful backends document the state and expose `reset()`

A skeleton conformance suite is shipped as part of this v0.3.x cycle. Until it lands, the L0 backends (`ollama_local`, `claude_code_cli`) are de-facto reference implementations — read those.

## Implementer's quick-start (Gemini API example sketch)

```python
# tests/fixtures/example_gemini_backend.py (illustrative — not in main)
import os, time
from core.reasoning.backends import CompletionResult, register_backend

class GeminiApiBackend:
    backend_id = "gemini_api"

    def __init__(self) -> None:
        # Lazy import — keep the SDK out of the module's import side-effects
        # so test harnesses can mock the backend without google-generativeai
        self._client = None

    def _ensure(self):
        if self._client is None:
            import google.generativeai as genai
            genai.configure(api_key=os.environ["GEMINI_API_KEY"])
            self._client = genai

    def complete(self, prompt, *, system="", max_tokens=1024, timeout=60.0,
                 model="gemini-1.5-pro", temperature=None, **opts):
        self._ensure()
        composed = f"{system}\n\n{prompt}" if system else prompt
        t0 = time.monotonic()
        try:
            m = self._client.GenerativeModel(model)
            cfg = self._client.types.GenerationConfig(
                max_output_tokens=max_tokens,
                temperature=temperature,
            )
            resp = m.generate_content(composed, generation_config=cfg,
                                      request_options={"timeout": timeout})
            text = resp.text or ""
            return CompletionResult(text=text, backend_id=self.backend_id,
                                    model=model,
                                    latency_ms=int((time.monotonic() - t0) * 1000))
        except Exception as exc:
            return CompletionResult(text="", backend_id=self.backend_id,
                                    model=model, error=f"{type(exc).__name__}: {exc}",
                                    latency_ms=int((time.monotonic() - t0) * 1000))

register_backend("gemini_api", GeminiApiBackend())
```

That ≈ 30 lines satisfies the contract.

## Versioning

This contract is `v0.3.x-1`. Future revisions:

- **Patch** (`v0.3.x-2`, `v0.3.x-3`): clarifications, additional reserved kwargs that backends are allowed to ignore, new conformance tests that the reference backends pass without change. Backwards-compatible.
- **Minor** (`v0.4.x-1`): new required behavior that existing backends must adopt (e.g. streaming support, token-by-token callbacks). Migration note shipped alongside.
- **Major** (`v1.0.0-1`): breaking changes to the Protocol surface — e.g. `complete` becomes `async def complete`. Coordinated with a JAMES major version bump.

The contract document itself lives at this path on `main`. Any change to the contract is a labeled diff entry at the bottom of this file, with the date and PR number — silent edits are not allowed once external backends exist in the wild.

## Where this contract does *not* apply

- **Multi-modal extensions** (`call_gemma_vision` in `llm/router.py`). Vision and audio backends will be a separate contract surface in v0.4 — the 1-D-text-in / 1-D-text-out Protocol above is intentionally narrow.
- **Streaming**. `complete(...)` returns one `CompletionResult` after the full response is buffered. Stream-token-by-token will be a v0.4 minor revision.
- **Function calling / tool calls.** The middleware layer (planner, tool router, reflection) handles tool dispatch via JSON outputs from `complete(...)`. Backend-native function-calling APIs (OpenAI tool calls, Anthropic computer use) are not first-class until v0.4.
- **The router-level cache** (`llm/router.py::RouterWrapper.get_cache_stats`). That's an artifact of the existing single-LLM path; in the multi-backend world, caching is the backend's responsibility (if it has one) or the application's (in `core/cache_manager.py`).

## Reference implementations on `main` at publish

- [`core/reasoning/backends/ollama_local.py`](../../core/reasoning/backends/ollama_local.py) — wraps `RouterWrapper.call_gemma`. Stateless. Always registered.
- [`core/reasoning/backends/claude_code_cli.py`](../../core/reasoning/backends/claude_code_cli.py) — spawns the `claude` CLI subprocess. Stateful (subprocess handle). Registered when `JAMES_ENABLE_CLAUDE_BACKEND=1`.

A new backend follows the shape of either — depending on whether it talks to a local daemon (like ollama) or invokes an external process (like Claude CLI).

## Diff log

(Add entries below as the contract evolves.)

| Date | Author | Change | PR |
|---|---|---|---|
| 2026-05-18 | Jiwon | Initial publication (v0.3.x-1), contract-only, no L1 wiring yet | #316 (this PR) |
