# Deferred & Future Items

> Items explicitly out of scope for the current implementation phases.
> Collected here for roadmap visibility and to preserve design context.

---

## Architectural Direction (Future)

Long-term goals documented for vision alignment. Not planned for implementation now. See [future-crate-first-architecture.md](future-crate-first-architecture.md) for the full design rationale.

### Crate-First Architecture

**Status**: Future vision — Phase 6 is the first step on this path.

Invert the current loading-driven architecture into a crate-driven one with three cleanly separated phases: **Collect** (resolve deps, fetch remote, gather all blueprints) → **Build** (construct one crate, optionally strip to transitive closure) → **Load** (same `load_from_crate()` on submitter and worker).

### Crate Stripping (Transitive Closure)

**Status**: Future — depends on Phase 6a's blueprint collector.

From the entry pipe, walk the transitive closure of pipe and concept dependencies to determine the minimal subset needed. Strip the crate to only those concepts, pipes, and domains. Once all blueprints (main + deps) are accumulated in 6a, stripping is a pure data transform before crate construction.

### Library Fingerprint Validation

**Status**: Future — no immediate need.

Verify that the worker's base library (PIPELEXPATH) matches the API's expectation. Detect version drift between API and worker deployments.

### Cross-Worker Cache via Shared Storage

**Status**: Future — complementary to crate stripping.

Cache loaded crates in shared storage (Redis, S3) so multiple workers don't redundantly load the same crate. Keyed by fingerprint. Smaller crates from stripping make this more effective.

---

## Distributed Tracing — Deferred Items

Items related to the event log and graph tracing system (Phase 4.5). Also documented in [distributed-tracing-and-reporting.md](distributed-tracing-and-reporting.md).

### Event Log Backends

| Item | Status | Context |
|---|---|---|
| DynamoDB backend | **Shipped** (2026, `feature/dynamodb-tracer`) | `pipelex/tracing/dynamodb_event_log.py` (+ `TEMPORAL_DYNAMODB` variant). Schema-compatible with `pipelex-api-infra`'s `TraceEventDynamoDBAdapter`. Selected via `[pipelex.tracing] backend = "dynamodb"`. |
| SQLite backend | **Not planned** — NDJSON + DynamoDB cover the matrix | Listed in the original analysis (`distributed-tracing-and-reporting.md`); not built. Re-evaluate only if a use case appears that NDJSON can't serve and DynamoDB is overkill. |

### Event Log Protocol Extensions

| Item | Status | Context |
|---|---|---|
| `subscribe()` on EventLogProtocol | **Deferred** — no consumer exists yet | Polling-based reads are sufficient for now; add when a real-time consumer exists. |
| Real-time progress consumer | **Deferred** — no consumer exists yet | The protocol supports it; build when needed. |
| TraceSinkProtocol abstraction | **Deferred** — EventLogProtocol is sufficient | The higher-level sink abstraction can wait for the cloud backend. |
| Large data externalization | **Deferred** — inline data is sufficient for now | Add StorageProvider offloading when traces grow too large. |

### Tracing Architecture

| Item | Status | Context |
|---|---|---|
| Unified code path (event log everywhere) | **Effectively unified** | `pipeline_run_setup.py:282` wires `set_event_log()` on the report delegate for direct mode too (using `workflow_id="direct"`). The same `EventLogProtocol` machinery runs in both modes; only the in-process vs. Temporal lifecycle differs. |
| Auto-cleanup of traces | **Deferred** — manual cleanup is safer for now | NDJSON traces persist on disk until manually deleted. Safer for debugging and allows usage reporting to run after graph generation. DynamoDB cleanup follows TTL on the table. |

### Known Issues

| Item | Status | Context |
|---|---|---|
| Causal event ordering in assembler | **Known limitation** — works for current topologies | `read_events()` sorts by `(workflow_id, sequence)` which groups by lexicographic workflow ID, not execution order. In parent/child workflow topologies, this can cause `_stuff_producer_map` overwrites in the wrong order during `GraphSpecAssembler.pass_one()`, producing incorrect DATA edge sources. Consider sorting by timestamp or processing events in a topology-aware order. Flagged by codex on PR #796. |
| PipeCondition SELECTED_OUTCOME wiring | **Deferred** — no production use yet | PipeCondition has no tracer calls in production code currently. If SELECTED_OUTCOME edges are needed, that's a separate change. |

---

## Schema Reconstruction Hardening

Items related to `SchemaToModelFactory` (`pipelex/cogt/content_generation/schema_to_model_factory.py`), which reconstructs Pydantic BaseModel classes from JSON schemas across process boundaries (Temporal payloads). The current implementation `exec()`'s code generated by `datamodel-code-generator` and ships that source verbatim as `__kajson_class_source__` so receivers can re-exec it. See the module docstring for the full threat model.

### `exec()` → `pydantic.create_model()` Migration

**Status**: Deferred — non-trivial refactor, no immediate threat-model trigger.

Replace `exec()` in `_exec_source_to_types` with programmatic class construction via `pydantic.create_model()`. Eliminates the exec primitive on both the sender path (`make_from_json_schema`) and the receiver path (`make_types_from_source`), and lets us delete `_make_restricted_builtins`, `_restricted_import`, and `_reject_unsafe_schema_extensions`.

Non-trivial because nested models, enums, and forward references currently rely on `datamodel-code-generator`'s output structure. A faithful port has to walk the JSON schema directly, build the class graph in dependency order, and apply `model_rebuild()` after the fact to resolve forward refs.

This is the right structural fix; everything below is a stopgap until it lands.

### Receiver-Side Schema Validation Gap

**Status**: Known gap — documented in `make_types_from_source`'s docstring; only worth closing if the migration above is materially delayed.

`make_from_json_schema` (sender) calls `_reject_unsafe_schema_extensions` before generating source — Layer 1. `make_types_from_source` (receiver) takes pre-generated source and execs it directly — only Layer 2 (`_make_restricted_builtins`) applies. An attacker who can inject a crafted `__kajson_class_source__` string into a Temporal payload bypasses Layer 1 entirely.

Interim mitigation: AST-scan the source before exec, rejecting any top-level node that isn't an allowlisted import, a class def, or a docstring (and any class body that isn't fields / `model_config` / docstring). Mirrors Layer 1's "reject loudly" failure mode at the structural level. Trade-off: tied to `datamodel-code-generator`'s emitted output shape, which is not a stable API. Skip if the `create_model()` migration is near-term; the migration closes the gap permanently.
