# Router and Dashboard State Taxonomy and Inventory

This document is the canonical inventory for restart-sensitive router and dashboard state.

Use it to answer three questions before adding or changing a stateful feature:

- what kind of state is this
- who owns it
- what durability and recovery behavior users should expect

## State Taxonomy

### `ephemeral_request_state`

- Per-request or per-connection working state that may live only in memory.
- Examples: extproc request context, in-flight streaming buffers, active SSE or WebSocket client maps, short-lived retry state.
- Losing this state on process restart is acceptable.

### `restart_safe_local_state`

- State that should survive a local process restart in one workspace or one node, but does not need to be shared across replicas by default.
- Examples: local-dev bootstrap outputs under `.vllm-sr/`, local auth/evaluation SQLite files, generated runtime config snapshots, local artifact directories.
- This is acceptable for local development and single-node tooling, but it is not a substitute for product durability in multi-user or multi-replica deployments.

### `shared_durable_workflow_state`

- State that users or operators expect to survive restart and remain authoritative across processes, containers, or replicas.
- Examples: user accounts, audit logs, workflow jobs, campaign state, chat history if presented as a product surface, vector-store metadata, file registries, response history, replay history, model-selection learning state.
- This state should live behind a server-owned storage contract, not browser storage or process memory.

### `audit_analytics_telemetry`

- Append-oriented facts about progress, lifecycle, usage, or recovery that operators need for debugging, audits, or reporting.
- Examples: typed workflow events, startup progress records, replay aggregates, deploy history, durable status transitions.
- This is distinct from request-local logs and must not depend on log scraping.

### `derived_projection_state`

- A query-oriented projection derived from a canonical source of truth.
- This state may be rebuilt, but it should still be persisted if operators or the dashboard rely on it.
- Examples: normalized tables for deployed models, signals, decisions, plugins, DSL snapshots, topology views, or current active config version.

## Source Of Truth Rules

- Keep canonical router intent in YAML and DSL, not in a second mutable primary database model.
- If the dashboard needs fast querying, filtering, audit, or joins, create a persisted projection from the active YAML or DSL rather than introducing dual-primary writes.
- Treat `.vllm-sr/` and `dashboard-data/` as local-dev adapters unless a feature is explicitly documented as single-workspace only.
- Keep live connection registries in memory, but move user-visible entities and workflow progress behind durable records.

## Current Inventory

| Surface | Primary owner | Current backend / default | Current durability class | Restart behavior today | Scale risk | Recommended direction |
| --- | --- | --- | --- | --- | --- | --- |
| Response API stored responses and conversations | router runtime, `src/semantic-router/pkg/responsestore/**` | Default `redis`; optional `memory` for local dev only | `shared_durable_workflow_state` | Response and conversation history survives restart when using the default Redis backend. The `memory` backend emits a startup warning and loses all data on restart. | Replica-local only when `memory` is explicitly selected; Redis backend is shared across replicas | Keep metadata and conversation chain in a durable server-owned store by default for product use. Prefer relational storage for metadata and queryability; keep large payloads in blob/object storage only if needed later. |
| Router replay records | router runtime, `src/semantic-router/pkg/routerreplay/**`, `src/semantic-router/pkg/extproc/router_replay_setup.go` | Default `postgres`; optional `redis`, `milvus`, `memory` (local dev only) | `audit_analytics_telemetry` with `shared_durable_workflow_state` default | Replay history survives restart when using the default Postgres backend. The `memory` backend emits a startup warning and loses all records on restart. | Postgres provides SQL queryability for audit and compliance. Redis available for lightweight deployments. | Keep metadata and replay records in a durable server-owned store by default. Postgres is the default for long-term audit retention and compliance. Keep Milvus only when semantic replay search is explicitly needed. |
| Semantic cache entries | router runtime, `src/semantic-router/pkg/cache/**` | Default `memory`; optional Redis, Milvus, hybrid | `ephemeral_request_state` in local dev; shared cache in scaled deploys | Restart flushes cache; replicas do not share hot entries by default | Cold-start latency, inconsistent cache hit rates, and uneven behavior across replicas | Keep this as cache, not a database table. Prefer Redis or hybrid shared backends for scaled deployments; document memory backend as local/dev or single-node only. |
| RAG retrieval result cache | router runtime, `src/semantic-router/pkg/extproc/req_filter_rag_cache.go` | Process-wide singleton in-memory LRU with TTL | `ephemeral_request_state` | Restart flushes cache; cache is global per process, not per tenant or replica | Hidden shared mutable state, no observability, no durability, and no multi-replica coherence | Keep as optional cache only. Move to a pluggable shared cache backend if this becomes performance-critical, or document as local process optimization. |
| Agentic memory vectors | router runtime via `routerruntime.Registry`, `src/semantic-router/pkg/memory/**` | Disabled by default; vector content leans on Milvus config when enabled | `shared_durable_workflow_state` when enabled | Depends on backend choice; not enabled by default. `InitWithRuntime` now resolves startup memory state only from the shared runtime registry; nil-registry callers still use the process-wide fallback. | Product semantics remain ambiguous between experimental memory and supported user data | Keep vector embeddings in Milvus or another vector store, but pair them with explicit metadata and lifecycle ownership in a durable server-owned contract. |
| Vector store collection registry | router runtime, `src/semantic-router/pkg/vectorstore/manager.go` | Pluggable `MetadataRegistry`: in-memory (default) or Postgres | `shared_durable_workflow_state` when `metadata_store: postgres`; otherwise ephemeral | With `metadata_store: memory`, collection metadata disappears on restart even if backend collections remain. With `postgres`, metadata survives restarts. API servers with a runtime registry resolve the vector manager, embedder, and ingestion pipeline only from the registry once present; process-wide globals are limited to nil-registry fallback callers. ExtProc vectorstore RAG resolves request-time manager/embedder dependencies only from the router runtime registry. | Set `vector_store.metadata_store: postgres` for durable registry; the CLI local runtime provisions Postgres and fills `metadata_postgres` defaults for this mode. A warning is emitted when `metadata_store` is `memory` with a durable backend (Milvus/Valkey/Qdrant). | Postgres registry tables: `vector_store_registry`, `file_registry`. Loaded on startup via `LoadFromRegistry`. |
| Vector store file registry | router runtime, `src/semantic-router/pkg/vectorstore/filestore.go` | File bytes on local disk; file metadata backed by `MetadataRegistry` | Mixed `restart_safe_local_state` and `shared_durable_workflow_state` | With `metadata_store: memory`, files may remain on disk while metadata vanishes on restart. With `postgres`, file metadata is durable. API servers with a runtime registry resolve the file store only from the registry once present; process-wide globals are limited to nil-registry fallback callers. | Set `vector_store.metadata_store: postgres` for restart-safe file metadata. File bytes remain on local disk. | Shares the same `MetadataRegistry` as the collection registry. |
| Startup readiness and model download progress | router runtime, `src/semantic-router/pkg/startupstatus/status.go`, `redis_writer.go`, `src/semantic-router/pkg/apiserver/route_startup_status.go` | Default `file`; recommended `redis` through `startup_status.store_backend` for production. Router exposes `GET /startup-status` and readiness consumes the same state resolver. | `audit_analytics_telemetry` with `shared_durable_workflow_state` when Redis backend is configured | Status survives restart and is shared across replicas when using the Redis backend. The `file` backend emits a startup warning and is not visible to the dashboard in containerized deployments. The API server starts before model downloads so `/startup-status` is available during the entire boot sequence, and `/ready` now reads the same Redis/file startup state. | File backend is replica-local and path-dependent; Redis backend is shared across replicas. Dashboard reads from `/startup-status` API first, falling back to file path. | Keep Redis as the recommended backend for containerized and multi-replica deployments. The `file` backend remains as a local-dev fallback. Status resolution uses only the `/startup-status` API and the file path. |
| Router Learning adaptation and protection state | router runtime via `global.router.learning.adaptation`, `global.router.learning.protection`, `src/semantic-router/pkg/extproc/router_learning_*.go`, and `routerruntime.LearningRuntime` | In-process runtime state for the first implementation; outcomes update model-targeted experience through the runtime registry | `ephemeral_request_state` for single-replica Router Learning states | Protection state preserves conversation/session continuity inside one router process. Adaptation state records model experience, reliability, latency, cache, cost, and outcome evidence in process. Missing state fails open and is recorded in learning diagnostics. | Multi-replica deployments need sticky routing for session/conversation protection; non-sticky replicas split online state and model experience. | Keep request-time reads local. For production multi-replica learning, add a designed shared-state seam with strict timeout, fail-open, and local fallback semantics. |
| Legacy Elo, RL-driven, and GMTRouter selector state | legacy selector implementations under `src/semantic-router/pkg/selection/**` | Historical local JSON or in-memory selector stores | Legacy only | Public `decision.algorithm.type: elo`, `rl_driven`, and `gmtrouter` config is removed from the clean Router Learning API and rejected by config loading. Reusable model-choice concepts belong in Router Learning adaptation, model experience, outcomes, and offline recipe learning. | Keeping these as public selector-owned stores would split learning state across algorithms and replicas. | Do not add new product behavior to legacy selector-local stores. Migrate reusable rating, reward, and personalization concepts into typed adaptation experience or offline recipe-learning artifacts. |
| Tools database | router integrations, `global.integrations.tools`, `config/tools_db.json` | JSON file path by default | `restart_safe_local_state` | Survives only as workspace file | Multi-user editing, audit, and HA are weak | If dashboard editing becomes first-class, add a durable metadata store or projection for tools; keep JSON as import/export and local-dev source. |
| Dashboard auth users, roles, permissions, audit logs | dashboard backend auth, `dashboard/backend/auth/store.go`; Helm persistence in `deploy/helm/semantic-router/templates/dashboard-pvc.yaml` | SQLite at `./data/auth.db` by default; Helm can mount dashboard-local state at `/app/data/auth.db` when `dashboard.persistence.enabled=true` | `restart_safe_local_state` | Restart-safe in one workspace or one mounted PVC; not shared across replicas. The Helm chart now fails template rendering when `dashboard.replicaCount > 1` because the current store is not an HA store. | Adequate for local stacks and one dashboard replica with a persistent volume; weak for HA or multi-instance deployments | Keep SQLite for local dev and single-replica Helm deployments. Add a relational production storage seam before allowing multi-replica Dashboard auth. |
| Dashboard auth session token | dashboard backend auth handlers and SQLite store, `dashboard/frontend/src/utils/authFetch.ts`, `dashboard/frontend/src/contexts/AuthContext.tsx`, `dashboard/frontend/src/contexts/authSession.ts`; Helm env wiring in `deploy/helm/semantic-router/templates/dashboard-deployment.yaml` | Login/bootstrap return a JWT response, set an HttpOnly `vsr_session` cookie, and persist a server-side session id for newly issued tokens; frontend stores only a bounded local token for token-based transports; Helm persistence wires `DASHBOARD_AUTH_DB_PATH` into the mounted dashboard-local PVC | `restart_safe_local_state` in the current SQLite auth store; intended `shared_durable_workflow_state` for production multi-replica deployments | Malformed or oversized local tokens are dropped before they can be mirrored into headers, cookies, or query-token transports; backend bearer, cookie, and query token inputs are normalized and bounded before JWT parsing; newly issued token session ids are checked against the server-side session record; logout clears local state, calls `POST /api/auth/logout`, revokes the current server-side session when present, and expires the server cookie; auth-store startup prunes expired or revoked sessions older than the retention window; provider startup probes `/api/auth/me` with same-origin credentials so a valid HttpOnly cookie can recover the user without readable local token material; Helm production values now persist the SQLite file while blocking unsupported multi-replica dashboard settings | Production multi-replica session-store policy is not complete; token-based protected transports still use bounded localStorage; tokens without a session id remain accepted until session-only auth policy is complete | Continue moving toward server-owned session contracts for production deployments. Current browser storage is bounded and sanitized, server cookie issue/clear includes explicit expiry semantics, backend token intake is bounded, newly issued sessions can be revoked, old inactive session rows are pruned, reload can recover from the server cookie, and Helm can preserve the local session DB across pod restarts; local token storage and SQLite-only session storage still need retirement or tighter scoping. |
| Evaluation task metadata | dashboard backend evaluation, `dashboard/backend/evaluation/db.go` | SQLite | `restart_safe_local_state` moving toward `shared_durable_workflow_state` | Tasks survive local restart in one workspace | HA and shared-operator workflows are limited to one mounted DB file | Keep task metadata in relational storage. SQLite is fine for local dev; add a production DB seam for shared deployments. |
| Evaluation progress fanout and cancellation | dashboard handlers, `dashboard/backend/handlers/evaluation.go` | In-memory `sync.Map` of SSE clients and cancel funcs | `ephemeral_request_state` | Active streams and cancel handles vanish on restart | Acceptable for live connections, but status history depends on the DB and in-flight runner state | Keep connection registries in memory, but ensure all progress and terminal states are durable and reconstructable from server-owned records. |
| ML pipeline jobs | dashboard backend ML pipeline, `dashboard/backend/mlpipeline/runner.go`, `dashboard/backend/workflowstore/**`, `dashboard/backend/handlers/mlpipeline.go` | SQLite `workflow.sqlite` (default `./data/workflow.sqlite`) for jobs and typed progress rows; SSE client maps in handler | `shared_durable_workflow_state` for job rows; `ephemeral_request_state` for live SSE | Job status and progress events survive dashboard restart; in-flight subprocess work is still lost without a worker resume story | Same SQLite file must be mounted for multi-replica dashboards | Keep subprocess execution separate; optional future: external job runner. |
| OpenClaw container registry | dashboard backend OpenClaw, `dashboard/backend/handlers/openclaw.go`, CLI `src/vllm-sr/cli/docker_services.py` | JSON file under OpenClaw data dir | `restart_safe_local_state` | Registry survives in one workspace, not shared across dashboards | Container control depends on local file convention and workspace mounts | Keep this as local-dev adapter for `vllm-sr serve`, but introduce a server-owned registry for multi-user dashboard control. |
| OpenClaw teams and workers | dashboard backend OpenClaw, `dashboard/backend/handlers/openclaw.go`, `openclaw_teams.go`, `openclaw_workers.go`, `dashboard/backend/workflowstore/**` | Rows in shared `workflow.sqlite`; one-time JSON import from `containers.json` / `teams.json` when tables are empty | `shared_durable_workflow_state` (restart-safe local SQLite) | Entities survive process restart when the DB file is preserved | HA requires shared DB backend (future seam) | Same as evaluation auth: SQLite for single-node; add relational seam for multi-replica. |
| OpenClaw rooms and chat messages | dashboard backend OpenClaw, `dashboard/backend/handlers/openclaw_rooms.go`, `dashboard/backend/workflowstore/**` | Room rows and append-only message rows in `workflow.sqlite`; one-time JSON import from `rooms.json` / `room-messages/*.json`; SSE/WS client maps in memory | `shared_durable_workflow_state` for persisted chat; `ephemeral_request_state` for live transport | Messages no longer rewrite whole JSON files; history survives restart with the DB | Very large rooms may need pagination APIs | Add list cursors / limits on message APIs as usage grows. |
| Dashboard chat history and queued tasks in Playground-style surfaces | dashboard frontend, `dashboard/frontend/src/hooks/useConversationStorage.ts`, `dashboard/frontend/src/hooks/conversationStorage.ts`, `dashboard/frontend/src/hooks/usePlaygroundQueue.ts`, `dashboard/frontend/src/components/ChatComponent.tsx` | Browser `localStorage` with local caps for saved conversations and queued playground tasks | Ambiguous today | Persists only in one browser; malformed conversation and queued-task records are dropped on restore, duplicate conversation IDs are normalized, and persistence is bounded before hydration and writes | No cross-device continuity, no audit, and no server recovery | Decide explicitly: either mark this demo-only and ephemeral, or move conversation and task queue state server-side if it is a supported product surface. |
| Config backups, generated runtime config, DSL snapshots | CLI and dashboard, `src/vllm-sr/cli/commands/runtime_support.py`, `dashboard/backend/handlers/config_backups.go`, `dashboard/backend/handlers/runtime_config_sync.go`, `src/semantic-router/pkg/apiserver/kb_persistence.go` | Files under `.vllm-sr/`, including generated runtime config and KB bootstrap markers | `restart_safe_local_state` and `derived_projection_state` | Survive in one workspace; generated KB bootstrap markers are treated as local restart state, corrupted marker YAML is ignored, marker rewrites use same-directory atomic replace, API startup with a runtime registry now reads only registry-published config instead of adopting `config.Get()` from another runtime, registry-backed classification refresh no longer republishes service config through `config.Get()`, and managed KB mutations update server-local/runtime-registry config without replacing the process-wide config cache | Files blur source-of-truth boundaries and are hard to audit in multi-user setups | Keep YAML and DSL as canonical intent. Add durable version/audit tables plus read-model projections for current models, signals, decisions, plugins, and DSL text. Do not make DB a second mutable primary writer yet. |
| Active deployed models, signals, decisions, plugins, DSL parse results | shared config contract, dashboard topology/config APIs, `src/semantic-router/pkg/config/**`, `src/semantic-router/pkg/dsl/**` | Derived from YAML/DSL at runtime; not stored as durable projection | Missing `derived_projection_state` | Recomputed ad hoc from files and live parse paths; runtime-registry-backed API startup, classification-service resolution, config refresh, and managed KB config mutation no longer fall back to or rewrite process-wide config before registry publication, ExtProc initial router construction, gRPC startup, and config-source watcher selection resolve from active router/runtime config before the process-wide fallback, empty-registry ExtProc construction parses the explicit config file instead of adopting a conflicting Kubernetes global, empty-registry ExtProc server config resolution also avoids adopting a conflicting process-wide config before runtime publication, registry-backed ExtProc file reload publishes through the runtime registry without rewriting `config.Get()`, and ExtProc modality/system-prompt request filters plus their shared request-time helper use router/runtime-owned config before process-wide fallback | Hard to query, audit, diff, or expose consistently across dashboard and future APIs | Add a persisted projection tied to deployed config version. Suggested projections: active config version, DSL snapshot, models, signals, decisions, plugins, and validation diagnostics. |

## Default Memory-Backed Surfaces To Treat As High Risk

- `global.stores.semantic_cache.backend_type = memory`
- `global.stores.vector_store.backend_type = memory` in dashboard defaults when enabled
- RAG `cache_results` in `src/semantic-router/pkg/config/rag_plugin.go`
- vector-store metadata in `src/semantic-router/pkg/vectorstore/manager.go`
- vector-store file registry in `src/semantic-router/pkg/vectorstore/filestore.go`
- model-selection learning state without configured storage in `src/semantic-router/pkg/selection/{elo.go,rl_driven.go,gmtrouter.go}`; `rl_driven` and `gmtrouter` decision-scoped `storage_path` now reaches runtime selector construction, API servers with a runtime registry no longer use selector globals before runtime publication, and RL-driven looper execution now receives the runtime selector registry, but shared-store parity remains open

## What Should Go To A Database First

- User accounts, roles, permissions, and audit logs
- Evaluation, ML pipeline, and model-research job metadata and typed progress events
- OpenClaw teams, workers, rooms, and room messages
- Router-visible metadata for vector stores and uploaded files
- Response API conversations and response metadata if they are exposed as supported product features
- Config version history, active config projection, deployed model/signal/decision/plugin projection, and DSL snapshots

## What Should Prefer Shared Cache Or Specialized Storage Instead Of A Database

- Semantic cache entries: shared cache such as Redis or the existing hybrid cache path
- RAG retrieval cache: shared cache if retained at all
- Vector embeddings and memory embeddings: vector backends such as Milvus or Llama Stack
- Large binary artifacts: local file/object storage with durable metadata in a database

## Execution Order

1. Publish and keep this inventory current.
2. Make router metadata and replay or response history restart-safe where the product already exposes those surfaces.
3. Move dashboard workflow state off in-memory maps and browser-only storage into server-owned durable records.
4. Add a persisted deployed-config projection so dashboard and future APIs stop reparsing YAML and DSL for every query path.
5. Add restart and recovery coverage in E2E for at least one router state surface and one dashboard workflow. (Response API restart-recovery E2E test added in `e2e/testcases/response_api_restart_recovery.go`, registered in Redis profile.)
