# Kage Memory Harness Research

Date: 2026-04-30

## Executive Verdict

Kage is pointed at the right problem, but the current system is not yet storing,
creating, or sharing memory in the right shape for an org-scale "hive memory"
experience.

The current design is useful as a prototype because it captures human-reviewed
Markdown nodes, commits project knowledge to git, and offers a public static
graph for common framework gotchas. The breakthrough version must become a
memory control plane:

1. It must observe work automatically, but never trust raw observation as memory.
2. It must convert coding sessions into structured, source-backed memory packets.
3. It must separate repo, personal, org, and public/global scopes.
4. It must retrieve context through one reliable tool, not through hard prompt
   pressure.
5. It must treat privacy, permissions, provenance, freshness, and review as core
   storage fields, not policy text.

The north star:

> After install, every agent working in a repo should behave like a senior
> teammate who knows how the codebase is structured, how to run it, what broke
> before, why decisions were made, what flows are fragile, and what public
> framework issues apply right now.

## What Kage Does Right Today

Kage has several strong primitives:

- Project memory lives in `.agent_memory/` and can travel with git.
- Pending memory requires review before becoming team-visible.
- The repo indexer concept is correct: every repo needs a compact architecture
  map before agent work starts.
- The public `kage-graph` model is promising: static JSON/Markdown over a CDN is
  cheap, simple, and easy for agents to consume.
- The system uses multiple memory tiers: project, personal, and global.

These are the bones. Keep them.

## What Is Wrong Today

### Creation Is Too Prompt-Governed

Today, memory creation depends heavily on the agent remembering to call
`kage-distiller` and on the distiller following Markdown instructions. This is
fine for demos, but brittle for teams. Schema validation, deduplication,
provenance, sensitivity scanning, and index generation need deterministic code.

Current reference:
`/Users/kushaljain/Documents/Codex/2026-04-30/i-m-working-on-the-framework/Kage/.claude/agents/kage-distiller.md`

### Storage Is Too Document-Like

Markdown nodes are human-readable, but the system needs a structured memory
packet behind each Markdown body. Otherwise retrieval, permissions, freshness,
supersession, and quality scoring stay approximate.

Markdown can remain the authoring and review surface. It should not be the only
database.

### Sharing Is Correct For Small Teams, Not Orgs

Git sharing works for repo memory. It does not solve:

- cross-repo org memory
- team permission inheritance
- private framework knowledge
- memory promotion workflows
- audit logs
- global/public sanitization

Org memory needs an authenticated registry or MCP service. Public global memory
needs explicit promotion and community review.

### Retrieval Is Too Rigid

The current `kage-memory` agent searches tiers in order and stops when it finds
2+ local nodes. That loses important org policy, global framework gotchas, and
fresh bug context. Retrieval should blend scopes and rank by relevance,
authority, freshness, path proximity, and permissions.

### The Current MCP Global Search Has Schema Drift

`Kage/mcp/index.ts` expects `catalog.domains[*].node_count`, while
`kage-graph/catalog.json` uses `domains[*].nodes`. That can cause broad MCP
search to miss domains even when nodes exist.

Fixing this kind of drift is urgent because schema mismatch at the global layer
will quietly erode trust.

## What Is Actually Worth Storing

Kage should store five memory classes, each with different creation and review
rules.

### 1. Codebase Map Memory

This is generated from code, not conversation.

Examples:

- architecture graph
- package boundaries
- route map
- data model
- auth flow
- queue/job flow
- config/env var index
- run/test/build commands
- deployment topology

Creation:

- generated by repo indexer
- source-backed by commit SHA and file paths
- refreshed on meaningful diffs

Review:

- light review, because it is derived from code
- stale automatically when source files change

### 2. Operational Runbook Memory

This tells agents and humans how to work in the repo.

Examples:

- how to run the app locally
- how to seed data
- how to run a specific test suite
- known flaky commands
- required env vars and where to get them
- common local setup failures

Creation:

- from successful/failed command episodes
- from README/package scripts/devcontainer/CI
- from user-approved session discoveries

Review:

- repo owner review
- strong freshness checks

### 3. Decision Memory

This records why the system is this way.

Examples:

- "Use refresh token rotation instead of long-lived JWTs"
- "Keep billing webhooks idempotent because Stripe retries"
- "Do not use Prisma migrations in CI preview DBs"

Creation:

- from explicit architecture decisions in sessions, PRs, issues, ADRs

Review:

- human required
- ideally linked to PR/issue/commit

### 4. Debugging And Bug Memory

This prevents rediscovery.

Examples:

- symptom -> root cause -> fix
- exact error message
- reproduction command
- files touched
- regression test added
- "never do this again" note

Creation:

- after successful fix, not during speculation
- from PR diffs, test failures, issue comments, session transcript

Review:

- human or PR-based approval
- duplicate detection against prior bug memories

### 5. Agent Procedure Memory

This teaches agents how to work in this codebase.

Examples:

- "Before editing auth, inspect `middleware.ts` and `auth/session.ts`"
- "Use factory fixtures, not direct DB inserts, in tests"
- "For billing changes, run webhook replay tests"

Creation:

- distilled from repeated successful workflows
- confirmed by maintainers

Review:

- strict, because procedural memory directly changes agent behavior

## What Should Not Be Stored

Do not store:

- raw transcripts by default
- secrets, tokens, internal URLs with credentials
- email addresses or user PII
- speculative ideas that did not become true
- temporary debugging guesses
- giant code summaries that duplicate source files
- stale "how to run" commands without validation
- personal preferences in repo/org memory
- public/global nodes based on private code

The rule is:

> Store resolved, reusable, source-backed knowledge. Do not store chat exhaust.

## Breakthrough Architecture: The Kage Memory Control Plane

The breakthrough design is a layered system:

```text
Agent / IDE / CLI
      |
      v
Kage Adapter Layer
  - Claude Code hooks
  - Cursor/Windsurf/Codex adapters
  - MCP client/server
  - GitHub/GitLab integration
      |
      v
Memory Kernel
  - observe events
  - propose memories
  - normalize schema
  - scan secrets/PII
  - dedupe/upsert
  - rank and assemble context
  - audit all reads/writes
      |
      v
Storage Tiers
  - session scratch
  - repo memory
  - personal memory
  - org memory
  - public/global graph
```

Kage should not be "Claude hooks plus Markdown." Claude Code should be one
adapter into the memory kernel.

## Core Primitive: Memory Packet

Every memory should compile into this canonical schema:

```yaml
schema_version: 2
id: repo:github.com/acme/app#bug-stripe-raw-body-2026-04-28
title: Stripe webhook signature fails if body parser runs first
summary: Express must expose raw request body before JSON parser for Stripe signature verification.
body: |
  Symptom, cause, fix, verification, never-do.
type: bug_fix
memory_kind: semantic|episodic|procedural
scope: session|repo|personal|org|public
visibility: private|team|org|public
sensitivity: public|internal|confidential|blocked
status: candidate|approved|deprecated|superseded
confidence: 0.87
freshness:
  ttl_days: 365
  last_verified_at: 2026-04-28
  verification: test_passed|source_seen|human_reviewed|unknown
source_refs:
  - kind: commit
    repo: github.com/acme/app
    sha: abc123
  - kind: file
    path: backend/webhooks/stripe.ts
    sha: abc123
  - kind: pr
    url: https://github.com/acme/app/pull/491
permissions:
  inherits_from: repo
  repo: github.com/acme/app
  teams: ["payments"]
paths: ["backend/webhooks", "backend/payments"]
tags: ["stripe", "webhook", "express", "raw-body"]
stack: ["express@>=4", "stripe-node@>=12"]
edges:
  - rel: depends_on
    target: repo:github.com/acme/app#api-routes
  - rel: supersedes
    target: repo:github.com/acme/app#old-webhook-note
quality:
  reviewer: "@alice"
  votes_up: 4
  votes_down: 0
  uses_30d: 19
created_at: 2026-04-28T10:22:00Z
updated_at: 2026-04-28T10:22:00Z
```

Markdown can be generated from this packet. JSON should be the canonical
machine-readable form.

## Capture Pipeline

### Step 1. Observe

Collect event streams:

- tool calls
- file reads/writes
- diffs
- tests run
- command failures/successes
- PRs/issues/comments
- CI failures
- user approvals
- explicit agent notes

Store raw events locally for a short retention period. Do not sync raw events by
default.

### Step 2. Propose Candidate Memories

Run a distiller that only proposes candidates. It should output structured
objects with reasons:

```json
{
  "candidate_type": "bug_fix",
  "why_save": "Resolved a repeated Stripe webhook failure with a verified fix",
  "evidence": ["test passed", "file changed", "error message seen"],
  "proposed_scope": "repo",
  "risk": "low"
}
```

### Step 3. Normalize

Turn candidates into the memory packet schema. Attach source refs and detect
whether it is:

- new memory
- update to existing memory
- duplicate
- supersession
- contradiction

### Step 4. Scan

Run deterministic scanners:

- secret scanner
- PII detector
- internal hostname detector
- public-safety classifier
- license/copyright check for public nodes

Anything flagged becomes `sensitivity: blocked` until edited.

### Step 5. Review

Different scopes require different gates:

- session: no review, ephemeral
- personal: user review optional
- repo: maintainer or PR review
- org: repo/team owner review
- public: maintainer/community review

### Step 6. Publish

Write to the right target:

- repo: `.agent_memory/packets/*.json` plus generated Markdown/indexes
- personal: encrypted local store or local SQLite
- org: authenticated registry
- public: static CDN bundle after review

## Retrieval Pipeline

The agent should call one tool:

```text
kage_recall(
  query,
  repo,
  branch,
  paths_touched,
  task_type,
  scopes=["repo","org","personal","public"],
  budget_tokens=1200
)
```

The tool returns a compact context block:

```text
Kage Recall

Pinned:
- [Repo Policy] Auth changes require refresh-token rotation tests.
- [Runbook] Use `pnpm test:api -- webhooks` for Stripe webhook tests.

Relevant:
1. [BugFix][repo][confidence .91][verified 2026-04-28]
   Stripe webhook signature fails if body parser runs first.
   Why matched: path backend/webhooks + query "stripe signature".
   Source: PR #491, backend/webhooks/stripe.ts@abc123

2. [Public Gotcha][fresh][score 82]
   Stripe requires raw request body for signature validation in Express.
   Source: kage-graph/payments/stripe-webhook-signature-raw-body

Actions:
- explain why
- fetch full node
- report wrong/stale
- save new learning
```

Retrieval ranking should combine:

- path overlap
- exact keyword/BM25 match
- embedding similarity
- graph distance from current domain
- freshness
- verification strength
- user/team/repo permission
- source authority
- prior successful use
- negative feedback

Important: retrieval should blend scopes. Do not stop just because repo memory
found two nodes. Some global bugs and org policies must still surface.

## Storage Design

### Repo Memory

Keep repo memory in git, but make it generated and validated:

```text
.agent_memory/
  packets/
    *.json          canonical memory packets
  nodes/
    *.md            generated human-readable views
  indexes/
    by-path.json
    by-tag.json
    by-type.json
    graph.json
  SUMMARY.md        generated digest
  kage.lock         schema + generator version
```

Human edits should happen through `/kage edit` or PRs that modify packet JSON.
Indexes should never be hand-authored.

### Local Cache

Each install gets:

```text
~/.kage/cache.sqlite
```

Use it for:

- BM25/FTS search
- recently recalled memories
- source refs
- pending candidates
- local embeddings when available

This gives speed without requiring a daemon.

### Org Memory

Org memory should be served by an authenticated Kage MCP server:

```text
kage-org-server
  auth: GitHub/GitLab/SSO
  permissions: repo/team inheritance
  storage: Postgres + object store, or private repo bundle first
  search: BM25 + vectors + graph edges
  audit: every read/write
```

The org server answers:

- what does this user have access to?
- what repo are they in?
- what memories match this task?
- what memories are mandatory policy?
- what can be promoted or shared?

### Public Global Graph

The global graph should be StackOverflow for agents:

- public gotchas
- known framework bugs
- version-specific config
- issue/documentation distilled knowledge
- agent workflow lessons

But it must not be a dump of private repo memory. Public promotion should require:

- no private source refs
- public-safe examples
- version/stack metadata
- citations to public docs/issues when possible
- maintainer review
- freshness TTL
- voting and usage telemetry
- conflict/supersession edges

## Global CDN Design

Use static distribution for read scale:

```text
catalog.json
domains/{domain}/index.json
domains/{domain}/nodes/{slug}.json
domains/{domain}/nodes/{slug}.md
tags/{tag}.json
versions/{package}/{range}.json
issues/{provider}/{project}/{issue_id}.json
```

Public nodes should look like:

```yaml
type: gotcha|pattern|config|decision|reference
domain: payments
stack: ["stripe-node@>=12", "express@>=4"]
symptom: "Webhook signature verification fails"
cause: "JSON body parser consumed raw body"
fix: "Register express.raw before express.json for the webhook route"
public_sources:
  - https://docs.stripe.com/webhooks/signature
freshness:
  ttl_days: 365
  last_checked: 2026-04-30
quality:
  score: 84
  accepted_answers: 3
  issue_links: 2
  uses_30d: 141
```

Public graph retrieval should support:

- exact error lookup
- framework/version lookup
- "known issue" search
- "how to implement X" search
- alternatives and conflicts
- superseded redirects

## Agent Contribution Flow

Agents should be able to contribute, but not publish directly.

Flow:

1. Agent solves issue.
2. Kage proposes memory candidates.
3. User approves repo memory.
4. Agent asks if any candidate is generic enough for org/public.
5. Kage sanitizes and rewrites to public form.
6. Kage attaches public citations.
7. Human reviews.
8. Bot opens PR to org/public graph.
9. CI validates schema, sources, safety, duplicates, and freshness.
10. Merge publishes to CDN.

The key is promotion, not automatic upload.

## Product Experience

### Install Experience

After install, users should see:

```text
Kage active for github.com/acme/app

Repo memory: 84 nodes
Org memory: 1,241 nodes available
Public graph: online
Last indexed: 2h ago

Claude/Codex/Cursor can now recall repo context with `kage_recall`.
```

No scary global hook patching without a clear prompt.

### Daily Experience

Developers should feel helped, not policed.

Good:

- "I found 3 relevant memories before editing billing."
- "This command failed before; use `pnpm test:api --runInBand`."
- "This bug was fixed in PR #491; want the summary?"
- "I learned something reusable. Save to repo memory?"

Bad:

- "You MUST invoke memory first before any file read."
- silent background upload
- unexplained context injection
- stale wrong memories with no feedback path

### Commands

```text
/kage recall <query>
/kage why <memory-id>
/kage save
/kage review
/kage validate
/kage index
/kage deprecate <memory-id>
/kage promote <memory-id> --to org|public
/kage audit
/kage forget <memory-id>
```

### Agent UX

Agents need a simple contract:

1. Recall before major decisions.
2. Save only resolved, reusable knowledge.
3. Cite source refs.
4. Ask before sharing beyond current scope.
5. Report stale or wrong memory.

## Evaluation Harness

Kage should be measured like a retrieval and agent productivity system.

### Offline Eval Set

Build tasks like:

- "Fix webhook signature bug"
- "Add auth endpoint"
- "Run local app"
- "Debug CI failure"
- "Add migration"
- "Implement feature following repo convention"

For each task, define:

- required memories
- harmful/stale memories
- expected commands
- expected files
- success criteria

### Metrics

- recall@k for required memory
- precision@k
- stale memory rate
- secret leakage rate
- duplicate memory rate
- task completion time
- file rediscovery time
- commands until first successful run
- number of user corrections
- memory approval rate
- memory reuse count

### Product Metric

The north-star metric:

> How often does an agent solve a repo-specific task without asking the user to
> re-explain prior knowledge?

## Roadmap

### Phase 0: Repair Current Prototype

- Fix MCP `node_count` vs `nodes` schema mismatch.
- Add deterministic `kage validate`.
- Remove stale legacy scripts or make them work.
- Fix Stop hook processed-session semantics.
- Generate indexes from packets, not manual appends.
- Add secret scanning before pending nodes are written.

### Phase 1: Repo Memory That Teams Can Trust

- Introduce packet schema v2.
- Store packet JSON plus generated Markdown.
- Add local SQLite FTS cache.
- Add dedupe/upsert.
- Add source refs to every memory.
- Add `/kage why` and `/kage deprecate`.

### Phase 2: Great Retrieval

- Build `kage_recall`.
- Blend repo, personal, org, and public scopes.
- Add path-aware ranking.
- Add pinned runbook/policy memory.
- Add token-budgeted context assembly.
- Add feedback: useful, stale, wrong, ignore.

### Phase 3: Org Memory

- Private org registry.
- SSO/GitHub auth.
- Repo/team permission inheritance.
- Audit logs.
- Org promotion workflow.
- Cross-repo pattern memory.

### Phase 4: Public StackOverflow-For-Agents Graph

- Public node schema with citations.
- PR validation and duplicate detection.
- Version compatibility checks.
- Votes, usage, freshness TTL.
- Supersession and conflict graph.
- CDN indexes optimized for exact error and framework lookup.

### Phase 5: Universal Adapters

- Claude Code adapter.
- Codex adapter.
- Cursor/Windsurf adapter.
- GitHub PR/issue adapter.
- CI adapter.
- MCP server/client.
- CLI and library API.

## Research Sources

- LangGraph memory distinguishes short-term/thread memory from long-term
  namespace stores and describes semantic, episodic, and procedural memory:
  https://docs.langchain.com/oss/python/concepts/memory
- LangChain context engineering frames reliability as getting the right context
  into the model at the right time:
  https://docs.langchain.com/oss/python/langchain/context-engineering
- Zep concepts describe temporal knowledge graphs, fact invalidation, user and
  group graphs, memory add/get, graph search, and fact ratings:
  https://help.getzep.com/concepts
- Zep memory docs recommend adding chat turns in order, retrieving an assembled
  context string, and using graph search for custom retrieval:
  https://help.getzep.com/v2/memory
- Graphiti describes temporal, incremental knowledge graphs with hybrid
  semantic, full-text, and graph search:
  https://help.getzep.com/graphiti/getting-started/overview
- MCP tools spec requires schemas, access controls, sanitization, rate limiting,
  and audit-oriented client behavior for tools:
  https://modelcontextprotocol.io/specification/draft/server/tools
- MCP elicitation spec emphasizes user control and says servers must not request
  sensitive information through elicitation:
  https://modelcontextprotocol.io/specification/2025-06-18/client/elicitation

## Final Answer

Kage should become the memory layer every coding agent plugs into. The winning
design is not "store more transcripts." It is:

- source-backed memory packets
- deterministic validation
- careful scope boundaries
- org-aware permissioning
- global public promotion
- hybrid retrieval
- tight context assembly
- explicit feedback and freshness

If Kage gets this right, the user experience is not "I installed another agent
tool." It is "my agent finally behaves like someone who has worked in this repo
for months."
