---
title: "Company Brain Compile Pipeline"
description: How raw memories become durable Company Brain pages — step by step, in plain language, then in technical detail.
---

import { Aside } from "@astrojs/starlight/components";

This doc is a walking tour of how raw memories become Company Brain pages — the distillation layer that turns scattered observations into structured, queryable pages. It's written to be read top to bottom; each step covers one thing the pipeline does, why it does it, and what can go wrong. The pipeline is the load-bearing implementation of the page promise: reproducible outputs from the same input set and a complete provenance trail from every page back to the source memories that wrote it.

The first half is plain-language narrative. The second half (["Under the hood"](#under-the-hood)) has the code paths, SQL, and prompt contracts for anyone who needs to modify or debug the pipeline itself.

## High-level shape

There are two compile passes that run in the same job:

1. **Leaf pass.** Reads new memories, decides which concrete _entity_ pages should be created or updated. Produces pages like `Momofuku Daishō`, `Franklin Barbecue`, `Biscuit (dog)`. This is where most wiki content originates.
2. **Aggregation pass.** Runs _after_ the leaf pass completes. Looks across the agent's recently-changed pages and decides which _hub_ topics deserve rollup sections (e.g. `Toronto` with a `Restaurants` section) and which sections have become dense enough to promote into their own page (e.g. a `Restaurants` section on `Austin` promotes into a standalone `Austin Restaurants` topic page).

Both passes are driven by one compile job. The job is always scoped to one `(tenant, owner)` pair. If anything fails, the cursor stays where it was and the next job safely replays.

<Aside type="tip">
  If you only remember one thing: **leaf creates entities, aggregation makes
  them cohere**. The leaf planner is conservative; the aggregation planner is
  what makes the wiki feel compounded instead of just piled up.
</Aside>

## Where ontology fits

The compile pipeline still starts from Hindsight memory and writes compiled pages. The [Business Ontology](/concepts/knowledge/business-ontology/) adds a governed meaning layer beside it:

- suggestion scans observe memory, current Brain pages, and external references to propose new entity types, relationship types, facet templates, and mappings;
- admins approve those proposals as ontology change sets in Ontology Studio;
- approval queues an ontology reprocess job that applies the approved definitions and materializes tenant-shared Brain pages/facets;
- future compiles and Brain retrieval use approved templates instead of relying only on generic page defaults.

This means "compile to wiki" is no longer the whole story. Hindsight remains the source memory substrate, owner-scoped compiled wiki pages remain stable, and the ontology-driven Brain becomes the shared business view that can be rebuilt when the tenant vocabulary changes.

## Pass 1: Leaf compile

### Step 1.1 — A compile job gets queued

A job row lands in `wiki_compile_jobs` in one of four ways:

- **After a retain.** The `memory-retain` Lambda fires `maybeEnqueuePostTurnCompile` after every successful `retainTurn`. Jobs dedupe on a 5-minute bucket, so a burst of turns collapses into one queued compile.
- **Admin trigger.** The `compileWikiNow(tenantId, ownerId)` mutation enqueues a job directly.
- **Lint sweep.** The nightly lint pass enqueues promotion jobs for unresolved mentions that have crossed the `≥ 3 mentions in 30 days` threshold.
- **CLI.** `thinkwork wiki compile` wraps the admin mutation for operator use.

### Step 1.2 — The compile worker wakes up

`wiki-compile` Lambda claims the pending job (FOR UPDATE SKIP LOCKED) and reads the cursor for that `(tenant, owner)` — a bookmark of the last memory the worker processed.

Then it reads the next 50 memories from Hindsight where `updated_at > cursor`. Hindsight is the canonical memory substrate; the wiki is a derived store.

### Step 1.3 — The planner reads memories in context

The batch of memories plus the scope's existing pages plus any open unresolved mentions get handed to the **leaf planner** — Bedrock via the Converse API, defaulting to Claude Haiku 4.5 but swappable via the `modelId` event parameter.

The planner sees:

- The new memories, in full.
- Titles, slugs, and summaries of every active page in scope.
- Every open unresolved mention with its accumulated context.

The planner is explicitly biased toward caution. Holding an alias as "unresolved" is almost free; creating a spurious page is expensive to undo. When in doubt, it holds.

### Step 1.4 — The planner produces a plan

The plan is a small JSON document with six arrays:

**Page updates.** "This existing page should have its Visits section rewritten given this new memory."

**New pages.** "There's enough evidence for a new page about Taberna dos Mercadores. Here's a title, sections, aliases."

**Unresolved mentions.** "Someone named 'Chef João' showed up but doesn't justify a page yet — hold the alias."

**Promotions.** "Chef João has been held for months and three new memories push him over the threshold. Promote to a real page, here are the sections."

**Page links.** "Taberna dos Mercadores should link to Lisbon. Both pages exist or are being created in this plan."

**Parent section updates** and **section promotions** are also emitted by the planner contract but the leaf planner rarely fills them — the aggregation pass does that work.

The planner never writes to the database. It produces a plan; the compiler applies it.

### Step 1.5 — The section writer patches one section at a time

For every section the plan wants to update, the compiler calls a second Bedrock model — the **section writer**. It's narrow: take the existing section markdown, the planner's proposed body, and the specific memories cited — produce final markdown for _this one section only_.

Before calling, the compiler checks `isMeaningfulChange` — a cheap edit-distance filter that skips sections when the proposal is effectively identical to the stored body. No point burning tokens polishing prose that already says the same thing.

### Step 1.6 — Hallucinated IDs and stray wikilinks get filtered

Before the model output is trusted:

- **`isValidUuid()`** gates every page-id and mention-id lookup. Smaller models sometimes emit truncated ids like `"814e6b70"`; Postgres would reject those as invalid UUIDs and kill the job. The guard silently skips invalid rows instead.
- **`stripWikilinks()`** removes `[[Obsidian-style]]` syntax from section bodies at the repo boundary. Models were told not to emit it, but they slip; the strip runs inside `upsertSections` so every write path is covered.
- **`linkifyKnownEntities()`** rewrites `**Entity Name**` bolded mentions into real `[**Entity Name**](/wiki/<type>/<slug>)` markdown links when the name matches a page in scope. Longest-title-first match prevents shadowing.
- **Invalid `pageLinks`** (bad page types, blank slugs) are dropped with a CloudWatch warning rather than failing the whole plan.

All five filters exist because _some_ model has produced the exact failure mode we're guarding against. Prompts are the first line of defense; these repo-level guards are the second.

### Step 1.7 — Provenance gets written with deliberate care

Every section carries a list of which memories were used as sources. The rule is strict: **only the specific memories the planner cited for this section**. Not "all the memories in the batch." Not "every memory that looks related."

An earlier version of the compiler fell back to "use the whole batch as sources" when the planner omitted citations. The result was nonsense: a memory about Austin weather got written as a source for unrelated CRM pages that happened to be in the same batch. The reverse-lookup (`MemoryRecord.wikiPages`) then returned those CRM pages when you tapped the weather memory. Everything downstream broke.

The fix: zero cited memories means zero source rows. **Blank provenance is always preferable to wrong provenance.** `resolveCitedRecords` encodes this rule and no path bypasses it.

### Step 1.7a — Alias dedupe on newPages

Before creating any page the leaf planner proposed in `newPages[]`, the compiler tries to merge the proposal into an existing page. The match runs in two passes:

1. **Exact alias match.** The proposed title + aliases are normalized and looked up against `wiki_page_aliases` in the same scope. A hit turns the proposal into a `pageUpdate` against the existing page and bumps `alias_dedup_merged`.
2. **Trigram fuzzy fallback.** If no exact alias matches, the compiler queries `wiki_page_aliases.alias` and `wiki_pages.title` with Postgres `pg_trgm` similarity. Matches at `similarity() >= 0.85` are accepted only when the matched page has the _same_ `type` as the proposal — entity into entity, topic into topic. Cross-type fuzzy matches are always rejected. Successful fuzzy merges bump `fuzzy_dedupe_merges`, tracked separately from exact because fuzzy is the over-collapse risk.

The repo helper is `findAliasMatchesFuzzy` in `packages/api/src/lib/wiki/repository.ts`; it sits on the `pg_trgm` GIN indexes added by migration `0015_pg_trgm_alias_title_indexes.sql`. The helper's internal try/catch makes a missing `pg_trgm` extension a graceful no-match rather than a job failure.

Without this step, planner-emitted `"Paris, France"` creates a second page when `"Paris"` already exists. With it, the proposal collapses into the existing `topic/paris` page and its new sections land there.

### Step 1.7b — Deterministic link emission

After the leaf plan applies, two deterministic emitters write additional `reference` links. Both are LLM-free, both gate on `WIKI_DETERMINISTIC_LINKING_ENABLED` (default `true`, pinned in terraform next to `WIKI_AGGREGATION_PASS_ENABLED`).

- **`emitDeterministicParentLinks`** — reads parent-expander candidates (Step 2.1's `deriveParentCandidates*`). For each `city` / `journal` candidate with an exact-title match in scope, it writes a `reference` edge from every leaf entity page sourced by that record into the parent topic. Context tag: `deterministic:city:<slug>` or `deterministic:journal:<slug>`.
- **`emitCoMentionLinks`** — reads `wiki_section_sources` (not the leaf planner's `pageLinks`) to find entity pages that share a `memory_unit` source. Emits reciprocal `reference` edges between those pages, capped at 10 directed edges per memory (slug-ascending order). Context tag: `co_mention:<memory_unit_id>`.

Both emitters persist through `upsertPageLink`'s `onConflictDoNothing` on the `(from_page_id, to_page_id, kind)` unique index, so a replay is a no-op. The context-tag format exists so targeted rollback can drop only the deterministic rows without touching planner-emitted references.

When the flag is off, the job records `deterministic_linking_flag_suppressed: true` in metrics and both `links_written_*` counters stay at 0. The rollback SQL lives in `docs/metrics/wiki-link-density.md` in the repo.

### Step 1.7c — R5 precision canary

Every job records `duplicate_candidates_count` — the count of `(owner_id, title)` groups with more than one active row, scoped to the tenant. A rising value means the dedupe step is losing ground (fuzzy gate too strict, new aliases slipping through). It's the first signal operators should watch after flipping `WIKI_DETERMINISTIC_LINKING_ENABLED`.

### Step 1.8 — The cursor advances

Once the plan applies cleanly, the worker moves its bookmark to the last memory it processed. If anything failed mid-batch, the bookmark stays and the next worker replays. Safe to retry.

### Step 1.9 — The worker loops

Keep fetching 50-memory batches until: cursor drains, hard cap (500 records / 25 new pages / 100 section rewrites), or Bedrock rate-limits. Then the job finishes and emits metrics.

Large backfills naturally spread across multiple jobs — cursor-preserving means no replay hazard.

## Pass 2: Aggregation

<Aside type="note">
  Aggregation is gated behind `WIKI_AGGREGATION_PASS_ENABLED`. When the flag is
  off (default), the pipeline stops at the leaf pass. Everything below runs only
  when the flag is on — currently per-agent allowlist until thresholds
  stabilize.
</Aside>

The leaf pass ends with fresh entity pages in the scope, but they're not yet _organized_. Marco now has 90 restaurants, events, and places — but there's no "Austin" topic page tying his Austin ones together, no "Restaurants" hub, no rollups you can scroll through. That's what the aggregation pass builds.

### Step 2.1 — Derive parent candidates

Two deterministic expanders scan the current job's records + the scope's recently-changed pages to suggest hubs that _obviously_ need to exist:

- **`deriveParentCandidates(records)`** — pulls city, journal, and tag clusters out of raw record metadata. Records with `place.city = "Austin"` group into an Austin candidate. Shared `journal_id`s group into a trip topic. Repeated tags group into a tag-collection hub.
- **`deriveParentCandidatesFromPageSummaries(pages)`** — scans each scope page's _summary_ text for city names. Catches "Korean restaurant in Toronto." style summaries where raw metadata isn't in the current batch. Walks comma-separated postal addresses from the end to pull `Toronto` out of `", Toronto, ON M6J 1G1, Canada."`.

Both expanders are cheap — no LLM, no extra DB hits beyond the already-hydrated pages. They exist so the LLM gets grounded seed hubs instead of inventing them.

### Step 2.2 — The aggregation planner

A second Bedrock call, with its own system prompt, receives:

- Every recently-changed page in scope (with summary, sections, existing aggregation metadata).
- Parent candidates from Step 2.1.
- Link neighborhoods (inbound link counts).

The prompt is strict about membership: **every page listed under a hub must have visible evidence in its summary**. An earlier version hallucinated an `Austin Restaurants` hub containing Toronto, Tokyo, and Bogotá restaurants — the prompt now requires geographic proof per linked page and refuses to create a hub when fewer than 3 pages clearly match.

Output: `parentSectionUpdates`, `sectionPromotions`, hub `newPages`, hierarchy `pageLinks`.

### Step 2.3 — Rollup bodies are rendered deterministically

When a `parentSectionUpdate` carries `linked_page_slugs`, the compiler **skips the section writer** and renders the body with `renderRollupList`:

```
- [**Franklin Barbecue**](/wiki/entity/franklin-barbecue) — BBQ joint in Austin, TX.
- [**Momofuku Daishō**](/wiki/entity/momofuku-daisho) — Korean-inspired restaurant in Toronto.
```

Why deterministic: the LLM was previously writing prose ("Franklin Barbecue is a BBQ joint…") that mentioned entity names as plain text with no hyperlinks. A structured list with real markdown links is cheaper, consistent, and clickable. Summaries are trimmed to the first sentence — the full summary lives on the child page.

The markdown URL `/wiki/<type>/<slug>` matches the mobile router path. Mobile's wiki detail view hooks `onLinkPress` to intercept these and route internally rather than handing off to the OS Linking API.

### Step 2.4 — Dedupe across sections on the same parent

A single parent page like `Austin Activities` previously listed `Lady Bird Lake Hike & Bike Trail` under _both_ `Overview` AND `Outdoor & Family Attractions`. The aggregation planner was emitting the same child under multiple sections.

The compiler now tracks `claimedChildrenByParent` per job — first section to claim a child wins; later sections on the same parent drop the duplicate before rendering. The aggregation prompt was also updated to forbid emitting the same child twice.

### Step 2.5 — Section promotion scoring

Every section with aggregation metadata gets scored by `scoreSectionAggregation`. A pure function, no LLM:

```
score = 0.25·linked_page_count_signal     // saturates at 20 child pages
      + 0.25·supporting_record_signal      // saturates at 30 records
      + 0.20·temporal_spread_signal        // saturates at 30 days
      + 0.15·coherence_signal              // tag overlap ratio
      + 0.15·readability_pressure_signal   // body length >1200 chars
```

Thresholds: `candidate >= 0.4`, `promote_ready >= 0.55`. When a section crosses `promote_ready`, its `aggregation.promotion_status` flips to `candidate` and it's eligible for promotion on the next aggregation pass. Stickiness: once `promotion_status = "promoted"`, never flaps back — operator override only.

### Step 2.6 — Section promotion

When the aggregation planner emits a `sectionPromotion`, the compiler:

1. Creates the new topic page from `newPage.seed`.
2. Calls `setParentPage({ pageId: newPage.id, parentPageId: parent.id })` — writes `parent_page_id` AND mirrors into `wiki_page_links` with `kind='parent_of'/'child_of'`.
3. Rewrites the parent's section body down to the `parentSummary` + `topHighlights` the planner wrote, followed by `See: [Title](/wiki/...)` as a real markdown link.
4. Marks the parent section `promotion_status = "promoted"` with the child's page id stored in `aggregation.promoted_page_id`.

This is the compounding loop — a section outgrows its parent, spins off its own page, and that new page can itself grow sections that eventually promote. Austin → Austin Activities → (eventually) Austin Kid-Friendly Activities, on and on as data accumulates.

### Step 2.7 — Hubness recompute

Every page the aggregation pass touched gets its `hubness_score` recomputed:

```
hubness = inbound_reference_links
        + 2 · promoted_child_count
        + floor(avg(section.supporting_record_count) / 10)
```

The score is coarse and monotonic — it's used for ordering, not as a precise ranking number. A page that gains a promoted child jumps ahead of one with only inbound references.

## What can go wrong

**Bedrock throttling.** `ThrottlingException` comes back when Claude's per-minute quota is exhausted. The job fails, cursor stays put, next invoke retries clean.

**JSON truncation.** Model output that exceeds `maxTokens` comes back malformed. `maxTokens` is clamped per-model via `MODEL_MAX_OUTPUT_TOKENS` so Nova Micro (10k hard limit) doesn't reject outright; Haiku/gpt-oss get the full 24k planner budget.

**Hallucinated IDs.** Covered in [Step 1.6](#step-16--hallucinated-ids-and-stray-wikilinks-get-filtered) — caught at repo boundary, not the prompt.

**Over-grouping in aggregation.** The `Austin Restaurants = Toronto restaurants` bug. Fixed by adding evidence requirements to the prompt _and_ showing the planner per-page summaries (previously only titles were visible to the LLM).

**Env vars reset on deploy.** Terraform apply on an unrelated merge re-applies Lambda env config, wiping `WIKI_AGGREGATION_PASS_ENABLED`. Known paper cut; pin in terraform is the fix.

**Cursor precision drift.** Postgres timestamps are microsecond-precision but JS `Date` only carries milliseconds. The cursor query `date_trunc('milliseconds', …)` on both sides so a record at `xx:yy:zz.689019` isn't seen as "after" a stored cursor of `xx:yy:zz.689` forever.

## Cost

Typical full recompile of an agent with ~700 memory units runs $0.50 – $1.00 in Bedrock costs. Per-batch cost: $0.04 – $0.12. Aggregation pass adds ~$0.01 – $0.05 per job on top of leaf cost.

Every job's `metrics` JSON records `input_tokens`, `output_tokens`, `cost_usd`, and aggregation-specific counters (`aggregation_planner_calls`, `deterministic_parents_derived`, `parent_sections_updated`, `sections_promoted`). Watch those when tuning prompts or thresholds.

Link-densification + alias-dedupe counters (added by the deterministic-linking and fuzzy-dedupe passes):

- `links_written_deterministic` — `reference` edges written by `emitDeterministicParentLinks`.
- `links_written_co_mention` — `reference` edges written by `emitCoMentionLinks`.
- `duplicate_candidates_count` — R5 precision canary (active `(owner, title)` groups with more than one row).
- `alias_dedup_merged` — `newPages[]` entries collapsed into an existing page by exact alias match.
- `fuzzy_dedupe_merges` — same, via `pg_trgm` similarity ≥ 0.85 (same-type gate).
- `deterministic_linking_flag_suppressed` — `true` when `WIKI_DETERMINISTIC_LINKING_ENABLED=false` skipped both emitters.

Ontology reprocess jobs have their own metrics on `ontology_reprocess_jobs.metrics`. Those metrics track applied definitions, mappings, facets, materialized pages, preserved higher-trust sections, and errors from the ontology materializer. Use them when an approved change set did not appear in Brain output.

## Related pages

- [Compiled Company Brain Pages](/concepts/knowledge/compounding-memory/) — the overview
- [Company Brain Page Model](/concepts/knowledge/compounding-memory-pages/) — the data model
- [Business Ontology](/concepts/knowledge/business-ontology/) — the governed business vocabulary that shapes tenant Brain materialization
- [Operating Compiled Company Brain Pages](/guides/compounding-memory-operations/) — hands-on recipes
- [Operating Business Ontology](/guides/business-ontology-operations/) — change-set and reprocess recovery workflow
- [Compiled Pages API](/api/compounding-memory/) — GraphQL reference

---

## Under the hood

### Triggers

- **Post-retain** — `maybeEnqueuePostTurnCompile` in `packages/api/src/lib/wiki/enqueue.ts`. Checks `tenants.wiki_compile_enabled`, confirms adapter is Hindsight, inserts a job row with dedupe key `${tenantId}:${ownerId}:${floor(epoch_s/300)}`, fires `Lambda.invoke` with `InvocationType: "Event"` and `{jobId}` payload.
- **Continuation (cap hit)** — when a job exits on any `cap_hit` without draining its cursor, `runCompileJob` enqueues a follow-up into the **next** dedupe bucket via `(parseCompileDedupeBucket(parent.dedupe_key) + 1) * DEDUPE_BUCKET_SECONDS`. The chained job inherits the parent's `trigger`, so `bootstrap_import` continuations keep the higher `MAX_RECORDS_PER_BOOTSTRAP_JOB = 1000` cap and self-complete without manual re-kick. Parsing the parent's dedupe bucket (not `Date.now()` or `created_at`) is what keeps the chain monotonic — see [`docs/solutions/logic-errors/compile-continuation-dedupe-bucket-2026-04-20.md`](/../../solutions/logic-errors/compile-continuation-dedupe-bucket-2026-04-20/).
- **Admin mutation** — `compileWikiNow(tenantId, ownerId, modelId?)` resolver.
- **Nightly lint** — `packages/api/src/handlers/wiki-lint.ts`.
- **CLI** — `thinkwork wiki compile --tenant … --owner …`.

Ontology uses a separate trigger path:

- **Suggestion scan** — `startOntologySuggestionScan` inserts an `ontology_suggestion_scan_jobs` row and invokes `ontology-scan`.
- **Approval reprocess** — `approveOntologyChangeSet` applies an approved ontology version, queues an `ontology_reprocess_jobs` row, and invokes `ontology-reprocess`.

Those jobs can change tenant-shared Brain materialization without advancing an owner-scoped `wiki_compile_cursors` row.

### Caps (leaf pass)

```ts
// packages/api/src/lib/wiki/compiler.ts
const RECORD_PAGE_SIZE = 50;
const MAX_RECORDS_PER_JOB = 500;
const MAX_RECORDS_PER_BOOTSTRAP_JOB = 1000; // applied only when trigger='bootstrap_import'
const MAX_NEW_PAGES_PER_JOB = 25;
const MAX_SECTIONS_REWRITTEN_PER_JOB = 100;
```

### Caps (aggregation pass)

```ts
const MAX_AGGREGATION_PAGES = 60; // top-N recently-changed pages fed to the aggregation planner
```

### Cursor SQL

```ts
// packages/api/src/lib/memory/adapters/hindsight-adapter.ts
WHERE bank_id = ${bankId}
  AND (
    date_trunc('milliseconds', COALESCE(updated_at, created_at)) > ${sinceTs.toISOString()}::timestamptz
    OR (
      date_trunc('milliseconds', COALESCE(updated_at, created_at)) = ${sinceTs.toISOString()}::timestamptz
      AND id::text > ${sinceId}
    )
  )
```

### Leaf planner output contract

```jsonc
{
  "pageUpdates": [
    {
      "pageId": "<uuid>",
      "sections": [
        {
          "slug": "overview",
          "proposed_body_md": "...",
          "source_refs": ["<memory_unit_id>"],
        },
      ],
      "aliases": ["..."],
    },
  ],
  "newPages": [
    {
      "type": "entity|topic|decision",
      "slug": "...",
      "title": "...",
      "summary": "...",
      "aliases": ["..."],
      "source_refs": ["<memory_unit_id>"],
      "sections": [
        {
          "slug": "overview",
          "heading": "Overview",
          "body_md": "...",
          "source_refs": ["<memory_unit_id>"],
        },
      ],
    },
  ],
  "unresolvedMentions": [
    {
      "alias": "Chef João",
      "suggestedType": "entity",
      "context": "mentioned when describing…",
      "source_ref": "<memory_unit_id>",
    },
  ],
  "promotions": [
    {
      "mentionId": "<uuid>",
      "type": "entity",
      "title": "...",
      "slug": "...",
      "sections": [
        /* same shape */
      ],
    },
  ],
  "pageLinks": [
    {
      "fromType": "entity",
      "fromSlug": "...",
      "toType": "topic",
      "toSlug": "...",
      "context": "why this link exists",
    },
  ],
  "parentSectionUpdates": [], // reserved for leaf planner, rarely populated
  "sectionPromotions": [], // same
}
```

### Aggregation planner output contract

```jsonc
{
  "parentSectionUpdates": [
    {
      "pageId": "<parent hub page id>",
      "sectionSlug": "restaurants",
      "heading": "Restaurants",
      "proposed_body_md": "...", // ignored when linked_page_slugs is populated
      "linked_page_slugs": [{ "type": "entity", "slug": "franklin-barbecue" }],
      "source_refs": ["<memory_unit_id>"],
      "observed_tags": ["bbq", "food"],
      "rationale": "…",
    },
  ],
  "sectionPromotions": [
    {
      "pageId": "<parent page id>",
      "sectionSlug": "restaurants",
      "reason": "Section has 20+ linked entities and 30+ days of provenance.",
      "parentSummary": "Short paragraph to leave on the parent section.",
      "topHighlights": ["- Franklin Barbecue", "- Uchi", "- Suerte"],
      "newPage": {
        /* same shape as leaf newPages[] */
      },
    },
  ],
  "newPages": [], // hub topic pages only — banned from creating leaf entities
  "pageLinks": [],
  // all other fields forced to [] — aggregation planner is prevented from re-entering the leaf planner's job
}
```

### Repo-boundary guards

```ts
// packages/api/src/lib/wiki/repository.ts
export function isValidUuid(raw: unknown): raw is string { … }
export function stripWikilinks(md: string | null | undefined): string { … }
// and inside compiler.ts:
export function linkifyKnownEntities(body, refs): string { … }
```

### Promotion scorer

```ts
// packages/api/src/lib/wiki/promotion-scorer.ts
export const DEFAULT_PROMOTION_THRESHOLDS = {
  candidate: 0.4,
  promoteReady: 0.55,
};
const WEIGHTS = {
  linked: 0.25,
  supporting: 0.25,
  temporal: 0.2,
  coherence: 0.15,
  readability: 0.15,
};
const SATURATION = {
  linkedPageCount: 20,
  supportingRecordCount: 30,
  temporalSpreadDays: 30,
  bodyLength: 1800,
};
```

### Parent expanders

```ts
// packages/api/src/lib/wiki/parent-expander.ts
export function deriveParentCandidates(records): DerivedParentCandidate[]; // record metadata
export function deriveParentCandidatesFromPageSummaries(
  pages,
): DerivedParentCandidate[]; // page summaries
export function mergeParentCandidates(...lists): DerivedParentCandidate[];
```

### Deterministic link emitters

```ts
// packages/api/src/lib/wiki/deterministic-linker.ts
export async function emitDeterministicParentLinks(args): Promise<number>; // city/journal → parent topic
export async function emitCoMentionLinks(args): Promise<number>; // entity ↔ entity via shared memory_unit
```

Gated on `WIKI_DETERMINISTIC_LINKING_ENABLED` (default `true`). Context tags `deterministic:city:<slug>`, `deterministic:journal:<slug>`, `co_mention:<memory_unit_id>` make the rows droppable via a single `DELETE ... WHERE context LIKE 'deterministic:%' OR ...` query — see `docs/metrics/wiki-link-density.md` for the rollback SQL.

### Fuzzy alias dedupe

```ts
// packages/api/src/lib/wiki/repository.ts
export async function findAliasMatches(args): Promise<AliasMatch[]>; // exact match (existing)
export async function findAliasMatchesFuzzy(args): Promise<AliasMatch[]>; // pg_trgm similarity ≥ 0.85
```

Backed by migration `packages/database-pg/drizzle/0015_pg_trgm_alias_title_indexes.sql` (`CREATE EXTENSION IF NOT EXISTS pg_trgm` + GIN trigram indexes on `wiki_page_aliases.alias` and `wiki_pages.title`). The compiler's `maybeMergeIntoExistingPage` tries exact first, falls through to fuzzy with a strict same-type gate, and returns `"exact" | "fuzzy" | null` so the caller bumps the right metric.

### Operator scripts

- `packages/api/scripts/wiki-link-density-baseline.ts` — per-agent density snapshot; appends a timestamped markdown file under `docs/metrics/` so before/after flag flips can be diffed.
- `packages/api/scripts/wiki-link-backfill.ts` — one-off `--tenant --owner [--dry-run]` that applies both deterministic emitters to the existing corpus. Idempotent via `onConflictDoNothing` on the `(from, to, kind)` unique index.

### Feature flags (wiki)

- `WIKI_COMPILE_ENABLED` — tenant-scoped, column on `tenants`.
- `WIKI_AGGREGATION_PASS_ENABLED` — Lambda env var, pinned in terraform.
- `WIKI_DETERMINISTIC_LINKING_ENABLED` — Lambda env var, pinned in terraform next to aggregation. Default `true`. `false` ⇒ both deterministic emitters skip and metrics record `deterministic_linking_flag_suppressed: true`.

### Model selection

- Default: `openai.gpt-oss-120b-1:0` (env: `BEDROCK_MODEL_ID`).
- Per-job override: `{ modelId: "…" }` in the compile Lambda event payload.
- Known-model output caps (Nova Micro's 10k limit, etc.) live in `MODEL_MAX_OUTPUT_TOKENS` inside `bedrock.ts` so large `maxTokens` requests get clamped, not rejected.

### Provenance rule

```ts
// packages/api/src/lib/wiki/compiler.ts
function resolveCitedRecords(
  byId: Map<string, ThinkWorkMemoryRecord>,
  ids: string[] | undefined,
): ThinkWorkMemoryRecord[] {
  if (!ids || ids.length === 0) return [];
  const out: ThinkWorkMemoryRecord[] = [];
  for (const id of ids) {
    const r = byId.get(id);
    if (r) out.push(r);
  }
  return out;
}
```

No fallback to batch-wide sources. Single most important correctness invariant in the pipeline.

### Measured cost + shape

Against a full recompile of one agent (~700 source memory units) running gpt-oss-120b with aggregation on:

- ~90 pages created (mix of entity and topic, ~5–10% topics).
- ~115 page links across the graph (`reference` + `parent_of`/`child_of`).
- 1–3 sections promoted per full rebuild depending on density.
- ~$0.50–$1.00 total compile cost.
- 4–6 aggregation rollup sections populated on hub pages.
- Per-batch cost $0.04–$0.12; aggregation pass adds $0.01–$0.05.
