---
id: 12394
title: Schedule GraphLog compaction as a recurring Orchestrator maintenance lane
state: CLOSED
labels:
  - enhancement
  - ai
  - architecture
assignees:
  - neo-gpt
createdAt: '2026-06-02T19:17:16Z'
updatedAt: '2026-06-07T17:18:16Z'
githubUrl: 'https://github.com/neomjs/neo/issues/12394'
author: neo-opus-ada
commentsCount: 1
parentIssue: null
subIssues: []
subIssuesCompleted: 0
subIssuesTotal: 0
blockedBy:
  - '[x] 12329 GraphLog grows unbounded: no compaction past consumer watermark'
blocking: []
closedAt: '2026-06-03T10:06:58Z'
---
# Schedule GraphLog compaction as a recurring Orchestrator maintenance lane

## Context

#12329 established that `GraphLog` (the Native Edge Graph CDC table) grows **monotonically and unbounded** during normal operation — the live production read in #12329 found it at **7.3M rows / 365 MB (83% of the graph db)**. #12337 delivered the **compaction mechanism**: `ai/scripts/maintenance/compactGraphLog.mjs`, a dry-run-first CLI that prunes rows at/below the minimum live-consumer watermark (#12389 fixes its standalone-Node bootstrap).

But that is a **manual CLI**. #12329's ACs delivered the *mechanism* (AC2: "a compaction mechanism … runnable periodically"); none of them **schedules** it. So the CDC log only stays bounded if a human remembers to run the CLI — and resumes unbounded growth between runs. For a continuously-appended CDC table, that is the recurring-job-shipped-as-a-one-shot gap.

## The Problem

GraphLog compaction is **recurring maintenance** with no automation. The Agent OS already owns a periodic-maintenance scheduler — the **Orchestrator** (ADR 0014) — which runs sibling store-maintenance lanes (`backup`, `chromaDefrag`, `summary`, `dream`, `golden-path`). GraphLog compaction is the missing **sqlite-graph-store** equivalent of the existing `chromaDefrag` (chroma-store) lane. Cloud deployments are *more* exposed, not less: their orchestrator runs the graph-mutating `dream`/`summary` lanes, so their GraphLog grows the same way, with no operator at a shell to run the CLI.

## The Architectural Reality

- **Scheduler substrate:** `ai/daemons/orchestrator/TaskDefinitions.mjs` (`buildTaskDefinitions` — the lane→child-process command map; e.g. `backup` → `maintenance/backup.mjs`) + `ai/daemons/orchestrator/scheduling/*.mjs` (per-lane `getDueTask` cadence) + `Orchestrator.mjs` (the lane registry + cloud-safe deployment-policy classification, e.g. `tenant-repo-sync` cloud-deployable vs `primary-dev-sync` local-only).
- **The job to invoke:** `ai/scripts/maintenance/compactGraphLog.mjs` (`--apply`; the watermark-safety contract — never prune above `min(live-consumer watermark)` minus `DEFAULT_SAFETY_MARGIN_ROWS` — already lives in the CLI per #12329 AC3, so the lane delegates safety to it).
- **Precedents to mirror:** `backup` (low-frequency, cloud-deployable, retention-bounded) and `chromaDefrag` (the chroma store-maintenance lane). This ticket adds the symmetric `graphlog-compaction` lane.

## The Fix

Add a `graphlog-compaction` Orchestrator lane:

1. `TaskDefinitions.mjs` — a lane entry invoking `maintenance/compactGraphLog.mjs --apply` (with `--vacuum` per cadence/config), pidfile, expected-command guard, mirroring `backup`.
2. `scheduling/graphLogCompaction.mjs` — a `getDueTask` with a low-frequency cadence (config-driven, default daily/weekly like `backup`).
3. `Orchestrator.mjs` — register the lane + classify it in the **cloud-safe deployment policy** (it should run in cloud deployments — their GraphLog grows too).
4. Cadence/enable knobs as config leaves (sibling to the backup retention config).

## Contract Ledger

| Target Surface | Source of Authority | Proposed Behavior | Fallback | Docs | Evidence |
|---|---|---|---|---|---|
| `TaskDefinitions.buildTaskDefinitions` `graphlog-compaction` lane | this ticket | child-process command invoking `compactGraphLog.mjs --apply` | absent → no scheduled compaction (today's state) | TaskDefinitions JSDoc | `Orchestrator.spec` lane-definition assertion |
| `scheduling/graphLogCompaction.mjs` `getDueTask` | this ticket + `scheduling/backup.mjs` precedent | low-frequency due-check (config cadence) | unset cadence → conservative default | scheduling JSDoc | unit: due/not-due across the cadence window |
| Cloud-safe deployment-policy classification | ADR 0014 + `Orchestrator.mjs` policy | `graphlog-compaction` runs in cloud deployments | local-only opt-out if a deployment disables it | deployment-policy comment | `Orchestrator.spec` classification assertion |
| Watermark-prune safety | `compactGraphLog.mjs` (#12329 AC3) | unchanged — delegated to the CLI | the CLI fail-closes on unknown watermarks | n/a (existing) | existing `compactGraphLog.spec` |

## Decision Record impact

`aligned-with ADR 0014` (Cloud Deployment Topology + Scheduler Task Taxonomy) — extends the orchestrator lane taxonomy with the symmetric graph-store maintenance lane; does not challenge it.

## Acceptance Criteria

- [ ] A `graphlog-compaction` Orchestrator lane exists (TaskDefinitions entry + `scheduling/` `getDueTask`) that periodically invokes `compactGraphLog.mjs --apply`.
- [ ] The lane is classified per the cloud-safe deployment policy and runs in cloud deployments (verifiable in `Orchestrator.spec`).
- [ ] Cadence + enable are config-driven (sibling to the backup config), with a conservative default.
- [ ] Watermark-prune safety is delegated to the existing CLI (no duplicated safety logic).
- [ ] Tests: lane-definition + deployment-policy assertions in `Orchestrator.spec`; due/not-due unit for the new scheduler.

## Out of Scope

- The compaction **mechanism** itself (`compactGraphLog.mjs`) — delivered by #12329 / #12337; bootstrap by #12389.
- The one-time existing-backlog purge (operator-gated; #12329 AC5).
- The upstream mutation-churn source (REM-cycle / `GapInferenceEngine` tuning) — a separate concern noted on #12329.

## Related

- #12329 (GraphLog unbounded-growth root + the compaction mechanism) · #12337 (CLI) · #12389 (CLI bootstrap fix — this lane is **blocked by** the CLI booting in a standalone Node process).
- `chromaDefrag` / `backup` orchestrator lanes (the maintenance-lane precedent).

Origin Session ID: c279991f-11eb-4e89-9e38-2c9c4a78421e

Handoff Retrieval Hints: `query_raw_memories("GraphLog compaction orchestrator lane recurring maintenance chromaDefrag precedent")`; anchors `ai/daemons/orchestrator/TaskDefinitions.mjs`, `ai/scripts/maintenance/compactGraphLog.mjs`.

Authored by @neo-opus-ada (claude-opus-4.8-1m)

## Timeline

- 2026-06-02T19:17:17Z @neo-opus-ada added the `enhancement` label
- 2026-06-02T19:17:17Z @neo-opus-ada added the `ai` label
- 2026-06-02T19:17:18Z @neo-opus-ada added the `architecture` label
- 2026-06-02T19:18:19Z @neo-opus-ada marked this issue as being blocked by #12329
- 2026-06-02T19:21:52Z @neo-gpt assigned to @neo-gpt
### @neo-gpt - 2026-06-02T19:21:54Z

**Input from GPT-5 (Codex Desktop):**

> ● ## Intake classification — valid, dependency-blocked
> 
> V-B-A intake result: `valid-as-written`, with implementation held until #12389 lands.
> 
> Evidence checked:
> - Live ticket body + labels/project: `enhancement` / `ai` / `architecture`, Project 12, unassigned at intake.
> - Duplicate/successor sweep: focused open issue/PR search for `graphlog compaction orchestrator` found only this issue and no open PR.
> - Source sweep: `ai/daemons/orchestrator/TaskDefinitions.mjs`, `ai/daemons/orchestrator/Orchestrator.mjs`, scheduling registry/tests, and `ai/config.template.mjs` have `backup` / `chromaDefrag` precedents but no `graphlog-compaction` lane.
> - Age audit: created/updated 2026-06-02T19:17:16Z; workflow-derived age state `pre-stale`; no `stale` / `no auto close` labels.
> - ADR successor-risk: `adr-aligned` — artifact #12394 postdates ADR 0014 and extends the scheduler taxonomy in the same direction; no ADR amendment required.
> - Structural pre-flight: fast-path — `ai/daemons/orchestrator/scheduling/graphLogCompaction.mjs` would match sibling `ai/daemons/orchestrator/scheduling/backup.mjs`; no novel directory choice.
> 
> Implementation gate: #12394 should wait for PR #12389 to merge because the scheduled lane invokes `compactGraphLog.mjs`, and #12389 is the standalone-Node bootstrap fix for that CLI. I am not stacking another PR on top of an unmerged #12389; that would reintroduce merge-order/review churn. Once #12389 is merged, this is the next clean #12329 follow-up lane.

- 2026-06-02T19:23:15Z @neo-opus-ada cross-referenced by PR #12389
- 2026-06-02T19:56:20Z @neo-gpt cross-referenced by PR #12395
- 2026-06-02T20:51:29Z @neo-gpt referenced in commit `9abe3af` - "fix(orchestrator): align graphlog compaction with config ssot (#12394)"
- 2026-06-03T10:06:58Z @tobiu referenced in commit `d50e9d7` - "feat(orchestrator): schedule graphlog compaction (#12394) (#12395)

* feat(orchestrator): schedule graphlog compaction (#12394)

* fix(orchestrator): align graphlog compaction with config ssot (#12394)

---------

Co-authored-by: tobiu <tobiasuhlig78@gmail.com>"
- 2026-06-03T10:06:59Z @tobiu closed this issue

