---
id: 9914
title: 'Epic: Native Edge Graph Auditing and Deduplication Pipeline'
state: CLOSED
labels:
  - enhancement
  - ai
  - architecture
assignees:
  - tobiu
createdAt: '2026-04-12T11:37:21Z'
updatedAt: '2026-04-12T20:22:29Z'
githubUrl: 'https://github.com/neomjs/neo/issues/9914'
author: tobiu
commentsCount: 1
parentIssue: null
subIssues: []
subIssuesCompleted: 0
subIssuesTotal: 0
blockedBy:
  - '[x] 9912 bug(ai): Prevent Vector Apoptosis from eradicating structural entities'
blocking:
  - '[ ] 9904 Epic: RLAIF Reward Function and Model Orchestration Pipeline'
closedAt: '2026-04-12T20:22:29Z'
---
# Epic: Native Edge Graph Auditing and Deduplication Pipeline

### Context
The SQLite memory matrix has rapidly scaled past 50,000 raw memory nodes. To stabilize RLAIF querying logic and reduce Context Bloat during RAG lookups, we must implement an automated database pruning daemon.

### Scope
- Cleanse duplicate string mappings.
- Identify orphaned scalar `CONTAINS` edges.
- Provide a `vacuum` / `compact` abstraction over the memory matrix to speed up memory-core operations.

## Timeline

- 2026-04-12T11:37:26Z @tobiu added the `epic` label
- 2026-04-12T11:37:27Z @tobiu added the `ai` label
- 2026-04-12T11:37:27Z @tobiu added the `architecture` label
- 2026-04-12T11:40:03Z @tobiu marked this issue as being blocked by #9912
- 2026-04-12T11:40:06Z @tobiu marked this issue as blocking #9904
### @tobiu - 2026-04-12T20:11:43Z

**Input from Antigravity (Gemini 3.1 Pro):**

> ✦ ### 🔍 Defect Diagnosis: 19,500 Duplicate `CONTAINS` Edges
> Agent Antigravity actively investigated the Graph bloat issue regarding enormous amounts of orphaned scalar `CONTAINS` edges.
> 
> **Root Cause Analysis:**
> The duplicate edges are actively generated by `FileSystemIngestor.mjs`. 
> When syncing the physical workspace structure to the Graph, the Ingestor creates `CONTAINS` edges from directories to files. To avoid duplicates, it uses:
> `GraphService.db.edges.items.find(e => e.source === parentId && ...)`
> 
> Because SQLite manages thousands of records natively, it does **not** load the full 36,000+ edge historical ledger into the `db.edges.items` RAM array. Therefore, `.find()` inevitably evaluates to `false` for existing structure edges, triggering recursive edge injection on every synchronization.
> 
> **Proposed Resolution within this Ticket:**
> 1. **Palliative Deduplication:** Directly utilize SQLite's atomic abilities to DELETE duplicate mappings `HAVING count(*) > 1`.
> 2. **Structural Fix:** Refactor `FileSystemIngestor.mjs` to bypass raw array `.find()` lookups. We will strictly route its edge insertion logic through `GraphService.linkNodes(parentId, nodeId, 'CONTAINS', 1.0)`, which relies on synchronous SQL lookup checks (`SELECT 1 FROM Edges WHERE...`) to guarantee 100% deduplicated data constraints directly on the disk.

- 2026-04-12T20:13:54Z @tobiu referenced in commit `0500d71` - "refactor(ai): Eliminate Edge topology duplicate bloat via native GraphService linking (#9914)"
- 2026-04-12T20:14:05Z @tobiu cross-referenced by PR #9943
- 2026-04-12T20:18:56Z @tobiu assigned to @tobiu
- 2026-04-12T20:19:10Z @tobiu removed the `epic` label
- 2026-04-12T20:19:10Z @tobiu added the `enhancement` label
- 2026-04-12T20:22:29Z @tobiu referenced in commit `d3e1a9f` - "refactor(ai): Eliminate Edge topology duplicate bloat via native GraphService linking (#9914) (#9943)"
- 2026-04-12T20:22:29Z @tobiu closed this issue

