# Phase 2: Data Backup & Recovery - Research

**Researched:** 2026-02-26
**Domain:** Incremental filesystem backup, SHA-256 integrity, scheduler integration, restore tooling, dashboard widget
**Confidence:** HIGH

## Summary

Phase 2 implements a complete backup and restore system for PortOS's `./data/` directory using `rsync` (already available on macOS as `openrsync` at `/usr/bin/rsync`), Node.js built-in `crypto` for SHA-256 manifests, and the existing `eventScheduler` service for scheduling. No new npm dependencies are required for the core backup engine — all critical primitives exist in the current stack.

The design follows a snapshot-per-run model: each backup run creates a dated directory under the external drive target path (e.g., `<dest>/snapshots/2026-02-26T14-30-00/`). Rsync's `--archive` and `--itemize-changes` flags handle incremental transfer. A SHA-256 manifest file is generated by Node.js `crypto` after each successful run. Restore operations re-use rsync in reverse with `--dry-run` for preview. The dashboard widget uses the existing `useAutoRefetch` hook pattern for live status.

**Primary recommendation:** Implement backup as a standalone `backupService.js` with a matching `backupScheduler.js` (mirroring `brainScheduler.js`), a `routes/backup.js` REST layer, and a `BackupWidget.jsx` dashboard component. Use Node.js `spawn('rsync', [...], { shell: false })` for all rsync invocations, consistent with how `commands.js` executes child processes. Store backup state in `data/backup/state.json`.

<phase_requirements>
## Phase Requirements

| ID | Description | Research Support |
|----|-------------|-----------------|
| BAK-01 | Incremental rsync backup copies changed files from `./data/` to configurable external drive path | rsync `--archive --itemize-changes` with snapshot dirs; `spawn()` from Node.js |
| BAK-02 | SHA-256 manifest tracks file integrity across backups | Node.js `crypto.createHash('sha256')` — no new deps; walk destination tree after rsync, write manifest.json |
| BAK-03 | Daily backup runs via existing scheduler with configurable interval | `eventScheduler.schedule({ type: 'cron', cron: '0 2 * * *', ... })` — directly uses existing scheduler |
| BAK-04 | Dashboard widget shows last backup time, next scheduled, health status | `BackupWidget.jsx` using `useAutoRefetch` hook + GET `/api/backup/status` endpoint |
| BAK-05 | One-click manual backup trigger from dashboard | POST `/api/backup/run` route; widget button dispatches it with toast notification |
| BAK-06 | Restore from named snapshot with dry-run mode showing what would change | `rsync --dry-run --archive <snapshot>/ <dest>/` captures itemize-changes output for preview |
| BAK-07 | Selective directory restore (e.g., only `data/brain/` without touching other dirs) | rsync `--include='brain/***' --exclude='*'` pattern applied to snapshot path |
</phase_requirements>

## Standard Stack

### Core

| Library | Version | Purpose | Why Standard |
|---------|---------|---------|--------------|
| `rsync` (system) | openrsync 2.6.9 compat | Incremental file sync to external drive | Already on macOS; `--archive`, `--itemize-changes`, `--dry-run`, `--link-dest` all verified working |
| `node:crypto` | Node built-in | SHA-256 hash of each file for manifest | Zero deps; `createHash('sha256').update(buf).digest('hex')` works in tests |
| `node:child_process` `spawn` | Node built-in | Execute rsync from server process | Consistent with `commands.js` pattern; `shell: false` prevents injection |
| `node:fs/promises` | Node built-in | Walk destination tree, write manifest/state | Already used everywhere in services |
| `eventScheduler` | Project service | Cron scheduling for daily backup | Existing scheduler; `schedule({ type: 'cron', cron: '0 2 * * *', ... })` |
| `zod` | ^3.24.1 | Validate backup config / restore request inputs | Project standard; in `lib/validation.js` |

### Supporting

| Library | Version | Purpose | When to Use |
|---------|---------|---------|-------------|
| `socket.io` | ^4.8.3 | Emit real-time progress during backup run | Already initialized; emit `backup:progress` events; widget listens optionally |
| `uuid` | ^11.0.3 | Generate snapshot IDs if needed | Already in package.json |

### Alternatives Considered

| Instead of | Could Use | Tradeoff |
|------------|-----------|----------|
| System rsync + Node.js crypto | `node-rsync` npm package | Extra dep with no meaningful benefit; direct spawn is simpler and already the project pattern |
| System rsync + Node.js crypto | `tar` + `gzip` archives | Tarballs don't support incremental transfers; harder to restore individual files |
| eventScheduler cron | `node-cron` npm package | Extra dep; `eventScheduler` already supports cron expressions and is the project-standard scheduler |
| Per-file SHA-256 in manifest | `md5` or `sha1` | SHA-256 is stronger, is the project requirement (BAK-02), and crypto is built-in |

**Installation:** No new packages required.

## Architecture Patterns

### Recommended Project Structure

```
server/
  routes/
    backup.js              # REST handlers: GET /status, POST /run, POST /restore, GET /snapshots
  services/
    backup.js              # Core logic: runBackup(), generateManifest(), restore(), listSnapshots()
    backupScheduler.js     # Scheduler init: mirrors brainScheduler.js pattern
  lib/
    validation.js          # Add: backupConfigSchema, restoreRequestSchema (extend existing file)

client/src/
  components/
    BackupWidget.jsx       # Dashboard widget: status, manual trigger, health indicator
  services/
    api.js                 # Add: getBackupStatus(), triggerBackup(), getBackupSnapshots(), restoreBackup()

data/
  backup/
    state.json             # { lastRun, nextRun, status, snapshotCount, lastError }
```

### Pattern 1: Snapshot Directory Layout on External Drive

**What:** Each backup run creates a dated subdirectory. rsync copies only changed files. Manifest is stored inside the snapshot directory.

**When to use:** Default for all runs.

```
<destPath>/                          # User-configured external drive mount
  snapshots/
    2026-02-26T02-00-00/
      data/                          # Mirror of ./data/
        brain/
        meatspace/
        ...
      manifest.json                  # { generatedAt, fileCount, files: { path: sha256 } }
    2026-02-25T02-00-00/
      data/
      manifest.json
  state.json                         # { lastSnapshot, snapshotCount }
```

**Why dated dirs vs. link-dest:** macOS `openrsync` supports `--link-dest` but snapshot dirs are simpler to browse and restore from. `--link-dest` can be added in v2 (BAK-10 deferred). For v1, simple dated copies are correct.

### Pattern 2: rsync Invocation (Incremental Transfer)

**What:** Spawn rsync with `--archive --itemize-changes` (no `--delete` in v1 to be safe). Capture stdout line-by-line for file tracking.

**When to use:** All backup runs.

```javascript
// Source: verified against openrsync 2.6.9 on macOS 24.6.0
import { spawn } from 'child_process';
import { join } from 'path';

function runRsync(srcDir, destDir, flags = []) {
  const args = [
    '--archive',          // recursive + preserve metadata
    '--itemize-changes',  // per-file change indicators for logging
    ...flags,
    `${srcDir}/`,         // trailing slash = copy contents, not dir itself
    destDir
  ];

  const child = spawn('rsync', args, { shell: false });
  const changed = [];

  child.stdout.on('data', (chunk) => {
    const lines = chunk.toString().split('\n').filter(Boolean);
    for (const line of lines) {
      if (line.startsWith('>') || line.startsWith('<')) {
        changed.push(line.trim());
      }
    }
  });

  return new Promise((resolve, reject) => {
    child.on('close', (code) => {
      if (code === 0 || code === 24) { // 24 = partial transfer (vanished files), acceptable
        resolve({ changed, code });
      } else {
        reject(new Error(`rsync exited with code ${code}`));
      }
    });
    child.on('error', reject);
  });
}
```

### Pattern 3: SHA-256 Manifest Generation

**What:** Walk the snapshot `data/` directory with `fs.readdir` recursively, hash each file, write `manifest.json`.

```javascript
// Source: Node.js built-in crypto (verified working on Node 20+)
import { createHash } from 'crypto';
import { readFile, writeFile, readdir, stat } from 'fs/promises';
import { join, relative } from 'path';

async function generateManifest(snapshotDataDir, manifestPath) {
  const files = {};
  await walkDir(snapshotDataDir, async (filePath) => {
    const content = await readFile(filePath);
    const hash = createHash('sha256').update(content).digest('hex');
    files[relative(snapshotDataDir, filePath)] = hash;
  });
  const manifest = {
    generatedAt: new Date().toISOString(),
    fileCount: Object.keys(files).length,
    files
  };
  await writeFile(manifestPath, JSON.stringify(manifest, null, 2));
  return manifest;
}
```

### Pattern 4: Scheduler Integration

**What:** Register a cron event with the existing `eventScheduler` at server startup. Mirror `brainScheduler.js` exactly.

```javascript
// Source: server/services/eventScheduler.js (project code, verified)
import * as eventScheduler from './eventScheduler.js';
import { runBackup } from './backup.js';
import { getSettings } from './settings.js';

let scheduledEventId = null;

export function startBackupScheduler() {
  const id = 'backup-daily';
  eventScheduler.schedule({
    id,
    type: 'cron',
    cron: '0 2 * * *',  // default: 2 AM daily; override from settings
    handler: async () => {
      console.log('💾 Backup scheduler: Running daily backup...');
      const settings = await getSettings();
      await runBackup(settings.backup?.destPath);
    },
    metadata: { source: 'backupScheduler' }
  });
  scheduledEventId = id;
  console.log('💾 Backup scheduler started');
}

export function stopBackupScheduler() {
  if (scheduledEventId) {
    eventScheduler.cancel(scheduledEventId);
    scheduledEventId = null;
  }
}
```

### Pattern 5: Dry-Run Restore Preview

**What:** Run rsync with `--dry-run` flag. Capture `--itemize-changes` output. Return list of files that would be changed before actually running.

```javascript
// Source: verified against openrsync on macOS (--dry-run supported, exit 0)
async function previewRestore(snapshotDataDir, targetDir, subdirFilter = null) {
  const flags = ['--dry-run'];
  if (subdirFilter) {
    // Selective restore: only data/brain/ for example
    flags.push(`--include=${subdirFilter}/***`, '--exclude=*');
  }
  const { changed } = await runRsync(snapshotDataDir, targetDir, flags);
  return changed; // array of itemize-changes lines describing what would change
}
```

### Pattern 6: Zod Validation for Backup Routes

**What:** Add backup schemas to `server/lib/validation.js` following the existing schema pattern.

```javascript
// Add to server/lib/validation.js
export const backupConfigSchema = z.object({
  destPath: z.string().min(1),                     // external drive mount path
  cronExpression: z.string().optional(),            // e.g., '0 2 * * *'
  enabled: z.boolean().optional().default(true)
});

export const restoreRequestSchema = z.object({
  snapshotId: z.string().min(1),                   // snapshot directory name
  subdirFilter: z.string().optional().nullable(),  // e.g., 'brain' for selective restore
  dryRun: z.boolean().optional().default(true)     // always default to safe dry-run
});
```

### Pattern 7: Dashboard Widget

**What:** React component using `useAutoRefetch` (existing hook) to poll `/api/backup/status`. Shows last backup time, next scheduled time, and health status (green/yellow/red). Follows `SystemHealthWidget.jsx` structural pattern.

```jsx
// Pattern from SystemHealthWidget.jsx and CosDashboardWidget.jsx
import { useAutoRefetch } from '../hooks/useAutoRefetch';
import * as api from '../services/api';

const BackupWidget = memo(function BackupWidget() {
  const { data: status, loading } = useAutoRefetch(
    () => api.getBackupStatus({ silent: true }).catch(() => null),
    60000  // refresh every 60s
  );

  // health: 'healthy' (last run < 25h ago, no error)
  //         'warning' (last run 25-49h ago or missed)
  //         'critical' (last run > 49h ago or errored)
  const health = computeHealth(status);  // green/yellow/red

  // Manual trigger button: POST /api/backup/run
  // Uses toast for feedback (no window.alert)
});
```

### Anti-Patterns to Avoid

- **Running rsync with `shell: true`:** Project convention in `commands.js` is `shell: false`. Rsync called directly with `spawn('rsync', argsArray, { shell: false })` prevents injection.
- **Using `window.alert` / `window.confirm`:** Project forbids this. All feedback via `react-hot-toast` (already imported in `api.js`). Confirmation of destructive restore goes via inline modal UI state, not a dialog.
- **Hardcoding `localhost`:** Widget fetches from `/api/backup/...` (relative), not `http://localhost:5554`. See CLAUDE.md.
- **Storing backup state outside `data/`:** Backup state JSON goes in `data/backup/state.json` so it is itself backed up.
- **Deleting old snapshots in v1:** `--delete` is deferred (BAK-10 retention policy is v2). v1 snapshots accumulate until disk fills — document this, provide snapshot list UI.
- **Giant try/catch blocks:** Project pattern (CLAUDE.md) is no try/catch; errors bubble to `asyncHandler` middleware.
- **Blocking the event loop with large file reads:** Manifest generation reads files sequentially to avoid memory spikes. For very large data dirs, use streaming hash: `fs.createReadStream()` piped to `crypto.createHash()`.

## Don't Hand-Roll

| Problem | Don't Build | Use Instead | Why |
|---------|-------------|-------------|-----|
| Incremental file sync | Custom file-diff logic | `rsync --archive --itemize-changes` | rsync handles timestamps, permissions, symlinks, partial transfers correctly |
| File integrity verification | Custom file comparison | Node.js `crypto.createHash('sha256')` | Built-in, battle-tested, no deps |
| Cron scheduling | Custom minute-check interval | `eventScheduler.schedule({ type: 'cron', ... })` | Already in project, handles edge cases, re-uses existing infrastructure |
| Progress streaming | Custom WebSocket protocol | Emit `backup:progress` on existing `socket.io` instance (`req.app.get('io')`) | Already initialized in `server/index.js`; pattern used by other services |

**Key insight:** The entire backup engine is assembling existing system tools and Node built-ins. The only novel logic is the state tracking, manifest generation, and schedule integration.

## Common Pitfalls

### Pitfall 1: macOS openrsync vs GNU rsync feature gaps
**What goes wrong:** openrsync (macOS default) has different flag support than GNU rsync. Some GNU flags (`--checksum` for file-content comparison, `--link-dest` in some forms) may behave differently.
**Why it happens:** Apple ships OpenBSD's rsync implementation, not the GNU one from rsync.samba.org.
**How to avoid:** Verified flags on this system: `--archive`, `--itemize-changes`, `--dry-run`, `--link-dest`, `--include`, `--exclude` all work (confirmed via manual test). Do NOT use `--checksum` for deduplication (not reliably supported). Do NOT use rsync's built-in checksums as a replacement for the Node.js SHA-256 manifest.
**Warning signs:** rsync exits with unexpected error codes or silently ignores flags — always check exit code in the spawn handler.

### Pitfall 2: External Drive Not Mounted
**What goes wrong:** Backup silently fails or creates directories in wrong location if the drive path doesn't exist.
**Why it happens:** External drives have non-deterministic mount paths on macOS (e.g., `/Volumes/BackupDrive`).
**How to avoid:** Before running backup, verify `destPath` exists with `fs.access(destPath)`. If missing: set state to `{ status: 'error', error: 'Destination path not found' }`, emit Socket.IO event for dashboard notification. Never auto-create the destPath (could silently write to wrong location).
**Warning signs:** Dashboard shows "drive not found" error state; no snapshot directories created.

### Pitfall 3: Concurrent Backup Runs
**What goes wrong:** If the cron fires while a manual backup is in progress, or if two manual triggers are fired rapidly, rsync instances conflict on the destination path.
**Why it happens:** No lock mechanism.
**How to avoid:** Use an in-memory `isRunning` flag in `backup.js`. Check before starting; return early if already running. Store `{ status: 'running', startedAt }` in state.json while active.
**Warning signs:** Two rsync processes with same source/dest in `ps aux`; corrupted snapshot directory.

### Pitfall 4: Large File Reads for Manifest on Large `./data/` Directories
**What goes wrong:** Reading entire files into memory for SHA-256 hashing causes memory spikes if `data/runs/` contains many large AI output files.
**Why it happens:** `readFile()` loads entire file into a Buffer.
**How to avoid:** Use streaming hash for files over a size threshold (e.g., > 1MB): `fs.createReadStream(path).pipe(hash)`. Or hash only metadata (path + mtime + size) for the manifest rather than content — acceptable approximation since rsync already handles content integrity.
**Warning signs:** Node.js memory usage spikes during manifest generation; OOM in PM2 logs.

### Pitfall 5: Rsync Exit Code 24 (Partial Transfer / Vanished Files)
**What goes wrong:** Treat exit code 24 as an error, causing backup to report failure when it actually succeeded.
**Why it happens:** Exit code 24 means "some source files vanished during transfer" — completely normal for an active system where JSON files are being written.
**How to avoid:** Accept exit codes 0 and 24 as success. Log the code but don't set error state. Only codes 1, 2, 3, etc. are real errors.
**Warning signs:** Backup always reports "failed" despite successful file transfer.

### Pitfall 6: Route Not Registered in index.js
**What goes wrong:** New backup routes exist but return 404 because they weren't imported in `server/index.js`.
**Why it happens:** The import + `app.use()` registration pattern requires explicit wiring.
**How to avoid:** After creating `routes/backup.js`, add to `server/index.js` alphabetically (project convention): `import backupRoutes from './routes/backup.js'` and `app.use('/api/backup', backupRoutes)`.
**Warning signs:** 404 on all `/api/backup/*` endpoints; widget shows loading forever.

### Pitfall 7: Backup Widget Hidden Due to Missing Data
**What goes wrong:** Widget renders nothing during initial load because status is null, leading to layout shift.
**Why it happens:** `useAutoRefetch` starts with `loading: true`, and widgets often return null while loading.
**How to avoid:** Unlike CosDashboardWidget (which hides when no activity), BackupWidget should always render since backup status is always relevant. Show a skeleton/loading state rather than returning null. If backup has never run, show "No backups yet" with configure button.

## Code Examples

### Run Backup (Core Flow)

```javascript
// server/services/backup.js
import { spawn } from 'child_process';
import { createHash } from 'crypto';
import { readFile, writeFile, readdir, stat, mkdir, access } from 'fs/promises';
import { join, relative } from 'path';
import { PATHS } from '../lib/fileUtils.js';

let isRunning = false;

export async function runBackup(destPath) {
  if (isRunning) {
    console.log('💾 Backup already running, skipping');
    return { skipped: true };
  }
  if (!destPath) throw new Error('No backup destination configured');

  // Verify destination exists
  await access(destPath).catch(() => {
    throw new Error(`Backup destination not found: ${destPath}`);
  });

  isRunning = true;
  const startedAt = new Date().toISOString();
  const snapshotId = startedAt.replace(/:/g, '-').replace(/\..+/, '');
  const snapshotDir = join(destPath, 'snapshots', snapshotId);
  const dataDestDir = join(snapshotDir, 'data');

  console.log(`💾 Backup starting: snapshot ${snapshotId}`);

  await mkdir(dataDestDir, { recursive: true });
  const { changed, code } = await runRsync(PATHS.data, dataDestDir);
  console.log(`💾 Backup rsync complete: ${changed.length} files changed (exit ${code})`);

  const manifest = await generateManifest(dataDestDir, join(snapshotDir, 'manifest.json'));
  console.log(`💾 Backup manifest generated: ${manifest.fileCount} files`);

  const completedAt = new Date().toISOString();
  await saveState({ lastRun: completedAt, lastSnapshotId: snapshotId, status: 'ok', filesChanged: changed.length });

  isRunning = false;
  return { snapshotId, filesChanged: changed.length, fileCount: manifest.fileCount };
}
```

### List Snapshots

```javascript
export async function listSnapshots(destPath) {
  const snapshotsDir = join(destPath, 'snapshots');
  const entries = await readdir(snapshotsDir).catch(() => []);
  const snapshots = await Promise.all(
    entries.map(async (name) => {
      const manifestPath = join(snapshotsDir, name, 'manifest.json');
      const manifest = await readFile(manifestPath, 'utf-8').then(JSON.parse).catch(() => null);
      return { id: name, createdAt: name, fileCount: manifest?.fileCount ?? null };
    })
  );
  return snapshots.sort((a, b) => b.id.localeCompare(a.id)); // newest first
}
```

### Restore with Dry-Run

```javascript
// server/services/backup.js
export async function restoreSnapshot(destPath, snapshotId, { dryRun = true, subdirFilter = null } = {}) {
  const snapshotDataDir = join(destPath, 'snapshots', snapshotId, 'data');
  const flags = dryRun ? ['--dry-run'] : [];
  if (subdirFilter) {
    // Only restore e.g. 'brain' directory
    flags.push(`--include=${subdirFilter}/***`, '--include=*/', '--exclude=*');
  }
  const { changed } = await runRsync(snapshotDataDir, PATHS.data, flags);
  console.log(`💾 Restore ${dryRun ? 'preview' : 'completed'}: snapshot ${snapshotId}, ${changed.length} files ${dryRun ? 'would change' : 'changed'}`);
  return { dryRun, snapshotId, subdirFilter, changedFiles: changed };
}
```

### REST Routes

```javascript
// server/routes/backup.js
import { Router } from 'express';
import { asyncHandler, ServerError } from '../lib/errorHandler.js';
import { validateRequest } from '../lib/validation.js';
import { restoreRequestSchema } from '../lib/validation.js';
import * as backup from '../services/backup.js';
import { getSettings } from '../services/settings.js';

const router = Router();

// GET /api/backup/status
router.get('/status', asyncHandler(async (req, res) => {
  const state = await backup.getState();
  const settings = await getSettings();
  const nextRun = backup.getNextRunTime(); // from eventScheduler
  res.json({ ...state, destPath: settings.backup?.destPath, nextRun });
}));

// POST /api/backup/run
router.post('/run', asyncHandler(async (req, res) => {
  const settings = await getSettings();
  const result = await backup.runBackup(settings.backup?.destPath);
  res.json(result);
}));

// GET /api/backup/snapshots
router.get('/snapshots', asyncHandler(async (req, res) => {
  const settings = await getSettings();
  const snapshots = await backup.listSnapshots(settings.backup?.destPath);
  res.json(snapshots);
}));

// POST /api/backup/restore
router.post('/restore', asyncHandler(async (req, res) => {
  const { snapshotId, subdirFilter, dryRun } = validateRequest(restoreRequestSchema, req.body);
  const settings = await getSettings();
  const result = await backup.restoreSnapshot(settings.backup?.destPath, snapshotId, { dryRun, subdirFilter });
  res.json(result);
}));

export default router;
```

### API Client Functions (to add to `api.js`)

```javascript
// client/src/services/api.js additions
export const getBackupStatus = (options) => request('/backup/status', options);
export const triggerBackup = () => request('/backup/run', { method: 'POST' });
export const getBackupSnapshots = (options) => request('/backup/snapshots', options);
export const restoreBackup = (data) => request('/backup/restore', { method: 'POST', body: JSON.stringify(data) });
```

### Settings Schema (extend existing `settings.json`)

```javascript
// Backup config stored in data/settings.json under 'backup' key
{
  "backup": {
    "destPath": "/Volumes/BackupDrive/PortOS",   // user-configured
    "cronExpression": "0 2 * * *",               // 2 AM daily default
    "enabled": true
  }
}
```

### Backup State File (`data/backup/state.json`)

```json
{
  "lastRun": "2026-02-26T02:00:00.000Z",
  "lastSnapshotId": "2026-02-26T02-00-00",
  "status": "ok",
  "filesChanged": 12,
  "nextRun": "2026-02-27T02:00:00.000Z",
  "error": null
}
```

## State of the Art

| Old Approach | Current Approach | When Changed | Impact |
|--------------|------------------|--------------|--------|
| Manual tar archives | rsync incremental with snapshot dirs | Standard for local backup | Faster, resumable, browsable |
| MD5 manifests | SHA-256 manifests | 2015+ (SHA-1 weaknesses) | More secure |
| cron daemon (system-level) | App-internal scheduler | Node.js ecosystem shift | No system cron config needed |
| Full restore only | Selective directory restore | Modern backup tools | Less disruptive restore |

**Deprecated/outdated:**
- System-level `crontab` entries: Project self-manages scheduling via `eventScheduler`. Don't add crontab entries.
- `node-cron` npm package: Project already has a custom `eventScheduler` with cron support. Redundant.

## Open Questions

1. **External Drive Detection**
   - What we know: Backup destination is user-configured (a path, not auto-detected)
   - What's unclear: Should the dashboard show available external drives for user to pick? Or just a text field for path?
   - Recommendation: Start with a simple text field in settings (via PUT `/api/settings`). Drive picker is UI complexity not in v1 requirements.

2. **Backup Config Location**
   - What we know: Settings live in `data/settings.json` managed by `settings.js`
   - What's unclear: Should backup config have its own file (`data/backup/config.json`) or extend the global settings object?
   - Recommendation: Extend global `settings.json` under a `backup` key — consistent with how brain settings work. No new file needed.

3. **Streaming Progress to Dashboard (BAK-03 / BAK-04)**
   - What we know: Socket.IO is available (`req.app.get('io')`) and `backup:progress` events can be emitted
   - What's unclear: Does the widget need real-time rsync file-by-file progress, or just start/complete?
   - Recommendation: Emit `backup:started` and `backup:completed` events only. File-by-file progress is scope creep for v1. Widget polls via `useAutoRefetch` every 60s; during active backup shows "Running..." from state.json.

4. **Manifest File Size**
   - What we know: `data/` contains `runs/` (AI run outputs, potentially hundreds of files) and `memory/` embeddings
   - What's unclear: Could manifest.json become very large (thousands of file hashes)?
   - Recommendation: Use streaming hash (`fs.createReadStream` piped to hash) for files > 512KB to prevent memory spikes. File count will likely be hundreds not thousands — acceptable.

5. **Rsync `rsync` not found on remote Tailscale access**
   - What we know: Server runs on the local machine; rsync is at `/usr/bin/rsync`
   - What's unclear: Will PATH include rsync when spawned from Node.js PM2 process?
   - Recommendation: Hardcode path in spawn: `spawn('/usr/bin/rsync', args, ...)` or use `which rsync` at startup to resolve path. Avoid PATH dependency in PM2 context.

## Validation Architecture

> `workflow.nyquist_validation` is not present in `.planning/config.json` — skipping this section.

## Sources

### Primary (HIGH confidence)

- System `rsync` tested directly (`/usr/bin/rsync`) — `--archive`, `--itemize-changes`, `--dry-run`, `--link-dest`, `--include`, `--exclude` all confirmed working on macOS 24.6.0
- Node.js built-in `crypto` — `createHash('sha256')` verified working in Node.js on this system
- `server/services/eventScheduler.js` — project source code, cron scheduling verified
- `server/services/brainScheduler.js` — scheduling pattern to mirror
- `server/services/commands.js` — `spawn` with `shell: false` pattern to follow
- `server/lib/commandSecurity.js` — `rsync` is NOT in current ALLOWED_COMMANDS (only matters if using the command executor — backup service will use spawn directly, not commandSecurity)
- `server/lib/fileUtils.js` — `PATHS.data`, `ensureDir`, `readJSONFile` utilities available
- `server/lib/validation.js` — Zod schema patterns for new backup schemas
- `client/src/hooks/useAutoRefetch.js` — polling hook for widget
- `client/src/components/SystemHealthWidget.jsx` — widget structural pattern
- `server/index.js` — route registration pattern; `app.use('/api/backup', backupRoutes)` location

### Secondary (MEDIUM confidence)

- rsync exit code documentation: exit 24 (partial transfer/vanished files) is acceptable for active systems — confirmed across multiple rsync documentation sources
- SHA-256 streaming pattern for large files — standard Node.js pattern, no verification needed

### Tertiary (LOW confidence)

- None — all research items confirmed against project source or live system testing

## Metadata

**Confidence breakdown:**
- Standard stack: HIGH — verified on live system, no external deps needed
- Architecture: HIGH — mirrors existing project patterns exactly
- Pitfalls: HIGH — most identified from reading project code patterns + live rsync testing

**Research date:** 2026-02-26
**Valid until:** 2026-08-26 (stable domain; rsync, Node.js crypto, and project patterns won't change significantly)
