---
phase: 04-cross-domain-insights-engine
plan: 01
type: execute
wave: 1
depends_on: []
files_modified:
  - server/services/insightsService.js
  - server/routes/insights.js
  - server/index.js
  - server/lib/validation.js
  - client/src/services/api.js
autonomous: true
requirements:
  - INS-01
  - INS-02
  - INS-04

must_haves:
  truths:
    - "GET /api/insights/genome-health returns genome markers grouped by clinical category with matched blood test values and confidence levels"
    - "GET /api/insights/themes returns cached LLM-generated taste-identity themes or triggers generation"
    - "POST /api/insights/themes/refresh regenerates theme analysis from current taste profile data"
    - "GET /api/insights/narrative returns cached cross-domain narrative with previous text for diff"
    - "POST /api/insights/narrative/refresh generates new narrative preserving previous text"
    - "All endpoints degrade gracefully when source data is missing (genome, blood tests, taste profile)"
  artifacts:
    - path: "server/services/insightsService.js"
      provides: "Insights engine with correlation, theme, and narrative functions"
      exports: ["getGenomeHealthCorrelations", "getThemeAnalysis", "generateThemeAnalysis", "getCrossDomainNarrative", "refreshCrossDomainNarrative"]
    - path: "server/routes/insights.js"
      provides: "REST API routes for insights"
      exports: ["default (Router)"]
    - path: "client/src/services/api.js"
      provides: "Client API functions for insights endpoints"
      exports: ["getGenomeHealthCorrelations", "getInsightThemes", "refreshInsightThemes", "getInsightNarrative", "refreshInsightNarrative"]
  key_links:
    - from: "server/routes/insights.js"
      to: "server/services/insightsService.js"
      via: "import * as insightsService"
      pattern: "insightsService\\."
    - from: "server/index.js"
      to: "server/routes/insights.js"
      via: "app.use('/api/insights')"
      pattern: "app\\.use.*insights"
    - from: "server/services/insightsService.js"
      to: "server/services/genome.js"
      via: "getGenomeSummary()"
      pattern: "getGenomeSummary"
    - from: "server/services/insightsService.js"
      to: "server/services/meatspaceHealth.js"
      via: "getBloodTests(), getBodyHistory()"
      pattern: "getBloodTests|getBodyHistory"
    - from: "server/services/insightsService.js"
      to: "server/services/taste-questionnaire.js"
      via: "getTasteProfile()"
      pattern: "getTasteProfile"
---

<objective>
Build the insights engine backend: a service layer that correlates genome markers with blood test data (rule-based), generates LLM taste-to-identity themes, and produces cross-domain narrative summaries. Wire up REST routes and client API functions.

Purpose: Establish the complete data and API layer for the Insights dashboard. All three insight domains (genome-health, taste-identity, cross-domain narrative) are served from this backend — the frontend plan consumes these endpoints.

Output: `insightsService.js` with 5 exported functions, `routes/insights.js` with 5 endpoints, Zod validation schemas, client API helpers, `data/insights/` cache directory.
</objective>

<execution_context>
@/Users/antic/.claude/get-shit-done/workflows/execute-plan.md
@/Users/antic/.claude/get-shit-done/templates/summary.md
</execution_context>

<context>
@.planning/PROJECT.md
@.planning/ROADMAP.md
@.planning/STATE.md

<interfaces>
<!-- Key types and contracts the executor needs. Extracted from codebase. -->

From server/services/genome.js:
```javascript
export async function getGenomeSummary()
// Returns: { uploaded: bool, savedMarkers: { [uuid]: { rsid, gene, category, status, name, description, implications, references } }, ... }
// status values: 'beneficial', 'typical', 'concern', 'major_concern', 'not_found'
```

From server/lib/curatedGenomeMarkers.js:
```javascript
export const MARKER_CATEGORIES = {
  longevity: { label: 'Longevity', icon: 'Sparkles', color: 'purple' },
  cardiovascular: { label: 'Cardiovascular', icon: 'HeartPulse', color: 'rose' },
  iron: { label: 'Iron Metabolism', icon: 'Droplet', color: 'red' },
  methylation: { label: 'Methylation', icon: 'Zap', color: 'blue' },
  nutrient: { label: 'Nutrient Metabolism', icon: 'Apple', color: 'emerald' },
  caffeine: { label: 'Caffeine', icon: 'Coffee', color: 'amber' },
  detox: { label: 'Detoxification', icon: 'Shield', color: 'green' },
  inflammation: { label: 'Inflammation', icon: 'Flame', color: 'orange' },
  tumor_suppression: { label: 'Tumor Suppression', icon: 'ShieldCheck', color: 'indigo' },
  cognitive: { label: 'Cognitive', icon: 'Brain', color: 'cyan' },
  cognitive_decline: { label: 'Cognitive Decline & Dementia Risk', icon: 'BrainCog', color: 'rose' },
  sleep: { label: 'Sleep & Circadian', icon: 'Moon', color: 'violet' },
  athletic: { label: 'Athletic Performance', icon: 'Dumbbell', color: 'sky' },
  skin: { label: 'Skin & UV Response', icon: 'Sun', color: 'yellow' },
  diabetes: { label: 'Blood Sugar & Diabetes', icon: 'Droplets', color: 'amber' },
  gut_health: { label: 'Gut Health & Digestion', icon: 'Salad', color: 'lime' },
  autoimmune: { label: 'Autoimmune Risk', icon: 'ShieldAlert', color: 'pink' },
  thyroid: { label: 'Thyroid & Hormones', icon: 'Activity', color: 'teal' },
  eye_health: { label: 'Eye Health', icon: 'Eye', color: 'sky' },
  mental_health: { label: 'Mental Health', icon: 'Brain', color: 'violet' },
  bone_health: { label: 'Bone Health', icon: 'Bone', color: 'stone' },
  pharmacogenomics: { label: 'Pharmacogenomics', icon: 'Pill', color: 'fuchsia' },
  // ... plus cancer-specific categories
};
// 117 total markers across these categories
```

From server/services/meatspaceHealth.js:
```javascript
export async function getBloodTests()
// Returns: { tests: [{ date, ...analytes }], referenceRanges: {} }

export async function getBodyHistory()
// Returns: [{ date, weight, bodyFat, ... }]
```

From server/services/taste-questionnaire.js:
```javascript
export async function getTasteProfile()
// Returns: { sections: [{ id, label, status, summary, progress }], completedCount, profileSummary }

// LLM call pattern (local to taste-questionnaire.js, must be replicated):
async function callProviderAISimple(provider, model, prompt, { temperature, max_tokens })
// Returns: { text: string } on success, { error: string } on failure
```

From server/services/providers.js:
```javascript
export async function getActiveProvider()
export async function getProviderById(id)
```

From server/services/appleHealthQuery.js:
```javascript
export async function getCorrelationData(from, to)
export async function getDailyAggregates(metricName, from, to)
```

From server/lib/fileUtils.js:
```javascript
export const PATHS = { data: '...', health: '...', meatspace: '...', ... };
export async function ensureDir(dir)
export async function readJSONFile(filePath, defaultValue)
```

From server/lib/errorHandler.js:
```javascript
export function asyncHandler(fn)
```

From client/src/services/api.js:
```javascript
async function request(endpoint, options = {})
// Base: /api, auto JSON headers, toast on error, returns parsed JSON
```
</interfaces>
</context>

<tasks>

<task type="auto">
  <name>Task 1: Create insightsService.js with genome-health correlation engine and LLM analysis functions</name>
  <files>server/services/insightsService.js</files>
  <action>
Create `server/services/insightsService.js` with 5 exported functions:

**1. `getGenomeHealthCorrelations()`** (INS-01 — pure rule-based, no LLM):
- Import `getGenomeSummary` from `genome.js`, `getBloodTests` and `getBodyHistory` from `meatspaceHealth.js`, `getCorrelationData` from `appleHealthQuery.js`, `MARKER_CATEGORIES` from `curatedGenomeMarkers.js`
- Call `getGenomeSummary()` — if `!summary.uploaded` return `{ available: false, reason: 'no_genome' }`
- Extract `savedMarkers` from summary, filter out `status === 'not_found'` markers
- Call `getBloodTests()` in parallel — extract latest value per analyte from `tests` array using a `getLatestBloodValues(tests)` helper that normalizes analyte names (lowercase, trim, common abbreviation mapping)
- Define `CATEGORY_BLOOD_MAP` object mapping marker categories to blood analyte names:
  ```
  cardiovascular: ['total_cholesterol', 'ldl', 'hdl', 'triglycerides', 'homocysteine']
  iron: ['ferritin', 'serum_iron', 'transferrin_saturation', 'tibc']
  methylation: ['homocysteine', 'b12', 'folate']
  diabetes: ['fasting_glucose', 'hba1c', 'insulin']
  thyroid: ['tsh', 't3', 't4']
  nutrient: ['vitamin_d', 'b12', 'folate', 'magnesium', 'zinc']
  ```
  (Other categories like longevity, cognitive, sleep, athletic, etc. have no direct blood analyte mapping — include them with `matchedBloodValues: []`)
- Define `CONFIDENCE_FROM_STATUS` mapping:
  ```
  beneficial: { level: 'strong', color: 'green', label: 'Strong Evidence' }
  typical: { level: 'moderate', color: 'yellow', label: 'Moderate Evidence' }
  concern: { level: 'weak', color: 'red', label: 'Elevated Risk Marker' }
  major_concern: { level: 'significant', color: 'red', label: 'Significant Risk Marker' }
  ```
- Group markers by `category` key. For each category group, produce:
  ```
  {
    category: key,
    label: MARKER_CATEGORIES[key].label,
    icon: MARKER_CATEGORIES[key].icon,
    color: MARKER_CATEGORIES[key].color,
    markers: [{ rsid, gene, name, status, description, implications, confidence: CONFIDENCE_FROM_STATUS[status], matchedBloodValues: [...], references }],
    notFoundCount: N
  }
  ```
  where `matchedBloodValues` is an array of `{ analyte, value, date, unit }` objects from the latest blood test data for that category's mapped analytes
- Return `{ available: true, categories: [...], totalMarkers: N, matchedMarkers: N, sources: ['23andMe', ...conditionally 'Apple Health', 'Blood Tests'] }`
- IMPORTANT: Never use language implying causation. Status labels show correlation strength, not causal claims.

**2. `getThemeAnalysis()`** (INS-02 — returns cached themes):
- Read `data/insights/themes.json` using `readJSONFile(path, null)`
- If null (no cached themes), return `{ available: false, reason: 'not_generated' }`
- Otherwise return the cached object as-is: `{ themes: [...], generatedAt, model }`

**3. `generateThemeAnalysis(providerId, model)`** (INS-02 — LLM generation):
- Import `getTasteProfile` from `taste-questionnaire.js`, `getActiveProvider`/`getProviderById` from `providers.js`
- Get taste profile, filter sections that have `summary` populated
- If no completed sections, return `{ available: false, reason: 'no_taste_data' }`
- Get provider (by ID or active). Get model (passed or `provider.defaultModel`)
- Build prompt: provide all section summaries as context, request JSON array of 3-5 theme objects with `title`, `narrative`, `evidence` (array of `{ preference, domain, connection }`), `strength` ("strong"/"moderate"/"tentative")
- Use analytical third-person tone instruction: "The data indicates a pattern of..."
- Replicate `callProviderAISimple` pattern inline (fetch to `provider.endpoint/chat/completions` with Authorization header, temperature 0.4, max_tokens 2000)
- Strip markdown code fences before JSON.parse: `raw.replace(/^```(?:json)?\n?/, '').replace(/\n?```$/, '')`
- Persist to `data/insights/themes.json` with `{ themes, generatedAt: ISO string, model }`
- Use `ensureDir` before writing
- Log: `console.log(\`🧠 Taste-identity themes generated: ${themes.length} themes\`)`
- Return `{ themes, generatedAt }`

**4. `getCrossDomainNarrative()`** (INS-04 — returns cached narrative):
- Read `data/insights/narrative.json` using `readJSONFile(path, null)`
- If null, return `{ available: false, reason: 'not_generated' }`
- Return cached: `{ text, previousText, generatedAt, previousGeneratedAt, model }`

**5. `refreshCrossDomainNarrative(providerId, model)`** (INS-04 — LLM generation with diff support):
- Load existing narrative from `data/insights/narrative.json` if it exists
- Gather context from all domains:
  - Genome correlations via `getGenomeHealthCorrelations()` — summarize top findings by category
  - Taste themes via `getThemeAnalysis()` — include theme titles and summaries
  - Apple Health summary via `getCorrelationData()` (if available)
- Build prompt requesting 2-3 paragraph analytical narrative with personal framing: "Your data shows..."
- Use `callProviderAISimple` pattern (temperature 0.5, max_tokens 2000)
- Strip markdown code fences from response
- Persist to `data/insights/narrative.json`:
  ```
  {
    text: newText,
    previousText: existingNarrative?.text ?? null,
    generatedAt: new Date().toISOString(),
    previousGeneratedAt: existingNarrative?.generatedAt ?? null,
    model
  }
  ```
- Log: `console.log(\`🔮 Cross-domain narrative generated (${newText.length} chars)\`)`
- Return the persisted object

**Helper: `callProviderAISimple(provider, model, prompt, options)`** — local async function:
- Copy pattern exactly from `taste-questionnaire.js` lines 951-986
- Handles API-type providers only, returns `{ text }` or `{ error }`

**Helper: `getLatestBloodValues(tests)`** — pure function:
- Takes array of blood test objects, returns Map of normalized analyte name → `{ value, date, unit }`
- Normalizes keys: lowercase, trim, replace common abbreviations (e.g., "LDL Cholesterol" → "ldl")
- Returns the latest (most recent date) value for each analyte

All file I/O uses `PATHS.data` + 'insights' for the cache directory. Use `ensureDir` before writes. Use `readJSONFile` for reads.
  </action>
  <verify>
    <automated>cd /Users/antic/github.com/atomantic/PortOS/server && node -e "import('./services/insightsService.js').then(m => { const fns = ['getGenomeHealthCorrelations','getThemeAnalysis','generateThemeAnalysis','getCrossDomainNarrative','refreshCrossDomainNarrative']; const missing = fns.filter(f => typeof m[f] !== 'function'); if (missing.length) { console.error('Missing exports:', missing); process.exit(1); } console.log('All 5 functions exported'); })"</automated>
  </verify>
  <done>insightsService.js exports 5 functions. getGenomeHealthCorrelations returns categorized markers with blood value matching. Theme and narrative functions handle cache read, LLM generation, and file persistence. All functions degrade gracefully with { available: false, reason } when source data is missing.</done>
</task>

<task type="auto">
  <name>Task 2: Create insights routes, Zod validation, mount in index.js, and add client API functions</name>
  <files>server/routes/insights.js, server/lib/validation.js, server/index.js, client/src/services/api.js</files>
  <action>
**1. Create `server/routes/insights.js`:**
- Import `Router` from express, `asyncHandler` from `../lib/errorHandler.js`, `* as insightsService` from `../services/insightsService.js`
- Import `validateRequest` and relevant schemas from `../lib/validation.js`

5 routes:

```
GET /genome-health → insightsService.getGenomeHealthCorrelations()
GET /themes → insightsService.getThemeAnalysis()
POST /themes/refresh → insightsService.generateThemeAnalysis(req.body.providerId, req.body.model)
  - Validate body with insightRefreshSchema (optional providerId string, optional model string)
GET /narrative → insightsService.getCrossDomainNarrative()
POST /narrative/refresh → insightsService.refreshCrossDomainNarrative(req.body.providerId, req.body.model)
  - Validate body with insightRefreshSchema
```

All routes wrapped in `asyncHandler`. POST routes use `validateRequest(insightRefreshSchema)` middleware.
Export default router.

**2. Add Zod schema to `server/lib/validation.js`:**
- Add `insightRefreshSchema` (z.object with optional `providerId` as z.string().optional() and `model` as z.string().optional())
- Export alongside existing schemas

**3. Mount in `server/index.js`:**
- Import `insightsRoutes` from `'./routes/insights.js'`
- Add `app.use('/api/insights', insightsRoutes);` — place alphabetically among existing route mounts (after `/api/instances` and before `/api/jira`)

**4. Add client API functions to `client/src/services/api.js`:**
- Add these exports (place in an `// Insights` section, alphabetically positioned among existing sections):
```javascript
// Insights
export const getGenomeHealthCorrelations = () => request('/insights/genome-health');
export const getInsightThemes = () => request('/insights/themes');
export const refreshInsightThemes = (providerId, model) => request('/insights/themes/refresh', {
  method: 'POST',
  body: JSON.stringify({ providerId, model })
});
export const getInsightNarrative = () => request('/insights/narrative');
export const refreshInsightNarrative = (providerId, model) => request('/insights/narrative/refresh', {
  method: 'POST',
  body: JSON.stringify({ providerId, model })
});
```
  </action>
  <verify>
    <automated>cd /Users/antic/github.com/atomantic/PortOS/server && node -e "import('./routes/insights.js').then(m => { const r = m.default; if (!r || !r.stack) { console.error('No router exported'); process.exit(1); } const routes = r.stack.filter(l => l.route).map(l => l.route.path + ' ' + Object.keys(l.route.methods).join(',')); console.log('Routes:', routes); if (routes.length < 5) { console.error('Expected 5 routes'); process.exit(1); } console.log('All 5 routes registered'); })"</automated>
  </verify>
  <done>5 REST endpoints mounted at /api/insights/*, Zod validation on POST bodies, client API functions exported. Server starts without errors. GET endpoints return graceful { available: false } when no data exists. POST endpoints accept optional providerId and model.</done>
</task>

</tasks>

<verification>
1. Server starts without import errors: `cd server && node -e "import('./index.js')"`
2. GET /api/insights/genome-health responds (200 with data or graceful degradation)
3. GET /api/insights/themes responds with `{ available: false, reason: 'not_generated' }` initially
4. GET /api/insights/narrative responds with `{ available: false, reason: 'not_generated' }` initially
5. POST /api/insights/themes/refresh with empty body returns 200 or appropriate error if no taste data
6. POST /api/insights/narrative/refresh with empty body returns 200 or appropriate error
7. Client api.js exports all 5 insight functions
</verification>

<success_criteria>
- insightsService.js exports 5 functions matching the interface contract
- All 5 routes return valid JSON responses
- Genome-health correlations group markers by MARKER_CATEGORIES with blood test value matching
- LLM functions persist results to data/insights/*.json with generatedAt timestamps
- Narrative refresh preserves previousText for client-side diff
- Empty states return { available: false, reason: '...' } not errors
- No causation language in genome correlation labels — only "evidence" and "risk marker"
</success_criteria>

<output>
After completion, create `.planning/phases/04-cross-domain-insights-engine/04-01-SUMMARY.md`
</output>
