# @harness-engineering/intelligence

Intelligence pipeline for spec enrichment, complexity modeling, and pre-execution simulation. Augments the hybrid orchestrator's routing with graph-backed analysis and tiered simulation.

**Version:** 0.2.2

## Installation

```bash
npm install @harness-engineering/intelligence
```

## Architecture

```
                         +---------------------+
                         |    Work Item Input   |
                         |  Roadmap . JIRA .    |
                         |  GitHub . Linear .   |
                         |  Manual text         |
                         +----------+----------+
                                    |
                                    v
                         +---------------------+
                         |      Adapters        |
                         |  toRawWorkItem()     |
                         |  jiraToRawWorkItem() |
                         |  githubToRawWorkItem |
                         |  linearToRawWorkItem |
                         |  manualToRawWorkItem |
                         +----------+----------+
                                    |
                                    v
+----------------------------------------------------------------+
|                    IntelligencePipeline                         |
|                                                                |
|  +----------------------------------------------------------+  |
|  |  SEL -- Spec Enrichment Layer                             |  |
|  |  LLM analysis + graph-validated system discovery          |  |
|  |  -> EnrichedSpec (intent, affected systems, unknowns)     |  |
|  +------------------------+---------------------------------+  |
|                           |                                    |
|                           v                                    |
|  +----------------------------------------------------------+  |
|  |  CML -- Complexity Modeling Layer                         |  |
|  |  Structural (graph blast radius) + Semantic (ambiguity)   |  |
|  |  + Historical (past outcome failure rate)                 |  |
|  |  -> ComplexityScore -> ConcernSignal[]                    |  |
|  +------------------------+---------------------------------+  |
|                           |                                    |
|                           v                                    |
|  +----------------------------------------------------------+  |
|  |  PESL -- Pre-Execution Simulation Layer                   |  |
|  |  Graph-only checks (quick-fix) or full LLM simulation     |  |
|  |  (guided-change) with abort on low confidence             |  |
|  |  -> SimulationResult (confidence, abort flag)             |  |
|  +----------------------------------------------------------+  |
|                                                                |
|  +----------------------------------------------------------+  |
|  |  Outcome Recording                                        |  |
|  |  ExecutionOutcomeConnector -> graph nodes + edges         |  |
|  |  Feeds back into CML historical dimension                 |  |
|  +----------------------------------------------------------+  |
+----------------------------------------------------------------+
```

## Quick Start

```ts
import {
  IntelligencePipeline,
  AnthropicAnalysisProvider,
  toRawWorkItem,
} from '@harness-engineering/intelligence';
import { GraphStore } from '@harness-engineering/graph';

// 1. Create the pipeline
const provider = new AnthropicAnalysisProvider({
  apiKey: process.env.ANTHROPIC_API_KEY!,
});
const store = new GraphStore();
await store.load('.harness/graph');

const pipeline = new IntelligencePipeline(provider, store);

// 2. Preprocess an issue (SEL -> CML -> signals)
const result = await pipeline.preprocessIssue(issue, scopeTier, escalationConfig);
// result.spec    -- EnrichedSpec with intent, affected systems, unknowns
// result.score   -- ComplexityScore with overall 0-1, risk level, reasoning
// result.signals -- ConcernSignal[] for routeIssue()

// 3. Simulate before dispatch (PESL)
if (result.spec && result.score) {
  const simulation = await pipeline.simulate(result.spec, result.score, scopeTier);
  // simulation.abort -- true if confidence too low
  // simulation.predictedFailures -- what might go wrong
  // simulation.testGaps -- missing test coverage
}

// 4. Record execution outcomes (feedback loop)
await pipeline.recordOutcome({
  issueId: 'CORE-42',
  result: 'success',
  retryCount: 0,
  failureReasons: [],
  durationMs: 45000,
  affectedSystemNodeIds: ['node-abc'],
});
```

## Tier-Based Behavior

The pipeline adapts its behavior based on the orchestrator's scope tier:

| Scope Tier                            | SEL  | CML  | PESL                | Routing                                 |
| ------------------------------------- | ---- | ---- | ------------------- | --------------------------------------- |
| `autoExecute` (quick-fix, diagnostic) | Skip | Skip | Graph-only          | Always dispatch locally                 |
| `signalGated` (guided-change)         | Run  | Run  | Full LLM simulation | Dispatch if no concern signals          |
| `alwaysHuman` (full-exploration)      | Run  | Skip | Skip                | Always escalate (enrichment as context) |

---

## Layers

### SEL -- Spec Enrichment Layer

Converts raw work items into structured `EnrichedSpec` objects via LLM analysis and graph validation.

```ts
import { enrich, GraphValidator } from '@harness-engineering/intelligence';

const validator = new GraphValidator(store);
const spec = await enrich(rawWorkItem, provider, validator);
// spec.intent -- what the task is trying to accomplish
// spec.affectedSystems -- graph-validated system references
// spec.unknowns -- explicitly identified knowledge gaps
// spec.ambiguities -- areas needing clarification
```

### CML -- Complexity Modeling Layer

Scores task complexity across three dimensions:

- **Structural** -- Graph blast radius via `CascadeSimulator`, normalized to 0-1
- **Semantic** -- Unknowns, ambiguities, and risk signals from SEL enrichment
- **Historical** -- Smoothed failure rate from past execution outcomes on the same affected systems

```ts
import { scoreCML, scoreToConcernSignals } from '@harness-engineering/intelligence';

const score = scoreCML(enrichedSpec, store);
// score.overall -- 0-1 weighted composite
// score.riskLevel -- 'low' | 'medium' | 'high' | 'critical'
// score.recommendedRoute -- 'local' | 'human' | 'simulation-required'

const signals = scoreToConcernSignals(score);
// signals fed into existing routeIssue() for signalGated tiers
```

### PESL -- Pre-Execution Simulation Layer

Simulates execution before code is written. Tiered by cost:

- **Graph-only** (quick-fix/diagnostic) -- `CascadeSimulator` blast radius + test gap detection. Fast (<2s), no LLM cost.
- **Full simulation** (guided-change) -- LLM-assisted plan expansion, failure injection, test projection. Merged with graph baseline.

```ts
import { PeslSimulator } from '@harness-engineering/intelligence';

const simulator = new PeslSimulator(provider, store);
const result = await simulator.simulate(spec, score, scopeTier);

if (result.abort) {
  // Confidence too low -- escalate to human instead of dispatching
  console.log('Predicted failures:', result.predictedFailures);
  console.log('Test gaps:', result.testGaps);
}
```

---

## Adapters

Pure mapping functions for converting external data into `RawWorkItem`:

```ts
import {
  toRawWorkItem, // Roadmap Issue -> RawWorkItem
  jiraToRawWorkItem, // JIRA issue -> RawWorkItem
  githubToRawWorkItem, // GitHub issue/PR -> RawWorkItem
  linearToRawWorkItem, // Linear issue -> RawWorkItem
  manualToRawWorkItem, // Free text -> RawWorkItem
} from '@harness-engineering/intelligence';
```

Each adapter accepts a pre-fetched data object and produces a `RawWorkItem`. No API clients are included -- adapters are pure functions.

---

## Effectiveness -- Agent Performance Introspection

Analyzes persona-attributed `execution_outcome` nodes in the graph to score per-persona accuracy, detect blind spots, and recommend which persona to route new issues to.

```ts
import {
  computePersonaEffectiveness,
  detectBlindSpots,
  recommendPersona,
} from '@harness-engineering/intelligence';

// Per-(persona, system) success rates with Laplace smoothing
const scores = computePersonaEffectiveness(store, { persona: 'backend-dev' });
// scores[0].successRate -- smoothed success rate in [0, 1]
// scores[0].sampleSize  -- total observations

// Blind spots: (persona, system) pairs with high failure rates
const spots = detectBlindSpots(store, { minFailures: 2, minFailureRate: 0.5 });
// spots[0].failureRate -- raw failure rate
// spots[0].failures    -- absolute failure count

// Recommend personas for an issue given its affected systems
const recs = recommendPersona(store, {
  systemNodeIds: ['module-auth', 'module-db'],
  candidatePersonas: ['backend-dev', 'fullstack'],
  minSamples: 3,
});
// recs[0].persona -- best candidate
// recs[0].score   -- mean smoothed success rate across requested systems
```

---

## Specialization -- Persistent Agent Expertise Tracking

Extends effectiveness with temporal decay, task-type categorization, consistency scoring, expertise levels, and persistent profile storage. Recent outcomes are weighted more heavily than old ones via exponential decay.

```ts
import {
  computeSpecialization,
  computeExpertiseLevel,
  buildSpecializationProfile,
  weightedRecommendPersona,
  decayWeight,
  temporalSuccessRate,
  loadProfiles,
  saveProfiles,
  refreshProfiles,
} from '@harness-engineering/intelligence';

// Compute specialization entries for (persona, system, taskType) tuples
const entries = computeSpecialization(store, {
  persona: 'backend-dev',
  taskType: 'bug-fix',
  temporal: { halfLifeDays: 30 },
});
// entries[0].score.composite         -- weighted combination of temporal, consistency, volume
// entries[0].expertiseLevel          -- 'novice' | 'competent' | 'proficient' | 'expert'

// Classify expertise from raw numbers
const level = computeExpertiseLevel(25, 0.8); // -> 'proficient'

// Build a full profile for a persona (strengths, weaknesses, overall level)
const profile = buildSpecializationProfile(store, 'backend-dev', {
  temporal: { halfLifeDays: 30 },
});
// profile.strengths   -- top 3 entries by composite score
// profile.weaknesses  -- entries with temporal success rate < 0.5
// profile.overallLevel -- median expertise across all entries

// Weighted recommendation incorporating specialization multipliers
const weighted = weightedRecommendPersona(store, {
  systemNodeIds: ['module-auth'],
  taskType: 'bug-fix',
});
// weighted[0].weightedScore -- baseScore * specializationMultiplier

// Temporal decay helpers
const weight = decayWeight(15, 30); // weight at 15 days with 30-day half-life
const rate = temporalSuccessRate([{ result: 'success', timestamp: '2026-04-01T00:00:00Z' }], {
  halfLifeDays: 30,
});

// Persist profiles to .harness/specialization-profiles.json
const profiles = loadProfiles('/path/to/project');
saveProfiles('/path/to/project', profiles);

// Recompute and persist profiles for all personas with outcomes
const refreshed = refreshProfiles('/path/to/project', store);
```

---

## Outcome Recording

Execution results feed back into the graph for future CML scoring:

```ts
import { ExecutionOutcomeConnector } from '@harness-engineering/intelligence';

const connector = new ExecutionOutcomeConnector(store);
await connector.ingest({
  issueId: 'CORE-42',
  result: 'failure',
  retryCount: 2,
  failureReasons: ['Migration script failed'],
  durationMs: 120000,
  affectedSystemNodeIds: ['module-auth'],
});
// Creates execution_outcome node with outcome_of edges to affected systems
// Future CML scoring queries these outcomes for historical failure rates
```

---

## Orchestrator Integration

The intelligence pipeline integrates into the orchestrator's tick cycle at two points:

1. **Pre-routing** (in `asyncTick`) -- `preprocessIssue()` runs SEL + CML, producing concern signals that feed into `routeIssue()`. For `alwaysHuman` tiers, the enriched spec is attached to the `EscalateEffect` as context for the human reviewer.

2. **Post-routing** (in `asyncTick`) -- `simulate()` runs PESL for locally-routed issues. If simulation recommends abort (`confidence < 0.3`), the dispatch is converted to an `EscalateEffect` instead.

3. **Post-execution** (in `emitWorkerExit`) -- `recordOutcome()` ingests the execution result into the graph, feeding the CML historical dimension for future scoring.

### Dashboard

The dashboard's **Attention** page displays escalated interactions. When the intelligence pipeline is enabled, escalations include enriched context:

- **Enriched spec summary** -- Intent, affected systems, unknowns
- **Concern signals** -- What triggered the escalation (high complexity, large blast radius, high ambiguity)
- **PESL abort reasons** -- Predicted failures and test gaps (when simulation recommends abort)

This context helps human reviewers make faster, more informed decisions about escalated work items.

---

## Configuration

The pipeline uses the same LLM connection as the orchestrator's agent backend -- Anthropic API, OpenAI API, or a local LLM (Ollama, LM Studio, etc.):

```yaml
intelligence:
  enabled: true
```

No separate API key needed. The pipeline derives its provider from the existing agent config:

| Agent Backend                     | Intelligence Uses                 |
| --------------------------------- | --------------------------------- |
| `anthropic` / `claude`            | Anthropic Messages API (same key) |
| `openai`                          | OpenAI Chat API (same key)        |
| `localBackend: openai-compatible` | Local endpoint (same URL)         |

Override models per-layer:

```yaml
intelligence:
  enabled: true
  models:
    sel: llama3.2 # fast model for enrichment
    pesl: deepseek-r1 # reasoning model for simulation
```

For thinking models (Qwen3, DeepSeek-R1), disable reasoning for structured output:

```yaml
intelligence:
  enabled: true
  promptSuffix: '/no_think' # Qwen3 -- disable thinking for structured output
  jsonMode: false # Qwen3 hangs with Ollama's JSON grammar constraint
  requestTimeoutMs: 90000 # fail fast instead of waiting 10 min
  failureCacheTtlMs: 300000 # don't retry failed analyses for 5 min
```

When `enabled: false` (default), the pipeline is completely skipped. See the [Intelligence Pipeline Guide](../guides/intelligence-pipeline.md) for full configuration reference.

---

## Dependencies

```
@harness-engineering/types  -> shared type definitions
@harness-engineering/graph  -> GraphStore, CascadeSimulator, node/edge types
@anthropic-ai/sdk           -> LLM calls via AnalysisProvider
openai                      -> OpenAI-compatible analysis provider
zod                         -> response schema validation
```

The intelligence package has **no dependency** on `@harness-engineering/orchestrator`. The dependency flows one way: `orchestrator -> intelligence -> graph -> types`.

---

## API Reference

### Pipeline

| Export                 | Description                                  |
| ---------------------- | -------------------------------------------- |
| `IntelligencePipeline` | Main pipeline class composing SEL, CML, PESL |
| `PreprocessResult`     | Return type of `preprocessIssue()`           |

### Adapters

| Export                | Description                      |
| --------------------- | -------------------------------- |
| `toRawWorkItem`       | Roadmap `Issue` -> `RawWorkItem` |
| `jiraToRawWorkItem`   | JIRA issue -> `RawWorkItem`      |
| `githubToRawWorkItem` | GitHub issue/PR -> `RawWorkItem` |
| `linearToRawWorkItem` | Linear issue -> `RawWorkItem`    |
| `manualToRawWorkItem` | Free text -> `RawWorkItem`       |

### Types

| Export             | Description                                          |
| ------------------ | ---------------------------------------------------- |
| `RawWorkItem`      | Generic work item input                              |
| `EnrichedSpec`     | SEL output with intent, affected systems, unknowns   |
| `ComplexityScore`  | CML output with overall score, risk level, reasoning |
| `SimulationResult` | PESL output with confidence, abort flag, predictions |
| `ExecutionOutcome` | Outcome data for graph ingestion                     |

### Effectiveness

| Export                        | Description                                                    |
| ----------------------------- | -------------------------------------------------------------- |
| `computePersonaEffectiveness` | Per-(persona, system) Laplace-smoothed success rates           |
| `detectBlindSpots`            | Find (persona, system) pairs with high failure rates           |
| `recommendPersona`            | Recommend personas for an issue given affected system node IDs |
| `PersonaEffectivenessScore`   | Type: smoothed success rate for a (persona, system) pair       |
| `BlindSpot`                   | Type: (persona, system) pair with consistent failures          |
| `PersonaRecommendation`       | Type: persona recommendation with score and coverage stats     |

### Specialization

| Export                       | Description                                                                     |
| ---------------------------- | ------------------------------------------------------------------------------- |
| `computeSpecialization`      | Compute specialization entries for (persona, system, taskType) tuples           |
| `computeExpertiseLevel`      | Classify expertise level from sample size and success rate                      |
| `buildSpecializationProfile` | Build a full profile for a persona (strengths, weaknesses, overall level)       |
| `weightedRecommendPersona`   | Persona recommendation with specialization multipliers                          |
| `decayWeight`                | Exponential decay weight for an outcome at a given age                          |
| `temporalSuccessRate`        | Temporally-weighted success rate from timestamped outcomes                      |
| `loadProfiles`               | Load persisted specialization profiles from disk                                |
| `saveProfiles`               | Save specialization profiles to `.harness/specialization-profiles.json`         |
| `refreshProfiles`            | Recompute and persist profiles for all personas with outcomes                   |
| `SpecializationScore`        | Type: composite score (temporal, consistency, volume, composite)                |
| `SpecializationEntry`        | Type: single entry for a (persona, system, taskType) bucket                     |
| `SpecializationProfile`      | Type: full persona profile with strengths, weaknesses, overall level            |
| `WeightedRecommendation`     | Type: recommendation with base score, specialization multiplier, weighted score |
| `ExpertiseLevel`             | Type: `'novice' \| 'competent' \| 'proficient' \| 'expert'`                     |
| `TaskType`                   | Type: task type categorization                                                  |
| `TemporalConfig`             | Type: temporal decay configuration (half-life, reference time)                  |
| `ProfileStore`               | Type: persisted store of specialization profiles                                |

### Analysis

| Export                             | Description                                |
| ---------------------------------- | ------------------------------------------ |
| `AnthropicAnalysisProvider`        | Anthropic-backed `AnalysisProvider`        |
| `OpenAICompatibleAnalysisProvider` | OpenAI/local LLM-backed `AnalysisProvider` |
| `enrich`                           | SEL enrichment function                    |
| `GraphValidator`                   | Graph-based affected system resolver       |
| `scoreCML`                         | CML scoring function                       |
| `scoreToConcernSignals`            | Score -> `ConcernSignal[]` conversion      |
| `PeslSimulator`                    | Tiered simulation facade                   |
| `ExecutionOutcomeConnector`        | Outcome graph ingestion                    |
| `computeHistoricalComplexity`      | CML historical dimension                   |
