# Plan: Daily Brief Signal Quality

## Overview

Improve the reporting pipeline so `topic-radar` separates candidate discovery
from signal quality, and `daily-brief` uses that quality metadata when deciding
what belongs in Topline versus `Early Watchlist`. The implementation keeps
source fetching and ranking inside `topic-radar`, keeps synthesis rules inside
`daily-brief`, and uses the Heuristic System only for retained feedback and
operation evidence. The goal is not to hide low-signal items; it is to label
them accurately and make future reports more useful after repeated corrections.

## Read First

- Primary source: `docs/plans/daily-brief-signal-quality/daily-brief-signal-quality-discussion-source.md`
- Source type: `discussion-to-implementation-doc`
- Open questions carried into execution:
  - Choose exact high/medium/watchlist numeric thresholds during implementation
    after reviewing fixtures and current sample output.

## Execution

- Execution state:
  `docs/plans/daily-brief-signal-quality/daily-brief-signal-quality-execution-state.md`

## Scope

- In scope: `topic-radar` signal metrics, optional public GitHub URL
  enrichment, `signalTier` classification, brief/watchlist JSON and Markdown
  output, `daily-brief` synthesis rules, focused tests, sample fixtures, and a
  compact Heuristic System operation record after validation.
- Out of scope: paid or authenticated APIs, broad ranking redesign, removing
  low-signal items from source output, changing personal daily-brief record
  storage, and nils-cli primitive changes.

## Assumptions

1. Public GitHub REST lookup is acceptable for enrichment only when a source
   item already links to a public repository. It must degrade through source
   errors or missing metrics without failing the whole digest.
2. `topic-radar` can add backward-compatible JSON fields without changing the
   existing CLI contract or requiring downstream consumers to update
   immediately.
3. Routine daily brief generation should not create retained `skill-usage`
   records by default. User correction and validation workflows should.
4. `Early Watchlist` is useful when the topic is relevant but popularity,
   adoption, source backing, or cross-source evidence is weak.
5. The first implementation should favor simple explainable thresholds over a
   complicated ranking model.

## Sprint 1: Signal Policy And Fixtures

**Goal**: Define the signal-quality model and create fixtures that make the
GateGraph failure mode reproducible before touching ranking behavior.
**Demo/Validation**:

- Command(s): `scripts/check.sh --docs`, `scripts/check.sh --markdown`
- Verify: source-strategy and skill docs describe signal tiers, watchlist
  placement, and the difference between heuristic rank and report importance.

**PR grouping intent**: group
**Execution Profile**: parallel-x2

### Task 1.1: Define signal-tier policy in source strategy

- **Location**:
  - `skills/tools/market-research/topic-radar/references/source-strategy.md`
  - `skills/tools/market-research/topic-radar/SKILL.md`
- **Description**: Add a compact signal-quality section that defines
  `high-signal`, `medium-signal`, and `early-watchlist`. Make the contract
  explicit: heuristic score is candidate triage, while `signalTier` is the
  presentation hint used by report writers.
- **Dependencies**:
  - none
- **Complexity**:
  - 2
- **Acceptance criteria**:
  - Docs say `recency + topic match` alone is not enough for Topline.
  - Docs define source-strength inputs: official/mainstream source,
    cross-source duplication, HN points/comments, GitHub stars/forks, and
    source age.
  - Docs preserve the existing warning that scores are not objective
    importance.
- **Validation**:
  - `scripts/check.sh --docs`
  - `scripts/check.sh --markdown`

### Task 1.2: Add low-signal reporting fixtures

- **Location**:
  - `skills/tools/market-research/topic-radar/tests/test_tools_market_research_topic_radar.py`
  - `skills/tools/market-research/topic-radar/bin/topic_radar.py`
- **Description**: Introduce a GateGraph-like fixture in tests: HN item, 1
  point, 0 comments, GitHub repository URL, 2 stars, 0 forks, no cross-source
  duplicate. The fixture should be usable by unit tests without live network
  calls.
- **Dependencies**:
  - Task 1.1
- **Complexity**:
  - 3
- **Acceptance criteria**:
  - Fixture captures HN metrics and GitHub repo metrics separately.
  - Tests can assert future classification without relying on live GitHub or HN.
  - Existing sample mode remains deterministic.
- **Validation**:
  - `scripts/test.sh -k tools_market_research_topic_radar`

### Task 1.3: Add daily-brief policy test expectations

- **Location**:
  - `skills/workflows/reporting/daily-brief/tests/test_workflows_reporting_daily_brief.py`
  - `skills/workflows/reporting/daily-brief/SKILL.md`
- **Description**: Add tests that require `daily-brief` docs to mention
  `signalTier`, Topline eligibility, `Early Watchlist`, and visible metrics for
  low-signal items.
- **Dependencies**:
  - Task 1.1
- **Complexity**:
  - 2
- **Acceptance criteria**:
  - Tests fail if the skill stops instructing agents to demote low-signal
    repos.
  - Tests keep the existing boundary that `daily-brief` consumes
    `topic-radar --format json` rather than implementing fetchers.
- **Validation**:
  - `scripts/test.sh -k workflows_reporting_daily_brief`

## Sprint 2: Topic Radar Metrics And Tiering

**Goal**: Add machine-readable metrics and presentation tiers to `topic-radar`
without removing any source items from the raw ranked list.
**Demo/Validation**:

- Command(s): `scripts/test.sh -k tools_market_research_topic_radar`
- Verify: JSON output includes signal metrics and a GateGraph-like item is
  classified as `early-watchlist`.

**PR grouping intent**: group
**Execution Profile**: serial

### Task 2.1: Extend RadarItem with normalized signal metrics

- **Location**:
  - `skills/tools/market-research/topic-radar/bin/topic_radar.py`
- **Description**: Add fields such as `signal_tier`, `signal_reasons`, and
  `signal_metrics` to `RadarItem`. Populate source-native metrics for HN
  points/comments and GitHub stars/forks instead of relying only on the
  formatted `reason` string.
- **Dependencies**:
  - Task 1.2
- **Complexity**:
  - 4
- **Acceptance criteria**:
  - `RadarItem.to_json()` emits stable fields such as `signalTier`,
    `signalReasons`, and `signalMetrics`.
  - Existing fields remain backward compatible.
  - HN and GitHub source fetchers preserve points/comments/stars/forks in
    machine-readable form.
- **Validation**:
  - `scripts/test.sh -k tools_market_research_topic_radar`

### Task 2.2: Enrich public GitHub repository URLs from non-GitHub sources

- **Location**:
  - `skills/tools/market-research/topic-radar/bin/topic_radar.py`
- **Description**: Add a small enrichment pass that detects public
  `github.com/<owner>/<repo>` URLs from HN/news/official items, fetches public
  repository metadata with existing cache/refresh behavior, and attaches stars,
  forks, pushed time, and repository age to `signalMetrics`. Rate limits or
  malformed responses should be reported as enrichment gaps without failing the
  digest.
- **Dependencies**:
  - Task 2.1
- **Complexity**:
  - 5
- **Acceptance criteria**:
  - HN items linking to GitHub repositories can expose GitHub star/fork metrics.
  - Enrichment is bounded by the current result set or a small fixed cap.
  - Enrichment uses public read-only requests and existing timeout/cache
    behavior.
  - Source errors identify enrichment failures without hiding the original item.
- **Validation**:
  - `scripts/test.sh -k tools_market_research_topic_radar`

### Task 2.3: Implement signal-tier classification

- **Location**:
  - `skills/tools/market-research/topic-radar/bin/topic_radar.py`
- **Description**: Add deterministic classification rules. Early rules should
  demote single-source HN/GitHub items with very low engagement and low GitHub
  stars, promote official/mainstream/cross-source items appropriately, and
  allow medium signal for moderately engaged community or open-source items.
- **Dependencies**:
  - Task 2.1
  - Task 2.2
- **Complexity**:
  - 5
- **Acceptance criteria**:
  - GateGraph-like fixture is `early-watchlist`.
  - Official release fixture is not demoted only because engagement is zero.
  - Cross-source duplicate fixture is at least medium signal.
  - Ranking score and `signalTier` are both present and have distinct
    explanations.
- **Validation**:
  - `scripts/test.sh -k tools_market_research_topic_radar`

### Task 2.4: Separate brief clusters from early watchlist output

- **Location**:
  - `skills/tools/market-research/topic-radar/bin/topic_radar.py`
- **Description**: Update `build_brief_clusters`, JSON rendering, and Markdown
  rendering so high/medium items appear in normal brief clusters while
  early-watchlist items appear in a distinct `brief.watchlist` or Markdown
  `Early Watchlist` section. Keep full `items` and `sections` output complete.
- **Dependencies**:
  - Task 2.3
- **Complexity**:
  - 4
- **Acceptance criteria**:
  - Early-watchlist items are not lost from JSON.
  - Markdown brief labels early-watchlist items with visible metrics.
  - Existing non-brief `Top Signals` output still lists ranked candidates with
    score and signal tier.
- **Validation**:
  - `scripts/test.sh -k tools_market_research_topic_radar`
  - `skills/tools/market-research/topic-radar/scripts/topic-radar.sh --sample --preset ai-news --format json`
  - `skills/tools/market-research/topic-radar/scripts/topic-radar.sh --sample --preset ai-news --format markdown`

## Sprint 3: Daily Brief And Heuristic Feedback Loop

**Goal**: Teach the user-facing daily workflow to use `signalTier` and preserve
the correction as durable Heuristic System learning.
**Demo/Validation**:

- Command(s): `scripts/test.sh -k workflows_reporting_daily_brief`,
  `scripts/check.sh --docs`
- Verify: the daily brief skill instructs agents to keep Topline high signal and
  to put low-star/low-engagement repos into `Early Watchlist`.

**PR grouping intent**: group
**Execution Profile**: parallel-x2

### Task 3.1: Update daily-brief synthesis rules

- **Location**:
  - `skills/workflows/reporting/daily-brief/SKILL.md`
- **Description**: Update the workflow section so daily synthesis consumes
  `signalTier`, `signalMetrics`, and `brief.watchlist` when present. Add clear
  instructions: Topline uses high/medium signals, `Early Watchlist` carries
  low-signal but relevant items, and watchlist bullets must show metrics such as
  GitHub stars and HN points/comments.
- **Dependencies**:
  - Task 2.4
  - Task 1.3
- **Complexity**:
  - 3
- **Acceptance criteria**:
  - The skill says low-star/low-HN items are topic samples, not popularity or
    adoption evidence.
  - The skill keeps source-grounded links and explicit inference language.
  - The skill does not duplicate ranking/fetching logic.
- **Validation**:
  - `scripts/test.sh -k workflows_reporting_daily_brief`
  - `scripts/check.sh --docs`

### Task 3.2: Add reporting-feedback recording guidance

- **Location**:
  - `HEURISTIC_SYSTEM.md`
  - `docs/runbooks/skills/SKILL_USAGE_RECORDING_V1.md`
  - `skills/tools/workflow-evidence/skill-usage/SKILL.md`
- **Description**: Add a narrow guidance note for reporting-skill corrections:
  routine briefs do not need retained skill-usage records, but user corrections
  that change skill behavior should be captured during implementation and then
  compressed into tests/docs/operation records.
- **Dependencies**:
  - Task 3.1
- **Complexity**:
  - 3
- **Acceptance criteria**:
  - Guidance preserves the boundary that `skill-usage` is a primitive and
    agent-kit owns judgment/promotion.
  - Guidance says to serialize record writes and verify final records.
  - Guidance discourages raw record commits unless curated or compressed.
- **Validation**:
  - `scripts/check.sh --docs`
  - `scripts/check.sh --markdown`

### Task 3.3: Create a compact operation record after validation

- **Location**:
  - `heuristic-system/operation-records/daily-brief-signal-quality.md`
- **Description**: After the implementation is validated, add a short operation
  record that links the GateGraph correction, diagnosis, durable changes, and
  validation commands. Keep it compressed and avoid copying raw daily brief JSON.
- **Dependencies**:
  - Task 2.4
  - Task 3.1
  - Task 3.2
- **Complexity**:
  - 2
- **Acceptance criteria**:
  - Record proves the heuristic loop operated: signal, diagnosis, promotion,
    durable fix, validation, retention outcome.
  - Record links source paths rather than duplicating raw output.
  - Record is small enough to remain useful after plan cleanup.
- **Validation**:
  - `scripts/check.sh --docs`
  - `scripts/check.sh --markdown`

### Task 3.4: Update docs tests for reporting feedback boundaries

- **Location**:
  - `tests/test_heuristic_system_docs.py`
  - `skills/tools/workflow-evidence/skill-usage/tests/test_tools_devex_skill_usage.py`
- **Description**: Add focused assertions that reporting corrections can be
  retained through `skill-usage` and operation records without requiring every
  routine report to create raw retained evidence.
- **Dependencies**:
  - Task 3.2
  - Task 3.3
- **Complexity**:
  - 3
- **Acceptance criteria**:
  - Tests fail if docs imply all daily briefs require skill-usage records.
  - Tests fail if docs blur primitive writing with skill-level policy.
- **Validation**:
  - `scripts/check.sh --tests -- tests/test_heuristic_system_docs.py`
  - `scripts/check.sh --tests -- -k tools_devex_skill_usage`

## Sprint 4: Validation, Compatibility, And Delivery Readiness

**Goal**: Prove the updated reporting pipeline is backward compatible,
source-grounded, and ready for a normal feature PR.
**Demo/Validation**:

- Command(s): `scripts/check.sh --pre-commit`
- Verify: topic-radar sample output, daily-brief docs, Heuristic System docs,
  markdown lint, and script smoke specs pass together.

**PR grouping intent**: per-sprint
**Execution Profile**: serial

### Task 4.1: Validate JSON compatibility and sample output

- **Location**:
  - `skills/tools/market-research/topic-radar/tests/test_tools_market_research_topic_radar.py`
  - `tests/script_specs/skills/tools/market-research/topic-radar/scripts/topic-radar.sh.json`
- **Description**: Extend sample-output assertions and smoke specs as needed so
  the new JSON fields are stable and the CLI help/output contract remains
  documented.
- **Dependencies**:
  - Task 2.4
- **Complexity**:
  - 3
- **Acceptance criteria**:
  - Sample JSON includes `signalTier` and `signalMetrics`.
  - Sample Markdown includes signal-tier language and `Early Watchlist` when the
    sample contains low-signal data.
  - Existing script smoke coverage remains aligned with the entrypoint.
- **Validation**:
  - `scripts/check.sh --tests -- -k tools_market_research_topic_radar`
  - `scripts/check.sh --tests -- -m script_smoke`

### Task 4.2: Run focused workflow gates

- **Location**:
  - `skills/tools/market-research/topic-radar/bin/topic_radar.py`
  - `skills/tools/market-research/topic-radar/SKILL.md`
  - `skills/tools/market-research/topic-radar/references/source-strategy.md`
  - `skills/tools/market-research/topic-radar/tests/test_tools_market_research_topic_radar.py`
  - `skills/workflows/reporting/daily-brief/SKILL.md`
  - `skills/workflows/reporting/daily-brief/tests/test_workflows_reporting_daily_brief.py`
  - `HEURISTIC_SYSTEM.md`
  - `docs/runbooks/skills/SKILL_USAGE_RECORDING_V1.md`
- **Description**: Run the focused test and docs gates that correspond to the
  touched workflow surfaces before attempting the aggregate pre-commit gate.
- **Dependencies**:
  - Task 3.4
  - Task 4.1
- **Complexity**:
  - 2
- **Acceptance criteria**:
  - Topic-radar tests pass.
  - Daily-brief tests pass.
  - Heuristic System and skill-usage docs tests pass.
  - Docs and markdown checks pass.
- **Validation**:
  - `scripts/test.sh -k tools_market_research_topic_radar`
  - `scripts/test.sh -k workflows_reporting_daily_brief`
  - `scripts/check.sh --tests -- tests/test_heuristic_system_docs.py`
  - `scripts/check.sh --docs`
  - `scripts/check.sh --markdown`

### Task 4.3: Run repository pre-commit gate and prepare delivery

- **Location**:
  - repository root
- **Description**: Run the preferred pre-commit gate, review changed files for
  scope, and prepare the final PR/change summary. If unrelated pre-existing
  failures appear, isolate them in the report and preserve focused validation
  evidence for this plan.
- **Dependencies**:
  - Task 4.2
- **Complexity**:
  - 2
- **Acceptance criteria**:
  - `scripts/check.sh --pre-commit` passes, or unrelated failures are clearly
    separated with focused checks passing.
  - Final summary names the behavior change, validation, and any compatibility
    notes.
  - No raw skill-usage records or raw daily-brief JSON are committed unless
    intentionally curated as fixtures.
- **Validation**:
  - `scripts/check.sh --pre-commit`

## Testing Strategy

- Unit: add focused tests for signal metrics, GitHub URL enrichment fallback,
  `signalTier`, and brief/watchlist separation in
  `skills/tools/market-research/topic-radar/tests/test_tools_market_research_topic_radar.py`.
- Contract/docs: update daily-brief and Heuristic System tests so workflow
  boundaries remain explicit.
- Script smoke: update topic-radar smoke specs only if the entrypoint help or
  deterministic sample expectations change.
- Manual sample: run `topic-radar.sh --sample --preset ai-news --format json`
  and Markdown output to inspect the generated `Early Watchlist` shape.

## Risks & gotchas

- GitHub URL enrichment can add network latency or rate-limit noise. Keep it
  bounded, cached, and non-fatal.
- Thresholds can become arbitrary. Keep them simple, documented, and covered by
  fixtures rather than implying objective importance.
- Over-filtering would lose useful emerging-topic signals. The target is
  demotion and labeling, not deletion.
- `daily-brief` must not parse source-specific URLs itself; it should consume
  `topic-radar` metadata.
- Raw `skill-usage` records should not become a new pile of report history.
  Compress corrections into tests/docs/operation records.
- Existing users of JSON output may ignore new fields. Add fields
  backward-compatibly and avoid renaming existing keys.

## Rollback plan

- Revert the topic-radar signal-tier/enrichment patch to return to current
  ranking-only output.
- Keep any pure docs warning about heuristic score if it remains accurate and
  useful.
- Remove the operation record if the implementation is reverted before
  validation; otherwise update it with the rollback outcome.
- Re-run `scripts/test.sh -k tools_market_research_topic_radar`,
  `scripts/test.sh -k workflows_reporting_daily_brief`, and
  `scripts/check.sh --docs` after rollback.
